Running User App 4.01 (no patches yet, gotta get base installed first)
on WebSphere on Linux.

Prod is fine, tried to fix a broken Dev, and now after many install
iterations we are getting an error JVM/JCE broken error. This seems to
indicate an issue with our JCE security files.

[5/17/13 17:16:47:924 EDT] 00000012 webapp E
com.ibm.ws.webcontainer.webapp.WebApp notifyServletContextCreated
SRVE0283E: Exception c
aught while initializing context: {0}
java.lang.RuntimeException:
com.sssw.fw.security.core.SigCertManagerException:
com.sssw.fw.exception.EboRun
timeException: JVM/JCE is broken.
at
com.sssw.fw.servlet.InitListener.contextInitialize d(InitListener.java:95)
at
com.ibm.ws.webcontainer.webapp.WebApp.notifyServle tContextCreated(WebApp.java:1588)
at
com.ibm.ws.webcontainer.webapp.WebApp.commonInitia lizationFinish(WebApp.java:350)
at
com.ibm.ws.webcontainer.webapp.WebAppImpl.initiali ze(WebAppImpl.java:292)
at
com.ibm.ws.webcontainer.webapp.WebGroupImpl.addWeb Application(WebGroupImpl.java:99)
at
com.ibm.ws.webcontainer.VirtualHostImpl.addWebAppl ication(VirtualHostImpl.java:167)
at
com.ibm.ws.webcontainer.WSWebContainer.addWebApp(W SWebContainer.java:722)
at
com.ibm.ws.webcontainer.WSWebContainer.addWebAppli cation(WSWebContainer.java:607)
at
com.ibm.ws.webcontainer.component.WebContainerImpl .install(WebContainerImpl.java:376)
at
com.ibm.ws.webcontainer.component.WebContainerImpl .start(WebContainerImpl.java:668)
at
com.ibm.ws.runtime.component.ApplicationMgrImpl.st art(ApplicationMgrImpl.java:1162)
at
com.ibm.ws.runtime.component.DeployedApplicationIm pl.fireDeployedObjectStart(DeployedApplicationImpl .java:1313)
at
com.ibm.ws.runtime.component.DeployedModuleImpl.st art(DeployedModuleImpl.java:611)
at
com.ibm.ws.runtime.component.DeployedApplicationIm pl.start(DeployedApplicationImpl.java:938)
at
com.ibm.ws.runtime.component.ApplicationMgrImpl.st artApplication(ApplicationMgrImpl.java:740)
at
com.ibm.ws.runtime.component.ApplicationMgrImpl.st art(ApplicationMgrImpl.java:2092)
at
com.ibm.ws.runtime.component.CompositionUnitMgrImp l.start(CompositionUnitMgrImpl.java:437)
at
com.ibm.ws.runtime.component.CompositionUnitImpl.s tart(CompositionUnitImpl.java:122)
at
com.ibm.ws.runtime.component.CompositionUnitMgrImp l.start(CompositionUnitMgrImpl.java:380)
at
com.ibm.ws.runtime.component.CompositionUnitMgrImp l.access$300(CompositionUnitMgrImpl.java:105)
at
com.ibm.ws.runtime.component.CompositionUnitMgrImp l$CUInitializer.run(CompositionUnitMgrImpl.java:92 8)
at
com.ibm.wsspi.runtime.component.WsComponentImpl$_A synchInitializer.run(WsComponentImpl.java:349)
at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.j ava:1527)
Caused by: com.sssw.fw.security.core.SigCertManagerException:
com.sssw.fw.exception.EboRuntimeException: JVM/JCE is broken.
at
com.sssw.fw.security.core.SigCertManagerBase.<init >(SigCertManagerBase.java:101)
at
com.sssw.fw.security.core.SigCertManagerKeyStoreIm pl.<clinit>(SigCertManagerKeyStoreImpl.java:81)
at java.lang.J9VMInternals.initializeImpl(Native Method)
at java.lang.J9VMInternals.initialize(J9VMInternals.j ava:200)
at java.lang.Class.forNameImpl(Native Method)
at java.lang.Class.forName(Class.java:136)
at
com.sssw.fw.security.core.SigCertManagerFactory.<c linit>(SigCertManagerFactory.java:63)
at java.lang.J9VMInternals.initializeImpl(Native Method)
at java.lang.J9VMInternals.initialize(J9VMInternals.j ava:200)
at
com.sssw.fw.servlet.InitListener.contextInitialize d(InitListener.java:89)
... 22 more
Caused by: com.sssw.fw.exception.EboRuntimeException: JVM/JCE is broken.
at com.sssw.fw.util.crypto.KeyManager.a(Unknown Source)
at com.sssw.fw.util.crypto.KeyManager.decrypt(Unknown Source)
at
com.sssw.fw.security.core.SigCertManagerKeyStoreIm pl.loadNauditKeys(SigCertManagerKeyStoreImpl.java: 228)
at
com.sssw.fw.security.core.SigCertManagerBase.<init >(SigCertManagerBase.java:99)
... 31 more
Caused by: java.lang.ClassNotFoundException:
com.sssw.fw.security.core.SigCertManagerKeyStoreIm pl
at java.lang.Class.forNameImpl(Native Method)
at java.lang.Class.forName(Class.java:136)
... 35 more


The WebSphere guy says he had used the JCE files from Prod. But to be
safe, we got rid of the IDMProv deploy, renamed the idm directory, and
copied in the JCE files from Prod (where it is running), to a horrible
path like:
/home/websphere/IBM/WebSphere/Dmgr/java/jre/lib/security

and we used the JVM at:
/home/websphere/IBM/WebSphere/Dmgr/java
in our install. In fact our configupdate.sh was updated to properly
call that JVM as well by the installer.

As I re-read this error, I noticed the last/first caused by is:
Caused by: java.lang.ClassNotFoundException:
com.sssw.fw.security.core.SigCertManagerKeyStoreIm pl

Well, I looked at the deploy directory under Websphere:
/home/websphere/IBM/WebSphere/Dmgr/profiles/idm/installedApps/testwebsph01Cell01/IDMProv_war.ear/IDMProv.war/WEB-INF/lib

And looking into the xcs-all.jar, no com.ssw.fw class in there.

Looking into the IDMfw.jar, found
com.ssw.fw.security.core, but it is empty. No contents.

Ok, found more in the IDMsec.jar:
com.sssw.fw.security.core and SigCertmanagerKeyStoreImpl.class is there.

And this is in the Deploy context already.

So not like it is not in the path?

Any ideas?