Downloaded and applied the 10.2.2 update to our Development ZENworks zone. The first stage updates our two primary servers in our zone, then the workstations in the second stage.

During the first stage, our first primary server stalled during the "Updating Deployment Packages" stage. After 3 days the update timed out and the status was changed to Error. The second primary completed successfully.

I have tried clearing the errors and restarting the deployment several times. Every time the first primary has failed. What I have found is that I cannot open the ZCC for the failed primary (browser error below):


--------------------------------------------------------------------------
HTTP Status 500 -
--------------------------------------------------------------------------

type Exception report

message

description The server encountered an internal error () that prevented it from fulfilling this request.

exception

javax.servlet.ServletException: Servlet execution threw an exception
com.patchlink.sapphire.web.pages.vulnerability.ses sion.HibernateSessionFilter.doFilter(HibernateSess ionFilter.java:95)


root cause

javax.servlet.ServletException: Servlet execution threw an exception
com.patchlink.sapphire.web.pages.vulnerability.ses sion.HibernateSessionFilter.doFilter(HibernateSess ionFilter.java:75)


root cause

java.lang.NoClassDefFoundError: org/hibernate/HibernateException
com.novell.zenworks.fw.web.util.PageUtil.getServic eStore(PageUtil.java:406)
com.novell.zenworks.fw.web.internal.ZENworksAccess Manager.checkAccess(ZENworksAccessManager.java:85)
com.novell.web.NJWCServlet.preService(NJWCServlet. java:148)
com.novell.zenworks.fw.web.internal.ZENworksServle t.service(ZENworksServlet.java:52)
javax.servlet.http.HttpServlet.service(HttpServlet .java:803)
com.patchlink.sapphire.web.pages.vulnerability.ses sion.HibernateSessionFilter.doFilter(HibernateSess ionFilter.java:75)


note The full stack trace of the root cause is available in the Apache Tomcat/5.5.23 logs.
--------------------------------------------------------------------------


When I check the status of all the zenworks services, I find that the zenloader service is dead, and cannot be restarted (it immediately fails). This last messages reported for the service are around the 17th of December when I first tried to apply the update:


--------------------------------------------------------------------------
[DEBUG] [12/17/09 4:47:11 PM] [] [Loader.QueueRunner] [] [An unknown error occurred.] [] []
[DEBUG] [12/17/09 4:47:11 PM] [] [Loader.QueueRunner] [] [com.novell.zenworks.datamodel.exceptions.InternalD ataModelException: org.hibernate.exception.SQLGrammarException: could not execute query
at com.novell.zenworks.datamodel.database.AbstractDat abaseTypeHelper.newStandardDataModelException(Abst ractDatabaseTypeHelper.java:436)
at com.novell.zenworks.datamodel.database.sqlanywhere .SQLAnywhereHelper.newStandardDataModelException(S QLAnywhereHelper.java:282)
at com.novell.zenworks.datamodel.database.AbstractDat abaseTypeHelper.throwStandardDataModelException(Ab stractDatabaseTypeHelper.java:421)
at com.novell.zenworks.datamodel.session.hibernate.Ab stractHibernateSessionImpl.handleException(Abstrac tHibernateSessionImpl.java:177)
at com.novell.zenworks.datamodel.services.queue.Queue AdminImpl.popNextActionId(QueueAdminImpl.java:1089 )
at com.novell.zenworks.datamodel.services.queue.Queue AdminImpl.pop(QueueAdminImpl.java:468)
at com.novell.zenworks.loader.modules.queue.runner.Qu eueRunner.processNextAction(QueueRunner.java:450)
at com.novell.zenworks.loader.modules.queue.runner.Qu eueRunner.run(QueueRunner.java:341)
at com.novell.zenworks.loader.ZENModuleThread.run(ZEN ModuleThread.java:111)
Caused by: org.hibernate.exception.SQLGrammarException: could not execute query
at org.hibernate.exception.SQLStateConverter.convert( SQLStateConverter.java:67)
at org.hibernate.exception.JDBCExceptionHelper.conver t(JDBCExceptionHelper.java:43)
at org.hibernate.loader.Loader.doList(Loader.java:214 8)
at org.hibernate.loader.Loader.listIgnoreQueryCache(L oader.java:2029)
at org.hibernate.loader.Loader.list(Loader.java:2024)
at org.hibernate.loader.custom.CustomLoader.list(Cust omLoader.java:118)
at org.hibernate.impl.SessionImpl.listCustomQuery(Ses sionImpl.java:1659)
at org.hibernate.impl.AbstractSessionImpl.list(Abstra ctSessionImpl.java:142)
at org.hibernate.impl.SQLQueryImpl.list(SQLQueryImpl. java:164)
at org.hibernate.impl.AbstractQueryImpl.uniqueResult( AbstractQueryImpl.java:756)
at com.novell.zenworks.datamodel.services.queue.Queue AdminImpl.popNextActionId(QueueAdminImpl.java:1076 )
... 4 more
Caused by: java.sql.SQLException: [[Sybase]][[ODBC Driver]][[SQL Anywhere]]Procedure 'zQueuePop' not found
at ianywhere.ml.jdbcodbc.IIPreparedStatement.executeQ uery(Native Method)
at ianywhere.ml.jdbcodbc.IPreparedStatement.executeQu ery(IPreparedStatement.java:201)
at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement .executeQuery(NewProxyPreparedStatement.java:76)
at org.hibernate.jdbc.AbstractBatcher.getResultSet(Ab stractBatcher.java:186)
at org.hibernate.loader.Loader.getResultSet(Loader.ja va:1669)
at org.hibernate.loader.Loader.doQuery(Loader.java:66 2)
at org.hibernate.loader.Loader.doQueryAndInitializeNo nLazyCollections(Loader.java:224)
at org.hibernate.loader.Loader.doList(Loader.java:214 5)
... 12 more
] [] []
--------------------------------------------------------------------------


I get no further error reports after that message. Lasetly, when I try to run "novell-zenworks-configure -c Start" I get the following errors messages:


--------------------------------------------------------------------------
java.lang.reflect.InvocationTargetException
at sun.reflect.NativeMethodAccessorImpl.invoke0(Nativ e Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Native MethodAccessorImpl.java:39)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(De legatingMethodAccessorImpl.java:25)
at java.lang.reflect.Method.invoke(Method.java:585)
at com.novell.zenworks.configure.ConfigureLoader.main (ConfigureLoader.java:39)
Caused by: java.lang.NoClassDefFoundError: org/jdom/transform/XSLTransformException
at com.novell.zenworks.configure.actions.StartConfigu reAction.<clinit>(StartConfigureAction.java:74)
at java.lang.Class.forName0(Native Method)
at java.lang.Class.forName(Class.java:242)
at com.novell.zenworks.configure.ZENworksConfigure.ne wInstanceOf(ZENworksConfigure.java:1036)
at com.novell.zenworks.configure.ZENworksConfigure.ge tActions(ZENworksConfigure.java:961)
at com.novell.zenworks.configure.ZENworksConfigure.ma in(ZENworksConfigure.java:1083)
... 5 more
--------------------------------------------------------------------------


It would seem that some Tomcat modules did not get updated, or possibly some corrupt configuration files. When I check the system-update log, the only errors I find are towards the end where its trying to delete files. A lot of the files that the update script could not find where in the ZENWORKS_SHARE directory.


--------------------------------------------------------------------------
DEBUG;;;Attempting to delete file: /opt/novell/zenworks/share/tomcat/webapps/zenworks-setup
DEBUG;;;looking for :/opt/novell/zenworks/share/tomcat/webapps/zenworks-setup. Exists: False
DEBUG;;;/opt/novell/zenworks/share/tomcat/webapps/zenworks-setup is a directory attempting to delete
INSTALLING;INSTALLING_PERCENTAGE;87;
--------------------------------------------------------------------------


Anyone seen this before or have any ideas?

I've tried comparing file sizes and dates between the working server and my dead primary, with no luck. All files seem to match between my two primaries. ZCC even reports that both servers have the newest update:

ZCM Version = 10.2.2.33386
ZAM Version = 10.2.2.44767
ZPM Version = 10.2.2.7

Thanks,

Patrick