Hey All,

Wasn't exactly sure which forum to put this one under since it's mostly a server issue, but also involves the client machines.

Our environment is Windows 7, ENGL ZToolkit 6.0.1 for imaging, and ZCM 10.3.

Our user source is eDirectory which is being communicated to via LDAP on port 636 (SSL).

First of all, the LDAP server set up for the user source is a single DNS name which points to a load balancer, directing the traffic to whichever LDAP server it happens to choose.

Second, the certificate set up for that user source is a "star" * certificate, which covers *.ourdomain.edu.

What is happening, is that on a regular basis, a workstation that is imaging with ENGL will attempt to register with ZCM, and fail to do so.

Registration happens in a scripted fashion during imaging, and uses a known-good username/password to register:
Code:
zac.exe /inituserdaemon reg -k lab.workstation -u installer@OUR_TREE -p ******** http://zcm2.ourdomain.edu
zmd-messages.log from that machine reports an invalid username/password.

I enabled CASA debug logging on the ZCM server, and found the following:

ats.log.2:
Code:
2010-04-16 14:03:07,918 DEBUG authtoksvc.SvcConfig Constructor()-
2010-04-16 14:03:07,918 DEBUG authtoksvc.SvcConfig Constructor()- SvcConfigPath = /etc/CASA/authtoken/svc
2010-04-16 14:03:07,921 DEBUG authtoksvc.SvcConfig getSetting()- Did not find setting SessionTokenLifetime
2010-04-16 14:03:07,921 DEBUG authtoksvc.SvcConfig getSetting()- Assigning default value 43200
2010-04-16 14:03:07,921 DEBUG authtoksvc.SvcConfig getSetting()- Did not find setting LifetimeShorter
2010-04-16 14:03:07,921 DEBUG authtoksvc.SvcConfig getSetting()- Assigning default value 5
2010-04-16 14:03:07,921 DEBUG authtoksvc.SvcConfig getSetting()- Did not find setting ReconfigureInterval
2010-04-16 14:03:07,921 DEBUG authtoksvc.SvcConfig getSetting()- Assigning default value 60
2010-04-16 14:03:07,922 DEBUG authtoksvc.SvcConfig getSetting()- Found setting IAConfigFile
2010-04-16 14:03:07,922 DEBUG authtoksvc.SvcConfig getSetting()- Setting value = /etc/CASA/authtoken/svc/iaRealms.xml
2010-04-16 14:03:07,925 DEBUG authtoksvc.SvcConfig getSetting()- Found setting ReconfigureInterval
2010-04-16 14:03:07,925 DEBUG authtoksvc.SvcConfig getSetting()- Setting value = 60
2010-04-16 14:03:07,926 DEBUG authtoksvc.EnabledSvcsConfig Constructor()-
2010-04-16 14:03:07,926 DEBUG authtoksvc.EnabledSvcsConfig Constructor()- SvcConfigPath = /etc/CASA/authtoken/svc
2010-04-16 14:03:07,926 DEBUG authtoksvc.EnabledSvcsConfig Constructor()- File /etc/CASA/authtoken/svc/auth.policy not found
2010-04-16 14:03:07,926 DEBUG authtoksvc.AuthTokenConfig AuthTokenConfig()-
2010-04-16 14:03:07,928 DEBUG authtoksvc.AuthTokenConfig getSetting()- Did not find setting TokenLifetime
2010-04-16 14:03:07,928 DEBUG authtoksvc.AuthTokenConfig getSetting()- Assigning default value 3600
2010-04-16 14:03:07,928 DEBUG authtoksvc.AuthTokenConfig getSetting()- Did not find setting LifetimeShorter
2010-04-16 14:03:07,928 DEBUG authtoksvc.AuthTokenConfig getSetting()- Assigning default value 5
2010-04-16 14:03:07,928 DEBUG authtoksvc.IdenTokenConfig Constructor()-
2010-04-16 14:03:07,930 DEBUG authtoksvc.EnabledSvcsConfig Constructor()- Host folder /etc/CASA/authtoken/svc/enabled_services/localhost is directory
2010-04-16 14:03:07,930 DEBUG authtoksvc.EnabledSvcsConfig Constructor()- Service folder /etc/CASA/authtoken/svc/enabled_services/localhost/com.novell.zenworks.IPFW_TREE
2010-04-16 14:03:07,931 DEBUG authtoksvc.EnabledSvcsConfig Constructor()- Service folder /etc/CASA/authtoken/svc/enabled_services/localhost/com.novell.zenworks.IPFW_TREE is directory
2010-04-16 14:03:07,931 DEBUG authtoksvc.AuthTokenConfig AuthTokenConfig()-
2010-04-16 14:03:07,931 DEBUG authtoksvc.AuthTokenConfig Constructor()- File /etc/CASA/authtoken/svc/enabled_services/localhost/com.novell.zenworks.IPFW_TREE/authtoken.settings not found
2010-04-16 14:03:07,931 DEBUG authtoksvc.EnabledSvcsConfig Constructor()- Exception accessing /etc/CASA/authtoken/svc/enabled_services/localhost/com.novell.zenworks.IPFW_TREE/authtoken.settings Exception=java.lang.Exception: AuthTokenConfig()- File not found
2010-04-16 14:03:07,931 DEBUG authtoksvc.IdenTokenConfig Constructor()-
2010-04-16 14:03:07,933 DEBUG authtoksvc.EnabledSvcsConfig Constructor()- Adding entry in map for localhost com.novell.zenworks.IPFW_TREE
2010-04-16 14:03:07,933 DEBUG authtoksvc.SvcConfig getSetting()- Found setting ConfigFolderPath
2010-04-16 14:03:07,933 DEBUG authtoksvc.SvcConfig getSetting()- Setting value = /etc/CASA/authtoken/svc
2010-04-16 14:03:07,933 DEBUG authtoksvc.Authenticate init()- Mechanism folder /etc/CASA/authtoken/svc/auth_mechanisms/PwdAuthenticate is directory
2010-04-16 14:03:07,934 DEBUG authtoksvc.AuthMechConfig Constructor()-
2010-04-16 14:03:07,935 DEBUG authtoksvc.AuthMechConfig getSetting()- Found setting ClassName
2010-04-16 14:03:07,936 DEBUG authtoksvc.AuthMechConfig getSetting()- Setting value = com.novell.casa.authtoksvc.PwdAuthenticate
2010-04-16 14:03:07,936 DEBUG authtoksvc.AuthMechConfig getSetting()- Found setting RelativeClassPath
2010-04-16 14:03:07,936 DEBUG authtoksvc.AuthMechConfig getSetting()- Setting value = WEB-INF/classes
2010-04-16 14:03:07,936 DEBUG authtoksvc.SvcConfig getSetting()- Found setting AppRootPath
2010-04-16 14:03:07,936 DEBUG authtoksvc.SvcConfig getSetting()- Setting value = /srv/www/casaats/webapps/CasaAuthTokenSvc/
2010-04-16 14:03:07,936 DEBUG authtoksvc.Authenticate init()- Mechanism path = /srv/www/casaats/webapps/CasaAuthTokenSvc/WEB-INF/classes
2010-04-16 14:03:07,936 DEBUG authtoksvc.SvcConfig getSetting()- Found setting AppRootPath
2010-04-16 14:03:07,937 DEBUG authtoksvc.SvcConfig getSetting()- Setting value = /srv/www/casaats/webapps/CasaAuthTokenSvc/
2010-04-16 14:03:07,937 DEBUG authtoksvc.SvcConfig getSetting()- Found setting AppRootPath
2010-04-16 14:03:07,937 DEBUG authtoksvc.SvcConfig getSetting()- Setting value = /srv/www/casaats/webapps/CasaAuthTokenSvc/
2010-04-16 14:03:07,938 DEBUG authtoksvc.Authenticate init()- Mechanism folder /etc/CASA/authtoken/svc/auth_mechanisms/ZcmScardAuthenticate is directory
I also found a number of the following in ats.trace (IPs removed):

Code:
2010-04-16 08:21:22,102 INFO [ClientAddr=] Authenticate Rpc, Mech=PwdAuthenticate, Realm=IPFW_TREE, Status=UNSUCCESSFUL
2010-04-16 08:34:58,523 INFO [ClientAddr=] Authenticate Rpc, Mech=PwdAuthenticate, Realm=IPFW_TREE, Status=UNSUCCESSFUL
2010-04-16 08:57:11,941 INFO [ClientAddr=] Authenticate Rpc, Mech=PwdAuthenticate, Realm=IPFW_TREE, Status=UNSUCCESSFUL
I also see the following a number of times in ats.log:
Code:
2010-04-16 14:40:08,921 DEBUG authtoksvc.Krb5Authenticate Constructor()- GSS Exception caught: No valid credentials provided (Mechanism level: Attempt to obtain new ACCEPT credentials failed!)
2010-04-16 14:40:08,921 WARN authtoksvc.Authenticate init()- Exception instantiating mechConfig or mechanism /etc/CASA/authtoken/svc/auth_mechanisms/Krb5Authenticate/mechanism.settings Exception=java.lang.Exception: Failed to instantiate needed GSS objects
Also, I have a student worker adding a contact to each of the hundreds of bundles in ZCM right now, and have found that he gets "error contacting the user source" whilst adding the contact at seemingly exactly the same time the machine fails to register.


Any ideas?

We're currently at a bit of a standstill with this problem. It appeared to go away with 10.3, but has now come back with full force.