After some indeterminate patch by the Linux level folk last night, one
node in our two node UA cluster stopped working.

We watch in Catalina.out on a restart, and it binds to 8080, 8443, 8009
in the log (3 or 4 lines) and reports Tomcat started in 3000 ms.

But no deploy of any of the WARs (dash, landing, IDMProv, sspr, osp).

When you try to stop it complains that localhost:8005 is not listening
and uses the kill process instead to stop Tomcat.

We do not know of any changes made overnight. When we reboot the second
node, it too dies. (That was not a wise choice in my mind, but was done).

IDM 4.5.2 engine as, UA, OSP, SSPR (3.3.1) all patched as far as
we can find, on Tomcat, Java8, on SLES 11.

There is nothing in the catalina.out, it just properly starts, but only
binds the various ports and calls it a day. (Autodeploy is true,
unpackWARs is true, default config beyond usual config choices).

Very odd.

netstat -an shows that 8005 is not bound anywhere.

On a working node:
lsof | grep IPv4 | less shows that the local DNS name is listening on
:mxi (which /etc/services maps mxi to 8005).

On a dead node, not there.

iptables -L shows nothing there.

Really strange.