Yeah, I know, it's 4.02, it's out of support, etc.. I'm working at a customer with this running in production, supporting it while the IDM46 upgrade project works its way through the system. I'm doing what I can with what I've got.

What I have is an IDM402 AE / RBPM, running on SLES11Sp4 with JBoss. RBPM appears to be patch "E". Back end database is Oracle 11g.

Until last week, this was working fine. No complaints about performance. Starting last week, sporadic performance problems have been reported. I'm trying to figure out where to dig.

Initially getting in to RBPM, either the "Identity Self-Service" or the "Work Dashboard", performance may be ok, or you may be in for a 3-5 minute wait. Sometimes you'll get a "This portlet did not respond in time!" error. This seems to be from the portlets that draw the screen, so depending on which one(s) time out, you may or may not get the top menu bar, the left side menu, or whatever the bottom portlet is doing.

If you get it to load ok, the only thing used in the Identity Self-Service is a a searchlist portlet. You can do a new search, fill in the criteria, click "search" and it'll either respond immediately, or you might get a 3-5 minute wait, with or without eventual "This portlet did not respond in time!" errors from the top, left, and/or bottom (the searchlist portlet always responds ok).

If the search returns immediately (ok), you can "revise search" and "search" again, and you'll either get an immediate return of results, or maybe a 3-5 minute wait, with or without "This portlet did not respond in time!" errors.

What I'm sure this isn't:

LDAP search performance. Watching ndstrace, once the search is finally sent to eDirectory, it returns within a couple of milliseconds. During the 3-5 minute wait, eDirectory hasn't yet received the search request.

Java memory / heap allocation. The JVM was set to -Xmx:1024M. We changed it to -Xmx:2048M with no change in symptoms. Watching the heap with jstat, I can see garbage collection doing its thing, and the heap does not seem to be overly utilized. Watching the Java process in top, it allocates about 2.4G to Java, which is what it was doing before changing -Xmx to allow for a bigger heap if needed.

CPU, disk, etc. utilization. The box (VM) is dedicated to running the application. The CPU utilization in top is 5% or less, and load average never gets above 1, it's usually around .1 to .2.

What am I hunting here? If I recall correctly, the portlets get their page data from the database. Am I hunting an Oracle/JDBC performance problem?

It looks like what happens, when you click the "submit" button, is that all four portlets get refreshed. The top, left, and bottom portlets, I assume, go get page data from wherever. The searchlist portlet, eventually, gets the message to go do something, it submits the eDirectory search via LDAP, gets its results, and displays them. The other three porlets may or may not get a timely response from wherever. If they don't, "This portlet did not respond in time!".