Hello,

we are facing some strange Problems when executing concurrent SOAP request to a POA.
It seems that GroupWise is generating invalid SOAP responses or is mixing different responses (not threadsafe?).
Our environment is:
GroupWise 8.0.2 on SLES 11.0 (x86_64)
Oracle Java 6 Client on Linux (x86_64)

We can rule out any problems on the client site.
We tried 10 threads within a single Java VM as well as 10 seperate threads each in a dedicated Java VM.
Each thread is iterating over the items of different mailboxes.
Already read items will be marked (modifyItemRequest) using a custom field containing a timestamp.
The authentication is done using a trusted app key.
We are not using gwws.jar but generating the client using JAXB.

Below you find two example stacktraces from the client site:

com.sun.xml.internal.ws.protocol.soap.MessageCreat ionException: Couldn't create SOAP message due to exception:
unexpected XML tag. expected: {http://schemas.xmlsoap.org/soap/envelope/}Body but found: {adminOnly}Body
at com.sun.xml.internal.ws.encoding.SOAPBindingCodec. decode(SOAPBindingCodec.java:283)
at com.sun.xml.internal.ws.transport.http.client.Http TransportPipe.process(HttpTransportPipe.java:180)
at com.sun.xml.internal.ws.transport.http.client.Http TransportPipe.processRequest(HttpTransportPipe.jav a:83)
at com.sun.xml.internal.ws.transport.DeferredTranspor tPipe.processRequest(DeferredTransportPipe.java:78 )
at com.sun.xml.internal.ws.api.pipe.Fiber.__doRun(Fib er.java:587)
at com.sun.xml.internal.ws.api.pipe.Fiber._doRun(Fibe r.java:546)
at com.sun.xml.internal.ws.api.pipe.Fiber.doRun(Fiber .java:531)
at com.sun.xml.internal.ws.api.pipe.Fiber.runSync(Fib er.java:428)
at com.sun.xml.internal.ws.client.Stub.process(Stub.j ava:211)
at com.sun.xml.internal.ws.client.sei.SEIStub.doProce ss(SEIStub.java:124)
at com.sun.xml.internal.ws.client.sei.SyncMethodHandl er.invoke(SyncMethodHandler.java:98)
at com.sun.xml.internal.ws.client.sei.SyncMethodHandl er.invoke(SyncMethodHandler.java:78)
at com.sun.xml.internal.ws.client.sei.SEIStub.invoke( SEIStub.java:107)
at $Proxy48.send(Unknown Source)
....
Caused by: com.sun.xml.internal.ws.streaming.XMLStreamReaderE xception: unexpected XML tag. expected:
{http://schemas.xmlsoap.org/soap/envelope/}Body but found: {adminOnly}Body
at com.sun.xml.internal.ws.streaming.XMLStreamReaderU til.verifyTag(XMLStreamReaderUtil.java:203)
at com.sun.xml.internal.ws.encoding.StreamSOAPCodec.d ecode(StreamSOAPCodec.java:216)
at com.sun.xml.internal.ws.encoding.StreamSOAPCodec.d ecode(StreamSOAPCodec.java:292)
at com.sun.xml.internal.ws.encoding.StreamSOAPCodec.d ecode(StreamSOAPCodec.java:118)
at com.sun.xml.internal.ws.encoding.SOAPBindingCodec. decode(SOAPBindingCodec.java:278)
... 27 more


com.sun.xml.internal.ws.protocol.soap.MessageCreat ionException: Couldn't create SOAP message due to exception: XML
reader error: javax.xml.stream.XMLStreamException: ParseError at [row,col]:[1,129]
Message: Element type "http:" must be followed by either attribute specifications, ">" or "/>".
at com.sun.xml.internal.ws.encoding.SOAPBindingCodec. decode(SOAPBindingCodec.java:283)
at com.sun.xml.internal.ws.transport.http.client.Http TransportPipe.process(HttpTransportPipe.java:180)
at com.sun.xml.internal.ws.transport.http.client.Http TransportPipe.processRequest(HttpTransportPipe.jav a:83)
at com.sun.xml.internal.ws.transport.DeferredTranspor tPipe.processRequest(DeferredTransportPipe.java:78 )
at com.sun.xml.internal.ws.api.pipe.Fiber.__doRun(Fib er.java:587)
at com.sun.xml.internal.ws.api.pipe.Fiber._doRun(Fibe r.java:546)
at com.sun.xml.internal.ws.api.pipe.Fiber.doRun(Fiber .java:531)
at com.sun.xml.internal.ws.api.pipe.Fiber.runSync(Fib er.java:428)
at com.sun.xml.internal.ws.client.Stub.process(Stub.j ava:211)
at com.sun.xml.internal.ws.client.sei.SEIStub.doProce ss(SEIStub.java:124)
at com.sun.xml.internal.ws.client.sei.SyncMethodHandl er.invoke(SyncMethodHandler.java:98)
at com.sun.xml.internal.ws.client.sei.SyncMethodHandl er.invoke(SyncMethodHandler.java:78)
at com.sun.xml.internal.ws.client.sei.SEIStub.invoke( SEIStub.java:107)
at $Proxy48.send(Unknown Source)
....
at java.util.concurrent.FutureTask$Sync.innerRun(Futu reTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.jav a:138)
at java.util.concurrent.ThreadPoolExecutor$Worker.run Task(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run (ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: com.sun.xml.internal.ws.streaming.XMLStreamReaderE xception: XML reader error:
javax.xml.stream.XMLStreamException: ParseError at [row,col]:[1,129]
Message: Element type "http:" must be followed by either attribute specifications, ">" or "/>".
at com.sun.xml.internal.ws.streaming.XMLStreamReaderU til.wrapException(XMLStreamReaderUtil.java:256)
at com.sun.xml.internal.ws.streaming.XMLStreamReaderU til.next(XMLStreamReaderUtil.java:84)
at com.sun.xml.internal.ws.streaming.XMLStreamReaderU til.nextContent(XMLStreamReaderUtil.java:99)
at com.sun.xml.internal.ws.streaming.XMLStreamReaderU til.nextElementContent(XMLStreamReaderUtil.java:89 )
at com.sun.xml.internal.ws.encoding.StreamSOAPCodec.d ecode(StreamSOAPCodec.java:219)
at com.sun.xml.internal.ws.encoding.StreamSOAPCodec.d ecode(StreamSOAPCodec.java:292)
at com.sun.xml.internal.ws.encoding.StreamSOAPCodec.d ecode(StreamSOAPCodec.java:118)
at com.sun.xml.internal.ws.encoding.SOAPBindingCodec. decode(SOAPBindingCodec.java:278)
... 27 more
Caused by: javax.xml.stream.XMLStreamException: ParseError at [row,col]:[1,129]
Message: Element type "http:" must be followed by either attribute specifications, ">" or "/>".
at com.sun.org.apache.xerces.internal.impl.XMLStreamR eaderImpl.next(XMLStreamReaderImpl.java:594)
at com.sun.xml.internal.ws.util.xml.XMLStreamReaderFi lter.next(XMLStreamReaderFilter.java:81)
at com.sun.xml.internal.ws.streaming.XMLStreamReaderU til.next(XMLStreamReaderUtil.java:65)
... 33 more


According to http://www.java.net/node/681145 we tried to use woodstox as STAX impl but the problem still exists:

com.sun.xml.internal.ws.protocol.soap.MessageCreat ionException: Couldn't create SOAP message due to exception: XML
reader error: com.ctc.wstx.exc.WstxUnexpectedCharException: Unexpected character '/' (code 47) (expected a name start
character)
at [row,col {unknown-source}]: [1,120]
at com.sun.xml.internal.ws.encoding.SOAPBindingCodec. decode(Unknown Source)
at com.sun.xml.internal.ws.transport.http.client.Http TransportPipe.process(Unknown Source)
at com.sun.xml.internal.ws.transport.http.client.Http TransportPipe.processRequest(Unknown Source)
at com.sun.xml.internal.ws.transport.DeferredTranspor tPipe.processRequest(Unknown Source)
at com.sun.xml.internal.ws.api.pipe.Fiber.__doRun(Unk nown Source)
at com.sun.xml.internal.ws.api.pipe.Fiber._doRun(Unkn own Source)
at com.sun.xml.internal.ws.api.pipe.Fiber.doRun(Unkno wn Source)
at com.sun.xml.internal.ws.api.pipe.Fiber.runSync(Unk nown Source)
at com.sun.xml.internal.ws.client.Stub.process(Unknow n Source)
at com.sun.xml.internal.ws.client.sei.SEIStub.doProce ss(Unknown Source)
at com.sun.xml.internal.ws.client.sei.SyncMethodHandl er.invoke(Unknown Source)
at com.sun.xml.internal.ws.client.sei.SyncMethodHandl er.invoke(Unknown Source)
at com.sun.xml.internal.ws.client.sei.SEIStub.invoke( Unknown Source)
at $Proxy48.send(Unknown Source)
....
at java.util.concurrent.FutureTask$Sync.innerRun(Unkn own Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run Task(Unknown Source)
at java.util.concurrent.ThreadPoolExecutor$Worker.run (Unknown Source)
at java.lang.Thread.run(Unknown Source)
Caused by: com.sun.xml.internal.ws.streaming.XMLStreamReaderE xception: XML reader error:
com.ctc.wstx.exc.WstxUnexpectedCharException: Unexpected character '/' (code 47) (expected a name start character)
at [row,col {unknown-source}]: [1,120]
at com.sun.xml.internal.ws.streaming.XMLStreamReaderU til.wrapException(Unknown Source)
at com.sun.xml.internal.ws.streaming.XMLStreamReaderU til.next(Unknown Source)
at com.sun.xml.internal.ws.streaming.XMLStreamReaderU til.nextContent(Unknown Source)
at com.sun.xml.internal.ws.streaming.XMLStreamReaderU til.nextElementContent(Unknown Source)
at com.sun.xml.internal.ws.encoding.StreamSOAPCodec.d ecode(Unknown Source)
at com.sun.xml.internal.ws.encoding.StreamSOAPCodec.d ecode(Unknown Source)
at com.sun.xml.internal.ws.encoding.StreamSOAPCodec.d ecode(Unknown Source)
... 28 more
Caused by: com.ctc.wstx.exc.WstxUnexpectedCharException: Unexpected character '/' (code 47) (expected a name start
character)
at [row,col {unknown-source}]: [1,120]
at com.ctc.wstx.sr.StreamScanner.throwUnexpectedChar( StreamScanner.java:639)
at com.ctc.wstx.sr.StreamScanner.parseLocalName(Strea mScanner.java:1798)
at com.ctc.wstx.sr.BasicStreamReader.handleStartElem( BasicStreamReader.java:2925)
at com.ctc.wstx.sr.BasicStreamReader.nextFromTree(Bas icStreamReader.java:2817)
at com.ctc.wstx.sr.BasicStreamReader.next(BasicStream Reader.java:1065)
at com.sun.xml.internal.ws.util.xml.XMLStreamReaderFi lter.next(Unknown Source)
... 34 more

Last but not least the POA dies when using multiple SOAP clients.
This does not happen immediately.
Depending on the number of client threads it can take 5 minutes (20 threads) up to 20 minutes 5 (threads).

Are there any known issues with this?
Can someone help?

Thank you in advance,
Thilo