3 Replies Latest reply on Apr 15, 2014 7:54 AM by jorgemoralespou_2

Problem with multiple modules deployed in EARs

jorgemoralespou_2 Apr 14, 2014 11:10 AM

Hi,

We have an application consisting on 4 different EARs.

In this EARs we have switchyard applications, as well as java modules and libraries. These ears depend on a common module.

We are experiencing a strange behaviour. We deploy one of our ears, that contain a switchyard app that can be invoked via SOAP, and execute tests against this application. They run fine. When we deploy another EAR module, the previous application stops working, and we get a timeout from the client, with the following trace:

------- Begin Message Trace -------
Consumer -> {urn:com.example.provisioning:mock:1.0}ProvisioningMockService
Provider -> [unassigned]
Operation -> provisionItem
MEP -> IN_OUT
Phase -> IN
State -> OK
Exchange Context -> 
    CamelCreatedTimestamp ..........................: Mon Apr 14 14:21:10 CEST 2014
    CamelFilterMatched .............................: false
    CamelToEndpoint ................................: direct://%7Burn:com.example.provisioning:mock:1.0%7DProvisioningMockService
    org.switchyard.bus.camel.consumer ..............: ServiceReference [name={urn:com.example.provisioning:mock:1.0}ProvisioningMockService, interface=SWITCHYARD010007: BaseServiceInterface [type=wsdl, operations=[SWITCHYARD010008: provisionItem : IN_OUT  : [{urn:com.example.provisioning:mock:1.0}string, {urn:com.example.provisioning:mock:1.0}resultCode, null]]], domain=ServiceDomain [name=null]]
    org.switchyard.bus.camel.contract ..............: org.switchyard.metadata.BaseExchangeContract@2b6b2f3d
    org.switchyard.bus.camel.dispatcher ............: org.switchyard.bus.camel.ExchangeDispatcher@2c73217f
    org.switchyard.bus.camel.labels ................: {org.switchyard.exchangeGatewayName=[org.switchyard.label.behavior.transient], org.switchyard.exchangeInitiatedNS=[org.switchyard.label.behavior.transient], org.switchyard.security.context.SecurityContext=[org.switchyard.label.behavior.transient]}
    org.switchyard.bus.camel.phase .................: IN
    org.switchyard.bus.camel.replyHandler ..........: org.switchyard.component.common.SynchronousInOutHandler@9ea6410
    org.switchyard.exchangeGatewayName .............: _ProvisioningMockService_soap_1
    org.switchyard.exchangeInitiatedNS .............: 20617887716211
    org.switchyard.security.context.SecurityContext : DefaultSecurityContext@2068798341[systemUUID=09087bee-7f5e-4ab1-a79f-32588da4b9bf, expirationMillis=0, credentials=[ConfidentialityCredential@385988610[confidential=false]], securityDomainsToSubjects={}]
Message Context -> 
    breadcrumbId ........................: ID-santander-56564-1397477741217-0-36
    org.switchyard.bus.camel.labels .....: {org.switchyard.contentType=[org.switchyard.label.behavior.transient], org.switchyard.bus.camel.messageSent=[org.switchyard.label.behavior.transient]}
    org.switchyard.bus.camel.messageSent : true
    org.switchyard.contentType ..........: {urn:com.example.provisioning:mock:1.0}string
    org.switchyard.messageId ............: ID-santander-56564-1397477741217-0-36
    org.switchyard.soap.messageName .....: provisionItem
Message Content -> 
<urn:string xmlns:urn="urn:com.example.provisioning:mock:1.0">Hello</urn:string>
------ End Message Trace -------
14:21:10,006 DEBUG [org.switchyard.internal.transform.BaseTransformerRegistry] (http-localhost.localdomain/127.0.0.1:8080-1) No compatible transformer registered: from 'java:com.sun.xml.messaging.saaj.soap.ver1_1.BodyElement1_1Impl' to 'java:java.lang.String'
14:21:10,006 INFO  [org.switchyard.handlers.MessageTraceHandler] (http-localhost.localdomain/127.0.0.1:8080-1) 
------- Begin Message Trace -------
Consumer -> {urn:com.example.provisioning:mock:1.0}ProvisioningMockService
Provider -> [unassigned]
Operation -> provisionItem
MEP -> IN_OUT
Phase -> IN
State -> OK
Exchange Context -> 
    CamelCreatedTimestamp ..........................: Mon Apr 14 14:21:10 CEST 2014
    CamelExceptionCaught ...........................: java.lang.IllegalStateException: Unable to find BeanManager. Please ensure that you configured the CDI implementation of your choice properly.
    CamelFailureEndpoint ...........................: direct://%7Burn:com.example.provisioning:mock:1.0%7DProvisioningMockService
    CamelFilterMatched .............................: false
    CamelToEndpoint ................................: direct://%7Burn:com.example.provisioning:mock:1.0%7DProvisioningMockService
    org.switchyard.bus.camel.consumer ..............: ServiceReference [name={urn:com.example.provisioning:mock:1.0}ProvisioningMockService, interface=SWITCHYARD010007: BaseServiceInterface [type=wsdl, operations=[SWITCHYARD010008: provisionItem : IN_OUT  : [{urn:com.example.provisioning:mock:1.0}string, {urn:com.example.provisioning:mock:1.0}resultCode, null]]], domain=ServiceDomain [name=null]]
    org.switchyard.bus.camel.contract ..............: org.switchyard.metadata.BaseExchangeContract@2b6b2f3d
    org.switchyard.bus.camel.dispatcher ............: org.switchyard.bus.camel.ExchangeDispatcher@2c73217f
    org.switchyard.bus.camel.labels ................: {org.switchyard.exchangeGatewayName=[org.switchyard.label.behavior.transient], org.switchyard.exchangeInitiatedNS=[org.switchyard.label.behavior.transient], org.switchyard.security.context.SecurityContext=[org.switchyard.label.behavior.transient]}
    org.switchyard.bus.camel.phase .................: IN
    org.switchyard.bus.camel.replyHandler ..........: org.switchyard.component.common.SynchronousInOutHandler@9ea6410
    org.switchyard.exchangeGatewayName .............: _ProvisioningMockService_soap_1
    org.switchyard.exchangeInitiatedNS .............: 20617887716211
    org.switchyard.security.context.SecurityContext : DefaultSecurityContext@2068798341[systemUUID=09087bee-7f5e-4ab1-a79f-32588da4b9bf, expirationMillis=0, credentials=[ConfidentialityCredential@385988610[confidential=false]], securityDomainsToSubjects={}]
Message Context -> 
    breadcrumbId ........................: ID-santander-56564-1397477741217-0-36
    org.switchyard.bus.camel.labels .....: {org.switchyard.contentType=[org.switchyard.label.behavior.transient], org.switchyard.bus.camel.messageSent=[org.switchyard.label.behavior.transient]}
    org.switchyard.bus.camel.messageSent : true
    org.switchyard.contentType ..........: {urn:com.example.provisioning:mock:1.0}string
    org.switchyard.messageId ............: ID-santander-56564-1397477741217-0-36
    org.switchyard.soap.messageName .....: provisionItem
Message Content -> 
<urn:string xmlns:urn="urn:com.example.provisioning:mock:1.0">Hello</urn:string>
------ End Message Trace -------
14:21:25,011 DEBUG [org.switchyard.component.soap.util.SOAPUtil] (http-localhost.localdomain/127.0.0.1:8080-1) javax.xml.soap.SOAPException: SWITCHYARD035452: Timed out after 15000 ms waiting on synchronous response from target service '{urn:com.example.provisioning:mock:1.0}ProvisioningMockService'.
  at org.switchyard.component.soap.InboundHandler.invoke(InboundHandler.java:321)
  at org.switchyard.component.soap.endpoint.BaseWebService.invoke(BaseWebService.java:112)
  at org.switchyard.component.soap.endpoint.BaseWebService.invoke(BaseWebService.java:41)
  at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
  at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
  at java.lang.reflect.Method.invoke(Method.java:606)
  at org.jboss.ws.common.invocation.AbstractInvocationHandlerJSE.invoke(AbstractInvocationHandlerJSE.java:108)

It looks like Camel doesn't have access to BeanManager, but before we deployed the other application, this just runned fine.

------- Begin Message Trace -------
Consumer -> {urn:com.example.provisioning:mock:1.0}ProvisioningMockService
Provider -> [unassigned]
Operation -> provisionItem
MEP -> IN_OUT
Phase -> IN
State -> OK
Exchange Context -> 
    CamelCreatedTimestamp ..........................: Mon Apr 14 14:15:54 CEST 2014
    CamelFilterMatched .............................: false
    CamelToEndpoint ................................: direct://%7Burn:com.example.provisioning:mock:1.0%7DProvisioningMockService
    org.switchyard.bus.camel.consumer ..............: ServiceReference [name={urn:com.example.provisioning:mock:1.0}ProvisioningMockService, interface=SWITCHYARD010007: BaseServiceInterface [type=wsdl, operations=[SWITCHYARD010008: provisionItem : IN_OUT  : [{urn:com.example.provisioning:mock:1.0}string, {urn:com.example.provisioning:mock:1.0}resultCode, null]]], domain=ServiceDomain [name=null]]
    org.switchyard.bus.camel.contract ..............: org.switchyard.metadata.BaseExchangeContract@490800e6
    org.switchyard.bus.camel.dispatcher ............: org.switchyard.bus.camel.ExchangeDispatcher@2c73217f
    org.switchyard.bus.camel.labels ................: {org.switchyard.exchangeGatewayName=[org.switchyard.label.behavior.transient], org.switchyard.exchangeInitiatedNS=[org.switchyard.label.behavior.transient], org.switchyard.security.context.SecurityContext=[org.switchyard.label.behavior.transient]}
    org.switchyard.bus.camel.phase .................: IN
    org.switchyard.bus.camel.replyHandler ..........: org.switchyard.component.common.SynchronousInOutHandler@3a9c2fa5
    org.switchyard.exchangeGatewayName .............: _ProvisioningMockService_soap_1
    org.switchyard.exchangeInitiatedNS .............: 20302034698840
    org.switchyard.security.context.SecurityContext : DefaultSecurityContext@103734144[systemUUID=09087bee-7f5e-4ab1-a79f-32588da4b9bf, expirationMillis=0, credentials=[ConfidentialityCredential@1850520491[confidential=false]], securityDomainsToSubjects={}]
Message Context -> 
    breadcrumbId ........................: ID-santander-56564-1397477741217-0-1
    org.switchyard.bus.camel.labels .....: {org.switchyard.contentType=[org.switchyard.label.behavior.transient], org.switchyard.bus.camel.messageSent=[org.switchyard.label.behavior.transient]}
    org.switchyard.bus.camel.messageSent : true
    org.switchyard.contentType ..........: {urn:com.example.provisioning:mock:1.0}string
    org.switchyard.messageId ............: ID-santander-56564-1397477741217-0-1
    org.switchyard.soap.messageName .....: provisionItem
Message Content -> 
<urn:string xmlns:urn="urn:com.example.provisioning:mock:1.0">Hello</urn:string>
------ End Message Trace -------
14:15:54,167 DEBUG [org.switchyard.internal.transform.BaseTransformerRegistry] (http-localhost.localdomain/127.0.0.1:8080-1) Selecting transformer: from '{urn:com.example.provisioning:mock:1.0}string' to 'java:java.lang.String'. Type: org.switchyard.transform.internal.TransformerUtil$1
14:15:54,168 DEBUG [org.switchyard.transform.TransformSequence] (http-localhost.localdomain/127.0.0.1:8080-1) Transforming Message (69758222) from '{urn:com.example.provisioning:mock:1.0}string' to 'java:java.lang.String' using transformer type 'org.switchyard.transform.internal.TransformerUtil$1'.
14:15:54,170 DEBUG [org.switchyard.internal.validate.BaseValidatorRegistry] (http-localhost.localdomain/127.0.0.1:8080-1) No compatible validator registered: name 'java:java.lang.String'
14:15:54,171 INFO  [org.switchyard.handlers.MessageTraceHandler] (http-localhost.localdomain/127.0.0.1:8080-1) 
------- Begin Message Trace -------
Consumer -> {urn:com.example.provisioning:mock:1.0}ProvisioningMockService
Provider -> {urn:com.example.provisioning:mock:1.0}ProvisioningMockService
Operation -> provisionItem
MEP -> IN_OUT
Phase -> IN
State -> OK
Exchange Context -> 
    CamelCreatedTimestamp ............................: Mon Apr 14 14:15:54 CEST 2014
    CamelFilterMatched ...............................: false
    CamelToEndpoint ..................................: direct://%7Burn:com.example.provisioning:mock:1.0%7DProvisioningMockService
    org.switchyard.bus.camel.consumer ................: ServiceReference [name={urn:com.example.provisioning:mock:1.0}ProvisioningMockService, interface=SWITCHYARD010007: BaseServiceInterface [type=wsdl, operations=[SWITCHYARD010008: provisionItem : IN_OUT  : [{urn:com.example.provisioning:mock:1.0}string, {urn:com.example.provisioning:mock:1.0}resultCode, null]]], domain=ServiceDomain [name=null]]
    org.switchyard.bus.camel.contract ................: org.switchyard.metadata.BaseExchangeContract@490800e6
    org.switchyard.bus.camel.dispatcher ..............: org.switchyard.bus.camel.ExchangeDispatcher@2c73217f
    org.switchyard.bus.camel.labels ..................: {org.switchyard.exchangeGatewayName=[org.switchyard.label.behavior.transient], org.switchyard.exchangeInitiatedNS=[org.switchyard.label.behavior.transient], org.switchyard.exchange.transaction.beforeInvoked=[org.switchyard.label.behavior.transient], org.switchyard.security.context.SecurityContext=[org.switchyard.label.behavior.transient]}
    org.switchyard.bus.camel.phase ...................: IN
    org.switchyard.bus.camel.provider ................: Service [name={urn:com.example.provisioning:mock:1.0}ProvisioningMockService, interface=SWITCHYARD010007: BaseServiceInterface [type=java, operations=[SWITCHYARD010008: provisionItem : IN_OUT  : [java:java.lang.String, java:com.example.common.model.codes.ResultCode, null]]], domain=ServiceDomain [name=null], metadata=org.switchyard.metadata.ServiceMetadataBuilder$ServiceMetadataImpl@472f1cc1]
    org.switchyard.bus.camel.replyHandler ............: org.switchyard.component.common.SynchronousInOutHandler@3a9c2fa5
    org.switchyard.exchange.transaction.beforeInvoked : true
    org.switchyard.exchangeGatewayName ...............: _ProvisioningMockService_soap_1
    org.switchyard.exchangeInitiatedNS ...............: 20302034698840
    org.switchyard.security.context.SecurityContext ..: DefaultSecurityContext@103734144[systemUUID=09087bee-7f5e-4ab1-a79f-32588da4b9bf, expirationMillis=0, credentials=[ConfidentialityCredential@1850520491[confidential=false]], securityDomainsToSubjects={}]
Message Context -> 
    breadcrumbId ..............................: ID-santander-56564-1397477741217-0-1
    org.switchyard.bus.camel.labels ...........: {org.switchyard.contentType=[org.switchyard.label.behavior.transient], org.switchyard.bus.camel.messageSent=[org.switchyard.label.behavior.transient], org.switchyard.transform.TransformSequence=[org.switchyard.label.behavior.transient]}
    org.switchyard.bus.camel.messageSent ......: true
    org.switchyard.contentType ................: java:java.lang.String
    org.switchyard.messageId ..................: ID-santander-56564-1397477741217-0-1
    org.switchyard.soap.messageName ...........: provisionItem
    org.switchyard.transform.TransformSequence : org.switchyard.transform.TransformSequence@6608e769
Message Content -> 
Hello
------ End Message Trace -------

It seems that when deploying another EAR, transformerRegistry gets messed up, and there is no available transformations, as well as maybe some other problems.

We'll try to do a sample test case (with 2 ears) but so far it has been very difficult to get same behavior.

Cheers,

1. Re: Problem with multiple modules deployed in EARs

rcernich Apr 14, 2014 11:21 AM (in response to jorgemoralespou_2)

Hey Jorge,

I think this is a know issue. Please verify whether or not this aligns with what you are seeing: [SWITCHYARD-1979] Threads started from Camel do not have correct namespace context - JBoss Issue Tracker.

Thanks in advance,
Rob
Actions
2. Re: Problem with multiple modules deployed in EARs

jorgemoralespou_2 Apr 14, 2014 12:03 PM (in response to rcernich)
Hi Rob,
This might be related, as I'm with the customer that triggered the bug, and we have been delivered a patch that it is not working or has other side effects. So far, and with the provided patch by Rick Wagner, it does not work.
It seems that problem changes/varies depending on:
If we deploy the second ear by hot deploy
If we start the server with the provided startup order.

I thought it also could have some dependencies on the "dynamic modules" we have, so I changed them to static, to allow for better start up of AS, but still some problems.
Major issue is that is is difficult to get a reproducer without providing the whole app, and it is not recommended. :-(

Thanks,
Actions
3. Re: Problem with multiple modules deployed in EARs

jorgemoralespou_2 Apr 15, 2014 7:54 AM (in response to rcernich)

Hi Rob,
I have created a reproducer project to demonstrate how it fails.
It is here fsw-demo/fsw-ear-problem at master · jorgemoralespou/fsw-demo · GitHub as well as all the steps to reproduce it.

I hope somebody can jump in on this, as it is a very ugly bug, that it is preventing a customer from product adoption.

Any further details, I can be reached here or IRC.
Actions

Go to original post