1 2 Previous Next 21 Replies Latest reply on Nov 11, 2015 12:20 AM by zhfeng

    Blacktie Data Buffers and Content Based Routing

    joicejoy

      We are considering Blacktie for migration from Tuxedo.

       

      We have an XML interface available currently which is as follows:

      <?xml version="1.0" encoding="UTF-8"?>

      <Request>

        <Header>

          <OperatorName e-dtype="string">arborsv</OperatorName>

          <AccountServer e-dtype="int">3</AccountServer>

          <ApplicationName e-dtype="string">ExternalIdEquipMapCreate</ApplicationName>

        </Header>

        <ExternalIdEquipMapCreate>

          <ExternalIdEquipMap>

            <ViewStatus e-dtype="int">2</ViewStatus>

            <ActiveDate e-dtype="dateTime">2014-12-10 00:00:00</ActiveDate>

            <CreateDt e-dtype="dateTime">2014-12-10 00:00:00</CreateDt>

            <ServiceExternalId e-dtype="string">1976:0:1003</ServiceExternalId>

            <ServiceExternalIdType e-dtype="int">999</ServiceExternalIdType>

            <ServiceInternalId e-dtype="int">200468</ServiceInternalId>

            <ServiceInternalIdResets e-dtype="int">0</ServiceInternalIdResets>

          </ExternalIdEquipMap>

        </ExternalIdEquipMapCreate>

      </Request>

       

      If I malloc the above into a char buffer and send, I get the error "A NON-BUFFER WAS ATTEMPTED TO BE SENT".

       

      The above XMl has no schema information included. Is it possible to send the above in tpcall.

        • 1. Re: Blacktie Data Buffers and Content Based Routing
          joicejoy

          Also does using our own XML buffer affect CBR in any way.

           

          Also what is the feasibility to use a binary buffer.

          I see [BLACKTIE-379] Support codec layer in Blacktie - JBoss Issue Tracker

           

          Is it possible to use binary buffers at all. If yes, then is CBR supported on them.

          I mean if an XML header with routing data is passed along with a binary payload.

          • 2. Re: Blacktie Data Buffers and Content Based Routing
            tomjenkinson

            At the moment I believe CBR is untested as we used to work with JBoss ESB but we have not been kept up to date so I wouldn't be able to say whether the CBR QS would still work.

             

            I can say that the intention was that only NBF buffers would be routable. Binary (X_OCTET) types were not intended for routing. What we did consider was support for converting NBF to XML on the wire to assist with routing, I don't know if that was done zhfeng would know better than I. If that was done it might be possible to add a new buffer type that indicates its already XML.

             

            zhfeng, any ideas?

            • 3. Re: Blacktie Data Buffers and Content Based Routing
              zhfeng

              yeah, the NBF buffer is actually the XML string on the wire. Currently, we need the schema to detect the type information of the attributes.

              I think that it could be possible to route the content based on XML. In original JBoss ESB, we use the XPath to select the element and match some rules to deliver to the different services. I believe that the Camel should have the similar function for routing and I will investigate in [JBTM-2496] Investigate using of camel for content based routing in the Blacktie - JBoss Issue Tracker

               

              Tom, I just wonder the different between the buffer type XML and X_OCTET if we consider to add a new type ?

              • 4. Re: Blacktie Data Buffers and Content Based Routing
                tomjenkinson

                Thanks for looking Amos. The reason I could see them being different is just in the sense that by tpalloc("XML", ...) it might be able to help us so we don't try to look in none-XML payload messages in the CBR system. But yeah, apart from that I don't think there would be a difference.

                • 5. Re: Blacktie Data Buffers and Content Based Routing
                  joicejoy

                  Do we have any nested examples for NBF Buffers. I would like to use the Data Routing provided by NBF. So suppose we have an NBF buffer with data routing fields plus an X_OCTET buffer containing our message as payload.

                  This could solve most of our problems.

                  Is this possible.

                  • 6. Re: Blacktie Data Buffers and Content Based Routing
                    tomjenkinson

                    We will have to ask Amos, but I would imagine an NBF can take a char* as one of the fields (assuming you can define a max length for it ahead of time).

                    • 7. Re: Blacktie Data Buffers and Content Based Routing
                      zhfeng

                      Hi Joy,

                       

                      you can take a look at narayana/TestBTNbf.cxx at master · jbosstm/narayana · GitHub and https://github.com/jbosstm/narayana/blob/master/blacktie/nbf/src/test/cpp/TestComplex.cxx

                      I think it could be same to route the NBF buffer and XML buffer since they are both the xml strings on the wire. I agree with Tom to add the new XML buffer to indicate it.

                       

                      So the example codes could be

                      long size = strlen(message);
                      char* sendbuf = tpalloc("XML", NULL, size);
                      memcpy(sendbuf, message, size);
                      
                      int rc = tpcall("EXAMPLE_SERVICE", sendbuf, ...);
                      

                       

                      and it could define the routing rules in the blacktie subsystem in the wildfly

                       

                      <subsystem xmlns="urn:jboss:domain:blacktie:1.0">
                          <routings>
                              <routing>
                                   <name>route to SERVICE_ONE</name>
                                   <desc>route the messages from EXAMPLE_SERVICE to SERVICE_ONE when account server is 1
                                   <rule from="EXAMPLE_SERVICE" to="SERVICE_ONE">
                                       <xpath>/Request/Header/AccountServer</xpath>
                                       <eq>1</eq>
                                   </rule>
                              </routing>
                          </routings>
                      </subsystem>
                      

                       

                      if it does solve your problem, we can raise the Jira to add the XML buffer and the routing rules in wildfly-blacktie subsystem.

                      • 8. Re: Blacktie Data Buffers and Content Based Routing
                        joicejoy

                        I am trying to send our XML message inside an attribute.

                         

                        But I am getting multiple errors if the BT_NBF buffer size exceeds 512 bytes. Seems like tpalloc only allocates 512 bytes. Tried tprealloc but no success.

                        I have changed the maxLength to 10000.

                         

                        This is the server side error:

                         

                        sh-4.1$ *** glibc detected *** ./server: free(): invalid next size (normal): 0x00007f51e000d000 ***

                        ======= Backtrace: =========

                        /lib64/libc.so.6[0x3ca0875366]

                        /lib64/libc.so.6[0x3ca0877e93]

                        /usr/lib64/libstdc++.so.6(_ZNSt19basic_ostringstreamIcSt11char_traitsIcESaIcEED0Ev+0xbb)[0x3ca649aa6b]

                        /fxtest/trep/btie/blacktie-5.2.2.Final/lib/cxx/liblog4cxx.so(_ZN7log4cxx7helpers17CharMessageBufferD1Ev+0x36)[0x7f51fae323b8]

                        /fxtest/trep/btie/blacktie-5.2.2.Final/lib/cxx/liblog4cxx.so(_ZN7log4cxx7helpers13MessageBufferD1Ev+0x43)[0x7f51fae3243b]

                        /fxtest/trep/btie/blacktie-5.2.2.Final/lib/cxx/libblacktie-xatmi.so(_get_tperrno+0x34a)[0x7f51fa56b8f8]

                        /fxtest/trep/btie/blacktie-5.2.2.Final/lib/cxx/libblacktie-xatmi.so(_ZN17ServiceDispatcher9onMessageE9message_t+0x1c8d)[0x7f51fa57e0ed]

                        /fxtest/trep/btie/blacktie-5.2.2.Final/lib/cxx/libblacktie-xatmi.so(_ZN17ServiceDispatcher3svcEv+0x90b)[0x7f51fa57fa4d]

                        /fxtest/trep/btie/blacktie-5.2.2.Final/lib/cxx/libblacktie-xatmi.so(+0x45275)[0x7f51fa590275]

                        /lib64/libpthread.so.0[0x3ca0c07851]

                        /lib64/libc.so.6(clone+0x6d)[0x3ca08e767d]

                        ======= Memory map: ========

                        00400000-00401000 r-xp 00000000 00:17 612606                             /fxtest/trep/nbf/server

                        00600000-00601000 rw-p 00000000 00:17 612606                             /fxtest/trep/nbf/server

                        01b62000-01c95000 rw-p 00000000 00:00 0                                  [heap]

                        3ca0000000-3ca0020000 r-xp 00000000 08:03 34359                          /lib64/ld-2.12.so

                        3ca021f000-3ca0220000 r--p 0001f000 08:03 34359                          /lib64/ld-2.12.so

                        3ca0220000-3ca0221000 rw-p 00020000 08:03 34359                          /lib64/ld-2.12.so

                        3ca0221000-3ca0222000 rw-p 00000000 00:00 0

                        3ca0400000-3ca0402000 r-xp 00000000 08:03 34367                          /lib64/libdl-2.12.so

                        3ca0402000-3ca0602000 ---p 00002000 08:03 34367                          /lib64/libdl-2.12.so

                        3ca0602000-3ca0603000 r--p 00002000 08:03 34367                          /lib64/libdl-2.12.so

                        3ca0603000-3ca0604000 rw-p 00003000 08:03 34367                          /lib64/libdl-2.12.so

                        3ca0800000-3ca0989000 r-xp 00000000 08:03 34361                          /lib64/libc-2.12.so

                        3ca0989000-3ca0b88000 ---p 00189000 08:03 34361                          /lib64/libc-2.12.so

                        3ca0b88000-3ca0b8c000 r--p 00188000 08:03 34361                          /lib64/libc-2.12.so

                        3ca0b8c000-3ca0b8d000 rw-p 0018c000 08:03 34361                          /lib64/libc-2.12.so

                        3ca0b8d000-3ca0b92000 rw-p 00000000 00:00 0

                        3ca0c00000-3ca0c17000 r-xp 00000000 08:03 34362                          /lib64/libpthread-2.12.so

                        3ca0c17000-3ca0e17000 ---p 00017000 08:03 34362                          /lib64/libpthread-2.12.so

                        3ca0e17000-3ca0e18000 r--p 00017000 08:03 34362                          /lib64/libpthread-2.12.so

                        3ca0e18000-3ca0e19000 rw-p 00018000 08:03 34362                          /lib64/libpthread-2.12.so

                        3ca0e19000-3ca0e1d000 rw-p 00000000 00:00 0

                        3ca1000000-3ca1083000 r-xp 00000000 08:03 34374                          /lib64/libm-2.12.so

                        3ca1083000-3ca1282000 ---p 00083000 08:03 34374                          /lib64/libm-2.12.so

                        3ca1282000-3ca1283000 r--p 00082000 08:03 34374                          /lib64/libm-2.12.so

                        3ca1283000-3ca1284000 rw-p 00083000 08:03 34374                          /lib64/libm-2.12.so

                        3ca1800000-3ca1807000 r-xp 00000000 08:03 34363                          /lib64/librt-2.12.so

                        3ca1807000-3ca1a06000 ---p 00007000 08:03 34363                          /lib64/librt-2.12.so

                        3ca1a06000-3ca1a07000 r--p 00006000 08:03 34363                          /lib64/librt-2.12.so

                        3ca1a07000-3ca1a08000 rw-p 00007000 08:03 34363                          /lib64/librt-2.12.so

                        3ca4000000-3ca416f000 r-xp 00000000 08:03 34403                          /lib64/libdb-4.7.so

                        3ca416f000-3ca436e000 ---p 0016f000 08:03 34403                          /lib64/libdb-4.7.so

                        3ca436e000-3ca4374000 rw-p 0016e000 08:03 34403                          /lib64/libdb-4.7.so

                        3ca4400000-3ca4426000 r-xp 00000000 08:03 34389                          /lib64/libexpat.so.1.5.2

                        3ca4426000-3ca4625000 ---p 00026000 08:03 34389                          /lib64/libexpat.so.1.5.2

                        3ca4625000-3ca4628000 rw-p 00025000 08:03 34389                          /lib64/libexpat.so.1.5.2

                        3ca5000000-3ca5016000 r-xp 00000000 08:03 34375                          /lib64/libgcc_s-4.4.6-20120305.so.1

                        3ca5016000-3ca5215000 ---p 00016000 08:03 34375                          /lib64/libgcc_s-4.4.6-20120305.so.1

                        3ca5215000-3ca5216000 rw-p 00015000 08:03 34375                          /lib64/libgcc_s-4.4.6-20120305.so.1

                        3ca6400000-3ca64e8000 r-xp 00000000 08:03 397010                         /usr/lib64/libstdc++.so.6.0.13

                        3ca64e8000-3ca66e8000 ---p 000e8000 08:03 397010                         /usr/lib64/libstdc++.so.6.0.13

                        3ca66e8000-3ca66ef000 r--p 000e8000 08:03 397010                         /usr/lib64/libstdc++.so.6.0.13

                        3ca66ef000-3ca66f1000 rw-p 000ef000 08:03 397010                         /usr/lib64/libstdc++.so.6.0.13

                        3ca66f1000-3ca6706000 rw-p 00000000 00:00 0

                        3ca6800000-3ca6804000 r-xp 00000000 08:03 34388                          /lib64/libuuid.so.1.3.0

                        3ca6804000-3ca6a03000 ---p 00004000 08:03 34388                          /lib64/libuuid.so.1.3.0

                        3ca6a03000-3ca6a04000 rw-p 00003000 08:03 34388                          /lib64/libuuid.so.1.3.0

                        3cb0400000-3cb0416000 r-xp 00000000 08:03 34401                          /lib64/libnsl-2.12.so

                        3cb0416000-3cb0615000 ---p 00016000 08:03 34401                          /lib64/libnsl-2.12.so

                        3cb0615000-3cb0616000 r--p 00015000 08:03 34401                          /lib64/libnsl-2.12.so

                        3cb0616000-3cb0617000 rw-p 00016000 08:03 34401                          /lib64/libnsl-2.12.so

                        3cb0617000-3cb0619000 rw-p 00000000 00:00 0

                        3cb0e00000-3cb0e5d000 r-xp 00000000 08:03 34385                          /lib64/libfreebl3.so

                        3cb0e5d000-3cb105c000 ---p 0005d000 08:03 34385                          /lib64/libfreebl3.so

                        3cb105c000-3cb105d000 r--p 0005c000 08:03 34385                          /lib64/libfreebl3.so

                        3cb105d000-3cb105e000 rw-p 0005d000 08:03 34385                          /lib64/libfreebl3.so

                        3cb105e000-3cb1062000 rw-p 00000000 00:00 0

                        3cb1200000-3cb1207000 r-xp 00000000 08:03 34386                          /lib64/libcrypt-2.12.so

                        3cb1207000-3cb1407000 ---p 00007000 08:03 34386                          /lib64/libcrypt-2.12.so

                        3cb1407000-3cb1408000 r--p 00007000 08:03 34386                          /lib64/libcrypt-2.12.so

                        3cb1408000-3cb1409000 rw-p 00008000 08:03 34386                          /lib64/libcrypt-2.12.so

                         

                         

                         

                        Are there any examples for tprealloc.

                        • 9. Re: Blacktie Data Buffers and Content Based Routing
                          zhfeng

                          yeah, if you use tpalloc("BT_NBF", "example", 0), the initial buffer size is 512. I think you should not use the tprealloc on the BT_NBF buffer

                          So your codes looks like the following ?

                          char* buf = tpalloc("BT_NBF", "example", 0);
                          btaddattribute(buf, "message", message, strlen(message));
                          

                           

                          can you send me your codes and the blacktie.log ?

                          • 10. Re: Blacktie Data Buffers and Content Based Routing
                            joicejoy

                            Right, I use tpalloc as you mentioned above.

                             

                            Please share how to allocate bigger buffers. Sent you the code.

                            • 11. Re: Blacktie Data Buffers and Content Based Routing
                              zhfeng

                              Thanks Joy, I will take a look at it.

                              • 12. Re: Blacktie Data Buffers and Content Based Routing
                                zhfeng

                                In your client.c sendbuf = tpalloc((char*)"BT_NBF", (char*)"employee", 0);

                                long tempLen = strlen(sendbuf);
                                btaddattribute(&sendbuf, (char*) "payload", (char*)inputxml, length);
                                tempLen = strlen(sendbuf);
                                btaddattribute(&sendbuf, (char*) "size", (char*)&length, sizeof(length));
                                

                                 

                                So do you add the "payload" and "size" element type in the buffers/employee.xsd ? I think we are using this scheme to get the type info of the attribute.

                                I have not test to add the xml string as a attribute and will try to run your client.c on my machine.

                                • 13. Re: Blacktie Data Buffers and Content Based Routing
                                  joicejoy

                                  When sending XML file in payload, the ParserHandlers give errors. So I tried sending large strings instead. I got errors when size of buffer exceeded 512 bytes.

                                  • 14. Re: Blacktie Data Buffers and Content Based Routing
                                    joicejoy

                                    I am trying to get the CBR quickstart running while the buffer issue is solved.

                                    https://github.com/jbosstm/quickstart/tree/14110d44b1cb49f35caa21cf969903e83b645745/blacktie/jatmibroker-xatmi/cbr

                                     

                                    But I face the following error:

                                    [INFO] Reploying /fxtest/trep/cbr/target/blacktie-quickstarts-cbr-5.2.3.Final-SNAPSHOT.esb to JBoss.

                                    [INFO] No server id specified for authentication - using defaults

                                    [INFO] ------------------------------------------------------------------------

                                    [INFO] BUILD FAILURE

                                    [INFO] ------------------------------------------------------------------------

                                    [INFO] Total time: 5.767 s

                                    [INFO] Finished at: 2015-09-29T00:43:18-06:00

                                    [INFO] Final Memory: 13M/303M

                                    [INFO] ------------------------------------------------------------------------

                                    [ERROR] Failed to execute goal org.codehaus.mojo:jboss-maven-plugin:1.5.0:redeploy (jboss-redeploy) on project blacktie-quickstarts-cbr: Mojo error occurred: http://10.68.215.247:8080/jmx-console/HtmlAdaptor?action=invokeOpByName&name=jboss.system:service%3DMainDeployer&methodName=redeploy&argType=java.net.URL&arg0=/fxtest/trep/cbr/target/blacktie-quickstarts-cbr-5.2.3.Final-SNAPSHOT.esb -> [Help 1]

                                    org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal org.codehaus.mojo:jboss-maven-plugin:1.5.0:redeploy (jboss-redeploy) on project blacktie-quickstarts-cbr: Mojo error occurred: http://10.68.215.247:8080/jmx-console/HtmlAdaptor?action=invokeOpByName&name=jboss.system:service%3DMainDeployer&methodName=redeploy&argType=java.net.URL&arg0=/fxtest/trep/cbr/target/blacktie-quickstarts-cbr-5.2.3.Final-SNAPSHOT.esb

                                            at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:216)

                                            at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)

                                            at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)

                                            at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)

                                            at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)

                                            at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)

                                            at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)

                                            at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)

                                            at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)

                                            at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)

                                            at org.apache.maven.cli.MavenCli.execute(MavenCli.java:862)

                                            at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:286)

                                            at org.apache.maven.cli.MavenCli.main(MavenCli.java:197)

                                            at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

                                            at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)

                                            at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)

                                            at java.lang.reflect.Method.invoke(Method.java:497)

                                            at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)

                                            at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)

                                            at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)

                                            at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)

                                    Caused by: org.apache.maven.plugin.MojoExecutionException: Mojo error occurred: http://10.68.215.247:8080/jmx-console/HtmlAdaptor?action=invokeOpByName&name=jboss.system:service%3DMainDeployer&methodName=redeploy&argType=java.net.URL&arg0=/fxtest/trep/cbr/target/blacktie-quickstarts-cbr-5.2.3.Final-SNAPSHOT.esb

                                            at org.codehaus.mojo.jboss.AbstractJBossDeployerMojo.doURL(AbstractJBossDeployerMojo.java:160)

                                            at org.codehaus.mojo.jboss.ReDeployMojo.doExecute(ReDeployMojo.java:78)

                                            at org.codehaus.mojo.jboss.AbstractJBossDeployerMojo.execute(AbstractJBossDeployerMojo.java:125)

                                            at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)

                                            at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)

                                            ... 20 more

                                    Caused by: java.io.FileNotFoundException: http://10.68.215.247:8080/jmx-console/HtmlAdaptor?action=invokeOpByName&name=jboss.system:service%3DMainDeployer&methodName=redeploy&argType=java.net.URL&arg0=/fxtest/trep/cbr/target/blacktie-quickstarts-cbr-5.2.3.Final-SNAPSHOT.esb

                                            at sun.net.www.protocol.http.HttpURLConnection.getInputStream0(HttpURLConnection.java:1835)

                                            at sun.net.www.protocol.http.HttpURLConnection.getInputStream(HttpURLConnection.java:1440)

                                            at org.codehaus.mojo.jboss.AbstractJBossDeployerMojo.doURL(AbstractJBossDeployerMojo.java:154)

                                     

                                     

                                    Also if I try :

                                    http://10.68.215.247:8080/jmx-console

                                     

                                    I get a error:

                                    404 - Not Found

                                     

                                    Please help with this

                                    1 2 Previous Next