1 2 Previous Next 18 Replies Latest reply on Oct 9, 2014 3:50 PM by harleybl

    HornetQ Stomp Subscription ID Mangling in JBoss EAP 6.2.4

    harleybl

      I am working on a project where I am attempting to use Apache.NMS.Stomp to connect to the HornetQ via Stomp.

      I had previously had problems with generating subscribers (Re: Stomp Client 1.5.4 error creating subscription to topic on JBoss HornetQ (Jboss 6.4.2 GA)) but was able to work through that and am looking to supply a patch to the Apache.NMS.Stomp project. The next problem I am running into is kind of weird and I don't fully understand where it is happening.

       

      I have a test case where I write a bunch of messages to the Queue and then pull them off the queue. At first the Stomp client was pulling the messages, but not Acking them or handling them because it didn't think it was subscribed because it appears that the Stomp handler on the JBoss side is mangling the subscription ids.

       

      I turned tracing in the Stomp client and here is what I see. First the library sends a SUBSCRIBE stomp frame:

       

      Debug:StompWireFormat - Writing StompFrame[ Command=SUBSCRIBE, Properties={ id=ID:MYMACHINE-50616-635482262727823605-1:1:1:1 receipt=2 destination=jms.queue.test.queue activemq.dispatchAsync=True transformation=jms-xml activemq.priority=0 activemq.maximumPendingMessageLimit=0 activemq.prefetchSize=1000 ack=client}, Content=]

       

      So far good - JBoss acknowledges receipt of the subscribe message and starts sending MESSAGE stomp frames:

       

      Debug:StompWireFormat - Received StompFrame[ Command=MESSAGE, Properties={ receipt=2 destination=jms.queue.test,queue NMSXDeliveryMode=true timestamp=1412643874233 expires=0 redelivered=false subscription=IDcMYMACHINE-50616-635482262727823605-1\c1c1\c1 message-id=95592 priority=5}, Content=System.Byte[]]

       

      If you look more closely at the ID that was sent with the SUBSCRIBE frame and then returned by the MESSAGE frame you see the following:

      Sent : ID:MYMACHINE-50616-635482262727823605-1:1:1:1

      Rec : IDcMYMACHINE-50616-635482262727823605-1\c1c1\c1

       

      It appears that some of the ':' are turned into plain 'c' and others are turned into '\c'.

      Since I am in the Apache.NMS.Stomp source code I put in a hack to replace both 'c' and '\c' with ':' but feel that is sub-optimal.

       

      I cloned the HornetQ git repository and read through the code and don't see a place where it is doing any modification on the headers or subscription.

      Does anyone have any ideas or clues as to why this is happening?

      Additionally I don't see way to turn up the logging to see the traffic, so if someone knows of a way to do that I would be most appreciative.

        • 1. Re: HornetQ Stomp Subscription ID Mangling in JBoss EAP 6.2.4
          jbertram

          I've not worked on the STOMP implementation at all, but a quick look at the STOMP 1.1 spec reveals http://stomp.github.io/stomp-specification-1.1.html#Value_Encoding which is probably related to your question.  Perhaps you could try using an ID with ":" characters.

          • 2. Re: Re: HornetQ Stomp Subscription ID Mangling in JBoss EAP 6.2.4
            harleybl

            I pulled down the HornetQ source code and unless I am reading it wrong it seems like the escaping code doesn't respect the Stomp specification as it is described at http://stomp.github.io/stomp-specification-1.2.html . It looks like the code is just putting slashes before special characters instead of converting them to the stomp equivalents.

             

            Here is the code I am talking about at https://github.com/hornetq/hornetq/blob/master/hornetq-protocols/hornetq-stomp-protocol/src/main/java/org/hornetq/core/protocol/stomp/StompFrame.java :

             

             public String escape(String str)
               {
                  int len = str.length();
            
            
                  char[] buffer = new char[2 * len];
                  int iBuffer = 0;
                  for (int i = 0; i < len; i++)
                  {
                     char c = str.charAt(i);
                     if (c == '\n')
                     {
                        buffer[iBuffer++] = '\\';
                        buffer[iBuffer] = 'n';
                     }
                     else if (c == '\\')
                     {
                        buffer[iBuffer++] = '\\';
                        buffer[iBuffer] = '\\';
                     }
                     else if (c == ':')
                     {
                        buffer[iBuffer++] = '\\';
                        buffer[iBuffer] = ':';
                     }
                     else
                     {
                        buffer[iBuffer] = c;
                     }
                     iBuffer++;
                  }
            
            
                  char[] total = new char[iBuffer];
                  System.arraycopy(buffer, 0, total, 0, iBuffer);
            
            
                  return new String(total);
               }
            
            
            
            
            
            
            
            
            
            

             

            So unless I am reading this wrong it appears that rather than converting ':' to '\c' it is converting it to \: .

            • 3. Re: HornetQ Stomp Subscription ID Mangling in JBoss EAP 6.2.4
              jbertram

              Couple of things...

              1. I see what you're saying, but I still don't understand where the 'c' characters would be coming from.  Is the Apache client doing that?
              2. I would need a reproducible test-case in order to fix this.  Perhaps you could add one to org.hornetq.tests.integration.stomp.StompTest?
              • 4. Re: HornetQ Stomp Subscription ID Mangling in JBoss EAP 6.2.4
                harleybl

                On #1 - That is confusing to me as well. It doesn't appear that the Apache client is doing that. I don't think that it is because I put some print debug statements right after the .NET client code converts the bytes it received from the stream to UTF8 and I see the 'c' characters. Another thing that is puzzling to me is the fact that some of them are converted to just 'c' and others are '\c'.

                 

                On #2 - Dumb Question -  Can you point me to some documentation on how to set up my development environment to work on the project?

                I am using Intellij Idea and the "Import project from pom.xmls" didn't work right out of the box.

                • 5. Re: HornetQ Stomp Subscription ID Mangling in JBoss EAP 6.2.4
                  jbertram

                  On #2 - Dumb Question -  Can you point me to some documentation on how to set up my development environment to work on the project?

                  I am using Intellij Idea and the "Import project from pom.xmls" didn't work right out of the box.

                  I use IntelliJ IDEA as well and all I do is File->Import Project...  Then I point to the directory where I cloned HornetQ and it detects all the Maven poms and does the rest.  I just click "Next" or whatever until the project loads.  I'm using 13.1.5.

                  • 6. Re: HornetQ Stomp Subscription ID Mangling in JBoss EAP 6.2.4
                    harleybl

                    I must be doing something wrong because I did exactly that and there are dependencies missing all over the place in terms of:

                    • Maven libraries not being assigned to projects
                    • Sub-projects not depending on the correct other projects
                    • Idea not being able to see test source from projects

                    What version of Maven are you using?

                    • 7. Re: Re: HornetQ Stomp Subscription ID Mangling in JBoss EAP 6.2.4
                      jbertram
                      $ mvn -v
                      Apache Maven 3.0.5
                      Maven home: /usr/share/maven
                      Java version: 1.7.0_67, vendor: Oracle Corporation
                      Java home: /usr/lib/jvm/java-7-oracle/jre
                      Default locale: en_US, platform encoding: UTF-8
                      OS name: "linux", version: "3.13.0-36-generic", arch: "amd64", family: "unix"
                      
                      • 8. Re: Re: HornetQ Stomp Subscription ID Mangling in JBoss EAP 6.2.4
                        gaohoward

                        Can you give us a test client?

                        • 9. Re: Re: HornetQ Stomp Subscription ID Mangling in JBoss EAP 6.2.4
                          harleybl

                          Yong - Welcome back from vacation.

                          I have a test client ready in a self contained zip file that has everything you need.

                          What is the best way for me to share that with you?

                          I don't see an option to attach a file here.

                          • 10. Re: HornetQ Stomp Subscription ID Mangling in JBoss EAP 6.2.4
                            jbertram

                            Were you not able to add a test to org.hornetq.tests.integration.stomp.StompTest?

                             

                            You can attach files by clicking the "Use advanced editor" link when composing a comment.

                            • 11. Re: Re: HornetQ Stomp Subscription ID Mangling in JBoss EAP 6.2.4
                              gaohoward

                              Yes you can attach it as Justin mentioned.

                              • 12. Re: Re: Re: HornetQ Stomp Subscription ID Mangling in JBoss EAP 6.2.4
                                harleybl

                                Here is the client.

                                I haven't been able to successfully compile the HornetQ source in idea.

                                I did add my test and run the maven build outside, but it hung before it got to my integration test.

                                I'll keep trying but at least you have this client in the mean time.

                                Check out the README.txt inside. You should be able to run it as is from the debug or release directory, but I included all the source in there as well.

                                • 13. Re: HornetQ Stomp Subscription ID Mangling in JBoss EAP 6.2.4
                                  jbertram

                                  I run Linux on my machine and I don't have access to a Windows box or VM so I won't be able to run your client.  In any event, the test would ideally not include Apache NMS so as to eliminate Apache NMS as a potential source of the bug.  That is one of the main reasons I wanted you to write a test in the HornetQ test-suite.

                                   

                                  I'm not sure what could be going wrong for you with HornetQ in IntelliJ IDEA.  I re-import the project all the time on my machine when I switch between major branches.  What branch are you trying to use?

                                  • 14. Re: Re: HornetQ Stomp Subscription ID Mangling in JBoss EAP 6.2.4
                                    harleybl

                                    I was finally able to get this to work after trying to pull the source again and import the project again with some different maven settings.

                                    Additionally I was able to create an integration test that reproduces the problem in the StompV11Test class.

                                    Here is my pull request for the integration test: https://github.com/hornetq/hornetq/pull/1900

                                     

                                    Additionally here is the code:

                                    @Test
                                        public void testSendAndReceiveWithEscapedCharactersInSenderId() throws Exception
                                        {
                                            connV11.connect(defUser, defPass);
                                            ClientStompFrame frame = connV11.createFrame("SEND");
                                            frame.addHeader("destination", getQueuePrefix() + getQueueName());
                                            frame.addHeader("content-type", "text/plain");
                                            frame.setBody("Hello World 1!");
                                    
                                    
                                            ClientStompFrame response = connV11.sendFrame(frame);
                                            assertNull(response);
                                    
                                    
                                            //subscribe
                                            StompClientConnection newConn = StompClientConnectionFactory.createClientConnection("1.1", hostname, port);
                                            newConn.connect(defUser, defPass);
                                    
                                    
                                            ClientStompFrame subFrame = newConn.createFrame("SUBSCRIBE");
                                            subFrame.addHeader("id", "ID\\cMYMACHINE-50616-635482262727823605-1\\c1\\c1\\c1");
                                            subFrame.addHeader("destination", getQueuePrefix() + getQueueName());
                                            subFrame.addHeader("ack", "auto");
                                    
                                    
                                            newConn.sendFrame(subFrame);
                                    
                                    
                                            frame = newConn.receiveFrame();
                                    
                                    
                                            System.out.println("received " + frame);
                                    
                                    
                                            assertEquals("MESSAGE", frame.getCommand());
                                            assertEquals("ID:MYMACHINE-50616-635482262727823605-1:1:1:1", frame.getHeader("subscription"));
                                            assertNotNull(frame.getHeader("message-id"));
                                            assertEquals(getQueuePrefix() + getQueueName(), frame.getHeader("destination"));
                                            assertEquals("Hello World 1!", frame.getBody());
                                    
                                    
                                            //unsub
                                            ClientStompFrame unsubFrame = newConn.createFrame("UNSUBSCRIBE");
                                            unsubFrame.addHeader("id", "ID\\cMYMACHINE-50616-635482262727823605-1\\c1\\c1\\c1");
                                            newConn.sendFrame(unsubFrame);
                                    
                                    
                                            newConn.disconnect();
                                        }
                                    
                                    
                                    
                                    
                                    
                                    
                                    
                                    
                                    
                                    
                                    
                                    
                                    
                                    
                                    
                                    
                                    
                                    
                                    
                                    
                                    
                                    1 2 Previous Next