6 Replies Latest reply on Aug 20, 2007 7:07 AM by Tom Fennelly

    Alternative Message Discussion

    Kurt Stam Master


      I have not posted a response to: the Message Body simplication tpic: http://www.jboss.com/index.html?module=bb&op=viewtopic&t=116278
      because I think there we need to finish the following discussion first.

      This subject of our Message Design has been brought up multiple times over the last year, and now that we are nearing completion on 4.2 it has become more and more clear to me that we should discuss changing the current design.

      1. Requirements (as far as I understand them):

      r1. The user should be able to define their own message type.

      r2. The message should be easy to understand and easy to use.

      r3. Should play nice with existing standards like SOAP and WS-* and may I add Java and JMS.


      Discussion
      Now what does r1 really mean? It means that the way the message gets serialized should be up to the users desire. We offer up 2 ways: Serialized java objects or XML.

      2. Issues with the current message implementation:

      i1. the body has a named map in which you can place objects or you can use a byteArray, well
      IMHO there is NO need for a byteArray. Everything is Java is an object, so you could simply add
      the byteArray in the map. It is causing all sort of misunderstood code in our own code base alone
      and we are supposed to understand it. Let's just get rid of the byteArray.

      i2. we have two different message implementations depending on the type of the Message (see r1). There
      is NO need for this! Only the un/marshal code should be different. There is no need to have this
      MessageFactory, there is no need for Message interfaces. It can all be done by some simple java objects
      with different pluggable serialization options.

      i3. The Message itself is constructed not using simple Java types: I mean when you do message.getBody() you are not getting the Map API. You are getting our interface. This is higly confusing and we should fix this. Also let's not reinvent "Property" once again. Let's simply use the one in Java.

      i4. Tools like MVEL cannot process our Message because of i3.

      3. Proposal

      Let discuss a simple Message implementation that satisfies our requirements and that does not have the issues above. We can keep on applying bandages on our current implementation, but I think it is broken by design. Yes it will be pain to our users to change it now, but as a Dutch saying goes ?Weak Doctors make stinking wounds?. I think we will be saving everyone involved a great deal of pain if we would decide to change it now.

      OK I'm all in Nomex.

      --Kurt

        • 1. Re: Alternative Message Discussion
          Kevin Conner Master

           

          "kurt.stam@jboss.com" wrote:

          because I think there we need to finish the following discussion first.


          I believe it has been mentioned a number of times that this is a discussion for 5 and not the 4.x codebase, of course I could be wrong.

          "kurt.stam@jboss.com" wrote:

          r1. The user should be able to define their own message type.


          Sorry but the message *type* should be irrelevant as far as the user is concerned. Only the contents of the message are of concern.

          "kurt.stam@jboss.com" wrote:

          r2. The message should be easy to understand and easy to use.

          r3. Should play nice with existing standards like SOAP and WS-* and may I add Java and JMS.


          Agreed.

          "kurt.stam@jboss.com" wrote:

          Now what does r1 really mean? It means that the way the message gets serialized should be up to the users desire. We offer up 2 ways: Serialized java objects or XML.


          The way in which a message is serialised should only be of relevance to a transport and not the user of the message. The real issue is that we still have major changes to be made to the architecture before we are close to the original design Mark and I proposed (especially in relation to the transport).

          "kurt.stam@jboss.com" wrote:

          i1. the body has a named map in which you can place objects or you can use a byteArray, well
          IMHO there is NO need for a byteArray. Everything is Java is an object, so you could simply add
          the byteArray in the map. It is causing all sort of misunderstood code in our own code base alone
          and we are supposed to understand it. Let's just get rid of the byteArray.


          I agree that there is no need for the byte array and it does just confuse the issue.

          The named map should also be changed IMHO. The real use of this map is to pass context data around but this should be handled by a separate (explicit) message context. This will allow us add other abilities to the context, such as the visibility of the context properties (i.e. whether it only applies to specific services, handlers, threads, VMs etc.).

          There is a big difference between message *context* and message *payload*.

          "kurt.stam@jboss.com" wrote:

          i2. we have two different message implementations depending on the type of the Message (see r1). There
          is NO need for this! Only the un/marshal code should be different. There is no need to have this
          MessageFactory, there is no need for Message interfaces. It can all be done by some simple java objects
          with different pluggable serialization options.


          Interfaces should be used to allow for flexibility in implementation. Different implementations may be possible to optimise for different requirements, e.g. in VM storage, DB passivation etc.

          The fact that the current implementations are focussed on serialising the message is the real issue.

          "kurt.stam@jboss.com" wrote:

          i3. The Message itself is constructed not using simple Java types: I mean when you do message.getBody() you are not getting the Map API. You are getting our interface. This is higly confusing and we should fix this. Also let's not reinvent "Property" once again. Let's simply use the one in Java.


          This goes back to separating out the message context which could easily extend the Map interface. I agree that reusing an appropriate (core) interface is preferable to reinventing our own.

          "kurt.stam@jboss.com" wrote:

          i4. Tools like MVEL cannot process our Message because of i3.

          Let discuss a simple Message implementation that satisfies our requirements and that does not have the issues above. We can keep on applying bandages on our current implementation, but I think it is broken by design. Yes it will be pain to our users to change it now, but as a Dutch saying goes ?Weak Doctors make stinking wounds?. I think we will be saving everyone involved a great deal of pain if we would decide to change it now.


          It is far too late in the day to change this now so this will have to stay for 4.x.

          We have an opportunity to handle this (and other issues) in the 5 stream and then migrate the 4.x stream towards this.

          Kev


          • 2. Re: Alternative Message Discussion
            Mark Little Master

             

            "kurt.stam@jboss.com" wrote:

            I have not posted a response to: the Message Body simplication tpic: http://www.jboss.com/index.html?module=bb&op=viewtopic&t=116278
            because I think there we need to finish the following discussion first.


            I'm fairly sure we had discussed leaving this sort of thing until after the 4.2.x release. Nothing of what we've got so far will be changing in this release. Everything is open to discussion for what comes after 4.2.x.


            This subject of our Message Design has been brought up multiple times over the last year, and now that we are nearing completion on 4.2 it has become more and more clear to me that we should discuss changing the current design.


            I'm fairly sure that this has been said time and time again: what we have was always a start and not the end result. I'm glad you agree with myself and others that we need to continue the evolution.


            1. Requirements (as far as I understand them):

            r1. The user should be able to define their own message type.


            Actually this has NOTHING to do with the user and everything to do with the service and the clients that interact with it. As I've said many times, the original intention was always that Message and the marshal/unmarshal approach would be hidden from the majority of users. Clients shouldn't be arbitrarily deciding what message format (Java Serialized, XML, ASN.1 etc.) to use when talking to a service: that's up to the service! And it may be different for different types of clients or even for different "operations" that the service supports.


            r2. The message should be easy to understand and easy to use.

            r3. Should play nice with existing standards like SOAP and WS-* and may I add Java and JMS.


            Agreed.


            Discussion
            Now what does r1 really mean? It means that the way the message gets serialized should be up to the users desire.


            Absolutely not.


            We offer up 2 ways: Serialized java objects or XML.


            We started off offering only one (XML). If you check the implementation of that, it allows for arbitrary plugins for marshalling and unmarshalling arbitrary data types, i.e., it doesn't assume serializable objects. So if anything the XML implementation is a superset of the Serializable one. The reason we also have the Serializable one is historic: that's all Rosetta supported when we inherited it. Feedback at the time from the Rosetta developers was that we should continue supporting Serializable, so we have.

            But ignoring that fact, let me re-iterate: the client should not (in 99% of cases) we setting the on-the-wire format for interactions with services. That information comes form the services themselves. This isn't new either. If you look at CORBA or some of the older RPC/MOM distributed systems such as Emerald, Argus, ANSAware, Arjuna2 etc. you'll see that they all supported flexible transports that were not (typically) exposed to the invoker: it was opaque.

            We currently expose a lot of what shouldn't be exposed simply because we have started building up and building out from Rosetta. This was always a first step, so people shouldn't make sweeping and incorrect assumptions based on what's currently there. There will always be a need for some developers to tinker with the low-level details of transports. At the moment we tend to cater for those individuals more than the ones who really don't care/shouldn't care, but as we build up the higher levels of abstraction we'll get there. That doesn't mean we will prevent the types of development style we currently have, only that it'll not be the norm.


            2. Issues with the current message implementation:

            i1. the body has a named map in which you can place objects or you can use a byteArray, well
            IMHO there is NO need for a byteArray. Everything is Java is an object, so you could simply add
            the byteArray in the map. It is causing all sort of misunderstood code in our own code base alone
            and we are supposed to understand it. Let's just get rid of the byteArray.


            I don't think it's such a big deal to have it or not. I do know that we overuse it when it's simply not necessary. From that perspective alone I'd agree to remove it (or deprecate it as we have to!)


            i2. we have two different message implementations depending on the type of the Message (see r1). There
            is NO need for this! Only the un/marshal code should be different. There is no need to have this
            MessageFactory, there is no need for Message interfaces. It can all be done by some simple java objects
            with different pluggable serialization options.


            Agreed. See my comment above about the way in which the XML implementation works. It's about 90% of the way you mention and the only differences are slight: at the serialization level. The name (XML message) is in no way meant to represent how the Message is laid down within the address space of the user, i.e., you're not manipulating XML data structures when you add/remove/replace elements. It only becomes XML when you hit the network. There is an issue to clarify the Message and hopefully this will become clearer in the documentation.


            i3. The Message itself is constructed not using simple Java types: I mean when you do message.getBody() you are not getting the Map API. You are getting our interface. This is higly confusing and we should fix this.


            Yes, we will definitely address this and several other issues in the rest of the code after 4.2.x is released.


            Also let's not reinvent "Property" once again. Let's simply use the one in Java.


            Agreed. And we do that elsewhere too.


            i4. Tools like MVEL cannot process our Message because of i3.


            MVEL support is not on the roadmap. So although this is useful information for when we come to re-evaluate, it is irrelevant for this release.


            3. Proposal

            Let discuss a simple Message implementation that satisfies our requirements and that does not have the issues above. We can keep on applying bandages on our current implementation, but I think it is broken by design. Yes it will be pain to our users to change it now, but as a Dutch saying goes ?Weak Doctors make stinking wounds?. I think we will be saving everyone involved a great deal of pain if we would decide to change it now.


            It is far too late to change this for the forthcoming release. I don't think it is that important an issue to reschedule the entire release process of the ESB and the SOA Platform. There are other higher priority issues that we need to address. Plus, feedback from end users is very important too. We can sit here in the forums contemplating what we believe is critical and what isn't, but feedback from in-the-field users is vitally important.


            OK I'm all in Nomex.


            Not sure I get the reference? Actually I am sure: I don't get it ;-)

            • 3. Re: Alternative Message Discussion
              Mark Little Master

               

              "Kevin.Conner@jboss.com" wrote:
              "kurt.stam@jboss.com" wrote:

              because I think there we need to finish the following discussion first.


              I believe it has been mentioned a number of times that this is a discussion for 5 and not the 4.x codebase, of course I could be wrong.


              No, you're not wrong. We're in the last phases of releasing 4.2 and the platform release is soon after that. We have QA, docs, productization etc. all running against what we've got and making good progress. No changes to the ESB at this stage can ever happen in isolation.


              "kurt.stam@jboss.com" wrote:

              r1. The user should be able to define their own message type.


              Sorry but the message *type* should be irrelevant as far as the user is concerned. Only the contents of the message are of concern.


              +1


              "kurt.stam@jboss.com" wrote:

              r2. The message should be easy to understand and easy to use.

              r3. Should play nice with existing standards like SOAP and WS-* and may I add Java and JMS.


              Agreed.

              "kurt.stam@jboss.com" wrote:

              Now what does r1 really mean? It means that the way the message gets serialized should be up to the users desire. We offer up 2 ways: Serialized java objects or XML.


              The way in which a message is serialised should only be of relevance to a transport and not the user of the message.


              Agreed. The transport is defined by the service as part of the contract definition.


              The real issue is that we still have major changes to be made to the architecture before we are close to the original design Mark and I proposed (especially in relation to the transport).


              Oh yes. Definitely.


              "kurt.stam@jboss.com" wrote:

              i1. the body has a named map in which you can place objects or you can use a byteArray, well
              IMHO there is NO need for a byteArray. Everything is Java is an object, so you could simply add
              the byteArray in the map. It is causing all sort of misunderstood code in our own code base alone
              and we are supposed to understand it. Let's just get rid of the byteArray.


              I agree that there is no need for the byte array and it does just confuse the issue.


              Let's deprecate it then.


              The named map should also be changed IMHO. The real use of this map is to pass context data around but this should be handled by a separate (explicit) message context.


              I may be missing something, but the Map in the Body implementation has nothing to do with context. That's what the Context element of the Message is for. Granted that we don't use that in the way we should (yes, that's another JIRA-ed issue ;-), and we tend to stuff context related information within the Body, but even if we fix that it doesn't remove the need for Body to maintain a mapping of content to name. The names of elements within the Body need to be unique, particularly if the Body can be manipulated by different independently developed service components (e.g., Actions). Whether the internal implementation of Body is a Map or not should be down to the Body implementation and not exposed to the end user.


              This will allow us add other abilities to the context, such as the visibility of the context properties (i.e. whether it only applies to specific services, handlers, threads, VMs etc.).

              There is a big difference between message *context* and message *payload*.


              +1


              "kurt.stam@jboss.com" wrote:

              i2. we have two different message implementations depending on the type of the Message (see r1). There
              is NO need for this! Only the un/marshal code should be different. There is no need to have this
              MessageFactory, there is no need for Message interfaces. It can all be done by some simple java objects
              with different pluggable serialization options.


              Interfaces should be used to allow for flexibility in implementation. Different implementations may be possible to optimise for different requirements, e.g. in VM storage, DB passivation etc.


              +1


              The fact that the current implementations are focussed on serialising the message is the real issue.


              Well to be honest the current XML implementation isn't focussed on Java serialized objects at all. With the exception of Attachments (simply hasn't been added) and Properties (obvious reasons), the XML message type doesn't mandate a content type for the elements within a Message. If there's not Java Serializable, then you just add a plugin to cope with the marshal/unmarshal for that data type. Pretty standard approach to be perfectly honest, and used in several distributed systems.

              I think the real issue is that we expose too much of this to developers. However, as I said in my response to Kurt, you've got to start somewhere and given the original Rosetta donation we started bottom-up. That means we haven't yet developed the higher level abstractions that we all (?) know and agree we need to have. But they'll come soon enough.


              "kurt.stam@jboss.com" wrote:

              i3. The Message itself is constructed not using simple Java types: I mean when you do message.getBody() you are not getting the Map API. You are getting our interface. This is higly confusing and we should fix this. Also let's not reinvent "Property" once again. Let's simply use the one in Java.


              This goes back to separating out the message context which could easily extend the Map interface. I agree that reusing an appropriate (core) interface is preferable to reinventing our own.

              "kurt.stam@jboss.com" wrote:

              i4. Tools like MVEL cannot process our Message because of i3.

              Let discuss a simple Message implementation that satisfies our requirements and that does not have the issues above. We can keep on applying bandages on our current implementation, but I think it is broken by design. Yes it will be pain to our users to change it now, but as a Dutch saying goes ?Weak Doctors make stinking wounds?. I think we will be saving everyone involved a great deal of pain if we would decide to change it now.


              It is far too late in the day to change this now so this will have to stay for 4.x.

              We have an opportunity to handle this (and other issues) in the 5 stream and then migrate the 4.x stream towards this.


              +1

              • 4. Re: Alternative Message Discussion
                Tom Fennelly Master

                I too think it's important that we revisit this stuff. But I also think we're just way too late for 4.2.x.

                Mark and I had an IM conversation on this topic on Friday morning last. At that time, I told him that I made an effort to gather my thoughts on this stuff a few weeks ago. I did it by way of putting together what I thought I'd like to see in the Message, taking the current message, my own experiences of using our Message, feedback from other users etc into consideration. I told Mark that I'd clean it up over the weekend.

                This is what I came up with. We obviously don't need to discuss anything in it right now. It's there for anyone to look at and we can discuss post 4.2.x.

                • 5. Re: Alternative Message Discussion
                  Mark Little Master

                   

                  "tfennelly" wrote:
                  I too think it's important that we revisit this stuff. But I also think we're just way too late for 4.2.x.


                  +1 (again).


                  Mark and I had an IM conversation on this topic on Friday morning last. At that time, I told him that I made an effort to gather my thoughts on this stuff a few weeks ago. I did it by way of putting together what I thought I'd like to see in the Message, taking the current message, my own experiences of using our Message, feedback from other users etc into consideration. I told Mark that I'd clean it up over the weekend.

                  <a href="http://wiki.jboss.org/wiki/attach?page=JBossESB%2Ftomfs-message-thoughts.zip">This is what I came up with</a>. We obviously don't need to discuss anything in it right now. It's there for anyone to look at and we can discuss post 4.2.x.


                  Thanks Tom. You're missing Context as a separate item in the Message and as I've said many times before, it is an important concept for SOA. Anyway, let's table the discussions for post 4.2.x when we all have more time to think and put our feet up ;-)

                  • 6. Re: Alternative Message Discussion
                    Tom Fennelly Master

                     

                    "mark.little@jboss.com" wrote:
                    Thanks Tom. You're missing Context as a separate item in the Message and as I've said many times before, <a href="http://www.idealliance.org/proceedings/xml05/ship/54/xml2005-wssessions.HTML">it is an important concept for SOA</a>.


                    Sure, I call this out as an "Issue/Question" off the overview page. I didn't have it in there simply because the one in our message was "just there" i.e. it was doing nothing. I can see what it's for long term for sure. Kev also made an interesting point re how the context could be used to carry around contextual info (Vs multiple named body payloads). I like that idea :-) Anyway, we can talk about it later :-)