1 2 Previous Next 22 Replies Latest reply on Jul 29, 2005 2:21 PM by ovidiu.feodorov

    Proposed Foundation for JBI

    pfricke

      A current proposal is JBoss Messaging (the replacement for JBossMQ) as the foundation extended to support the JBI Service Provider Interfaces (SPI) and bindings. Services built to SPI include BPM, Transformation, Rules, B2B. Bindings include SOAP, JMS, ebXML and AS2.

      JBoss Messaging 1.0 is targeted for Dec 2005. I'm working on the product plan with Ovidiu now.

      JBI also will involved jBPM, JBossWS, and external transformation and rules (Drools is a possibility).

      SOAP and JMS bindings are highest priority, followed by AS2. ebXML can be a community deal (so can AS2, possibly).

      Our public statements to date have been -

      We like the JBI architecture. It is similar in philosophy to our JBoss microkernel architecture. It has potential to enable a mass-market integration foundation. Integration has been inaccessible to a large base of users due to price, complexity and lack of standards. We are developing our Integration and JBI strategy with the goal of having a JBI foundation available in 2006 that is consumable by a large audience of users and developers.




        • 1. Re: Proposed Foundation for JBI

          Reading the above message makes it very obvious that it is necessary to keep the specification (JBI) separate from the product in order for the discussion to stay somehow sensible. Reading statements such as "JBI also will involved jBPM, JBossWS, and external transformation and rules (Drools is a possibility)" is just plain confusing.

          What is needed is a product name. JBI ain't it.

          • 2. Re: Proposed Foundation for JBI
            admin

            JBI is a specification of a container. It allows various applications to be plugged into the "bus", either via a Service Engine, or a Binding Component. The jBPM Pierre talks about will provide orchestration services and will be plugged into the bus via its own service engine. JBossWS provides support for WSDL, SOAP binding components and what not. Etcaetera.

            As for product name, since JBus is taken, what about JBoos :)

            • 3. Re: Proposed Foundation for JBI
              mbbickel

               

              "juha@jboss.org" wrote:
              What is needed is a product name. JBI ain't it.


              How about JBoss ESB?


              • 4. Re: Proposed Foundation for JBI
                ovidiu.feodorov

                The "JBI is a specification of a ..." post was mine. This site sends some tenacious cookies ...

                • 5. Re: Proposed Foundation for JBI
                  ovidiu.feodorov

                  JBoss Integration Platform?

                  • 6. Re: Proposed Foundation for JBI
                    pfricke

                     

                    "juha@jboss.org" wrote:
                    Reading statements such as "JBI also will involved jBPM, JBossWS, and external transformation and rules (Drools is a possibility)" is just plain confusing.

                    What is needed is a product name. JBI ain't it.


                    JBI isn't a product name, nor was it implied as one. We don't have a product name yet. So it was used as a place holder. Suggestions are welcome. "JBoss ESB" for the basic JBI implementation. "JBoss Integration" for a larger platform including service extensions and more bindings. These are a couple of possibilities.

                    • 7. Re: Proposed Foundation for JBI

                       

                      "admin" wrote:
                      JBI is a specification of a container. It allows various applications to be plugged into the "bus", either via a Service Engine, or a Binding Component. The jBPM Pierre talks about will provide orchestration services and will be plugged into the bus via its own service engine. JBossWS provides support for WSDL, SOAP binding components and what not. Etcaetera.


                      Right. And components such as rules engines don't have anything to do with the JBI specification per se, anymore than a JBoss Microkernel has to do with J2EE.

                      Therefore, there needs to be a separation of the spec (JBI) and the product (No Name Yet) to keep the discussion comprehensible.

                      Otherwise no one will understand what JBI is.



                      • 8. Re: Proposed Foundation for JBI
                        tom.baeyens

                        as the foundations for JBI, we should create a new POJO ESB foundation. just like the microcontainer is the pojo foundation for the new server.

                        binding components (of course), service engines (of course) *and* transport mechanisms should be pluggable.

                        i would avoid to base our implementation directly on JMS or any other transport implementation. that would make the coupling to the transport too hard.

                        regards, tom.

                        • 9. Re: Proposed Foundation for JBI
                          tom.baeyens

                          i like JBoss ESB.

                          just name it like it is:
                          JBoss Portal
                          JBoss Messaging
                          JBoss App Server
                          JBoss Cache
                          JBoss AOP

                          in that spirit we should propose following name changes :-)
                          JBoss ORM
                          JBoss BPM

                          regards, tom.

                          • 10. Re: Proposed Foundation for JBI

                             

                            "tom.baeyens@jboss.com" wrote:
                            as the foundations for JBI, we should create a new POJO ESB foundation. just like the microcontainer is the pojo foundation for the new server.

                            ...

                            i would avoid to base our implementation directly on JMS or any other transport implementation. that would make the coupling to the transport too hard.


                            That's a very good point, Tom.

                            I think it is too easy to forget that the NMS in JBI is not distributed. I recall the sketch we did in Paris about 6 months ago where at the architecture level the distribution occurs via binding components (when the NMS bus spans multiple processes). I haven't read the final specification, but I noticed the explicit point in "Future Directions" section:


                            Distributed JBI. JBI, as currently defined, is not distributed. Distributed implementations may require support in the JBI specification to address issues that arise only in distributed systems.


                            That supports your view that the foundation for JBI implementation (if we agree the foundation revolves around NMS) does not require a full blown JBoss Messaging implementation but an implementation similar to the MicroContainer although with fundamentally different architecture -- event (message) based instead of thread based (due to the reliability requirements in the specification).

                            JBoss Messaging would be the foundation for an isolated binding component to the JBI foundation (NMS) to provide the distributed service implementation. Do you agree with that?

                            So it all comes back to how to design OO (POJO) systems and how to design distributed systems and understanding that these are completely separate design considerations. The foundation should be designed around POJO's (as Tom suggests) and distribution should be designed into BC's and not into NMS directly (and that's where the similarity to current Microkernel and the new MicroContainer comes from, separation of bus and invokers). Unfortunately the reliability requirements are not supported by the current kernel (and I doubt the new MC).

                            So the question is do we have a POJO based non-distributed core (fast in-VM event system without group communication protocol) in JBoss Messaging that can be reused for JBoss ESB?

                            • 11. Re: Proposed Foundation for JBI
                              ovidiu.feodorov

                              JSR 208 doesn't specify anything about a distributed NMR. The example you mention (two JBI containers connected via two crossing Binding Components) is actually included in the final release of the specification.

                              JBoss Messaging as we know it is actually a JMS facade sitting in top of the Messaging Core, which is a distributed, reliable (transactional) messaging transport system. It doesn't have an JMS API, it has its own API, which is probably is looked at as a bad thing, until one realizes that it is very easy to add ANY semantically similar API in top of it, as another facade. The Messaging Core is built based on the concept of Channel, which can span address spaces, LANs and WANs.

                              Part of the MC requirements is reliability. A Channel is (or it can be configured to be) transactional, as long as it has access to reliable storage (more details here: http://wiki.jboss.org/wiki/Wiki.jsp?page=JBossMessagingCore
                              and http://wiki.jboss.org/wiki/Wiki.jsp?page=JBossMessagingCoreRevision1)

                              An ESB definitely needs JMS bindings, and JBoss Messaging can be used for that, however, when it comes to distributed asynchronous backbone, the Messaging Core could play a central role.

                              • 12. Re: Proposed Foundation for JBI

                                A thought on the reliability requirements within a local JBI Message Exchange Protocol:

                                What if the reliability requirements are left as the responsibility of BC and SE containers to implement?

                                Think of Resource Managers in XA.

                                I'm playing with a couple of assumptions here:

                                1) Failure scenario occurs at
                                - JVM level
                                - OS level
                                - Hardware level

                                2) Communication in JBI based system occurs between
                                - BC to BC
                                - BC to SE
                                - SE to BC
                                - SE to SE

                                From these we can get:
                                - NMS does not require reliability
                                - NMS can be synchronous

                                That says Microcontainer could be used as NMS. Reliability is a requirement for messages ENTERING the system and messages LEAVING the system (local JBI node):

                                Messages ENTER at BC (incoming message).
                                Messages ENTER at SE (new invocation to BC or SE, or MEP response msg)
                                Messages LEAVE at BC (distributed NMS)
                                Messages LEAVE at SE (processed and consumed msg)

                                BC is made reliable by storing the incoming Invocation payload. ENTER log.

                                BC pushes the message to NMS. No logging required. Given the failure scenario assumptions above, NMS does not fail in isolation from the local BC or SE (JVM, O/S, or Hardware failure).

                                Crash between BC and NMS can be recovered due to ENTER log and detected by lack of invalidation or removal of ENTER log entry.

                                SE or BC receives message from NMS. No logging required. Same assumptions as previously. We can recover from the receiving BC's log (or SE's log if SE is creating invocation/response).

                                SE or BC consumes message. SE processed the message and acknowledges work done (one-way) or creates response message. Invalidate existing ENTER log. Create ENTER log of response msg. BC consumes message by receiving ACK from distributed BC (Messaging Core) that the receiver has written ENTER log on on its own process space.

                                Crash between NMS to BC or SE can again be recovered from the initial BC's or SE's log.

                                That gives us 2 writes per message (enter + invalidate). Two for a simple one-way MEP (single msg), more writes on other MEPs depending on the exchange protocol.

                                It would be possible to reduce it to 2 writes per MEP (rather than msg) if the SE is configured to the equivalent of DUPS_OK in JMS in terms of its MEP.

                                Asynchronicity would be the property of Message Exchange Protocols, not NMS.

                                Existing Microcontainer implementation could be reused. The equivalent of invoker would be a BC container that provides the message logging. The equivalent of MBean or EJB container would be a SE container that provides the message logging.

                                Delivery of single message from BC to BC, BC to SE, SE to BC, and SE to SE would be synchronous in-VM. The fact the MEP defines multiple messages in its exchange protocol makes it async at higher API level.

                                JBoss ESB would provide BC and SE "containers" that handle these requirements and wrap 3rd party BC and SE implementations.

                                Reliability features would be properties of SE and BC containers, instead of NMS or messages. Reliable vs not reliable depends on my SE or BC configuration, not message properties (I create 'a' message at development time, without regard to QoS, QoS is defined at deployment time). Maybe the BC I use creates a log, maybe I route to BC that does not (again, deployment/runtime decision), maybe my SE allows duplicate messages, in which case I deploy with specific JBI container configuration which may reduce the number of log writes, etc.

                                • 13. Re: Proposed Foundation for JBI
                                  ovidiu.feodorov

                                   

                                  That says Microcontainer could be used as NMS. Reliability is a requirement for messages ENTERING the system and messages LEAVING the system (local JBI node):

                                  Messages ENTER at BC (incoming message).
                                  Messages ENTER at SE (new invocation to BC or SE, or MEP response msg)
                                  Messages LEAVE at BC (distributed NMS)
                                  Messages LEAVE at SE (processed and consumed msg)



                                  Why would be reliability a requirement for messages leaving the system? According to the distribution model described by the specification, where two JBI nodes are connected via BCs, once the consumer BC invokes the provider BC, the provider BC logs the message so in case of failure, the message it is recoverable. Following your logic, it seems that only entering messages need to be persisted.


                                  BC is made reliable by storing the incoming Invocation payload. ENTER log.


                                  Each BC/SE needs access to reliable storage in order to be able to provide reliability.

                                  Existing Microcontainer implementation could be reused. The equivalent of invoker would be a BC container that provides the message logging. The equivalent of MBean or EJB container would be a SE container that provides the message logging.


                                  How would this scale in case of a distributed ESB? The current clustering model relies on an external service (not part of the microcontainer) that offers a distributed RPC facility other services can use independently. For a JBI 1.0-based ESB, if we are to follow the specification recommendations, the distribution would be achieved by interconnecting BCs into a graph of JBI nodes. I didn't analyze in detail the implications of this approach, but I can think an alternative architecture where NMR is distributed and reliable. What JBI 1.0 specifies as NMR could actually be an NMR peer, capable of asynchronously send messages locally or remotely.



                                  Reliability features would be properties of SE and BC containers, instead of NMS or messages. Reliable vs not reliable depends on my SE or BC configuration, not message properties (I create 'a' message at development time, without regard to QoS, QoS is defined at deployment time).


                                  An equally simple mechanism could be imagined where a message sent by a BC/SE could be made reliable or unreliable at runtime based on a deployment configuration of the said BC/SE.



                                  • 14. Re: Proposed Foundation for JBI

                                     


                                    Why would be reliability a requirement for messages leaving the system?


                                    No, the point is that given the failure assumptions above, I don't care what happens INTERNALLY within a single process space, between local NMS, BC's and SE's.

                                    In order to be able to recover the invocation I only have to log the message entering/created in the node and leaving/consumed in the node.

                                    I am questioning the benefit of building an asynchronous communication foundation (NMS) within a single JVM when the protocols provide asynchronizity.

                                    In other words, if I have in-JVM communication between two local SE's using a MEP with in and out messages, what is the benefit of transferring the two individual messages asynchronously when they are targeting local service instances? From the service implementors perspective (developer writing to the API) the communication is asynchronous because of the two separate messages.

                                    EJB's already created a ton of stupidity by defining remote interface that forced pass-by-copy semantics. Turned out most communication is NOT distributed, but local within the same VM (because it's faster that way). Then we get more stupidity with local interfaces...

                                    So you can look at JBI (and not talking about ESB here, in my view ESBs ar e distributed) as a service container that allows you to integrate engines not part of J2EE into your platform in a standard fashion. These include workflow engines, rules engines, etc, etc. Now if my service X deployed on JBoss wants to talk to workflow engine (let's say jBPM with a SE wrapper) then chances are I am running within the same process space.

                                    In that case I just want the comm to be fast, distribution never enters the picture.

                                    On the other hand if you look at JBI mainly as an ESB where you are integrating existing systems running in their separate process spaces and just need to have a standard way for these processes to communicate then whatever happens internally is not going to matter since everything is going to be slow as molasses, relatively, due to the distribution.

                                    So I've heard JBI touted as a service container, and I've heard it touted as an ESB. The problem is that these two goals come with very different profiles in terms of distribution, and distribution tends to be a significant factor to take into account in the architecture and implementation.

                                    Would be nice to hear from Tom as an EG member if he has a definite view to what JBI is supposed to be? ;-)



                                    According to the distribution model described by the specification, where two JBI nodes are connected via BCs, once the consumer BC invokes the provider BC, the provider BC logs the message so in case of failure, the message it is recoverable. Following your logic, it seems that only entering messages need to be persisted.


                                    Well, a message is only persisted once for a node, yes. Then there's the second write to log the fact that some other node acknowledges persisting the message as well, so we can get rid of our own copy.


                                    Each BC/SE needs access to reliable storage in order to be able to provide reliability.


                                    Yes.


                                    How would this scale in case of a distributed ESB? The current clustering model relies on an external service (not part of the microcontainer) that offers a distributed RPC facility other services can use independently.


                                    No change there. It is the BC that provides distribution. Microcontainer provides the NMS (or NMR as they now seem to call it). In other words, the invoker <--> bus separation.

                                    So two things come up with this:

                                    1) We take JBI as an SPI for services that had no "home" in J2EE (e.g. workflow engine with no standardized API vs. transaction manager with a defined JTA). That means there will be significant local traffic, and we can look at NMR as a local entity in physical view (single node) but distributed in logical view (multiple nodes) via BCs.

                                    2) Take JBI as an ESB to integrate processes across IT enterprise, no local "container" view for engines, might as well distribute the whole thing at NMR .



                                    For a JBI 1.0-based ESB, if we are to follow the specification recommendations, the distribution would be achieved by interconnecting BCs into a graph of JBI nodes.


                                    Yes.


                                    I didn't analyze in detail the implications of this approach, but I can think an alternative architecture where NMR is distributed and reliable.


                                    To me that was the first obvious architecture. And I am guessing that might also be the case for most JMS (now ESB) vendors as well.

                                    Again, I think it depends how you mainly see JBI being used for: integrating service engines locally to your middleware stack, or building vast, distributed integration networks of applications.

                                    PS. the JBI spec says the producer and consumer are never sharing the same thread ;-)



                                    1 2 Previous Next