1 2 3 Previous Next 79 Replies Latest reply on Sep 14, 2009 4:33 AM by tfennelly

    Http Gateway - requirements please...

    tfennelly

      The HttpGateway added in v4.5 was based on an embedded tomcat, which has proven to be troublesome due to the sheer complexity of embedding tomcat. Because of this, we have decided to go to a model similar to that of EBWS, where we generate an actual war and deploy it as a sub deployment of the containing esb deployment.

      From a usability point of view, what would people like to see in a new HttpGateway? What I mean is ... what would they like to be able to configure, and how? What do we want to inherit from the existing HttpGateway config?

      From an implementation point of view, what would people like to see? Exactly the same as EBWS again, or should we try and create a new mechanism that abstracts this type of thing (creating sub deployments) away from the deployer and then use this as an opportunity to start evolving the EBWS specific code out of the deployer (where I would think it doesn't belong)? Do we want to merge deployments of the same type as in this case, where the EBWS and the new HttpGateway both require war deployments? What would that gain and would the added complexity be worth it?

      Apart from potential implementation similarities, what else should the new HttpGateway have in common with the EBWS functionality?

        • 1. Re: Http Gateway - requirements please...
          tfennelly

          So just looking at some configuration aspects (not implementation).... with the current http gateway (based on the embedded tomcat), you specify the listen address and port as part of defining a provider. Then on the bus definition, you specify the context e.g.

          <http-provider name="myhttp" host="localhost" port="9999">
           <http-bus busid="http1" context="/orders" />
          </http-provider>


          With the non embedded tomcat approach that we are proposing, a http bus will use the AS/ESB Server's http services and so there will be no configuring of the host and port. As already stated, the ESB deployment process will create a war sub deployment for the gateway. The gateway will then be deployed as with the normal war deployment process. So the provider/bus definition could be something as simple as:

          <http-provider name="http">
           <http-bus busid="orders" context="/orders" />
          </http-provider>


          Or just infer the context from the busid ala:

          <http-provider name="http">
           <http-bus busid="orders" />
           <http-bus busid="queries" />
          </http-provider>


          The listener config at the moment looks like this:

          <http-listener name="httplistener" busidref="orders" is-gateway="true" />


          That works fine I think. One downside is that it seems to mean that we're restricted to a 1:1 bus:listener relationship for http. What if we added a "urlPattern" type config (mapping to the same in the web.xml) option on the listener (or provider perhaps?). That would allow a single bus to be used across multiple listeners e.g.

          <!-- "Dogs" order Service.... -->
          <http-listener name="httplistener" busidref="orders" urlPattern="/dogs/*" is-gateway="true" />
          
          <!-- "Cats" order Service.... -->
          <http-listener name="httplistener" busidref="orders" urlPattern="/cats/*" is-gateway="true" />



          • 2. Re: Http Gateway - requirements please...
            tfennelly

            So that was some basic configuration. How about adding security etc?

            I think whatever we do here will translate into adding appropriate configurations to the web.xml of the generated .war. Seems like there are 2 approaches to this:

            1. Add a strict config model inside the ESB config file whereby we only allow certain configurations and we manually map those appropriately to the web.xml going into the generated .war sub deployment.

            2. We allow the user to specify a suplemental web.xml that gets merged into the generated web.xml. The user would define the "extra" configs as per the web.xml schema. We could allow them specify this inline in the esb config, or externally. We might want to restrict the config types we allow in this e.g. you can't define any servlet/filter configs etc??

            Same would apply to jboss-web.xml.

            So what are the pros/cons to these approaches? Other approaches?

            The existing http gateway also supports configuration of allowed http methods. Do we need to bring this along?

            • 3. Re: Http Gateway - requirements please...
              tfennelly

              Implementation....

              Do we want to replicate what was done for EBWS? After all, it's basically the same type of thing! Do we want to consider merging the EBWS and Http Gateway solutions?

              The EBWS implementation has EBWS specific code inside the JBoss ESB deployer. I think that's not so great and it hurts me to replicate something like that. Personally I would prefer to see us come up with a better long term solution that doesn't involve implementation specific code inside the deployer class. Kev's ultimate goal is to have all of http related features (EBWS, http listener, webservice proxy etc) build off the same basic solution and have all related http artifacts for a single deployment bundled in a single war sub deployment. Hope my wording makes sense - not Kev's exact words, but I think that's basically what he's after :)

              I think this is possible, but not without a new approach. I think trying to do this with the current EBWS impl would just lead to more hacks, which I don't think we're after, right?

              So how would we structure the code such that we can keep the deployer clean? I had thinkered with the idea of having a "Bus Deployment Factory" that the main deployer picks up on for creating sub-deployments at deploytime. We can go into that more later if we want to, but what do others think? Do we move the EBWS implementation to use a possible new/refactored approach for 4.6, or do we leave it for now and look to evolve it later, using the new/refactored approach only for the http listener and webservice proxy for now?

              • 4. Re: Http Gateway - requirements please...
                scottdawson

                Tom,
                I've been using the jbr-provider/listener for HTTP, not the http-provider, so I'm not sure if this functionality is provided by the http-provider, but a couple of settings I've used with the jbr-listener are the "synchronous" and "asyncResponse" properties, in order to configure a OneWay MEP. And, in my opinion, when "synchronous" is set to false, the HTTP response code should be set to 202 (http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.2.3) or else be configurable.

                Any plans for that type of functionality?
                Thanks,
                Scott

                • 5. Re: Http Gateway - requirements please...
                  tfennelly

                  Right Scott... that was something I forgot to bring up, so thanks!! How do we handle the whole sync/async response thing?

                  So I had thought that sync/async could be indicated by the mep and not a seperate config (such as the "synchronous" config on the JBR gateway). So when the Service MEP is "RequestResponse", we send back a 200 plus the output message of the pipeline as the response and when it's "OneWay", we just send back a 202 like you suggest. Apparently we can't do this and I don't really get why. Can you explain your logic here Kev please?

                  • 6. Re: Http Gateway - requirements please...
                    beve

                     

                    2. We allow the user to specify a suplemental web.xml that gets merged into the generated web.xml. The user would define the "extra" configs as per the web.xml schema. We could allow them specify this inline in the esb config, or externally. We might want to restrict the config types we allow in this e.g. you can't define any servlet/filter configs etc?

                    This sounds like a flexible solution to me, merging this with the generated web.xml.

                    I might be missing something obvious here but why can't we let users specify servlet filters?
                    Would this not give users some flexibility regarding transforming the response (compression perhaps) or adding logging.

                    /Daniel

                    • 7. Re: Http Gateway - requirements please...
                      kconner

                      There are quite a lot of comments on this thread but we seem to be straying away from what we were originally after in the work for the FP release. Let me go through these postings first, then I will follow up with a summary of what we were originally after.

                      Kev

                      • 8. Re: Http Gateway - requirements please...
                        kconner

                         

                        "tfennelly" wrote:
                        Exactly the same as EBWS again, or should we try and create a new mechanism that abstracts this type of thing (creating sub deployments) away from the deployer and then use this as an opportunity to start evolving the EBWS specific code out of the deployer (where I would think it doesn't belong)?


                        EBWS and the http gateway are essentially the same animal and should be shared. There are very few differences between the two, especially once we extend EBWS to support what will be available to HTTP.

                        We should definitely take this code out of the deployer but going to the extent of a fully extensible framework is not required. We only need to support HTTP and EBWS.

                        "tfennelly" wrote:
                        Do we want to merge deployments of the same type as in this case, where the EBWS and the new HttpGateway both require war deployments? What would that gain and would the added complexity be worth it?

                        Yes, these should be merged into the same war file so that we have a single deployment. Even with the current EBWS code we already have an issue where we conflict with other deployments, Expanding the number of these deployments, especially when not necessary, seems strange to me.

                        BTW what added complexity do you get from adding more servlets into a deployment? Can you expand on where you think the issue lies?

                        "tfennelly" wrote:
                        Apart from potential implementation similarities, what else should the new HttpGateway have in common with the EBWS functionality?

                        I'll try and cover this later when I post about the original plans.

                        Kev

                        • 9. Re: Http Gateway - requirements please...
                          kconner

                           

                          "tfennelly" wrote:
                          <http-provider name="myhttp" host="localhost" port="9999">
                           <http-bus busid="http1" context="/orders" />
                          </http-provider>

                          The context would not be part of this as the requirement is to keep this to a single deployment. It would share the same context as the EBWS, i.e. based on the esb deployment name by default.

                          "tfennelly" wrote:
                          That works fine I think. One downside is that it seems to mean that we're restricted to a 1:1 bus:listener relationship for http. What if we added a "urlPattern" type config (mapping to the same in the web.xml) option on the listener (or provider perhaps?). That would allow a single bus to be used across multiple listeners e.g.

                          URL patterns should definitely be supported.

                          Kev

                          • 10. Re: Http Gateway - requirements please...
                            kconner

                             

                            "tfennelly" wrote:
                            So that was some basic configuration. How about adding security etc?

                            Yes, security has to be included. It needs to cover authentication, transport guarantees, security constraints and integrate into the security work done by Daniel.

                            "tfennelly" wrote:
                            1. Add a strict config model inside the ESB config file whereby we only allow certain configurations and we manually map those appropriately to the web.xml going into the generated .war sub deployment.

                            This allows us to remain in control of what we support and is the preferred method.

                            "tfennelly" wrote:
                            2. We allow the user to specify a suplemental web.xml that gets merged into the generated web.xml. The user would define the "extra" configs as per the web.xml schema. We could allow them specify this inline in the esb config, or externally. We might want to restrict the config types we allow in this e.g. you can't define any servlet/filter configs etc??

                            This is likely to lead to problems unless the restrictions are so strict as to end up with the equivalent of the first suggestion. Adding explicit support, especially under our control, is much easier than disabling parts.

                            "tfennelly" wrote:
                            The existing http gateway also supports configuration of allowed http methods. Do we need to bring this along?


                            Yes, we need to retain support for the following verbs, DELETE, HEAD, GET, POST and PUT. We also need to provide automatic support for OPTIONS so that someone can determine what capabilities are present.

                            Only TRACE, as currently implemented, is not required.

                            Kev


                            • 11. Re: Http Gateway - requirements please...
                              kconner

                               

                              "tfennelly" wrote:
                              Apparently we can't do this and I don't really get why. Can you explain your logic here Kev please?

                              I know we have discussed this on a number of occasions, Monday being the last. All I can do is repeat what was said before.

                              (12:18:47) Tom Fennelly: another quick one... the mep issue
                              (12:18:55) Tom Fennelly: what was the issue with the?
                              (12:19:26) Tom Fennelly: you don't wanna use it to decide whether or not a respnse is sent back?
                              (12:19:33) Kevin Conner: The mep of the service has no bearing on the http gateway, it is an implementation config
                              (12:19:36) Kevin Conner: You can't
                              (12:19:57) Kevin Conner: A response can be sent back from a OneWay mep if it chains the request
                              (12:20:13) Kevin Conner: Service A (OneWay) -> Service B (RequestResponse)
                              (12:20:37) Kevin Conner: It is an implementation detail and not part of the service contract




                              • 12. Re: Http Gateway - requirements please...
                                kconner

                                Sorry, meant to add the following to my last.

                                As the example above shows, the mep on the service is an implementation detail specific to that service and is used by the pipeline to determine how it should handle requests.

                                The original rosetta code assumed that any message getting to the end of the pipeline was a response and, therefore, sent it whether it was required or not. This lead to duplicate actions (returning messages or nulls), duplicate replies being sent etc.

                                The mep was introduced to control this, providing explicit instruction to the pipeline stating how it should handle any responses.

                                This is nothing to do with a client contract, which we don't really have, and cannot be used to determine client behaviour.

                                Look through the async continuation stuff for more examples of how easy it is to use OneWay services to implement request/response behaviour.

                                Kev

                                • 13. Re: Http Gateway - requirements please...
                                  kconner

                                   

                                  "beve" wrote:
                                  I might be missing something obvious here but why can't we let users specify servlet filters?
                                  Would this not give users some flexibility regarding transforming the response (compression perhaps) or adding logging.

                                  The only downside is that it introduces another model, servlets in addition to esb, but this may be something that they already have in house.

                                  It is certainly something to bear in mind.

                                  Kev

                                  • 14. Re: Http Gateway - requirements please...
                                    kconner

                                    I have just had a chat with Tom and we feel that this topic needs further discussion before we proceed with integrating the gateway. We would both like to get this right rather than rush something into the release.

                                    As a result I am going to move this out of the 4.6 release, at least for now, but we cannot afford to let this go on for too long. Lets give this a week to progress then document what is expected.

                                    Kev

                                    1 2 3 Previous Next