3 Replies Latest reply on Jan 7, 2009 11:21 AM by dmlloyd

    Remoting 3: Decision time...

    dmlloyd

      OK, so I just released Beta2. We need to decide what else, if anything, we need to add before releasing 3.0.0.GA. Right now we've got:

      * Basic protocol (intended to be a model for implementations, not for primary usage)
      * Multiplex protocol (this will be the basis of JRPP which is multiplex + SSH + some option negotiation protocol)
      * A complete discovery API, but no (production-grade) mechanisms yet
      * Transporters, at least on a basic level (I can envision many more features to add, but this isn't a priority for us)
      * A complete streaming API (via JBoss Marshalling), but no (production-grade) mechanisms yet

      Things that are currently scheduled for 3.1 or later and for the supporting projects:

      * SSH + negotiation protocol to round out the multiplex transport
      * Discovery mechanism based on SLP (RFC 2608) implementation (this is has an SLP component and a Remoting component)
      * Streaming mechanisms for Multiplex+SSH
      * Maybe some generic streaming mechanisms for streaming directly over TCP?
      * Protocol compatibility layer for older Remoting wire protocols
      * Discovery mechanism based on JGroups (?)
      * An improved standalone API to make simple clients and servers be very easy to write
      * Remoting MC integration
      * Strategy for creating EJB2, EJB3, Remote JMX, Remote JDBC, Remote JNDI etc. services, including addressing compatibility

      Inclusion into JBossAS hinges on most (if not all) of the future scheduled tasks being complete as well.

      What do you guys think? Is 3.0 production ready? Is there a target audience for it as it is right now? I'm inclined to put it out there more or less as it is now, personally, and get a move on the other features I've listed for 3.1, locking in to a 12- or 16-week release cycle.

        • 1. Re: Remoting 3: Decision time...
          trustin

          The following is the log of the chat David and I had on Monday.

          trustin: Yes, but I didn't have a chance to reply yet.
          trustin: I think the road map you mentioned there is pretty much same to what we've discussed so far.
          trustin: The question would probably be .. 'is it ready to go CR?'
          dmlloyd: probably not without more tests
          dmlloyd: featurewise, yes
          trustin: I think we can go CR first and write tests to cover the code and stablize it.
          trustin: as long as it is stable featurewise
          dmlloyd: it is, unless someone can think of anything I missed
          trustin: I never think you missed something. :)
          trustin: I think there are potential audiences for 3.0 anyway
          trustin: It can be configured without JBossMC and we could provide a simple bootstrapper so that one can run it standalone quickly.
          trustin: then we can attract users who want to use a flexible remoting layer for their application first and then try JBossMC integration if nice tutorial is provided.
          dmlloyd: I have examples of how to configure it manually
          dmlloyd: not many examples, but I do have examples :)
          trustin: Yes. What I meant was a very simple 1-3 liner helper to attract people ;)
          trustin: such a helper could cover most simple 'default' setup and some additional parameters that about 80% (?) users will need.
          dmlloyd: the tricky part is that there is no standard protocol yet
          dmlloyd: the multiplex handler is raw - no security
          dmlloyd: things like classloading have been put off
          trustin: Could we make the helper extensible so that a user can choose a transport and specify parameters dynamically (with reasonable default values)?
          dmlloyd: sure, but I'm just saying that there really aren't any reasonable default values yet
          trustin: the basic protocol? and we could change the default in 3.1 as soon as the SSH-based one is ready?
          dmlloyd: I don't want people to use the basic protocol in production though. It mainly exists so that there's a model for other protocol implementations to follow
          dmlloyd: basic protocol uses a connection per client rather than maintaining a connection independently of services and clients
          trustin: hmm then who would be a target audience?
          dmlloyd: people who want to use the API to begin defining services, I guess
          dmlloyd: and clients
          dmlloyd: while deferring connection implementation details
          dmlloyd: the framework is designed to be used in an IoC environment and it shows, I think
          trustin: but you don't want them to use the basic protocol in production - I'm confused.
          dmlloyd: right, in production people should be using the multiplex protocol with some kind of security layer
          dmlloyd: the thing is, all the features that would make R3 useful have been deferred to 3.1 or later
          trustin: then the target audience for 3.0 would be limited to someone who is in the course of developing a service and will be in production after we release 3.1?
          dmlloyd: yes, that was my conclusion
          trustin: We have built R3 on top of flexible marshalling layer and XNIO-based scalable networking layer, so I think it will attract some people who wants to use it in a private network only? We could add SSL support for the basic protocol (via XNIO) in 3.0 though.
          trustin: I might be missing something but from my understanding, current R3 has more features than plain RMI or other related technologies except for R2.
          dmlloyd: well, it has more potential features anyway :)
          dmlloyd: it's the most pluggable framework out there
          dmlloyd: XNIO doesn't have SSL support currently
          trustin: dmlloyd: Let's steal some from Netty
          trustin: Shall I implement one? ;)
          trustin: I actually think SSL is not even necessary to attract users with 3.0.
          dmlloyd: I suppose what we need is a user guide
          trustin: we could focus on SSH instead of SSL to save some time (not much)
          dmlloyd: yeah, SSH will be a big priority
          trustin: Yes. We need it otherwise going GA wouldn't help us attract people IMHO.
          dmlloyd: maybe we could just live with not attracting many people with 3.0 :)
          trustin: :)
          dmlloyd: as far as SSL goes - I did not yet implement it because I wanted to be sure that the API is correct
          dmlloyd: and there's a lot to know about how SSL can be used before being able to say that definitively
          trustin: How SSL can be used?
          dmlloyd: yes
          trustin: to assure confidentiality of communication?
          dmlloyd: special cases, unusual usage patterns etc.
          trustin: from XNIO standpoint?
          dmlloyd: yes
          trustin: Ah I see.
          dmlloyd: I want the API to be as simple as possible
          trustin: We are talking about 3~4 projects when we talk about R3 :D
          dmlloyd: yes :)
          dmlloyd: if SSL support appeared in XNIO, R3 would be able to use it immediately
          trustin: Yes, with no sweat :)
          dmlloyd: but for streaming over a single channel I think we will need full SSH support, since it gives the ability to open independent channels
          dmlloyd: though I could enhance the multiplex protocol to support independent channels as well
          trustin: Agreed.
          trustin: I guess the conversation so far seems to answer your questions in the forum pretty enough. Did we miss something?
          dmlloyd: how shall we promote it
          dmlloyd: I suppose that in the announcement I should make it clear what it can and cannot do
          trustin: You mean promoting to JBoss.org developers or users?
          dmlloyd: and explain that the target audience are those who want to start developing applications and services in anticipation of 3.1 and 3.2
          dmlloyd: they're one and the same from my perspective :)
          trustin: ah OK. :)
          trustin: I like the idea about the announcement which shows the road map.
          trustin: I think the road map should be shown in the front page of our project home page, too.

          • 2. Re: Remoting 3: Decision time...
            trustin

            The following is the consequent chat regarding this topic:

            ron_sigal Do you think it's in a usable state?
            dmlloyd usable, sure
            dmlloyd I mean, you could use it
            dmlloyd er :)
            dmlloyd there's no security features to speak of
            dmlloyd no authentication, no authorization, no encryption
            dmlloyd also you have to wire all the pieces together by hand (like the samples demonstrate)
            ron_sigal Since it's not AS ready, this release would be for the general publc ...
            dmlloyd yes
            trustin Not so pretty but here's the log - http://www.jboss.com/index.html?module=bb&op=viewtopic&p=4198393#4198393
            ron_sigal Suppose you had an app based on R2. How hard would it be to upgrade to R3?
            ron_sigal If it's easy, it might attract more users ...
            trustin Perhaps rewrite where it depends on R2?
            dmlloyd you'd have to basically replace R2 with R3 - I think it would be a net subtraction of code
            dmlloyd but there would be code changes, probably substantial
            dmlloyd also it depends on *how* the app is using it
            ron_sigal So that might narrow the audience to new users ...
            dmlloyd wiring the pieces together to get a connection and services and so forth is actually quite
             a bit of boilerplate code
            dmlloyd but actually *using* it is very simple
            dmlloyd the intent is that a container would generally manage the wiring
            trustin new users and early adopters
            trustin or.. upgraders :)
            ron_sigal But it's not mc ready yet?
            trustin I thought it is now?
            dmlloyd you could deploy it in the MC using <bean> tags
            dmlloyd but I intend to have a separate XML schema for remoting deployments
            trustin ah, I see.
            dmlloyd tags to create endpoints, tags to set up connections and servers etc
            ron_sigal Do you think the boilerplate is easy to reproduce? I guess my question is: how steep is
             the learning curve?
            dmlloyd https://svn.jboss.org/repos/jbossremoting/remoting3/trunk/samples/src/main/java/org/jboss/remoting/samples/simple/MultiplexClientExample.java <- here's a trivial multiplex client
            dmlloyd I dunno, it's kind of a pain in my opinion
            dmlloyd but that's *correct*
            dmlloyd I expect you could do a quick-and-dirty version with a lot less messing around
            trustin the code is quite simple imo
            trustin many indented blocks make it look complicated at the first glance though.
            dmlloyd yeah
            dmlloyd everyone has their own style with dealing with resources - I like to use final variables
             + try/finally
            trustin IIRC, we didn't find a cool way to simplify this.
            trustin What about adding IoUtils.safeClose(Closeable... closeables) ?
            dmlloyd for 3.1 I'd like to have a negotiation protocol that can be used to configure things like
             marshalling
            ron_sigal Yeah, my first reaction was: that's scary. But upon reading it, it's not so bad, I think.
            trustin nvm, my suggestion doesn't work
            dmlloyd anyway I'd like to make at least a client simpler, if possible
            dmlloyd here's the server:
            dmlloyd https://svn.jboss.org/repos/jbossremoting/remoting3/trunk/samples/src/main/java/org/jboss/remoting/samples/simple/MultiplexServerExample.java
            dmlloyd actually a bit simpler, oddly :)
            trustin looks quite similar to the client side one
            dmlloyd a lot of it is the same. Either way, you're setting up an endpoint and a connection
            dmlloyd the server actually acts as a server in two senses: it acts as a TCP server to accept the
             multiplex connection, and it has a service registered as well which can be accessed from
             a remote client
            dmlloyd I could have put the service on the client I suppose
            trustin without modifying the current R3 but modifying the client side example?
            dmlloyd right
            trustin I see. Cool.
            dmlloyd multiplex just connects two endpoints. You could register many services on each end of
             the connection
            ron_sigal I think a user's guide is important. For people like me. :)
            dmlloyd yeah, I was thinking that maybe I should at least start a user guide before the 3.0 final
             release...
            ron_sigal +1
            dmlloyd and go through and make sure the javadoc is complete
            trustin Assuming that 3.1 user guide will be based on 3.0 user guide
            dmlloyd yes, I think the user guide should live in its own trunk
            ron_sigal 3.0 could present general principles.
            trustin yeah that's a good idea.
            dmlloyd I don't want to tie user guide releases to code releases
            ron_sigal ??
            dmlloyd I'd like to make sure that in the user guide, if something is only available, say, after 3.2, I'd say "In 3.2 or greater, you can frob the gibble using the new frobnicator blah blah"
            dmlloyd that way, corrections can make their way back :)
            dmlloyd otherwise I'd never release for fear of having an imperfect user guide
            *dmlloyd knows dmlloyd
            ron_sigal That sounds like the guide *is* tied to the release. Missing something.
            trustin hmm
            dmlloyd the latest user guide should always be available
            ron_sigal I like the "In 3.2 or greater, ..."
            dmlloyd if we bundle the user guide in trunk with the code, then we'll end up having to manage
             the user guide in branches as well
            ron_sigal Ah.
            dmlloyd I'd rather just keep it independent, and update the one on the website every, say, 2-4
             weeks or something
            ron_sigal I misunderstood "tie". Cool.
            ron_sigal Yeah, I'm always updated a 2.2 and a 2.5 guide.
            ron_sigal updated -> updating
            trustin perhaps we could maintain the documentation in a different trunk/branch and use
             svn:externals?
            dmlloyd what would we use svn:external for?
            trustin at least we need to include the documentation in the distribution
            dmlloyd I see it as fully independent
            dmlloyd hm, I dunno
            dmlloyd we could download it as an artifact too
            trustin or just a URL?
            dmlloyd I don't know if it's really that useful including it right in the distribution
            dmlloyd might be more useful as a separate download
            trustin I see. yeah separate download makes more sense.
            dmlloyd well, getting 3.0 out leaves us with the 3.1 tasks to get going on
            dmlloyd management (JMX and JOPR), HTTP, SSH, compatibility layer
            dmlloyd those are the big items
            trustin Yes.
            dmlloyd a 16-week release schedule, and a 3.0.0.GA release in the week of Jan.12 would mean that
             we'd need to have CR1 by mid-april
            ron_sigal I started looking into an HTTP transport at one time ...
            trustin You mean 16 weeks to the first CR of 3.1?
            dmlloyd yes
            dmlloyd no
            dmlloyd 16 weeks from GA to GA
            trustin ah OK I miscalculated :)
            dmlloyd oh, the other important item is completing the AS integration, with a JCA RA
            ron_sigal When's Weston coming back? :)
            dmlloyd :)
            dmlloyd I don't even ever see him on IM anymore
            ron_sigal ah
            trustin I'm not sure we can finish all the stuff scheduled for 3.1 in 16 weeks. I just guess I
             need to try.
            trustin not saying negative nor positive. Just don't have a clue on this.
            ron_sigal We know dmlloyd is salivating over SSH. :)
            dmlloyd well, we'll tackle one thing at a time
            dmlloyd yeah, I've got SSH underway
            dmlloyd I figure I'll need to get XNIO-SSH 1.0.0.Beta or CR out by early Feb to get a working
             transport on top of it by april...
            trustin I guess we also need netty cr so that ron can work on http
            ron_sigal I tried to create an indirection layer so I could use different implementations, so I
             could get started without netty, probably.
            ron_sigal I had something that worked with HTTPUrlConnection and apache HttpClient, IIRC.
            trustin OK. Netty HTTP implementation is ready to go. I think you can get started with it now
             although it's alpha.
            ron_sigal ok, cool.
            ron_sigal I'll just start reading the R3 users guide. :)
            dmlloyd it'll probably take me longer to get docbook set up than it will take me to get the guide
             roughed out :)
            ron_sigal Maybe you can copy from R2?
            dmlloyd I'll probably just copy whatever ales did for the jbossmc user guide
            dmlloyd there's a new thing now
            dmlloyd it's all maven-based
            ron_sigal ah
            dmlloyd with the new look&feel and so forth
            trustin you might want to take a look at the customization I've done for NEtty user guide.
            trustin I figured out how to override the default settings and insert the version number
             dynamically etc etc.
            dmlloyd I see
            dmlloyd I think I will copy yours :)
            trustin :D
            ron_sigal trustin has the hacker gene. :)
            trustin lol
            dmlloyd I don't know if I need version numbers, but yours looks better than the jbossmc one


            • 3. Re: Remoting 3: Decision time...
              dmlloyd

              One thing I do want to do before the final release - I think that Transporters should be separate from the core. The reason is that I think it is more of a "beta" feature - there are many things that haven't been considered (like remote reference management, distributed GC, etc.) that should be considered before this feature is considered "stable". Also I'd want to find at least one or two users of the feature before releasing Transporters as stable.