JBossDeveloper

After reading that post, I feel even more strongly that we should have jmx remoting use invokers as its connector transport. There should be some kind of information for the client Invoker to use to find its server (lets call it a SLP URL). As party of the Invoker's Invocation payload, the client may include a SLP URL for the server to make callbacks (or notifications in the case of JMX). The server could then use an Invoker to make the callback calls in the same manner the client originally setup the connection to the server.

Another aspect to consider is some kind of built in callback polling mechanism built into the Invoker where the client Invoker would actually poll the Invoker server for any callbacks it needs to be notified of. This would be needed in the case where a client invoker called out of the network to a invoker server could not get a network connection back to the client (because of firewall).

> An Interceptor is just a pass through point in an
> invocation call stack.
>
> An Invoker is just a transport handler. Up to now JMX

You can view the invokers as just system wide interceptors that "transport" the invocation from VM to VM and across any transport mechanism. They are just specific interceptors in our case :)

> has been brought into the Invoker layer due to
> Invokers being MBeans for their configuration (a good
> thing), and as the mechanism by which the transport
> agnostic invocation was passed to the Invoker target.
> The latter requires that Invoker targets be MBeans
> which is an uneccessary requirement.

I agree. I made that requirement and it seemed like a good idea at the time. I think this unification is now the next logical step to making the interceptor known as "remoteness" available across the board (pojos).

A lot of our effort is in fact applying what we learned from EJB to MBeans (JBoss 3.x) to POJOs (JBoss 4.x). That trend is clear and tying invokers to MBeans goes against the grain.

> If JMX is not going to be used as the namespace and
> invocation layer for Invokers, the only other
> suitable choice is something like JNDI and
> reflection.

negative. Any map with logical names will do. I view all the naming specs (say JNDI or JMX) as overhead. I mean I DON"T NEED A SPECIAL OBJECT to be the identifier and then the "serialization" requirement of JNDI is overhead in a big way.

> Using JMX as a naming service has been
> problematic so I can see that a valid criticism. What
> are the others and what is the proposal for replacing
> it?

Let's keep it very simple.

If we create a specific assembly for a POJO we include in the client side interceptor a key for the object (say the hashcode). That hashcode is mapped to the head of the interceptor chain on the server side. The serverside invoker retrieves the key from the payload, looks up the object in a static map on invokers (I would break down the map by class at least so as to speed it up). What is retrieved is the front interceptor with invoke(invocation). And then usual stuff.

marcf

> Sounds like it makes sense (at least at first glance)
> to have JMX Remoting use Invokers as its transport
> mechanism, since there is definite duplication there
> now. This will also give us a head start on security
> within JMX Remoting.

The interceptor approach before the target and after the invoker is what will buy you the security model. It will also give us a head start on specs. Meaning that it looks like every spec is going to come up with its own security specifying (for example the declarative model of EJB vs servlets) and here is a chance for us to unify some of it. We can then secure access to components and in fact any pojo based on methods and proxy like scott did. Interceptors give us a clean way to reuse cross-cutting functionality.

JMX on JBoss was pushing for interceptors being JMX native constructs. While I still believe that the chain of interceptors should be manageable (and that discussion was pushed in the JMX days) the interceptors shouldn't be JMX construct for 4.x going forward. That much is clear.

> We will also want to have the ability for automatic
> remote class loading (which we already have now in
> JMX Remoting). I think Invokers already has this in
> the works as well (so would make sense to consolidate
> as well).

Absolutely.

> The other big requirement we will have is a discovery
> mechanism. I am assuming that Invokers will need
> this as well (if not already there). We already have
> a multicast and unicast detector we use currently.

how does that compare to the multicast of java-groups for the clustering? I haven't really put any thought into this but it seems clear that having multicast discovery of invokers is a pre-requisite for the virtualized JBoss where we run large farms of JBoss'es.

marcf

Originally we tried to use JavaGroups' ReplicatedHashtable for detections. At the time we had a problem where if many instances tried to create the inital table within a few seconds of each other, there would be multiple owners that had problems merging the data into a single table viewable to all (thus if could be possibility of multiple servers coming up for the first time at the same time and not seeing each other). I started to work with Bela on this, but once we had our own multicast implemented, was less of a priority.

I know of the JMX remote implementation, will be important for us to have several differnt types of detectors (some that do dynamic discovery on controlled network and some that will do lookup based on predefine registry). But all should use same interface so can be pluggable as to which implementation is used.

BTW, what is POJO?

>
> JMX on JBoss was pushing for interceptors being JMX
> native constructs. While I still believe that the
> chain of interceptors should be manageable (and that
> discussion was pushed in the JMX days) the
> interceptors shouldn't be JMX construct for 4.x going
> forward. That much is clear.

If interceptors are changed not to be JMX construct, all these interceptors still have to be hosted somewhere in a component framework, be it an aspect server(in the sense of a mbean server) or whatever. Then a whole slew of classloading and invocation dispatching and object lifecycle management logic and maybe other service have to be added to this seperate framework(aspect framework). Maybe a middle ground can be found by hosting some of these stuff through Mbean server, but two frameworks at work is still questionable.

The beauty I see in JBoss 3 is it's using JMX microkernal and intercepter based EJB implementation. That gives JBoss a standard component framework and dynanmic nature and all the advantages. AOP is all good, but its essence in a service framework is it can provide better declaretive service weaving capability for clients. An AOP client view(container in EJB sense) is more powerful but backend service(remoting, transaction, etc) can still be JMX powered, that is, AOP containter is just another service mbean that gateways the client into the service world of JBoss. Changing JBoss core to extend AOP to POJOs just to give the programmers the power of AOP maybe not a good idea. A seperate AOP framework at the application level(in the form of an AOP liberary) serves this purpose better.

I'm very much less than convinced that we should detach invokers from jmx. Obviously it is possible. I would like a convincing demonstration that the results will be simpler than always using jmx, and I'd like to see a killer use case for it. KISS.

Since you can fairly easily wrap a POJO into an xmbean, I find the "POJO" argument pretty much completely unconvincing.

I think the biggest risk in jboss 4 will be managing the complexity of all the different projects going on. We now have at least 5 implementations of interceptors:

ejb
client
mbean
Hiram's aspects
Bills aspects.

Unless this proposal for decoupling invokers from jmx clearly results in a dramatic simplication of some part of the jboss core I suggest we leave it alone until the aspect work has progressed to the point where we have ______ONE______ kind of interceptor.

Currently several things I would like to do are waiting for the situation with aspects/interceptors to clarify a bit.

Without a compelling use case I have to regard this as a gold plated requirement that is likely to distract us from implementing clearly useful functionality.

My arguments are purely technical,
we know JMX will work but...

Decoupling is an aspect
For a service, it makes sense to wrap the POJO
(Plain Old Java Object) in an {X}MBean to decouple the
client from the service.

Decoupling has overhead
Every access goes through the MBeanServer with a
lookup into the registry to find the mbean
using the ObjectName.

In general we are talking about a POJO which we want to
make "remote". Do we wrap it in an XMBean, give it an
ObjectName and register that with the MBeanServer?
What if there are 1000 of these POJOs
each representing a jmx notification listener,
subscription to a jms topic or a client side cache
invalidator?

To avoid the 1000 mbeans, do we use an AOP remoting
service MBean instead?
That would have three indirections:

Invoker -- Registry --> AOP remoting ObjectName
MBeanServer -- MBeanRegistry --> AOP remoting MBean
MBean -- object id --> AOP wrapped POJO

What about optimizing local access? It now goes through
the extra layer of the MBeanServer, only because the
invokers must talk JMX.
Remember I don't need the additional decoupling,
just the ability to hand the invocation over to the
correct flow/POJO.

Why not the more direct
Invoker -- Registry (object id) --> handler --> POJO

if my config includes the decoupling
handler -- MBeanServer (objectname) --> MBean/POJO

That way we don't pay for what we don't use/need.

Ideally, the payload marshalling and registry access
should be optimized away locally, but I don't understand
the implication this has for the current interceptors
that assume the transaction/principal/etc are in the
invocation.

I started this by talking about clients generating
a remote proxy allowing the server to asynchronously
respond.
What if the client is running in a sandbox (e.g. an applet)?
We might not have the correct priviledges to create
an MBeanServer or register in an MBeanServer that is
already present.
There are other problems with sandboxes.

And finally, in remote mode, the invoker/handler has to
know the correct classloader before it tries to unmarshall
the payload.
RMI does a bad job at this, assuming the classloader is
where it loaded the stub, rather than the TCL of
the application that exported the object.
It looks like jmx1.2 is making a similar mistake with
MBeanServer.getClassLoaderFor(ObjectName)

Regards,
Adrian

Connectors are slightly different to the invokers,
but serve the same purpose.

Connectors expose the MBeanServer{Connection} interface
to clients, requiring the ObjectName to be known by the
client.
ObjectNames are quite large objects and they are
slow to serialize/deserialize.

The invokers require no JMX code on the client side,
using a "target" which identifies the ObjectName
on the server side.

Connectors must handle async callbacks for
addNotificationListener().

Regards,
Adrian

>
> negative. Any map with logical names will do. I view
> all the naming specs (say JNDI or JMX) as overhead.
> I mean I DON"T NEED A SPECIAL OBJECT to be the
> identifier and then the "serialization" requirement
> of JNDI is overhead in a big way.
>
The use of a hashCode is not guarenteed to be unique, is a bitch to debug and JNDI has no serialization requirements. There does need to be a meaningful name object to represent a target object.

> Let's keep it very simple.
>
> If we create a specific assembly for a POJO we
> include in the client side interceptor a key for the
> object (say the hashcode). That hashcode is mapped
> to the head of the interceptor chain on the server
> side. The serverside invoker retrieves the key from
> the payload, looks up the object in a static map on
> invokers (I would break down the map by class at
> least so as to speed it up). What is retrieved is
> the front interceptor with invoke(invocation). And
> then usual stuff.
>
Ok, so let's get a prototype of this using a meaningful key. A HashMap is O(1) for the key so just make it a unique string and quite worrying about the extra overhead of 4 bytes vs whatever.

We need to get something in place so we can port the RMI/JRMP, RMI/IIOP, RMI/HTTP, ... to this to see what works and what does not. We can then also look at the asynchronous JMS invokers to further generalize this.

> My arguments are purely technical,

I am sold on them. I like clear stuff.

> make "remote". Do we wrap it in an XMBean, give it
> an
> ObjectName and register that with the MBeanServer?
> What if there are 1000 of these POJOs
> each representing a jmx notification listener,
> subscription to a jms topic or a client side cache
> invalidator?

Forcing POJOS to be represented as XMBeans is overkill.

And as you say it ties the client to JMX. JMX should be on the client if the client wants an object framework with services.

> Remember I don't need the additional decoupling,
> just the ability to hand the invocation over to the
> correct flow/POJO.

That is correct. The decoupling makes sense at services level. You want the microkernel decoupling, it has to be an mbean but that behavior has little to do with using an invoker.

> Why not the more direct
> Invoker -- Registry (object id) --> handler --> POJO

clearly. Trick as Scott points out is going to be in clever implementations of the lookup.

> Ideally, the payload marshalling and registry access
> should be optimized away locally, but I don't
> understand
> the implication this has for the current
> interceptors
> that assume the transaction/principal/etc are in the
> invocation.

Good point but easy to solve.

On the invokers. The invokers are unique per server so we need the identifier to find the correct target. Only the invokers lookup the target and you are correct in pointing out that this lookup is not there in the case of local invocation.

On the payload. Yes you are correct in pointing out that in the case of client side we do encode some of the information in the payload by way of client side interceptors. If we are in the same VM then we could potentially have the TX interceptor work from the thread. I don't know.

marcf

As far as JMS goes, you have to have a way for the server to invoke the client in order to support asynch message delivery (onMessage).

You might have seen that I added a Axis SOAP connector for jboss-mx (which broke the build, sorry for that). We have a few more weeks worth of work that we would like to do so we can put out a jboss-mx release with the JMX remoting. I think using Invokers still makes more sense long term, but want to get out a working release of JMX remoting in the meantime. Once the release it out, I would like to join in the effort to consolidating JMX remoting with Invokers. There shouldn't be any visibility from a users perspective when we change out the implementation, so should be a smooth transition.

> > Let's keep it very simple.
> >
> > If we create a specific assembly for a POJO we
> > include in the client side interceptor a key for
> the
> > object (say the hashcode). That hashcode is
> mapped
> > to the head of the interceptor chain on the server
> > side. The serverside invoker retrieves the key
> from
> > the payload, looks up the object in a static map
> on
> > invokers (I would break down the map by class at
> > least so as to speed it up). What is retrieved is
> > the front interceptor with invoke(invocation).
> And
> > then usual stuff.
> >
> Ok, so let's get a prototype of this using a
> meaningful key. A HashMap is O(1) for the key so just
> make it a unique string and quite worrying about the
> extra overhead of 4 bytes vs whatever.
>

I would use a Long as the key to the Map of remoted objects. A sequence of remote object ids would be easy to generate and would be cheap to serialize. The downside is that client proxys would be invalid after a server restart. Is this acceptable (??)

I don't think that it is acceptable as a general rule.

But once again, let's ask the grand'father what we should do: CORBA.

In corba, you have both transient and persistent objects.

Transient object references are to be considered invalid after a server restart (so a long (starting from System.currentTimeMillis() and going modulo for example) would be fine).

Persistent object do have the same name between server restarts (such as EJB container ObjectName right now).