Multithreaded services and their EPR(s)

[kurtstam]

If a service is multithreaded, should each thread be registered as a separate EPR, or should there only be 1 EPR for the entire set of threads for this service?

The current issue we have is the following:
In the current implementation we only have one EPR for each service, which currently eventhough we only allow 1 thread, is sometimes causing us issues when the configuration is reloaded. It turns out that in this case the new thread B is created before thread A has ended, and the same EPR gets registered twice. Upon unregistering thread A both these EPRs are removed from the registry and upon unregistering thread B the registry throws an exception since no matching EPR exists any longer.

The issue can be resolved by registering each thread as it's own EPR. This is easily done simply by adding a UID to the EPR, since upon unregistering we simply do a string comparison on the xml-serialized EPR. However personally I am worried that we will end up with a lot of 'dead' EPRs. If an unclean shutdown happens, the EPRs will never be removed from the registry. Therefore personally I think that the parent of all the threads should be responsible for registering the service with it's EPR and it should not get unregistered until the last thread ends. Can this be done in the current implentation?

Any other ideas are welcome!

Cheers,

--Kurt

[estebanschifman]

The problem, in my view, goes beyond just multithreaded listeners. It will in fact be common to all situations in which more than one listener is registered to the same [category,name,EPR], which could be quite frequent. E.g. processes in different address spaces servicing the same JMS queue.

So there might not be something like "...the parent of all the threads ... responsible for registering the service..." Multithreading has been the circumstance that surfaced the problem, but in my opinion it will occur in other scenarios as well.

We could consider the idea of expanding the registering key to [category,name,EPR,UID] which would solve the problem for good.

Thanks.

Esteban

[marklittle]

First, the EPR is on a per service basis, not a per thread basis. Threads are an implementation issue that should not be exposed to users/clients. If there are hundreds of threads working on behalf of a given service (aka business logic), it would be wrong to then have hundreds of EPRs.

In some ways I agree with Esteban, but in others I think this is related to the thread pool issue. I'm assuming that irrespective of the reloading issue, there is a single listener thread at the "bottom" of the hierarchy that does the registration for the service and all it does upon receipt of a message is to spawn a new worker thread. That's pretty much what most distributed systems have been doing since the mid 1990's when threads (or lightweight processes as they were known then) became more widely available. Assuming we follow this model and ignore reload for now, then we should be fine.

Now back to the reload issue. As I've said before, I think there are good reasons for reloading and reinstantiating the configuration/setup, but that is an implementation specific concern for services. It shouldn't be something that is mandated for all listener implementations. In fact, by default I think all that should happen is a reload of the parameters and if anything has changed we trigger an event and leave it up to the event handler to figure out. If that handler wants to re-instantiate, then it's up to it to figure out how to do it without screwing up anything else. We could provide a handler for JMS, but does anything else really need it at the moment?

Assuming we can agree on what the issue(s) is(are) and potential solutions, the next question is: what can we do for the GA that is quick and gives us a solution? It may not be the best solution, but we can revisit that after the release.

[kurtstam]

1. I think we agree that the registration of the EPR should *not* be done in the thread.

2. However as Esteban pointed out that if we register multiple listeners in different address spaces we still have a problem. And I think it is actually useful to know from a registry position that we have mutliple physical listeners, for load balancing and failover scenarios. So should we have these listeners register as 'different' EPRs of the same service?

--Kurt

[estebanschifman]

When Mark first explained the concept of EPR to me, he assimilated an EPR to a simple 'address'. If many people live in the same address, even in the same apartment/suite, and we are looking for one of them in particular, we need to know the name of the person we are seeking (UID). One address, many people = one EPR, many UIDs

I don't share the description of many listeners in the same EPR being "...hundreds of EPRs...". It is the very same EPR with hundreds of listeners servicing it: each registration with a unique identifier. This concept is independent of if the many listeners are several threads in the same JVM, or distributed in disparate address spaces. If this is solved with one entry, or hundreds, in the registry underlying repository, it will be completely hidden for the user. They could't care less.

Having a UID added to the registration process would not alter the underlying architecture of alpha1. It would be a requirement of the registration API. That's why in a post on the public forum I made on friday, there was a suggestion for the Registry to return a UID at registration time, and that same UID be required at de-registration time.

E

[estebanschifman]

Here's yet another twist to the same problem:

The way we are using the registry today is to add an entry when a listener comes up, and remove it when the process stops listening. Some of us have talked about potential benefits of having entries in the registry that go beyond the uptime of any particular listener (or listener instance). This could be the case for all Message transports that are themselves 'durable': local or remote filesystems, JMS and other 'guaranteed delivery' messaging systems, sql tables, e-mail, etc. The listener does not need to be up and running for invokers to send a message to that EPR (provided that the service is not expected to give a response in 'near real time': that the service is asynchronous)

We also have transports that require a listener to be up and running (e.g. sockets). I think that in most of these (if not all of them) the conversation flow between invoker and provider will be bidirectional. Entries for these listener's EPR must thus be dynamic, so (just as our listeners do today) they must register as soon as they're ready to provide the service, and de-register before they finish.

This is also the case for those services that, even on 'durable' transports, expect a synchronous response. A good example of this is how the CbrProxy action class interacts with the content based router service: JMS as transport, but invoker will wait until a response comes back.

My guess is that when the service does not require response (not even an acknowledgement that a Message has been received), and the EPR transport is durable, we could (may be we even must) do with a 'durable' Registry entry. Listeners servicing such an entry would use the Registry not to register themselves, but to find out where to go pick up their incoming messages. The rest of services (ack or response required, or volatile transports) need register themselves whey they're ready to listen, and unregister when done (a volatile registry entry).

This surfaces the need for a friendly interactive way for administrators to add/change/remove durable entries to the registry. A simple call to the appropriate method in the registry API would suffice, but it might be useful to have a friendly user interface so ESB administrators could do it. (Kurt: is there an off the shelf juddi user interface that we could use ? Or do we have to develop our own registry browser/editor ?)

Having said all this, we might not need at all a UID for the registration process. Durable registry entries would not need it as they would be manually handled by the ESB administrator. Volatile entries not only don't need it, but should check if there's an entry for the same EPR they're about to register.

The next question would be if we need the registry to 'know' the difference between durable and volatile entries. When the time comes to have a sophisticated registry browser it might be useful to see which are and which are not durable. We might also require that volatile entries 'refresh' themselves every so often or else be purged by a clock driven process. This would avoid dead or zombie processes leaving behind stale registry entries.

[marklittle]

1. I think we agree that the registration of the EPR should *not* be done in the thread.

2. However as Esteban pointed out that if we register multiple listeners in different address spaces we still have a problem. And I think it is actually useful to know from a registry position that we have mutliple physical listeners, for load balancing and failover scenarios. So should we have these listeners register as 'different' EPRs of the same service?

--Kurt

First, I've mentioned this several times, but this paper http://http://www.orablogs.com/pavlik/archives/The-Session-Concept-in-Web-Services.pdf is extremely important to the general debate on EPRs and Services (don't let the title fool you - it is relevant to our architecture]. Although an EPR is an address, it should NOT be an object-level address (c.f., CORBA IORs). That is the way to close-coupling of clients to services. We need to maintain a loose-coupling. Things like multiple threads in the same "service" are back-end implementation issues. If I have multiple objects within my service and they are able to be dispatched too separately then that information should be encoded within the body of the message and not in the EPR. WS-Addressing lets you do the latter, but then you end up with something like WS-RF which is essentially the CORBA object-key approach. This leads to brittle applications in that if we change the way the back-end implementation works, the EPRs that all clients have then need to change.

Everyone should read the paper.

As for clustering support, that's an interesting question. Logically we want a single service name that can map to multiple EPRs. (Let's ignore the issue of stateful versus stateless services at this point, because that really does complicate the discussion - we'll assume stateless). It could be supported with multiple EPRs registered with the same service name within the EPR (c.f., DNS). Alternatively, you could have a single EPR and use CBR to dispatch across different physical endpoints. Obviously the CBR would then need to maintain the mapping information. There are trade-offs to both approaches. For now, we don't support clustering so we can leave this as an excercise for the informed user of JBossESB 4.0 GA.

[marklittle]

"estebanschifman" wrote:
When Mark first explained the concept of EPR to me, he assimilated an EPR to a simple 'address'. If many people live in the same address, even in the same apartment/suite, and we are looking for one of them in particular, we need to know the name of the person we are seeking (UID). One address, many people = one EPR, many UIDs

Check out the paper I referenced in my last post. In a SOA, the fact that there are many "people" in the service would be encapsulated within the message. Mixing metaphors, you'd send the letter to the house; the letter is within an envelope that has the house address on it and within the letter is the identity of the ultimate receiver. Services are course grained entities, not fine grained objects. It's the service that is registered with the registry, not the individual elements/threads/objects/components within it.

I don't share the description of many listeners in the same EPR being "...hundreds of EPRs...". It is the very same EPR with hundreds of listeners servicing it:

Now this may be slightly different. If we're talking about the same service that is accessible through different message delivery implementations, then it's the same service, with different EPRs.

I suspect that our definition of a service in 4.0 is not perfect at this stage. But this is only our first GA release after all.

each registration with a unique identifier. This concept is independent of if the many listeners are several threads in the same JVM, or distributed in disparate address spaces.

A single VM can host many services. If each listener is a unique service, then it should definitely have a different EPR to other services within the same VM as well as a different service name. If each listener is logically tied to the same service (business logic, implementation object, etc.) then they will all have different EPRs but the same service name.

Here's an example: I have a single process on my machine that provides a Web Service transaction service and a shared calendar. The WS-T service can only be interacted with via SOAP/HTTP because that's the only binding defined by the standard. Within the WS-T service, there are many different threads running to do work, e.g., start transactions, commit transactions etc. However, those are back-end implementation choices. The WS-T service is known by a single service name and has a single EPR for contacting it initially.

Now the calendar service is available on SOAP/HTTP, but also plain HTTP and JMS, because I want it to be available to a wider range of clients. This means that it is known by 3 EPRs, but all keyed to the same service name.

If this is solved with one entry, or hundreds, in the registry underlying repository, it will be completely hidden for the user. They could't care less.

Yes, users don't care what the EPR looks like (in fact, a lot of it is supposed to be opaque to a user and only of use to the receiver, i.e., the service). Users will typically look up an EPR given a service name.

Having a UID added to the registration process would not alter the underlying architecture of alpha1.

If you really mean Alpha1 as in the code prior to Rosetta, then it would change the architecture, because it appears as though the need for these threads to do the registration is not based on the same definition of a service.

It would be a requirement of the registration API. That's why in a post on the public forum I made on friday, there was a suggestion for the Registry to return a UID at registration time, and that same UID be required at de-registration time.

Each service should register itself once with the registry and deregister itself only when it goes out of service(!) If each listener is bound to the same service, then obviously we need to support multiple EPRs per service name, but I believe we already do that.

Why do we re-register/de-register when the listeners re-read their configuration information?

[marklittle]

"estebanschifman" wrote:
Here's yet another twist to the same problem:

The way we are using the registry today is to add an entry when a listener comes up, and remove it when the process stops listening. Some of us have talked about potential benefits of having entries in the registry that go beyond the uptime of any particular listener (or listener instance). This could be the case for all Message transports that are themselves 'durable': local or remote filesystems, JMS and other 'guaranteed delivery' messaging systems, sql tables, e-mail, etc. The listener does not need to be up and running for invokers to send a message to that EPR (provided that the service is not expected to give a response in 'near real time': that the service is asynchronous)

In that case, the queue is part of the service. The service encapsulates everything from the high-level business object down to the communication infrastructure. Whether or not the business object (or even listener in our case) is active is immaterial as long as the recipient of the message is active. We don't make a distinction between the receiving of a message and acting upon it.

Whether or not an EPR has to be removed from the registry in this case is implementation dependant. WS-RX would suggest you do not have to, for example, with SOAP/JMS, but you must with SOAP/HTTP.

We also have transports that require a listener to be up and running (e.g. sockets). I think that in most of these (if not all of them) the conversation flow between invoker and provider will be bidirectional. Entries for these listener's EPR must thus be dynamic, so (just as our listeners do today) they must register as soon as they're ready to provide the service, and de-register before they finish.

This is also the case for those services that, even on 'durable' transports, expect a synchronous response. A good example of this is how the CbrProxy action class interacts with the content based router service: JMS as transport, but invoker will wait until a response comes back.

My guess is that when the service does not require response (not even an acknowledgement that a Message has been received), and the EPR transport is durable, we could (may be we even must) do with a 'durable' Registry entry.

I think if you take the definition of the service to be everything I've outlined above, then whether or not you remove an EPR from the registry does come back to: you remove it if the service is out of order, because the queue is part of the service and in this case the queue is down too. If the queue remains up, but the listeners/objects/whatever that do the real work aren't there yet (maybe they are created lazily upon receipt of a message) then the service is still "up".

Listeners servicing such an entry would use the Registry not to register themselves, but to find out where to go pick up their incoming messages.

The listener is part of the service in this case.

The rest of services (ack or response required, or volatile transports) need register themselves whey they're ready to listen, and unregister when done (a volatile registry entry).

This surfaces the need for a friendly interactive way for administrators to add/change/remove durable entries to the registry. A simple call to the appropriate method in the registry API would suffice, but it might be useful to have a friendly user interface so ESB administrators could do it. (Kurt: is there an off the shelf juddi user interface that we could use ? Or do we have to develop our own registry browser/editor ?)

As I said earlier, our definition of a service is not quite right and is unlikely to be so for this GA because it will almost certainly need changes to the architecture. However, what we have so far is certainly good enough for users.

Having said all this, we might not need at all a UID for the registration process. Durable registry entries would not need it as they would be manually handled by the ESB administrator.

No. Durability is not an administrative issue. It's an implementation issue. Put simply, the basic rule is:

(i) when a service comes up (something can receive a message), the EPR is registered with the registry.

(ii) when a service goes down (messages that are attempted to be delivered will return a failure to the sender), the EPR is removed from the registry.

[marklittle]

It could be supported with multiple EPRs registered with the same service name within the EPR (c.f., DNS).

Should say "within the registry".

[kurtstam]

The registry already assigns a UID to the ServiceBinding (In which we store the EPR). Maybe I should just return this (for now?, or for always?).

--Kurt

[marklittle]

So we would stick with the register/de-register as now?

[kurtstam]

It's funny you use de-register. Anyway what we should change is that you need that UID when you unregister that EPR, and the UID is returned at registration time.

--Kurt

[marklittle]

;)

[estebanschifman]

+1