EJB/War deployer ordering problem

[pmuir]

Trunk seems to be starting the war before the ejb3s are started.

This causes a problem, as Seam uses a ServletListener to boot itself, and can try look up EJB3s using JNDI (as described http://docs.jboss.org/ejb3/app-server/tutorial/jndibinding/jndi.html).

This is reproduced by running

cd jbossas/testsuite
./build.sh _jars-seam
cp output/lib/jboss-seam-dvd* ../build/output/jboss-5.0.0.Beta4/server/default/deploy
cd ../build/output/jboss-5.0.0.Beta4
./bin/run.sh

You should get a NamingException from Seam, as it fails to look up the EJB3.

Note, the deployment order is different if you deploy to the already running app server.

[alesj]

Note, the deployment order is different if you deploy to the already running app server.

What do you mean by deployment order?

Here is my analysis of the problem:

Indexer EJB depends on persistence.units:ear=jboss-seam-dvd.ear,jar=jboss-seam-dvd.jar,unitName=dvdDatabase

persistence.units:ear=jboss-seam-dvd.ear,jar=jboss-seam-dvd.jar,unitName=dvdDatabase depends on jboss.jca:name=dvdDatasource,service=DataSourceBinding

jboss.jca:name=dvdDatasource,service=DataSourceBinding is picked up by RARDeployer which creates matching ServiceMetaData

TomcatDeployer also creates ServiceMetaData
And it's TomcatDeployer that deploys/starts Seam applications (for those unfamiliar with Seam).
And in there Indexer EJB is being looked-up in JNDI. Resulting in failure.

So, both TomcatDeployer and RARDeployer, need to be before ServiceDeployer (which deploys ServiceMetaData)

And therefore why Indexer EJB is only fully installed after TomcatDeployer.

I currently don't see how to solve this w/o Seam knowing about the dependency.

[wolfc]

Seam only does a simple JNDI lookup. Which is perfectly allowed.

A JNDI lookup can happen anywhere at any time without MC knowing any explicit dependency.

One way to solve this is to hook JNP to MC, so that if something is not found open deployments are resolved first until either the JNDI name appears or MC is done with deploying. In the later case it would result in a NameNotFoundException.
(It can even remember (persistently) which bean made it pop-up so future deployments will be faster.)

[starksm64]

A jndi lookup is not a sufficient expression of dependency for the mc to detect this. In general we are not going to support lookup(x) implies a dependnecy on x because there is no guarantee x resolves to a component as know to the mc. We already have jndi dependency implementations, but you have to know what component provides the jndi binding to use it. If a component is doing a jndi lookup and it fails, this would have to roll back the current war deployment and add a dependency on the name. That is a big change to deployment processing.

The real issue is that war components should be started after all other non-wars in the current deployment unit tree. Need to look into how that could be done.

[alesj]

The real issue is that war components should be started after all other non-wars in the current deployment unit tree. Need to look into how that could be done.

The things here is that a mbean component (in this case EJB container) must delay the whole war processing.
Which is luckily possible with the introduction of MC dependency into deployers.

This means we need to gather all possible information about the components that are candidates to do some outside war calls and put dependencies to war.
Perhaps staring with EJBs will for now do the trick.

btw: Scott, since I know you're busy with the JBW presentation :-), I can have a crack at this first thing in the morning ... and we can discuss it at AS5/MC call

[starksm64]

Ok. In general this is an implicit dependency that should require an @EJB to resolve. You won't be able to run the profile service (or any other vendor deployment tool) and analyze the deployment for unsatisfied component usage/dependencies. The only reason to not require an @EJB or ejb-ref in this case is because wars generally should be started after everything else.

Of course the first support case will be about an ejb depending on a webservice.

[alesj]

"alesj" wrote:
I can have a crack at this first thing in the morning ... and we can discuss it at AS5/MC call

Yup, I'm totally lost there. :-(
And with Carlo already bitc... about regression, and Adrian cursing all the existing EJB deployers hacks, I don't want to hack something that will break things in the final minute.

So, Scott, if you can pull another

 DependencyPolicy dependsPolicy = container.getDependencyPolicy();
 dependsPolicy.addDependency("jboss.ejb:service=EJBTimerService");

out of your hat, that would be great.

[alesj]

OK, this does what Adrian was explaining:

public class ExplicitOrderWARStructure extends WARStructure
{
 private int contextOrder = Integer.MAX_VALUE;

 protected ContextInfo createContext(VirtualFile root, String metaDataPath, StructureMetaData structureMetaData)
 {
 ContextInfo contextInfo = super.createContext(root, metaDataPath, structureMetaData);
 contextInfo.setRelativeOrder(contextOrder);
 return contextInfo;
 }

 public void setContextOrder(int contextOrder)
 {
 this.contextOrder = contextOrder;
 }
}

But where to put it?
Since if I want to put it to bootstrap-beans it needs to be in bootstrap classpath.

[alesj]

"alesj" wrote:

But where to put it?
Since if I want to put it to bootstrap-beans it needs to be in bootstrap classpath.

Adding it to MC will do, since we'll have to update it anyway.

But it doesn't help here. ;-(
This only orders contexts inside deployment.
Where in our case we have two seperate deployments - application and -ds.xml file.

Or I misunderstood Adrian.

[alesj]

"alesj" wrote:

Where in our case we have two seperate deployments - application and -ds.xml file.

I've added a comparator to MainDeployerImpl's top DeploymentContexts for process method.

 if (undeploy.isEmpty() == false)
 {
 // Undeploy in reverse order (subdeployments first)
 undeployContexts = new ArrayList<DeploymentContext>(undeploy.size());
 for (int i = undeploy.size() - 1; i >= 0; --i)
 undeployContexts.add(undeploy.get(i));
 Collections.sort(undeployContexts, reverted);
 undeploy.clear();
 }
 if (deploy.isEmpty() == false)
 {
 deployContexts = new ArrayList<DeploymentContext>(deploy);
 Collections.sort(deployContexts, comparator);
 deploy.clear();
 }

and also a way to set relative order of the context info (which then translates to relative order of deployment context) in StructureDeployer.
From AbstractStructureDeployer.createContext():

 if (result != null && contextInfoOrder != null)
 result.setRelativeOrder(contextInfoOrder);

So, now by setting contexInfoOrder property on FileStructure and WarStructure we can push -ds.xml before Seam .ear (and its .war).

[alesj]

Just to state this clearly, this is still a hack. :-)

e.g. first deploying Seam app, which is then picked up by HDScanner, a few moments later, in a new HDScanner cycle we deploy -ds.xml ... it's still gonna break

We need to get real dependencies in place.
But like we discussed, there has to be a limit on what we can handle w/o explicit user dependencies.

[starksm64]

I thought it was the ejbs that the app was doing an explicit lookup on, not the ds. From what I thought Adrian was talking about, the org.jboss.web.deployers.AbstractWarDeployer should be setting the AbstractDeploymentContext of the war unit. I don't see that this gets picked up from the structure information currently.

[alesj]

I thought it was the ejbs that the app was doing an explicit lookup on, not the ds.

Indexer EJB depends on persistence.units (probably having @PersistenceContext field), persistence.units depend on DS.
DS is deployed as a separate deployment unit, outside .ear.
Since -ds.xml is deployed after .ear file, there is nothing stopping TomcatDeployer to go fwd.
And that's where Seam apps get 'started', starting all of its components, where some of the are EJBs.
But the actual EJB container is not deployed until there is DS mbean present.

I don't see that this gets picked up from the structure information currently.

I just added this yesterday, to MC.

[starksm64]

If the war deployer depends on the ejbs which depend on the ds, then war should be ok. The ejb3 layer is creating proper dependencies on the ds for persistence contexts.

[alesj]

If the war deployer depends on the ejbs which depend on the ds, then war should be ok.

How do you define that a war depends on ejbs?
Putting that horrible unnecessary xml in war.xml?

Most of, if not all, Seam apps don't contain any classes in the war.
It's only jsp, xhtmls, css, ...

Seam initialization starts when the .war starts.
The Scanner scans for components inside .ear classloader.
This means it also finds components that are EJBs.
If the EJB has, besides @(ejb markup annotation), also a Seam @Startup annotation, Seam will at that point try to start that component, which in this case means first looking it up in the JNDI.

What about if Seam at that point added a JNDI EventListener?
http://java.sun.com/products/jndi/1.2/javadoc/javax/naming/event/NamingEvent.html
Do we support that?
And got notified later on that an EJB was registered and only then continued with startup?