-
1. RE: LB requests to a processing farm
slaboure Oct 11, 2002 4:57 AM (in response to slaboure)Hello Andrew and Jack,
How are you?
> What we are trying to do is load balance a data load operation.
> I believe
> Jack talked to you about this at the
> NY Advanced training class. Here is a quick rehash:
> 3 machines each with a web server and app server.
> 1 request to load say 10,000 files (in production it could be
> 1,000,000/day).
>
> The web/appserver that gets the initial request divides
> the files into smaller chunks that are then handed off to
> SLSB's to parse the files and update the DB. We would like to have a
> RoundRobin load balancing for the 2 boxes
> that did not get the initial request.
OK, first, you say that you need to process 10'000 files, but does the request comming from the web will say "Process 10'000 files" or do you have 10'000 requests saying "Process a file"?
> I believe you told Jack that we might have to change some
> code. If you could point to a package we should look at it would be a big
> help.
I think that what I said was that if you really just need to distribute the load in the farm, why not simply have a front-end (with possible weak computers), that simply dispatches processing requests to a strong backend of appservers. In this case, you don't need to do anything: you will simply cluster your SLSB found in your backend (and your backend would have no relationship with your front-end) and, once you have a SLSB remote interface you simply call (from the front-end):
pseudo-code:
while (incomingRequest)
{
remote = getRemote(); //possible cached
remote.processRequest (incomingRequest);
}
I don't know if it is stable enough yet, but I suggest that you do some testing with 3.2 directly as the load-balancing mechanism has been improved.
Cheers,
Sacha -
2. Re: RE: LB requests to a processing farm
andrewboyd Oct 11, 2002 9:13 AM (in response to slaboure)It is one request that says process 10,000 files.
Not multiple request.
The code I have is like so:
...
// Inside DataMiner POJO
public void run(){
try{
// DMHandler is our SLSB
DMHandler dmHandler = svLocator_.getDMHandler();
dmHandler.setLoadingtime(this.loadingtime_);
//processDM is where all the work happens
// searchResultLineItems_ has a subset of 10k files.
dmHandler.proceessDM(searchResultLineItems_);
...
//ServiceLocator call
public DMHandler getDMHandler() throws ServiceLocatorException{
DMHandler dmHandler = null;
try{
dmHandler = ((DMHandlerHome) getHomeService(DMHANDLER_JNDI_NAME)).create();
} catch(javax.ejb.CreateException e3){
e3.printStackTrace();
throw new ServiceLocatorException("Caught a CreateException in getDMHandler().");
} catch(java.rmi.RemoteException e4){
e4.printStackTrace();
throw new ServiceLocatorException("Caught a RemoteException in getDMHandler().");
} catch(Exception ex){
ex.printStackTrace();
throw new ServiceLocatorException(getInternals());
}
return dmHandler;
}
Every DMHandler that is created is on the same box as our DataMiner POJO and we would like it to not be. -
3. Re: RE: LB requests to a processing farm
slaboure Oct 11, 2002 9:31 AM (in response to slaboure)OK, but have you thought about my proposal i.e. having front-end boxes and back-end boxes?
I guess you've understood that the underlying problem is that as the EJB is also deployed locally, the clustered proxy will see that and make a direct local optimized call instead of a LB RMI call. As there is no-way currently to say "don't optimize", the easiest solution, if you cannot have a front- and back-end, is to deploy on each box, two JBoss instances:
- one running Jetty+whatever you need
- one running your file-processing EJBs
If it is a problem for you (because you will loose optimized behaviour that you also want for some EJB), then I will have to add a new setting.
Cheers,
sacha -
4. Re: RE: LB requests to a processing farm
andrewboyd Oct 11, 2002 9:57 AM (in response to slaboure)We have thought/tried to do what you said but I'm affraid we obviously did not know what we were doing.
We took our jboss dir structure which is the same on
all three boxes and on the "front-end box" we wanted
to remove the ejb's so we removed the ejb.xml file.
That didn't work as we then got exception stating it didn't know anything about the DMHandler.
Some direction is clearly needed, please. :-)
On the new setting front I was thinking that something
called <force.load.balancing> might work.
Thanks,
Andrew -
5. Re: RE: LB requests to a processing farm
slaboure Oct 11, 2002 10:08 AM (in response to slaboure)Yes, sure.
On the front-end boxes, remove your myEjb.jar from /deploy (so that the EJBs are not deployed) BUT put a JAR containing at least the EJB interfaces (home, remote, pk, etc.)
Then, we you need to do the lookup from the front-end (to access your EJB), you have several choices:
1) create an InitialContext (p) explicitling giving the Properties in parameters. The Properties should target the JNDI service of the other JBoss instance (not the local JNDI service)
2) in your deployment descriptor, you can map the JNDI names used in your code to the one of your deployment environment. An example for an EJB accessing other EJBs on another host is available here:
http://www.jboss.org/online-manual/HTML/ch05s13.html
("External EJB Reference", see the "jnp://otherserver/..." usage)
Cheers,
sacha -
6. Re: RE: LB requests to a processing farm
hyao Oct 14, 2002 3:45 PM (in response to slaboure)But if we use external EJB reference in jboss.xml, we can refer to the bean only on one app server box since it has ip address bind to jndi-name. What we want really is to do EJB load balancing: multiple app server boxes(and they are different machins too) have the exactly same EJB set deployed.
Any clue?
hongmei -
7. Re: From AndrewBoyd: LB requests to a processing farm
hyao Oct 14, 2002 3:49 PM (in response to slaboure)But if we use external EJB reference in jboss.xml, we only can refer to the bean only one app server box since it has ip address bind to jndi-name tag. What we really want is to do EJB loading balancing: multiple appserver boxes(and they are different machine too) have the exactly EJB set deployed.
Any clue?
hongmei -
8. Re: RE: LB requests to a processing farm
slaboure Oct 16, 2002 11:40 AM (in response to slaboure)Then choose solution 1) above