WildFly 9 Remote EJB Invocation Marshalling Issues
jfisherdev Oct 11, 2016 8:06 PMMy organization is currently migrating from JBoss AS 4.2.2 to WildFly 9.0.2 final.
We have been seeing some issues with remote EJB invocations from standalone Swing client applications that seem to be with marshalling. The EJB client approach for remote EJB access is being used in this case.
We have seen a significant decrease in performance in remote EJB invocations that return certain objects used to represent large data sets in some cases where this was not the case on JBoss AS 4.2.2. One common element with the objects where this has been observed is that they typically contain Object type fields or collections typed to Object. Note that all objects contained in these fields or collections are serializable.
Examples:
public class PojoModel implements Serializable { private Map<String, Object> attributes; ... }
public class TableDataModel implements Serializable { private Object[][] tableData; ... }
These are the most common types of objects returned by remote EJB invocations in our application suite and we have not really tested this with other types of objects used for this purpose, so it may not be the case, but is there possibly an issue in JBoss Marshalling with serializing Object type fields or instances of generic classes typed to Object? If it were a case of an object not being serializable, an exception would be thrown to indicate this; however, that is not the case here.
Here is an example of another object called PojoModelList that was sometimes causing issues when it was the return type for a remote EJB invocation:
public class PojoModelList implements Serializable { private static final long serialVersionUID = 1L; private List<String> properties; private Object[][] data; private transient List<PojoModel> pojos; public PojoModelList(List<? extends PojoModel> pojos) { if (pojos != null && !pojos.isEmpty()) { properties = pojos.get(0).getProperties(); int colCount = properties.size(); data = new Object[pojos.size()][colCount]; int row = 0; for (PojoModel pojo : pojos) { int col = 0; for (String p : properties) { data[row][col++] = pojo.getProperty(p); } row++; } } } public synchronized List<PojoModels> getPojos() { if (pojos== null) { int rowCount = (data == null) ? 0 : data.length; pojos = new ArrayList<PojoModel>(rowCount); for (int row=0; row<rowCount; row++) { Object[] rowData = data[row]; pojos.add(new PojoModel(rowData, properties)); } } return pojos; } private void writeObject(ObjectOutputStream oos) throws IOException { GZIPOutputStream gzip = new GZIPOutputStream(oos); ObjectOutputStream oos2 = new ObjectOutputStream(gzip); oos2.writeUnshared(properties); oos2.writeUnshared(data); oos2.flush(); gzip.finish(); } @SuppressWarnings({"unchecked"}) private void readObject(ObjectInputStream ois) throws IOException, ClassNotFoundException { GZIPInputStream zip = new GZIPInputStream(ois); ObjectInputStream ois2 = new ObjectInputStream(zip); properties = (List) ois2.readUnshared(); data = (Object[][]) ois2.readUnshared(); } }
While using PojoModelList, we were seeing rather cryptic client errors like this:
java.io.EOFException: Read past end of file at org.jboss.marshalling.SimpleDataInput.eofOnRead(SimpleDataInput.java:151) at org.jboss.marshalling.SimpleDataInput.readByte(SimpleDataInput.java:232) at org.jboss.ejb.client.remoting.ProtocolMessageHandler.readAttachments(ProtocolMessageHandler.java:48) at org.jboss.ejb.client.remoting.MethodInvocationResponseHandler$MethodInvocationResultProducer.getResult(MethodInvocationResponseHandler.java:106) at org.jboss.ejb.client.EJBClientInvocationContext.getResult(EJBClientInvocationContext.java:276) at org.jboss.ejb.client.EJBObjectInterceptor.handleInvocationResult(EJBObjectInterceptor.java:64) at org.jboss.ejb.client.EJBClientInvocationContext.getResult(EJBClientInvocationContext.java:290) at org.jboss.ejb.client.EJBHomeInterceptor.handleInvocationResult(EJBHomeInterceptor.java:88) at org.jboss.ejb.client.EJBClientInvocationContext.getResult(EJBClientInvocationContext.java:290) at org.jboss.ejb.client.TransactionInterceptor.handleInvocationResult(TransactionInterceptor.java:46) at org.jboss.ejb.client.EJBClientInvocationContext.getResult(EJBClientInvocationContext.java:290) at org.jboss.ejb.client.ReceiverInterceptor.handleInvocationResult(ReceiverInterceptor.java:129) at org.jboss.ejb.client.EJBClientInvocationContext.getResult(EJBClientInvocationContext.java:265) at org.jboss.ejb.client.EJBClientInvocationContext.awaitResponse(EJBClientInvocationContext.java:453) at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:204) at org.jboss.ejb.client.EJBInvocationHandler.doInvoke(EJBInvocationHandler.java:183) at org.jboss.ejb.client.EJBInvocationHandler.invoke(EJBInvocationHandler.java:146)
We eventually switched to using Lists of PojoModel objects, as opposed to PojoModelList objects and have not seen the issue since.
I am also curious if any considerations need to be made for NIO/XNIO in either the applications or WildFly configuration. This is not something we are especially familiar with, and it was not something that was a consideration in JBoss AS 4.2.2.
One thing we did do that was helpful was set the jboss.remoting.pooled-buffers=false system property to work around the issue described here [REM3-200] Memory leak in org.xnio.ByteBufferSlicePool - JBoss Issue Tracker.
Any information about this would be very much appreciated.