Cannot load null class!

[meetoblivion]

hey there

tinkering with Infinispan 4.1 Hotrod client/server mode.  Run into a few issues.

1. Does infinispan make any assumptions about how the client is connecting? e.g. does it expect that it's a bind, send data, receive data, unbind type of behavior or can it work if the client stays connected for a long time? (Or does it not matter?)

2. When I create a new RemoteCacheManager("server:port",boolean), if boolean = true, I can connect once, but then further connections get refused with an IOException.  When boolean is false, I get Cannot load null class! Caused by: HotRodServerException{messageId=0, errorStatusCode=0} org.infinispan.client.hotrod.exceptions.HotRodClientException: Could not instantiate class: null

Thoughts?  I am able to connect to the server and it writes the first time, just not any other times.

[supinko]

I got something similar when it turned out I was instantiating multiple RemoteCacheManagers and one or more hadn't been started.

HTH,

Supin

[meetoblivion]

agreed, after some more testing i noticed that it would consistently failed if start was false and i did not call start.

however, my other problem remains.  after doing some more testing i found that there's a sort of idle timeout on the remote cache manager, or something like that.  i set up a singleton ejb that created a remote cache manager, if i hit it often and quickly, it stayed up.  if i left the app running, went talked to someone about the weather for 10 minutes and came back, the result was an EOFException.    

[manik]

Are you guys using 4.1.0.Beta1?  Or is this a snapshot?

[meetoblivion]

I am using 4.1.0.Beta1

[mircea.markus]

2. When I create a new RemoteCacheManager("server:port",boolean), if  boolean = true, I can connect once, but then further connections get  refused with an IOException.  When boolean is false, I get Cannot load  null class! Caused by: HotRodServerException{messageId=0,  errorStatusCode=0}  org.infinispan.client.hotrod.exceptions.HotRodClientException: Could not  instantiate class: null

Starting cache manager before using the caches associated with it is mandatory. The boolean flag does just that: indicates weather the cache manager should be started or not. That's not clear in the javadoc nor in the exception message. I've created ISPN-440 for that.

[meetoblivion]

What about my first issue, that seems like a bigger problem to me.  Seems like I randomly lose connection back to a remote cache manager.    

[mircea.markus]

Looking into that right now. I've tried to reproduce it with an 10mins delay between calls, no luck. Trying with 20 mins. Any estimate on the time difference between the two calls? Also if you get a stack trace would be very helpful.

[meetoblivion]

Here's a sample EJB that can consistently break it for me

@Singleton
@Startup
public class SingletonRCM {
    private RemoteCacheManager rcm;

    @PostConstruct
    public void init() {
        rcm = new RemoteCacheManager("hell:11311",true);
    }

    public Cache<String,String> getStagingCache() {
        return rcm.getCache("staging");
    }
}

The exception itself is a broken pipe

Caused by: java.io.IOException: Broken pipe
        at sun.nio.ch.FileDispatcher.write0(Native Method)
        at sun.nio.ch.SocketDispatcher.write(SocketDispatcher.java:29)
        at sun.nio.ch.IOUtil.writeFromNativeBuffer(IOUtil.java:104)
        at sun.nio.ch.IOUtil.write(IOUtil.java:75)
        at sun.nio.ch.SocketChannelImpl.write(SocketChannelImpl.java:334)
        at java.nio.channels.Channels.write(Channels.java:60)
        at java.nio.channels.Channels.access$000(Channels.java:47)
        at java.nio.channels.Channels$1.write(Channels.java:134)
        at java.io.OutputStream.write(OutputStream.java:58)
        at java.nio.channels.Channels$1.write(Channels.java:115)
        at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.writeByte(TcpTransport.java:96)
        ... 117 more

Here's what the client does (JSF Action method)

@EJB SingletonRCM rcm;
    public void doSomething() {
        Cache<String,String> dataCache = rcm.getStagingCache();
        dataCache.put(cacheKey,cacheValue);
        FacesContext ctx = FacesContext.getCurrentInstance();
        ctx.addMessage(null, new FacesMessage("Put data successfully."));
    }

And a little bit more of the exception

Caused by: org.infinispan.client.hotrod.exceptions.TransportException: java.io.IOException: Broken pipe
        at org.infinispan.client.hotrod.impl.transport.VHelper.writeVLong(VHelper.java:48)
        at org.infinispan.client.hotrod.impl.transport.tcp.TcpTransport.writeVLong(TcpTransport.java:40)
        at org.infinispan.client.hotrod.impl.HotrodOperationsImpl.writeHeader(HotrodOperationsImpl.java:276)
        at org.infinispan.client.hotrod.impl.HotrodOperationsImpl.sendPutOperation(HotrodOperationsImpl.java:254)
        at org.infinispan.client.hotrod.impl.HotrodOperationsImpl.put(HotrodOperationsImpl.java:113)
        at org.infinispan.client.hotrod.impl.RemoteCacheImpl.put(RemoteCacheImpl.java:146)
        at org.infinispan.CacheSupport.put(CacheSupport.java:30)

hopefully this is a bit more info.

[mircea.markus]

Thanks for the detailed info.

The problem might be solved by having an async thread that would send ping requests on the persistent TCP connections to the server, when they are idle:  ISPN-441

[meetoblivion]

So just tinkering with things a little bit this morning.

I created

https://jira.jboss.org/browse/ISPN-443

there's a few typos in in startServer.sh

I noticed that when I pass --idle_timeout=0, my issues seem to go away.  I'll have to test it more thoroughly on the work dev server, but it seems like it is resolved by disabling timeouts.

Are there any long term ramifications to having timeouts disabled?

[mircea.markus]

good stuff!  ISPN-441 can also be used for that, as it performs a time-based check on the persistent connections by sending an hotrod-ping command to the HR server. This also solves the problem of server restarts (the connection will be replaced) and topology changes.