13 Replies Latest reply on Nov 4, 2012 1:47 PM by jpalka

    Modeshape 3 and jgroups XML configuration

    jpalka

      All,

       

      I am trying to configure JGroups for clustering in configRepository.json using channelConfiguration. How should I include my JGroups XML config in JSON file?

       

      Regards,

      Jarek

        • 1. Re: Modeshape 3 and jgroups XML configuration
          hchiorean

          Hi Jarek,

           

          Unfortunatelly the current code assumes that the value of the "channelConfiguration" key is the actual XML configuration string. In other words, smth. like this:

           

          "channelConfiguration" : "<config>                       <UDP>                             mcast_addr=..."

           

          Since this is inside a JSON formatted file, it's less than..ideal. Could you please open an issue against 3.1 so that we also support an external (classpath) configuration file ?

           

          Thanks

          • 2. Re: Modeshape 3 and jgroups XML configuration
            jpalka

            Loading JGroups config from classpath solves the problem, butI wonder if we can also have JGroups config embbeded in JSON, simply convert XML into JSON fragment, this way we will have less files to worry about.

            • 3. Re: Modeshape 3 and jgroups XML configuration
              hchiorean

              The easiest way for us to do that, would be if JGroups had a ProtocolStackConfigurator implementation (similar to XmlConfigurator) which supported JSON. I don't know if they have such an option atm.

              • 4. Re: Modeshape 3 and jgroups XML configuration
                jpalka

                Thanks for information, I was able to configure clustering in Modeshape JSON file, but it doesn't work. I have JBoss with Modeshape and Tomcat with WebDAV Modeshape servlet. They both see each other, but once I add node in JBoss it is not visible in WebDAV servlet, until restart of WebDAV Tomcat. Do I have to enable clustering in infinispan config as well? Should I use the same channel configuration and same cluster name to get changes propagated over Inginispan grid?

                 

                Regards,

                Jarek

                • 5. Re: Modeshape 3 and jgroups XML configuration
                  rhauch

                  Can you explain why you're using separate JBoss AS and Tomcat instances?

                   

                  Which JBoss AS version are you using? And is there a reason why you're not deploying ModeShape's WebDAV WAR file to that JBoss AS instance? If you're using JBoss AS7 and install ModeShape as a subsystem (with our kit), then ModeShape, the REST service, and the WebDAV service will all be installed and ready to be used. And you'll only need to cluster AS7 if you want multiple server instances; a single process will contain all of the ModeShape components.

                   

                  Anyway, to answer your question about clustering: there are actually two ways of setting up a small cluster with Tomcat and JBoss AS.

                   

                  1. ModeShape is running in both and should be clustered via JGroups, and Infinispan is running in both and should be clustered (actually, I'd suggest configuring the cache as replicated).
                  2. ModeShape is running in both and should be clustered via JGroups, but the Infinispan cache in Tomcat can use Infinispan's remote client mechansim to talk remotely to the Infinispan instance running in JBoss AS.

                   

                  Hope that helps.

                  • 6. Re: Modeshape 3 and jgroups XML configuration
                    jpalka

                    Randall,

                     

                    I am running JBoss 4.2.3 (yeah I know, don't laugh). So I cannot benefit from all beauty of JBoss 7. We are running WebDAV on Tomcat because we expect massive load from client side (mostly read operations), and we want to separatethe traffic. So JBoss (actually back office) will be exposed to smaller group of users which will manage repository content and Tomcat through WebDAV will be exposed to a massive traffic from the outside world.

                     

                    I will go with first option and will check how it works for us.

                     

                    Regards,

                    Jarek

                    • 7. Re: Modeshape 3 and jgroups XML configuration
                      jpalka

                      One addtional thing, we would like to use NFS for binary storage and Infinispan storage. Do you expect any problems with this setup?

                       

                      Regards,

                      Jarek

                      • 8. Re: Modeshape 3 and jgroups XML configuration
                        rhauch

                        One addtional thing, we would like to use NFS for binary storage and Infinispan storage. Do you expect any problems with this setup?

                        A couple of things to watch for:

                         

                        • Not all NFS configurations support file locks, which we use quite a bit inside our filesystem-based binary store. A simple test should verify it works correctly on your NFS mount.
                        • Some NSF configurations optimize for reads, which means that writes might take some time to propagate. Again, test how this works on your NFS mount.
                        • 9. Re: Modeshape 3 and jgroups XML configuration
                          jpalka

                          How stable is JDBM store implementation, do you have any experience?

                           

                          Regards,

                          Jarek

                          • 10. Re: Modeshape 3 and jgroups XML configuration
                            rhauch

                            I have minimal experience with it. Last time I checked, Infinispan was using the 1.x version of JDBM, which has since been rewritten (the latest is JDBM4) to be significantly faster and concurrent. I was hoping to have time to add a cache store (based on the JDBM cache store implementation), but haven't yet had time. I think there's a lot of promise there.

                             

                            If you venture to give it a try, please share your experiences.

                            • 11. Re: Modeshape 3 and jgroups XML configuration
                              jpalka

                              Randall,

                               

                              I am playing with JDBM4 at the moment, but I see strange exception with clustering enabled in Modeshape and Infinispan. I get EOFException thrown when I am adding new node: I have attached my configuration and logs with exception.

                              • 12. Re: Modeshape 3 and jgroups XML configuration
                                jpalka

                                Randall,

                                 

                                It looks like there is a problem in our session and repository objects handling, I will try to run this configuration in a isolated environment, outside of containers world.

                                 

                                Regards,

                                Jarek

                                • 13. Re: Modeshape 3 and jgroups XML configuration
                                  jpalka

                                  I tried this setting outside containers, and still see the same exception:

                                   

                                  19:22:50,924 ERROR InvocationContextInterceptor:141 - ISPN000136: Execution error

                                  org.infinispan.CacheException: Problems invoking command.

                                            at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:206)

                                            at org.jgroups.blocks.RequestCorrelator.handleRequest(RequestCorrelator.java:456)

                                            at org.jgroups.blocks.RequestCorrelator.receiveMessage(RequestCorrelator.java:363)

                                            at org.jgroups.blocks.RequestCorrelator.receive(RequestCorrelator.java:238)

                                            at org.jgroups.blocks.MessageDispatcher$ProtocolAdapter.up(MessageDispatcher.java:543)

                                            at org.jgroups.JChannel.up(JChannel.java:716)

                                            at org.jgroups.stack.ProtocolStack.up(ProtocolStack.java:1026)

                                            at org.jgroups.protocols.pbcast.STATE_TRANSFER.up(STATE_TRANSFER.java:178)

                                            at org.jgroups.protocols.pbcast.GMS.up(GMS.java:881)

                                            at org.jgroups.protocols.FRAG.up(FRAG.java:165)

                                            at org.jgroups.protocols.pbcast.STABLE.up(STABLE.java:244)

                                            at org.jgroups.protocols.UNICAST.handleDataReceived(UNICAST.java:656)

                                            at org.jgroups.protocols.UNICAST.up(UNICAST.java:317)

                                            at org.jgroups.protocols.pbcast.NAKACK.up(NAKACK.java:595)

                                            at org.jgroups.protocols.VERIFY_SUSPECT.up(VERIFY_SUSPECT.java:140)

                                            at org.jgroups.protocols.FD.up(FD.java:273)

                                            at org.jgroups.protocols.MERGE2.up(MERGE2.java:205)

                                            at org.jgroups.protocols.Discovery.up(Discovery.java:355)

                                            at org.jgroups.protocols.TP.passMessageUp(TP.java:1174)

                                            at org.jgroups.protocols.TP$IncomingPacket.handleMyMessage(TP.java:1722)

                                            at org.jgroups.protocols.TP$IncomingPacket.run(TP.java:1704)

                                            at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(Unknown Source)

                                            at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)

                                            at java.lang.Thread.run(Unknown Source)

                                  Caused by: java.io.EOFException

                                            at org.jboss.marshalling.UTFUtils.readUTFBytes(UTFUtils.java:157)

                                            at org.jboss.marshalling.river.BlockUnmarshaller.readUTF(BlockUnmarshaller.java:334)

                                            at org.infinispan.schematic.internal.document.Paths$Externalizer.readObject(Paths.java:477)

                                            at org.infinispan.schematic.internal.document.Paths$Externalizer.readObject(Paths.java:453)

                                            at org.infinispan.marshall.jboss.JBossExternalizerAdapter.createExternal(JBossExternalizerAdapter.java:48)

                                            at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1273)

                                            at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:272)

                                            at org.jboss.marshalling.river.BlockUnmarshaller.readObject(BlockUnmarshaller.java:153)

                                            at org.jboss.marshalling.river.BlockUnmarshaller.readObject(BlockUnmarshaller.java:139)

                                            at org.infinispan.schematic.internal.delta.AddValueOperation$Externalizer.readObject(AddValueOperation.java:121)

                                            at org.infinispan.schematic.internal.delta.AddValueOperation$Externalizer.readObject(AddValueOperation.java:108)

                                            at org.infinispan.marshall.jboss.JBossExternalizerAdapter.createExternal(JBossExternalizerAdapter.java:48)

                                            at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1273)

                                            at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:272)

                                            at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)

                                            at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37)

                                            at org.infinispan.marshall.exts.LinkedListExternalizer.readObject(LinkedListExternalizer.java:57)

                                            at org.infinispan.marshall.exts.LinkedListExternalizer.readObject(LinkedListExternalizer.java:45)

                                            at org.infinispan.marshall.jboss.ExternalizerTable$ExternalizerAdapter.readObject(ExternalizerTable.java:401)

                                            at org.infinispan.marshall.jboss.ExternalizerTable.readObject(ExternalizerTable.java:290)

                                            at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:351)

                                            at org.jboss.marshalling.river.BlockUnmarshaller.readObject(BlockUnmarshaller.java:153)

                                            at org.jboss.marshalling.river.BlockUnmarshaller.readObject(BlockUnmarshaller.java:139)

                                            at org.infinispan.schematic.internal.SchematicEntryDelta$Externalizer.readObject(SchematicEntryDelta.java:101)

                                            at org.infinispan.schematic.internal.SchematicEntryDelta$Externalizer.readObject(SchematicEntryDelta.java:86)

                                            at org.infinispan.marshall.jboss.JBossExternalizerAdapter.createExternal(JBossExternalizerAdapter.java:48)

                                            at org.jboss.marshalling.river.RiverUnmarshaller.doReadNewObject(RiverUnmarshaller.java:1273)

                                            at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:272)

                                            at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)

                                            at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37)

                                            at org.infinispan.marshall.exts.ReplicableCommandExternalizer.readParameters(ReplicableCommandExternalizer.java:118)

                                            at org.infinispan.marshall.exts.ReplicableCommandExternalizer.readObject(ReplicableCommandExternalizer.java:106)

                                            at org.infinispan.marshall.exts.ReplicableCommandExternalizer.readObject(ReplicableCommandExternalizer.java:57)

                                            at org.infinispan.marshall.jboss.ExternalizerTable$ExternalizerAdapter.readObject(ExternalizerTable.java:401)

                                            at org.infinispan.marshall.jboss.ExternalizerTable.readObject(ExternalizerTable.java:290)

                                            at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:351)

                                            at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)

                                            at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37)

                                            at org.infinispan.marshall.exts.ReplicableCommandExternalizer.readParameters(ReplicableCommandExternalizer.java:118)

                                            at org.infinispan.marshall.exts.CacheRpcCommandExternalizer.readObject(CacheRpcCommandExternalizer.java:164)

                                            at org.infinispan.marshall.exts.CacheRpcCommandExternalizer.readObject(CacheRpcCommandExternalizer.java:68)

                                            at org.infinispan.marshall.jboss.ExternalizerTable$ExternalizerAdapter.readObject(ExternalizerTable.java:401)

                                            at org.infinispan.marshall.jboss.ExternalizerTable.readObject(ExternalizerTable.java:290)

                                            at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:351)

                                            at org.jboss.marshalling.river.RiverUnmarshaller.doReadObject(RiverUnmarshaller.java:209)

                                            at org.jboss.marshalling.AbstractObjectInput.readObject(AbstractObjectInput.java:37)

                                            at org.infinispan.marshall.jboss.AbstractJBossMarshaller.objectFromObjectStream(AbstractJBossMarshaller.java:148)

                                            at org.infinispan.marshall.VersionAwareMarshaller.objectFromByteBuffer(VersionAwareMarshaller.java:110)

                                            at org.infinispan.marshall.AbstractDelegatingMarshaller.objectFromByteBuffer(AbstractDelegatingMarshaller.java:84)

                                            at org.infinispan.remoting.transport.jgroups.MarshallerAdapter.objectFromBuffer(MarshallerAdapter.java:50)

                                            at org.infinispan.remoting.transport.jgroups.CommandAwareRpcDispatcher.handle(CommandAwareRpcDispatcher.java:196)

                                            ... 23 more

                                  Caused by: an exception which occurred:

                                            in object of type org.infinispan.schematic.internal.document.Paths$MultiSegmentPath