7 Replies Latest reply on Nov 30, 2009 8:42 AM by Manik Surtani

    does infnispan support distributed processing?

    Elhanan Maayan Novice

      hi..

      let me explain ,we need a cache system, that won't replicate nodes, but allow a distributed processing of separate data between systems, and return results back to the main node.

      the system should very fast and lightweight.

        • 2. Re: does infnispan support distributed processing?
          Elhanan Maayan Novice

          thanks :) how long do you think what would take, i have doubts my project admin would accept it, he is so worried about performance he even considered doing it in GCC
          but i wonder if we could join forces on this, if he accepts this.

          another manager asks, what makes infinispan so different the just another cache system? (which is not so mature as other cache systems?)

          for example can microfocus cobol system communicate it and start and oracle xa transaction that would end in java server? (i know about tuxido, i'm just looking at alternatives)

          • 3. Re: does infnispan support distributed processing?
            Manik Surtani Master

             

            "Elhanan" wrote:
            thanks :) how long do you think what would take, i have doubts my project admin would accept it, he is so worried about performance he even considered doing it in GCC
            but i wonder if we could join forces on this, if he accepts this.


            My plan was to have this done by mid-next year, but it actually should be pretty simple to implement so if you do feel like helping out we could include it in an earlier release.

            In terms of performance, the network will always be the bottleneck. I can't see how implementing it in C instead of Java would help at all. ;)

            "Elhanan" wrote:
            another manager asks, what makes infinispan so different the just another cache system? (which is not so mature as other cache systems?)

            Scope. We're looking at the 1000 node use case, no other open source cache systems deal with this level of scope. A couple of commercial ones do, but these are expensive. Also, scalability. We're targeting linear scalability as you add nodes; again not something others do. And, of course, in feature set and API richness, we're more than just a basic cache - as you can see, with distributed execution, querying, JPA-like APIs, we're not just a simple cache.

            "Elhanan" wrote:
            for example can microfocus cobol system communicate it and start and oracle xa transaction that would end in java server? (i know about tuxido, i'm just looking at alternatives)

            Perhaps. At the moment the only integration we have for non-Java environments is the REST API. Soon, we will have a more powerful client/server wire protocol that you can write hooks into from most systems.

            HTH,
            Manik

            • 4. Re: does infnispan support distributed processing?
              Elhanan Maayan Novice

              regarding c implementation

              becouse we don't know if we could take an object in java with about 25 long fields (200 bytes in size) , with around 2.5 million of these objects ( thus reaching 500MB in memroy ) and compress it in 3 seconds.

              • 5. Re: does infnispan support distributed processing?
                Manik Surtani Master

                Compression of byte streams? We can handle this. JGroups - the network stack we use - can use GZIP compression of streams on the fly, and is pretty efficient.

                • 6. Re: does infnispan support distributed processing?
                  Elhanan Maayan Novice

                  you mean jgroups can compress this amount of data in 3 seconds, on which hardware are we talking about?

                  • 7. Re: does infnispan support distributed processing?
                    Manik Surtani Master

                    JGroups has stream compression. Performance will vary, as you say, based on hardware, concurrent processes, etc. I'm just saying it's worthwhile benchmarking. Or even writing a JGroups protocol to handle compression/decompression using JNI and calls to some C code, if you feel it performs better. Either way, if compression performance is your argument to use a C based implementation, then use the C based implementation for compression *only*, and not everything. :)