How to migrate an application at runtime

[giacomo.genovese]

Hi all,

I doing some test running one instance of Capedwarf with several applications running. Each Application is a separated web service.

At certain point, I would like to be able to migrate one of those applications from a Capedwarf to another one at run-time keeping its state and datastore.

I didn't find any useful documentation.

Do you have some suggestion for me?

Thanks.

Giacomo

[alesj]

We have this tool:

* capedwarf-blue/BulkLoader.java at master · capedwarf/capedwarf-blue · GitHub

[giacomo.genovese]

Thanks.

Some instruction about how to use it?

[alesj]

https://github.com/capedwarf/capedwarf-blue/blob/master/tools/src/test/java/org/jboss/test/capedwarf/tools/test/BulkLoaderTest.java

[giacomo.genovese]

I'm sorry Ales but I need more information about how to use/run this tool. I'm new in this Jboss environment.

Only code there.

I found some documentation about how migration is done in a AppScale environment but nothing for Capedwarf.

Thanks.

[alesj]

Try to see how the test does it -- it should not be too hard.

[giacomo.genovese]

Hi Ales,

When I try to run BulkLoader, I type :

$cd capedwarf-blue-master/tools/src/main/java/org/jboss/capedwarf/tools/

//to decompile

$javac BulkLoader.java

and I get error:

BulkLoader.java:32: error: package org.apache.http does not exist

import org.apache.http.HttpResponse;

                      ^

BulkLoader.java:33: error: package org.apache.http.client does not exist

import org.apache.http.client.HttpClient;

                             ^

BulkLoader.java:34: error: package org.apache.http.client.methods does not exist

import org.apache.http.client.methods.HttpGet;

                                     ^

BulkLoader.java:35: error: package org.apache.http.client.methods does not exist

import org.apache.http.client.methods.HttpPut;

                                     ^

BulkLoader.java:36: error: package org.apache.http.entity does not exist

import org.apache.http.entity.ContentType;

                             ^

BulkLoader.java:37: error: package org.apache.http.entity does not exist

import org.apache.http.entity.FileEntity;

                             ^

BulkLoader.java:38: error: package org.apache.http.impl.client does not exist

import org.apache.http.impl.client.CloseableHttpClient;

                                  ^

BulkLoader.java:39: error: package org.apache.http.impl.client does not exist

import org.apache.http.impl.client.HttpClients;

                                  ^

BulkLoader.java:72: error: cannot find symbol

    private static void doUpload(Arguments args) {

                                 ^

  symbol:   class Arguments

  location: class BulkLoader

BulkLoader.java:103: error: cannot find symbol

    private static void sendPacket(UploadPacket packet, HttpClient client, String url) throws IOException {

                                   ^

  symbol:   class UploadPacket

  location: class BulkLoader

BulkLoader.java:103: error: cannot find symbol

    private static void sendPacket(UploadPacket packet, HttpClient client, String url) throws IOException {

                                                        ^

  symbol:   class HttpClient

  location: class BulkLoader

BulkLoader.java:112: error: cannot find symbol

    private static void doDump(Arguments args) {

                               ^

  symbol:   class Arguments

  location: class BulkLoader

BulkLoader.java:165: error: cannot find symbol

    private static Integer getPacketSize(Arguments args) {

                                         ^

  symbol:   class Arguments

  location: class BulkLoader

BulkLoader.java:169: error: cannot find symbol

    private static String getFilename(Arguments args) {

                                      ^

  symbol:   class Arguments

  location: class BulkLoader

BulkLoader.java:173: error: cannot find symbol

    private static String getUrl(Arguments args) {

                                 ^

  symbol:   class Arguments

  location: class BulkLoader

BulkLoader.java:51: error: cannot find symbol

        Arguments arguments = new Arguments(args);

        ^

  symbol:   class Arguments

  location: class BulkLoader

BulkLoader.java:51: error: cannot find symbol

        Arguments arguments = new Arguments(args);

                                  ^

  symbol:   class Arguments

  location: class BulkLoader

BulkLoader.java:78: error: cannot find symbol

        try (DumpFileFacade dumpFileFacade = new DumpFileFacade(new File(filename))) {

             ^

  symbol:   class DumpFileFacade

  location: class BulkLoader

BulkLoader.java:78: error: cannot find symbol

        try (DumpFileFacade dumpFileFacade = new DumpFileFacade(new File(filename))) {

                                                 ^

  symbol:   class DumpFileFacade

  location: class BulkLoader

BulkLoader.java:81: error: cannot find symbol

            try (CloseableHttpClient client = HttpClients.createDefault()) {

                 ^

  symbol:   class CloseableHttpClient

  location: class BulkLoader

BulkLoader.java:81: error: cannot find symbol

            try (CloseableHttpClient client = HttpClients.createDefault()) {

                                              ^

  symbol:   variable HttpClients

  location: class BulkLoader

BulkLoader.java:83: error: cannot find symbol

                UploadPacket packet = new UploadPacket(partFile);

                ^

  symbol:   class UploadPacket

  location: class BulkLoader

BulkLoader.java:83: error: cannot find symbol

                UploadPacket packet = new UploadPacket(partFile);

                                          ^

  symbol:   class UploadPacket

  location: class BulkLoader

BulkLoader.java:92: error: cannot find symbol

                            packet = new UploadPacket(partFile);

                                         ^

  symbol:   class UploadPacket

  location: class BulkLoader

BulkLoader.java:104: error: cannot find symbol

        HttpPut put = new HttpPut(url);

        ^

  symbol:   class HttpPut

  location: class BulkLoader

BulkLoader.java:104: error: cannot find symbol

        HttpPut put = new HttpPut(url);

                          ^

  symbol:   class HttpPut

  location: class BulkLoader

BulkLoader.java:106: error: cannot find symbol

        put.setEntity(new FileEntity(packet.getFile(), ContentType.create("application/capedwarf-data")));

                          ^

  symbol:   class FileEntity

  location: class BulkLoader

BulkLoader.java:106: error: cannot find symbol

        put.setEntity(new FileEntity(packet.getFile(), ContentType.create("application/capedwarf-data")));

                                                       ^

  symbol:   variable ContentType

  location: class BulkLoader

BulkLoader.java:107: error: cannot find symbol

        HttpResponse response = client.execute(put);

        ^

  symbol:   class HttpResponse

  location: class BulkLoader

BulkLoader.java:125: error: cannot find symbol

        try (DumpFileFacade dumpFileFacade = new DumpFileFacade(file)) {

             ^

  symbol:   class DumpFileFacade

  location: class BulkLoader

BulkLoader.java:125: error: cannot find symbol

        try (DumpFileFacade dumpFileFacade = new DumpFileFacade(file)) {

                                                 ^

  symbol:   class DumpFileFacade

  location: class BulkLoader

BulkLoader.java:126: error: cannot find symbol

            try (CloseableHttpClient client = HttpClients.createDefault()) {

                 ^

  symbol:   class CloseableHttpClient

  location: class BulkLoader

BulkLoader.java:126: error: cannot find symbol

            try (CloseableHttpClient client = HttpClients.createDefault()) {

I tried the same for BulkLoaderTest, but I suppose that "javac" and "java" are not the right commands to run the scripts.

Could you suggest the right way to do it?

Thanks a lot.

[alesj]

Try running the test from an IDE.

Then simply try to mock / adjust what the test does.

And probably running things from IDE is the easiest, as it will already setup the classpath for you (if you get your Maven config right).

[giacomo.genovese]

Hi all,

I'm investigating on Wildfly HA Clustering in order to migrate the entire PaaS Capedwarf Instance.

I created two Standalone Instances, into two different machines, deploying the same applications and enabling Clustering.

Once the cluster is on, I executed some read and write operations on the Instance 1. Then, I decided to switch off such instance to force migration of services to Instance 2 and everything seems to work well. The Instance 2 maintains an updated status of all the operations done.

I would ask you:

• May I consider this operation like a migration of a CapeDwarf Instance?
• Do I missing something that make loss data in the Instance 2?

Thanks a lot for your support.

Regards,

Giacomo.

[alesj]

Good question.

Imo, it should work.

Let me check with Infinispan devs.

[sannegrinovero]

Hi Giacomo, I'm not a CapeDwarf expert, but have good experience with Infinispan.

Yes the approach you suggest sounds good: Infinispan will immediately start transferring a full replica of all state to the other node as soon as they are connected. To avoid data loss, just make sure that the state transfer operation is complete before you shut down or disconnect the first node. I usually check on state transfer by keeping an eye on its logs, but with more recent versions there are several better options; for example the Health API can be accessed in various ways: http://blog.infinispan.org/2017/03/checking-infinispan-cluster-health-and.html That would allow you to make a 100% replica of the Infinispan stored data.

There's another problem you might have though: Infinispan is a key/value store, so if the Key(s) used by CapeDwarf include any application-specific metadata, one would need to make sure these didn't change, or the entries will still exist in practice, but the application layer wouldn't be able to retrieve them from the distributed map. I am not sure if this could be a problem in practice, one would need to have a look into the CD code, or test the process.

HTH

[giacomo.genovese]

Thanks a lot for your comment.

About this issue:

There's another problem you might have though: Infinispan is a key/value store, so if the Key(s) used by CapeDwarf include any application-specific metadata, one would need to make sure these didn't change, or the entries will still exist in practice, but the application layer wouldn't be able to retrieve them from the distributed map. I am not sure if this could be a problem in practice, one would need to have a look into the CD code, or test the process.

alesj, please, do you have any answer?

Thanks a lot for your contribution!

[alesj]

Like I said, imo, it should work.

Try it, and provide feedback in the case of any failures.

[giacomo.genovese]

Hi Sanne,

I'm trying to find the best way to check the Cache status before shutdown the old instance.

The goal is to speed up the procedure and free the space in the VM of the old machine as soon as possible.

I found this link too.

My problem is that I don't have CacheContainerHealth object in my Mbeans view in CapeDwarf.

It should be an integrated module of Infinispan, isn't it? Can I add it?

Thanks a lot.