2 DC in same cluster. Web(Admin) console only comes up for one DC.

[jeet.bavishi]

Hi,

We are using infinispan-9.1.1. Final.

Following is my structure:

- One DC in my PC.

- One DC in my colleague's PC.

I am starting both DC's by running domain.bat. And since both the DCs are in same network they are going to be in same cluster as both the DC has same configurations too.

WHY 2 DC?

We are creating a structure where we have 2 DCs actively running and both will have some host configured for them. In such cases where one of the DCs go down we have mentioned other DC as backup. So lets say, if DC in my PC goes down then all the hosts which were earlier connected to this DC will now try and connect to other DC in my colleague's PC so we are providing fail-over for the DC.

Here the issue is that only one web(Admin) console comes up(For the DC that starts first) and when we start second DC, web console for the second DC does not come up however second DC starts successfully as I can see in the logs the cluster view and also re-balancing of caches.  

When I inspected the issue for second DC in chrome,it says following:

{

    "outcome" : "failed",

    "failure-description" : "WFLYCTL0030: No resource definition is registered for address [\n    (\"host\" => \"JBOSS1IND1\"),\n    (\"server\" => \"server-one\"),\n    (\"subsystem\" => \"datagrid-infinispan\"),\n    (\"cache-container\" => \"clustered\")\n]",

    "rolled-back" : true

}

Following is the request payload which is giving above 500 internal server error:

1. 1. address:["host", "JBOSS1IND1", "server", "server-one", "subsystem", "datagrid-infinispan", "cache-container",…]
      1. 0:"host"
      2. 1:"JBOSS1IND1"
      3. 2:"server"
      4. 3:"server-one"
      5. 4:"subsystem"
      6. 5:"datagrid-infinispan"
      7. 6:"cache-container"
      8. 7:"clustered"
   2. name:"sites-online"
   3. operation:"read-attribute"

Can you please help us here? We are stuck and need immediate help. And also, can you help in suggesting how should we solve the issue of DC fail over if above mentioned way is not correct?

I am attaching my domain.xml here which is same for both the DCs.

Please reply ASAP

[jeet.bavishi]

Hi galder.zamarreno and nadirx,

Can you help me with this?

[nadirx]

You cannot run two DCs like this.

Please read EAP's docs on DC failover

https://access.redhat.com/documentation/en-us/jboss_enterprise_application_platform/6.3/html/administration_and_configur…

Proper automatic promotion of the DC will be implemented by [WFCORE-338] Auto-promotion of slave HC to master - JBoss Issue Tracker

[galder.zamarreno]

Sorry but don't understand what you are asking...

Maybe you can provide some test/code and some logs of what you're doing?

If two nodes the same cluster, that's just a 2 node cluster. A remote client, using Java Hot Rod client, will be available to failover transparently.

The concept of DC or data centres, we use for talking about separate clusters (e.g. enabling cross-site replication that talk to each other, but I don't see any of your configuration really using this.

[jeet.bavishi]

Hi galder.zamarreno and nadirx,

Thanks for the quick reply. Here's our structure in detail.

We have two pairs. Each pair has one domain controller and a host controller. This means each pair will have 2 host(Because each domain controller will also start a local host) and one Domain controller so in essence all 4 host and 2 domain controller are in one cluster.  In case of Domain controller failure, the host will connect to another domain controller in same cluster.

For each remote host, I have written domain controller configuration as follows:

<remote protocol="remote" host="${jboss.domain.master.address:X}" port="${jboss.domain.master.port:9999}" security-realm="ManagementRealm">

                 <discovery-options>

                         <static-discovery name="newdiscovery" protocol="${jboss.domain.master.protocol:remote}" host="${jboss.domain.master.address:Y}" port="${jboss.domain.master.port:9999}"/>

                 </discovery-options>

nadirx: is this the same way you are asking to do domain controller fail over?

My issue is that web(Admin) console only comes up for one domain controller and not for another. Can you give me any suggestions?

[galder.zamarreno]

What is exactly the web(Admin) console you're talking about? What do you mean that it doesn't come up? You should check the links Tristan pointed out.

[jafarre.viewnext]

Hello,

I am having the same problem, but in my case I have only one Domain Controller.

It is a regular installation of Infinispan in domain mode.

When I connect through with my browser to WildFly console at http://myHost:9990/console I am asked to log in.

After succesful login, I get a totally blank page.

With the browser developer tools, I can see many POSTs succeeding, but later some of them fail with error 500 and an equivalent failure description in the payload:

{

    "outcome" : "failed",

    "failure-description" : "WFLYCTL0030: No resource definition is registered for address [\n    (\"host\" => \"infinispan-HS2-1\"),\n    (\"server\" => \"infinispan-HS2-1\"),\n    (\"subsystem\" => \"datagrid-infinispan\"),\n    (\"cache-container\" => \"clustered\")\n]",

    "rolled-back" : true

}

This is with Firefox, Chrome and IE, so it does not seem to be browser related.

Server versions:

Infinispan 9.4.15

OpenJDK 1.8.0_212

CentOS 7

Client versions:

Windows 10

Firefox 68.0

Chrome 75.0.3770.100

Internet Explorer 11

More information:

Appart of this, the infinispan server is working correctly.

I also have a very similar installation of Keycloak 6.0.1 and WildFly console works perfectly.

Keycloak packages WildFly 16 and Infinispan, WildFly 14.

Thank you very much for your help!