7.1.3.Final feedbacks

[silver06]

Hi guys,

I just gathered and build 7.1.3.Final which will the basis for EAP 6.0.1 apparently and I encounter 3 different issues.

As it's not officially releases I wonder if my errors are normal or not:

1)  when I start my host controller with about 10 servers I've got a XNIO error saying ""[Host Controller] 14:43:10,988 ERROR [org.jboss.remoting.remote.connection] (Remoting "nceaptint03:MANAGEMENT" read-1) JBREM000200: Remote connection failed: org.xnio.channels.ReadTimeoutException: Read timed out""

I didn't have that with either 7.1.1 or EAP 6.0.0

2) I can't declare more than 8 server groups in the domain.xml file. If I declare more server groups they don't show up in the admin console

3) When I deploy a WAR through the admin console I've got a marshal as xml error

[Host Controller] Caused by: org.jboss.as.controller.persistence.ConfigurationPersistenceException: JBAS014680: Failed to write configuration

[Host Controller]       at org.jboss.as.controller.persistence.AbstractConfigurationPersister.marshallAsXml(AbstractConfigurationPersister.java:122) [jboss-as-controller-7.1.3.Final.jar:7.1.3.Final]

[Host Controller]       at org.jboss.as.controller.persistence.FilePersistenceResource.<init>(FilePersistenceResource.java:59) [jboss-as-controller-7.1.3.Final.jar:7.1.3.Final]

[Host Controller]       ... 50 more

[Host Controller] Caused by: java.lang.IllegalArgumentException

[Host Controller]       at org.jboss.dmr.ModelValue.asPropertyList(ModelValue.java:100) [jboss-dmr-1.1.1.Final.jar:1.1.1.Final]

[Host Controller]       at org.jboss.dmr.ModelNode.asPropertyList(ModelNode.java:360) [jboss-dmr-1.1.1.Final.jar:1.1.1.Final]

[Host Controller]       at org.jboss.as.messaging.MessagingSubsystemParser.writeContent(MessagingSubsystemParser.java:1393)

[Host Controller]       at org.jboss.as.messaging.MessagingSubsystemParser.writeContent(MessagingSubsystemParser.java:123)

[Host Controller]       at org.jboss.as.host.controller.parsing.DomainXml.writeProfile(DomainXml.java:869) [jboss-as-host-controller-7.1.3.Final.jar:7.1.3.Final]

[Host Controller]       at org.jboss.as.host.controller.parsing.DomainXml.writeContent(DomainXml.java:169) [jboss-as-host-controller-7.1.3.Final.jar:7.1.3.Final]

[Host Controller]       at org.jboss.as.host.controller.parsing.DomainXml.writeContent(DomainXml.java:105) [jboss-as-host-controller-7.1.3.Final.jar:7.1.3.Final]

[Host Controller]       at org.jboss.staxmapper.XMLMapperImpl.doDeparse(XMLMapperImpl.java:88) [staxmapper-1.1.0.Final.jar:1.1.0.Final]

[Host Controller]       at org.jboss.staxmapper.XMLMapperImpl.deparseDocument(XMLMapperImpl.java:83) [staxmapper-1.1.0.Final.jar:1.1.0.Final]

[Host Controller]       at org.jboss.as.controller.persistence.AbstractConfigurationPersister.marshallAsXml(AbstractConfigurationPersister.java:116) [jboss-as-controller-7.1.3.Final.jar:7.1.3.Final]

Did I miss something about the 7.1.3.Final configuration ?

Thanks in advance for your information

Regards

[silver06]

I corrected the third error thanks to this JIRA. My messaging subsystem was empty which is not supported in 7.1.3.Final

https://issues.jboss.org/browse/AS7-5767

However I keep having the two first errors

The limited number of server groups (8 maximum) is only in the "Manage Deployments->Server groups" view

[ctomc]

Hi,

few more questions

1) are all 10 servers running same version of AS?

2) do you see servers in CLI?

--

tomaz

[silver06]

1) Yes, all these servers are declared in the same host.xml of a 7.1.3.Final host controller that tries to register them to a 7.1.3.Final domain controller (DC)

It's when they connect to the DC that it happens.

2) I see servers through the CLI or through the DC's admin console but they are not active

Note that if I decrease the number of servers to 4/5 it works perfectly I don't have any error.

I used two new properties as well that I was used to setup manually in 7.1.1 or EAP 6.0.0 (30000 ms by default)

-Djboss.host.server.connection.timeout=90000

-Djboss.host.domain.connection.timeout=90000

If I don't configure these system properties I've got other connection timeout as well

[jaikiran]

fabien tillay wrote:

Hi guys,

I just gathered and build 7.1.3.Final which will the basis for EAP 6.0.1 apparently and I encounter 3 different issues.

As it's not officially releases I wonder if my errors are normal or not:

1)  when I start my host controller with about 10 servers I've got a XNIO error saying ""[Host Controller] 14:43:10,988 ERROR [org.jboss.remoting.remote.connection] (Remoting "nceaptint03:MANAGEMENT" read-1) JBREM000200: Remote connection failed: org.xnio.channels.ReadTimeoutException: Read timed out""

Can you post or attach that entire exception stacktrace and other relevant configurations and logs, please?

fabien tillay wrote:

2) I can't declare more than 8 server groups in the domain.xml file. If I declare more server groups they don't show up in the admin console

What does the domain.xml look like? Can you attach it?

[silver06]

Here are the requested information

1)

console.log

host-controller.log

process-controller.log

2)

Domain.xml (about 50 server groups)

The result in the admin console

[harald.pehl]

I can confirm that the admin console only shows the first 8 server groups in "Runtime / Manage Deployments / Server Groups". I created the issue https://issues.jboss.org/browse/AS7-6208 to address this.

[silver06]

Ok Harald thanks I'm not crazy though ;-)

I hope this will be fixed for EAP 6.0.1 because it will be hard to work without that.

Regarding the XNIO issue do you have any clue ? If I decrease the number of servers in my host.xml to 4/5 it works perfectly. It seems that when we have a specific amount of servers, jboss.host.server.connection.timeout and jboss.host.domain.connection.timeout properties are useless and something else needs to be increased.

I'm gonna browse the source code to find where the problem is but I would appreciate any help

Thx

[emuckenhuber]

fabien tillay wrote:

Regarding the XNIO issue do you have any clue ? If I decrease the number of servers in my host.xml to 4/5 it works perfectly. It seems that when we have a specific amount of servers, jboss.host.server.connection.timeout and jboss.host.domain.connection.timeout properties are useless and something else needs to be increased.

From the logs it seems that the HC closes the connection if too many servers are getting registered concurrently. You could try to set auto-start=false and start them one-by-one for now. We may should switch to a more sequential startup process by default, but i need to do some testing before. Since i am not entirely sure that is what happens.

[ctomc]

From the logs it seems that the HC closes the connection if too many servers are getting registered concurrently.

Is it possible that there is some firewall or IDS running on network that is preventing too many concurrent connections?

[silver06]

Emanuel: We have more than 100 servers in total through different host controllers, having to start them (one by one for the time being) after the registration process is not appropriate to us.

It was working fine with 7.1.1 and EAP 6.0.0 so it seems to be a regression to me no ?

Tomaz: I don't have any firewall/ids between HC and DC.

[silver06]

Just to let you know that EAP 6.0.1 has been released this morning and we have exactly the same two big problems:

- Not possible to see more than 8 server groups in the "Manage Deployments->Server groups" view

JIRA : https://issues.jboss.org/browse/AS7-6208

- XNIO timeout message when we have more than 5 servers in a host controller.

Today we are thus blocked and we can't use this version unfortunately which is a pity because we are waiting for it since a long time now

Thanks in advance for you help

Regards

[jaikiran]

Fabien Tillay wrote:

Just to let you know that EAP 6.0.1 has been released this morning and we have exactly the same two big problems:

Please file a support case via the support channel. Since you are an EAP customer, depending on the fix, the chances are, you might get a patch for this without having to wait for a release.

[silver06]

jaikiran pai wrote:

Please file a support case via the support channel. Since you are an EAP customer, depending on the fix, the chances are, you might get a patch for this without having to wait for a release.

Yes this is why I did a few days ago for XNIO and today for the paging issue

Hope there will be something developed ;-)

[emuckenhuber]

Fabien Tillay wrote:

- XNIO timeout message when we have more than 5 servers in a host controller.

Today we are thus blocked and we can't use this version unfortunately which is a pity because we are waiting for it since a long time now

I tried to reproduce, but i could not get it to fail even having 20 local servers. Can you provide a bit more information about your enviroment you are running?

I think i saw at least the connection failures a long time ago while doing some testing, so i created a jira for tracking this: https://issues.jboss.org/browse/AS7-6230