java.lang.VerifyError when using byteman in JBOSS AS7

[aceyin]

Hi folks,

i am trying to use byteman in JBOSS AS7, but i always get the "java.lang.VerifyError" when i using "bmsubmit.sh" to submit a script.

Below is the full error message:

bmsubmit.sh -l aaa.btm

install rule aaa

VerifyError during retransformation : some rules may not have been correctly injected or uninjected!

java.lang.VerifyError

at sun.instrument.InstrumentationImpl.retransformClasses0(Native Method)

at sun.instrument.InstrumentationImpl.retransformClasses(InstrumentationImpl.java:144)

at org.jboss.byteman.agent.Retransformer.installScript(Retransformer.java:136)

at org.jboss.byteman.agent.TransformListener.handleScripts(TransformListener.java:337)

at org.jboss.byteman.agent.TransformListener.loadScripts(TransformListener.java:260)

at org.jboss.byteman.agent.TransformListener.handleConnection(TransformListener.java:213)

at org.jboss.byteman.agent.TransformListener.run(TransformListener.java:146)

Below is the script file i am trying to submit:

RULE aaa

CLASS com.abc.ddm.hwa.monitoring.handler.ApplicationMonitorHandler

METHOD getTime

IF true

DO System.out.println("AAAA")

ENDRULE

Below is the command i used to install the agent into JBOSS JVM:

bminstall.sh -b -Dorg.jboss.byteman.transform.all /opt/abc/local/jboss/jboss-modules.jar

I also tried to use the PID instead of jar file , but also failed.

Below is the command i used to check the script:

bmcheck.sh  -cp /opt/abc/xxx/manager/webapps/monitor.war/WEB-INF/lib/ddm.jar aaa.btm

the check result indicates the script is OK:

Checking rule aaa against class com.abc.ddm.hwa.monitoring.handler.ApplicationMonitorHandler

Parsed rule "aaa" for class com.abc.ddm.hwa.monitoring.handler.ApplicationMonitorHandler

Type checked rule "aaa"

TestScript: no errors

But when i use bmsubmit to submit the script, the exception in the head of this post occurs.

And when i tried to submit below script, it successed and the trace message can printed successfully:

RULE trace thread start

CLASS java.lang.Thread

METHOD start()

IF true

DO traceln("*** start for thread: "+ $0.getName())

ENDRULE

does someone has idea about this issue?

Below is the environments used in my test:

JBOSS :

jboss-as-7.1.3.Final

JAVA :

java version "1.7.0_04"

Java(TM) SE Runtime Environment (build 1.7.0_04-b20)

Java HotSpot(TM) 64-Bit Server VM (build 23.0-b21, mixed mode)

OS version:

Linux 2.6.32-71.el6.x86_64

Byteman Version:

byteman-2.1.3

The source code fragment of com.abc.ddm.hwa.monitoring.handler.ApplicationMonitorHandler :

package com.abc.ddm.hwa.monitoring.handler;

import java.text.SimpleDateFormat;

import java.util.Calendar;

import java.util.HashMap;

import java.util.List;

import java.util.Map;

import org.springframework.web.client.RestClientException;

import org.springframework.web.client.RestTemplate;

import com.abc.ddm.hwa.monitoring.hwa.HwaClusterInfo;

import com.abc.ddm.hwa.monitoring.hwa.HwaMonitorConfig;

import com.abc.ddm.hwa.monitoring.hwa.MonitorApis;

import com.abc.ddm.hwa.monitoring.servlet.FilterData;

import com.abc.ddm.hwa.monitoring.servlet.HostedWebAppInfo;

import com.abc.ddm.hwa.monitoring.servlet.HostedWebAppInfoXmlHelper;

import com.abc.ddm.hwa.monitoring.servlet.HttpClient;

import com.abc.ddm.hwa.monitoring.servlet.MonitorDataSummaryHelper;

public class ApplicationMonitorHandler extends AbstractHandler {

     ......

    public String getTime() {

        SimpleDateFormat sdf = new SimpleDateFormat(TIME_FORMAT);

        return sdf.format(Calendar.getInstance().getTime());

    }

    ......

}

[adinn]

Hi Ace,

Sorry for not replying sooner. I don't understand why you are seeing the results you get here. The fact that the type checker accepts your rule while the runtime rejects it is hard to understand. The type checker does almost exactly what Byteman does at runtime i.e. it injects the rule into the target class using a ClassTransformer. The only significant difference is that it does not use the same class loader setup as you will be using in your application -- no war file and no module loader. I suspect the verify error is arising because of a linking error but I don't really see ow that can happen. The transformed code will only refer to the Byteman classes and they should be visible to your war file.

Have you modified your standalone.conf or reset JAVA_OPTS when you start JBoss AS? The visibility of Byteman classes is determined by a setting in standalone.conf which ensures that the JBoss module system delegates loads of classes in the Byteman package space. If you look at the default standalone.conf you should see -Djboss.modules.system.pkgs=org.jboss.byteman being added to JAVA_OPTS. Perhaps you could check that it is still set when you start your JBoss AS instance.

[pasturel]

Hi Ace,

does the rule run with the classic -javaagent:<to Byteman.jar> ...  set in standalone.conf file of JBoss?

If not, perhaps the -Djboss.modules.pkg is not set in  standalone.conf  of JBoss, as Andrew Dinn says.

[aceyin]

Hi Andrew,

thank you very much for your reply, i will try your suggestion later.

i have tried to deploy the rule to the JBoss server running on my laptop (which is a simple and clean JBoss instance , the problematic one is also clean, but the configuration is much more complex than the one on my laptop) , the rule can run successfully on this JBoss server.

so i think the problem should be relevant to my JBoss configuration.

[adinn]

Ace Yin said:

i have tried to deploy the rule to the JBoss server running on my laptop (which is a simple and clean JBoss instance , the problematic one is also clean, but the configuration is much more complex than the one on my laptop) , the rule can run successfully on this JBoss server.

so i think the problem should be relevant to my JBoss configuration.

Thanks for replying, Ace.I'm glad to hear that it is working with a clean JBoss AS instance. It does seem that the problem is in your configuration. Good luck finding a fix :-)

[aceyin]

Hi Jean ,

i will try to use the "-javaagent" option

thanks for your tips

[aceyin]

hi Andrew,

thanks for your previous answer, it solved my issue

the -Djboss.modules.system.pkgs=org.jboss.byteman is the key parameter in my case