Loading Byteman Rules dynamically

[gaurav6281]

Hi,

One of my Java process (A) is running on box A . Another process (B) is running from box B.

Process A executes multiple steps one by one , including loading byteman agent dynamically to Process B. Second step in Process A is submitting rules dynamically to Process B

below are the steps which will be followed from Process A(from java class which I have yet to write using ProcessBuilder , currently I am just doing POC)

(1)  SET BYTEMAN_HOME

C:\Users\N662245>SET BYTEMAN_HOME=C:\Users\N662245\byteman-download-3.0.6-bin\by
teman-download-3.0.6

(2) SET BYTEMAN_HOME_OPTS  ( I know I can pass helper as a psrt of bmsubmit too but I have tried this way)

C:\Users\N662245>SET BYTEMAN_HOME_OPTS=-Xbootclasspath/a:"C:/Users/N662245/Bytem
apHelper.jar"

(3) Installing agent dynamically

C:\Users\N662245\byteman-download-3.0.6-bin\byteman-download-3.0.6\bin>

bminstall.bat -b -s 15656    (where 15656 is the Java PID of the Process B)

(4) Now when I am trying to load rules dynamically using bmsubmit, I am getting syntax error, but I opened the batch file and checked the usage too.

C:\Users\N662245\byteman-download-3.0.6-bin\byteman-download-3.0.6\bin>

bmsubmit.bat -l -c script:C:\Users\N662245\threadInteraction.btm

usage : Submit [-o outfile] [-p port] [-h hostname] [-l|-u] [scriptfile . . .]
        Submit [-o outfile] [-p port] [-h hostname] [-b|-s] jarfile . . .
        Submit [-o outfile] [-p port] [-h hostname] [-c]
        Submit [-o outfile] [-p port] [-h hostname] [-y] [prop1[=[value1]]. . .]

        Submit [-o outfile] [-p port] [-h hostname] [-v]
        -o redirects output from System.out to outfile
        -p specifies listener port
        -h specifies listener host
        -l (default) with scriptfile(s) means load/reload all rules in scriptfil
e(s)
                     with no scriptfile means list all currently loaded rules
        -u with scriptfile(s) means unload all rules in scriptfile(s)
           with no scriptfile means unload all currently loaded rules
        -b with jarfile(s) means add jars to bootstrap classpath
        -s with jarfile(s) means add jars to system classpath
        -c prints the jars that have been added to the system and boot classload
ers
        -y with no args list all byteman config system properties
           with args modifies specified byteman config system properties
             prop=value sets system property 'prop' to value
             prop= sets system property 'prop' to an empty string
             prop unsets system property 'prop'
        -v prints the version of the byteman agent and this client

C:\Users\N662245\byteman-download-3.0.6-bin\byteman-download-3.0.6\bin>

bmsubmit.bat Submit -l -c script:C:\Users\N662245\threadInteraction.btm

Failed to process request: java.lang.Exception: Invalid rule file: Submit
-- Args were: [Submit, -l, -c, script:C:\Users\N662245\threadInteraction.btm]
java.lang.Exception: Invalid rule file: Submit
        at org.jboss.byteman.agent.submit.Submit.getRulesFromRuleFiles(Submit.ja
va:829)
        at org.jboss.byteman.agent.submit.Submit.addRulesFromFiles(Submit.java:5
54)
        at org.jboss.byteman.agent.submit.Submit.main(Submit.java:1147)

C:\Users\N662245\byteman-download-3.0.6-bin\byteman-download-3.0.6\bin>

Whats the missing bit ???

[filippe.spolti]

Hi, If I recall correctly you need to start the byteman agent using the flag listener=true

The default address to use us host localhost and port 9091

[gaurav6281]

Hi Filippe,

In the programmer guide (Page 49) it is clearly mentioned below :-

"The installed bin directory contains a script called bmsubmit which can be used to communicate with the agent listener started up with option listener:true is passed as an option to the Bytemap agent(recall that this option is always enabled by the bminstall and bmjava scripts)"

As you will see I have already used bminstall for loading agent dynamically in another process. The error seems to be more of  how to invoke bmsubmit batch file from my windos prompt .It seems more like command usage issue.

[filippe.spolti]

OK, did you check the script with the bmcheck script?

Note if it returns any warning or error the rule will not work.

[gaurav6281]

Filippe , problem is not whether my rule is getting executed or not. Problem is in the very first place how to submit my rule dynamically using bmsubmit command.  Moreover nowhere in the bmsubmit.bat file , any option exists where to submit the rules. So where will my rules be deployed , in which java process ?

[adinn]

Hi Gaurav,

It seems you are supplying these arguments to the submit command

bmsubmit.bat -l -c script:C:\Users\N662245\threadInteraction.btm

that doesn't actually follow the correct syntax

Firstly, option -l needs to be followed by the name of the script file.You have inserted option -c between the -l and the name. So, get rid of that. Secondly, you have provided the name of the script file but with the prefix script:. That is not the name of one or more script files. so, ditch the prefix.

bmsubmit.bat -l C:\Users\N662245\threadInteraction.btm

Finally, I need to ask you to describe more clearly your current setup because your description of the setup is extremely unclear. You say

Process A executes multiple steps one by one , including loading byteman agent dynamically to Process B. Second step in Process A is submitting rules dynamically to Process B

Are you running the bmsubmit command on the same machine as you ran the bminstall command? Are process A and B separate Java processes running application code that yyou have provided or do you mean that process A is simply the Java code executing bminstall or bmsubmit.

regards,

Andrew Dinn

[gaurav6281]

Hi Andrew

Let me explain in detail what is our requirement:-

(1) There will be 2 machines . Machine A and Machine B. (Both Linux boxes)

(2) On Machine A, cucumber Test will be run which will call below steps:-

     (a) Restart one remote JVM on different Machine B via java class

     (b) Once that remote java main class is started by test running on Machine A, byteman agent will be dynamically be installed on running JVM of Machine B

     (c) Then bytemap helper class as well as script file will also be pushed dynamically on the remote JVM on machine B

     (d) Based on bytemap rules executed on remote JVM, some test validations will happen on machine A cucumber Test by checking with database which is accessible from both Machine A and Machine B.

Please Note:-  Cannot use JUNIT as we only have cucumber tests in place and moreover we don't want any BMUnit specific functionality for validation as you must have read from above steps.

Above Step (a) is not under my POC as that can easiely be done.

(1) Regarding Above step (b) , I was thinking to use bminstall shell script with bytemap jars to be pushed dynamically to remote machine via java program which can call unix shell script of whatever command I pass .

My understanding of the command is as below:-

bminstall.bat -b C:\Users\N662245\byteman-download-3.0.6-bin\byteman-download-3.0.6\lib\byteman.jar -s -Dorg.jboss.byteman.verbose -sys:C:/Users/N662245/BytemapHelper.jar 5400  (5400  is just java PID on my local windows box for testing, on remote Linux box, this will be replace by process Id or process name)

Please correct me ?

(2) Regarding Above step (c) , I was thinking to use bmsubmit shell script to load rules dynamically

bmsubmit.bat  -l C:\Users\N662245\threadInteraction.btm   (But  I am confused there is no syntax in bmsubmit to mention that I want my rules for particular JVM process id or process name ???)

Please correct me.

Please Note:- (1) In above design , I will not  be needing bytemap agent to be installed  on any machine and hence everything I am working on POC is on fly.

Kindly suggest if My above steps of design is feasible or not.

Thanks and Regards,

Gaurav Bhatnagar

[adinn]

Hi Gaurav,

Ok, so I think I understand the setup you are trying to adopt.

  You start a (Cucumber) test in a JVM on machine A.

  The test uses a remote invocation to start a JVM on machine B.

  You want the byteman agent to be installed into the JVM on remote machine B and you want rules to be uploaded to that agent.

  Finally, you are not in a position to control how the JVM on machine B is created (i.e. you cannot add the agent and specify the rule file on the command line)

1) Installing the agent

The first thing to note is that the agent install cannot be executed directly machine A. You have to do the install on machine B. As you say, you can do that by ensuring that the byteman jars and bminstall script are installed on machine B and fork a shell to execute the bminstall script. However, you don't have to do the install by running the shell script. If you look inside the script you will see that this is the last line

  java -classpath $CP org.jboss.byteman.agent.install.Install $*

That command simply invokes the main method of class Install which is a Byteman class located in the byteman-install jar. CP is the value used for the classpath which must reference byteman.jar and byteman-install.jar. $* is just the command line arguments you pass to bminstall.sh i.e. the PID and any other values you specify like -b, -Dorg.jboss.byteman.verbose, etc. So, if your test program on machine A knows the PID of the JVM on machine B then instead of forking a 2nd JVM on machien which forks a shell to run the bminstall script it can simply fork a 2nd JVM which runs the main method of Install.

2) Submitting the rules

If you want the rules uploaded into the JVM on machine B then there are two ways to do this. Well, actually, there are four because the bmsubmit script, like bminstall, really just runs a Java program. So, for each of the two ways of doing it you can fork a shell to run bmsubmit.sh or (the better option) you can just run the main method of class BMSubmit.

The first way to do it is to execute the submit request on machine B. So, you need to fork a 3rd JVM using class BMSubmit as the main class. You need to have byteman.jar and byteman.submit.jar in the classpath to do this. Here si the last line of the bmsubmit script to show how what you need to execute

  java -classpath ${BYTEMAN_JAR}:${BYTEMAN_SUBMIT_JAR} org.jboss.byteman.agent.submit.Submit $*

As before $* is the script arguments. So, you would need to pass arguments something like -l /path/to/myscript.btm when you start this 3rd JVM on machine B. If you do it this way the you have to make sure that the script is on machine B and the path identifies its location on that machine.

The second way to do it is to run the Submit program on machine A but get it to talk to the agent on machine B. This is only possible if you get the agent to start listening for submit requests on a non-local network interface (e.g. local ethernet or wireless). You also need to tell submit to use the same network interface. Normally, the agent listens on localhost:9090 i.e. on the loopback network. Since that interface is private to machine B you cannot use it to talk to the agent from machine A.

So, what you need to do is tell Install and Submit to open their serer and client sockets using the hostname of the machine on the network that A and B both sit on. Let's assume your local wireless address is 192.168.1.155 and that your machine appears on the network with name gaurav.my.domain. When you start the agent on machine B you need to start the JVM using something like this

  java classpath ${BYTEMAN_JAR}:${BYTEMAN_INSTALL_JAR} org.jboss.byteman.agent.install.install -h 192.168.1.155 pid

(where pid is the process id of the process you want to install the agent into). The agent will open a socket on the interface with address 192.168.1.155 using port 9090. If you cannot use that port because it is restricted try specifying a port  number as well , using a value that is usually unrestricted like 12345

  java classpath ${BYTEMAN_JAR}:${BYTEMAN_INSTALL_JAR} org.jboss.byteman.agent.install.install -h 192.168.1.155 -p 12345 pid

Now, when you want to upload the rule from machine A you can start a JVM on machine A running a command like this

  java -classpath ${BYTEMAN_JAR}:${BYTEMAN_SUBMIT_JAR} org.jboss.byteman.agent.submit.Submit -h 192.168.1.155 -p 12345 -l /path/to/myscript.btm

Note that when you run it this way the path to the script refers to the location on machine A not machine B.

[gaurav6281]

Hi Andrew,

Thanks for the detailed response. To me second option seems to be the best i.e. just call direct java command from machine A only . No knowledge of byteman at all on Machine B.

(1) So, I will install agent as below:-

java classpath ${BYTEMAN_JAR}:${BYTEMAN_INSTALL_JAR} org.jboss.byteman.agent.install.install -h 192.168.1.155 -p 12345 -Dorg.jboss.byteman.verbose -sys:/path_on_machine_A/BytemapHelper.jar pid   (where BYTEMAN_JAR and BYTEMAN_INSTALL_JAR variables are on Machine A only and not on machine B)

(2) I will call submit as below:-

java -classpath ${BYTEMAN_JAR}:${BYTEMAN_SUBMIT_JAR} org.jboss.byteman.agent.submit.Submit -h 192.168.1.155 -p 12345 -l /path/to/myscript.btm (where BYTEMAN_JAR and BYTEMAN_INSTALL_JAR variables are on Machine A only and not on machine B)

Kindly confirm below:-

(1) BYTEMAN_JAR and BYTEMAN_SUBMIT_JAR  variables should be present only on Machine A. No need for such variables on machine B at all.

(2) As by calling java , new JAVA process will be created on Machine A, hence any logging , tracing or any output of rules will be on Machine A and no on Machine B ?

Thanks and Regards,

Gaurav Bhatnagar

[adinn]

Hi Gaurav,

Thanks for the detailed response. To me second option seems to be the best i.e. just call direct java command from machine A only . No knowledge of byteman at all on Machine B.

Ok, that sounds like a good idea.

(1) So, I will install agent as below:-

java -classpath ${BYTEMAN_JAR}:${BYTEMAN_INSTALL_JAR} org.jboss.byteman.agent.install.install -h 192.168.1.155 -p 12345 -Dorg.jboss.byteman.verbose -sys:/path_on_machine_A/BytemapHelper.jar pid

(where BYTEMAN_JAR and BYTEMAN_INSTALL_JAR variables are on Machine A only and not on machine B)

I am afraid you have a few things wrong here.

Firstly, you have to run that command on machine B. As I said in my previous note, if you want to install the agent into a JVM on machine B then you also have to run the Install program in a JVM on machine B. It is not possible to install the agent over the network by running program Install on machine A. So, what you need to do is have the JVM on machine A start a 2nd remote JVM on  machine B. That 2nd remote JVM has to execute program Install.

However, the implication of that is that byteman jar and byteman-install.jar need to be pre-installed on machine B otherwise the 2nd JVM will not be able to find class Install nor will it be able to find the byteman agent code. So, the  two variables BYTEMAN_JAR and BYTEMAN_INSTALL_JAR used to construct the classpath need to identify the location of these jars on machine B (not on machine A).

Finally, where did you get the -sys option for command Install? There is no such option (there is a -s option but that is to do with security managers). If you want to add a helper jar to the classpath for the JVM on machine B then you can do that using the Submit program. You can even run the Submit program from machine A, so long as you use -h an d-p to talk using a public socket. However, the helper jar will also need to be installed on machine B.

So, your 2nd JVM started on machine B needs to run using command line

  -classpath ${BYTEMAN_JAR}:${BYTEMAN_INSTALL_JAR}  org.jboss.byteman.agent.install.Install -h 192.168.1.155 -p 12345 -Dorg.jboss.byteman.verbose pid

where

  192.168.1.155 is the network address of machine B

  12345 is an available port on machine B

  BYTEMAN_JAR and BYTEMAN_INSTALL_JAR identify the location of byteman.jar and byteman-install.jar on machine B

  pid is the id of the first JVM you started on machine B that is running your application code

  (note also I have corrected the class name to use a capital I)

Output from this command will be written to System.out and System.err for the 2nd remote JVM.

After you have done this you can add your helper jar to the classpath and load your rule script by running program Submit on machine A

(2) I will call submit as below:-

java -classpath ${BYTEMAN_JAR}:${BYTEMAN_SUBMIT_JAR} org.jboss.byteman.agent.submit.Submit -h 192.168.1.155 -p 12345 -l /path/to/myscript.btm

You don't need byteman.jar in the path just byteman-submit.jar. Also, you need to install the helper jar first and then install the rules. So, you need to run Submit twice. If you run it by creating a JVM on machine A then you need to execute them using the following arguments:

  -classpath ${BYTEMAN_SUBMIT_JAR} org.jboss.byteman.agent.submit.Submit -h 192.168.1.155 -p 12345 -s /path_on_machine_B/helper.jar

  -classpath ${BYTEMAN_SUBMIT_JAR} org.jboss.byteman.agent.submit.Submit -h 192.168.1.155 -p 12345 -l /path_on_machine_A/myscript.btm

Notice that the helper jar must be pre-installed on machein B and the path you specify is a path on machine B. The rule script is read and uploaded by class Submit. So, the path to the script is specified on machine A.

Any messages or errors from running these commands will go to System.out and System.err.

It is also possible for you to execute the submit calls from your test JVM on machine A. So long as you have byteman-submit.jar in your tests JVM's classpath your test program can create an instance of class Submit and use it to install the helper jar and load the script. You will need to do something like this

  ...

  Submit submit = new Submit(hostname, post);

  List<String> syspaths = new List<string>();

  syspaths.add("/path_on_machine_B/helper.jar");

  submit.addJarsToSystemClassloader(syspaths);

  List<String> scriptpaths = new List<string>();

  scriptpaths.add("/path_on_machine_A/myscript.btm");

  submit.addRulesFromFiles(scriptpaths);

  ...

If you use code like this then messgaes and errors will go to System.out and System.err. However, if you want you can pass a PrintStream in the constructor for Submit to receive these messages. I suggest you look at the javadoc for byteman-submit.jar for full details.

regards,

Andrew Dinn