It seems that you want to start servers in domain mode. Where you will define 3 server groups with 3 server each.
This can be configured in domain.xml for each server group in:
<server-groups> <server-group name="main-server-group" profile="full"> <jvm name="default"> <heap size="1000m" max-size="1000m"/> </jvm> <socket-binding-group ref="full-sockets"/> </server-group>
Or in host.xml if you need to set it up for each server in server group.
As Miroslav mentioned you are able to define jvm settings in different places.
The are inherited; domain-default - host-default - server-group - server.
So for each you can add or modify the settings. I recommend to use the server group if you cluster servers. Because each server should be able to perform similar otherwise it will be complicated to adjust load-balancing.
For the standalone mode you need to modify standalone.conf as these settings are needed for the JVM startup, so the xml config will be too late