EJB3 uses standardwise JBoss Remoting and by default the socket communication on port 3873. This is configured in $JBOSS_HOME/server/default/deploy/ejb3.deployer/META_INF/jboss-service.xml.
So I assume whatever you have set in jva.rmi.server.hostname has no impact at all.
And yes, you need to open up this port.
I admit I have a hard time understanding the "UsingJBossBehindAFirewall" wiki. It refers to services, which I know little about (users guide is too much for my head). Do I need "RmiPort" (1098)? Do I need jrmpInvoker or pooledInvoker (4444/4445), or both? I don't have a clue.
Do these ports need to be open on the client side? Do they have inbound or outbound traffic on both sides? About these issues I could not find information I could understand. :(