Is there a way to upgrade log4j there?
Ah - that did it. I replaced JBOSS_HOME/lib/log4j-boot.jar with a newer version. Before I was replacing the wrong one, it seems. Thanks Clerbert.
You should probably upgrade soon.
As you're working with that already, maybe you will be better on waiting JBoss7. JBoss 7 will be really awesome from what I have seen.