I'm running JBoss 7.1.1final on Windows XP 64bit with jdk1.6.0_22. Any attempt to call System.console() within the AS causes the thread to hang on a native method (istty()):
java.io.Console.istty(Native Method)
java.io.Console.access$500(Console.java:75)
java.io.Console$2.console(Console.java:506)
java.lang.System.console(System.java:187)
This seems to be WinXP specific issue - it also happens on the 32bit version, but not on Windows 7 or any of the Linux distributions I've tried so far.
It also seems to be unaffected of the jdk version I use (I've tried jdk1.6.0_24, 26 and 32) or the JB7 version (7.1.1, 7.1.0CR1b and 7.0.0final). Also, if I run a standalone app trying to get the console using the same jdk, it works ok.
I'm attaching a jar which could be used to reproduce the issue - it contains an ejb exposed as mbean, which contains a single operation (takeConsole()). That operation launches a new thread, which tries to get the console and print its classname on the standard output.