Not quite sure what you are trying to do but here are some excerpts of our routines from an MBean:

try
{
X509EncodedKeySpec publicKeySpec = new X509EncodedKeySpec(publicKeyBytes);
KeyFactory keyFactory = KeyFactory.getInstance("RSA", "BC");
publicKey = keyFactory.generatePublic(publicKeySpec);
}
catch(Exception e)
{
log.error("The public key is badly formatted.");
return false;
}

So we're using the Bouncy Castle RSA routines for generating the public key - all through the JCE API. The JBoss security policies are fine (java.policy) - we don't touch them.

bcprov-jdk14-118.jar lives in JAVA_HOME/jre/lib/ext.

In JAVA_HOME/jre/lib/security/java.security add the line,
security.provider.6=org.bouncycastle.jce.provider.BouncyCastleProvider

And then the routine works. So we're not doing anything too special or different from the JCE guidelines.

We're just speculating on things here as you'd need to detail further your code for a clear understanding.