Thanks for the pointers, Martin. Unfortunately, I still can't seem to get my provider loaded. Here's what I've tried:

I added ssl.SocketFactory.provider=<my.class.name> to /usr/lib/jvm/java-8-openjdk-amd64/jre/lib/security/java.security, on the basis that readlink -f `which java` pointed to /usr/lib/jvm/java-8-openjdk-amd64/jre/bin/java.

I also enabled SSL debugging to see what's going on, by adding this line to my standalone.conf:

  JAVA_OPTS="$JAVA_OPTS -Djavax.net.debug=ssl

The debugging output was produced, however I still saw no evidence of my code being used (it ought to be printing to stdout). I also can't see the log lines from this function https://hg.openjdk.java.net/jdk8/jdk8/jdk/file/687fd7c7986d/src/share/classes/javax/net/ssl/SSLSocketFactory.java#l88.

I also tried adding my security property using this addition to my conf file:

  JAVA_OPTS="$JAVA_OPTS -Djavax.net.debug=ssl -Djava.security.properties=<path to a simple properties file>"

but still no dice. It doesn't seem like WildFly is noticing my property. Or perhaps it's just ignoring it. I'd expect to see failures in the SSL logs if my class failed to load.

I also wrote a quite Java file to test my provider is registered correctly:

public static void main(String[] args) throws Exception {

  SocketFactory factory = SSLSocketFactory.getDefault();
   System.out.println(factory.getClass().getCanonicalName());
}

Executing this with java -Djavax.net.debug=ssl ... produced the expected results. I could see the log files indicating my provider was loaded, plus the outputs from my provider.

Do you have any further suggestions to debug this? It seems like WildFly is not creating SSL sockets in the way I'm expecting and potentially ignoring my java.security settings.

I've solved my problem by creating a security provider that implements "SSLContext.TLS". This provides me a hook to intercept how socket factories are constructed.

By placing this provider at the top of the java.security providers list, I can intercept the things I need.