I just ran into this exact same problem.
I tried to work around by explicitly putting the jar with my own Comparator in the run.sh classpath, but I get the following error:
11:44:28,526 INFO [ServiceConfigurator] Problem configuring service jboss.deployment:type=DeploymentScanner,flavor=URL
org.jboss.deployment.DeploymentException: Exception setting attribute javax.management.Attribute@433aa8 on mbean jboss.deployment:type=DeploymentScanner,flavor=URL; - nested throwable: (java.lang.NoClassDefFoundError: org/jboss/deployment/DeploymentSorter)
Which just seems crazy to me, because I can set the configuration back to the regular DeploymentSorter and it finds that class fine.
At this point I think I'm just going to explode jboss-system.jar where the default DeploymentSorter and PrefixDeploymentSorter live and put my own class in there, but that *can't* be the right way to do it. Anyone know where you're supposed to put this?