Trying to get in-jvm ActiveMQ with dynamic topics working in Wildfly 10

[erik777]

I'm porting a standalone app that works great to WildFly to give it a web UI.  Note that I'm using an in-memory/in-jvm implementation of MQ because I do not need persistence and do not want the performance impact of disk i/o.  You accomplish this by setting java.naming.provider.url = vm://localhost in your InitialContext properties.  I'm also using dynamic queue/topic creation, which ActiveMQ supports when you prefix your topic name with "dynamicTopics/".  Note that I'm creating my own listener.  I prefer not to complicate things by using an MDB, but will use one if I have no choice. 

When I try to run it the way it works as a standalone app, where it creates its own InitialContext to give it the settings ActiveMQ requires, it responds with:

java.lang.ClassCastException: org.apache.activemq.ActiveMQConnectionFactory cannot be cast to javax.jms.TopicConnectionFactory

and

java.lang.ClassCastException: org.apache.activemq.command.ActiveMQTopic cannot be cast to javax.jms.Topic

OK, so, Wildfly uses ActiveMQ, right?  Then how do I do this via Wildfly?  I begin by using new InitialContext() instead of my own, which should provide a context from Wildfly in the web context it is executing under. I'd like to start by using the default connection factory required by the spec, which is in-vm, although it is not clear that this is also in-memory and not persistent, which is also a requirement, since these things tend to have their own databases.

But, it fails on the lookup for the connection factory and the dynamic topic.  I tried both what I was using before, "ConnectionFactory", and "java:/ConnectionFactory".  I could not find a working example of the simple use-case online or in the forums.  Here are the lookup errors:

javax.naming.NameNotFoundException: ConnectionFactory -- service jboss.naming.context.java.ConnectionFactory

and

javax.naming.NameNotFoundException: dynamicTopics/apiStream -- service jboss.naming.context.java.dynamicTopics.apiStream

[jbertram]

OK, so, Wildfly uses ActiveMQ, right?

Wildfly 10 uses Apache ActiveMQ Artemis.  It appears you're using what I would call "classic" Apache ActiveMQ.  The two are different brokers altogether (although Artemis does support many of the same features (including OpenWire) that classic supports).

I'd like to start by using the default connection factory required by the spec, which is in-vm, although it is not clear that this is also in-memory and not persistent, which is also a requirement, since these things tend to have their own databases.

To be clear, a connection factory isn't in-memory or persistent.  That's a quality of a message, not a connection factory.

[erik777]

Thanks.  I got it running.

Regarding in-memory vs persistent, I think with classic ActiveMQ, a connection factory in-memory would result in durable topics as not being so durable because the entire MQ implementation is in-memory.  The analogy would be an in-memory database, which Derby supports, which "persists", works with JPA, does transactions, etc,...  But, when the instance is killed, it's gone, because it is never written to disk.

Because of the tremendously high throughput requirement and data intensiveness of this application, the goal is to do 99.99% in memory, and only persist to ensure it can return to operation on an instance recycle.  Minimal disk I/O is critical.  In the end, WildFly itself my be copied to a memory drive before booting to ensure it does not use any disk I/O in its own processing.

Wildfly supports an "in-jvm" connector.  But, it's not clear if this persists locally when durable, to offload non-durable to cache to reduce memory contstraints, or for any other reason.  I suspect it uses a local DB, and durable is truly durable -- persisted to disk.

Regardless, I used an in-jvm connection factory and configured WildFly with a non-durable topic.  Hopefully, this doesn't waste disk I/O for anything.  This is ultimately going to be running on a computer with Terabytes of RAM.  The primary purpose of this is to provide in-jvm pub/sub with JMS as a nice standard interface offering diverse configuration should other options be needed, such as publishing across the network or scalable load handling.

For the topic, I had to parameterize it so that the web uses one defined by WildFly while the standalone continues to use the dynamic topic.

In the end, the standalone continues to use "classic" ActiveMQ and the Web uses Artemis, both using the same code with the exception of the topic being parameterized to permit WildFly to provide it.  An extension class for the Web uses the JNDI required for WildFly topic lookup, overriding the defaults used by standalone where JNDI properties define the MQ instance as in-memory and the topic is dynamically created.

Also, for anyone reading this, I replaced the class ActiveMQ jar with a client jar I found in WildFly: artemis-jms-client-1.1.0.wildfly-011.jar.  I also had to add jboss-jms-api_2.0_spec-1.0.0.Final.jar, which I also found in WildFly.  Not sure this is the correct method or ideal jars, but it worked. 

[jbertram]

Wildfly supports an "in-jvm" connector.  But, it's not clear if this persists locally when durable, to offload non-durable to cache to reduce memory contstraints, or for any other reason.  I suspect it uses a local DB, and durable is truly durable -- persisted to disk.

As I noted previously, the connector's configuration has nothing to do with whether or not the message will be persisted to disk.  The durability of the message is defined by the message itself.  Durable messages are persisted to disk by default, but that behavior can be changed by marking the destination as non-durable or disabling persistence completely on the broker.  In any event, if the message is marked as non-durable then it won't be persisted to disk.  If you want to ensure that nothing is persisted to disk no matter what (e.g. some client sends a durable message by mistake) then you should set the "persistence-enabled" attribute on the "server" element to "false".

This is ultimately going to be running on a computer with Terabytes of RAM.

I'm curious to know what JVM and what GC configuration you're using to maximize performance over this amount of heap space.  Could you share any of those details?

Also, for anyone reading this, I replaced the class ActiveMQ jar with a client jar I found in WildFly: artemis-jms-client-1.1.0.wildfly-011.jar.  I also had to add jboss-jms-api_2.0_spec-1.0.0.Final.jar, which I also found in WildFly.  Not sure this is the correct method or ideal jars, but it worked. 

A simpler approach might be to just use the jboss-client.jar shipped in Wildfly's bin/client directory.

[erik777]

Haven't gotten into the details of Terabyte deployment, yet.  But, there are options today for scaling:

Home - Azul Systems, Inc. Azul Systems, Inc.

(what's up with the forum expanding a URL against your will?)

Also, it won't be the only thing using all that RAM, and there could be multiple JVMs running together.

Thanks for the jboss-client.jar tip.  i wasn't sure if that still existed.  Should of done a locate for it.  Expected it to be called wildfly-client or something new.

[jbertram]

Haven't gotten into the details of Terabyte deployment, yet.  But, there are options today for scaling:

Home - Azul Systems, Inc. Azul Systems, Inc.

You mentioned options (i.e. plural), but just mentioned Azul.  Are there other options you've considered?

In terms of your use-case do you believe that messaging will be your performance bottleneck?  Have you done any benchmarking so far?  If so, could you share any results.  I'm always interested in high-performance messaging use-cases that are out of the ordinary.

(what's up with the forum expanding a URL against your will?)

I don't know.  I just use the forum; I don't run it.

[erik777]

"Haven't gotten into the details of Terabyte deployment"

I just threw one out there for you in case you still believed that Java itself couldn't handle large heap sizes.  Azul proved otherwise.

Another fundamental option is increased distribution of parallel processing.  This is one reason I chose to use Rserve to connect R to Java instead of using a Java implementation.  By using TCP/IP to connect to R, I can potentially distribute it across many nodes.

The bottom line is that at this point I'm designing for multiple possible deployment scenarios and am just being sure I do not create bottlenecks early in development.  I'm a big fan of in-memory for just about anything.  That has become very common in higher performance computing, in large part due to the price/performance of memory.  Disk reads/writes need only to be part of system recovery. 

The pub/sub I'm currently using is about integrating various computing parts in Java, so that today they can be deployed in a single JVM and use in-memory as much as possible, but one day they could possibly be distributed and load-balanced. Even with the latter, there will likely be a lot of in-JVM pub/sub that should operate in-memory and not do any disk I/O. The difference will be fine-grained intra-module communication versus inter-module communications. 

[jbertram]

I just threw one out there for you in case you still believed that Java itself couldn't handle large heap sizes.  Azul proved otherwise.

I have a passing familiarity with Azul's work as well as with stuff like OpenJDK's Shenandoah so I know Java can handle really big heaps.

I understand where you're coming from.  I was just curious if you had any hard data to justify your in-memory-vs-disk concern for 2 reasons:

1. Like I said, I'm interested in out-of-the-ordinary messaging performance use-cases.
2. I've seen time and time again where developers will spend time on unnecessary optimization.  As Knuth said, "Premature optimization is the root of all evil."

[erik777]

Yeah, I hear you and appreciate the clarification.  Scalability and performance will ultimately be required in order for this to succeed.  I don't want to go back rewriting low-level coding, which I've done way too many times in the past.  And, a little extra forethought in design is not the same as writing extra code for "features" you may never use.  What I'm doing has become the norm in big data computing.  And, there is a lot online comparing disk I/O to memory.  The fastest SSDs today are PCI-E.  I'm just following the trend of getting all that data in memory and minimizing I/O.

So, I'll flip the question around:

Why would I want to read the same unchanged data multiple times from disk instead of memory, or write to disk something that does not need to persist (be available on reboot)?

[jbertram]

I have no doubt that memory is faster than disk in general and that, all things being equal, you would always want to leverage that speed.  Furthermore, all things being equal, you wouldn't ever want to read the same unchanged data multiple times from disk instead of memory or write to disk something that does not need to persist.  However, this is digressing from my question which was about whether or not you had hard performance data for your specific use-case.  As I noted previously, I've seen lots of users waste resources due to a lack of hard data to inform optimization priorities.

My questions are more along the lines of:

1. Does the speed difference between memory and disk matter in your specific use-case?
2. If the speed different does matter in your specific use-case how much does it matter?
3. How does the amount that the speed difference matters compare to other factors impacting performance?

You can probably answer question #1 without hard performance data for your use-case, but the rest of the questions can't - at least not with much precision.

In any event, we've gone far afield from your original question.  Good luck with your work!