Design notes for NoSQL integration on WildFly

Version 28

    Goal

     

    The goal is no-nonsense improvements focused on making the developer’s life easier when using NoSQL on WildFly.  In particular, not to hide and abstract the underlying access APIs but to give simpler and more natural access to them.  The configuration allows NoSQL database connection profiles to be defined, to be used by deployed applications.  The CDI access simplifies application access to the defined NoSQL database connection profiles.

     

    Requirements

    • NoSQL connection authentication support.
    • CLI can be used to add/remove NoSQL connection profiles.
    • NoSQL connection profiles, identify the remote NoSQL server(s), profile name and JNDI name.
    • NoSQL (native) connections may be (CDI) injected or referenced from JNDI.
    • The underlying NoSQL driver provides connection pooling.
    • Custom NoSQL driver modules addresses the need for tailoring the NoSQL dependencies to include additional modules.  Custom NoSQL driver modules may be referenced by NoSQL connection profile definitions.
    • NoSQL integration support will be merged into WildFly.

     

    What is optional

    • The underlying NoSQL driver may provide asynchronous (background) processing, which can be used as long as no leaks are caused.
    • If/when platform level management capabilities are exposed by the NoSQL driver(s), we will integrate with capabilities that makes sense, to improve the developer experience using NoSQL on WildFly.
    • Management (WEB) console support is optional, users can instead use CLI to add/remove NoSQL connection profiles.  The management console support can come in later, time permitting.
    • NoSQL driver jars are optional.  We wouldn't ship the libraries, instead we can document how to add the NoSQL driver jars.

     

    Expected use cases

    • Users will use supported versions of the NoSQL driver libraries and inject native NoSQL class instances into their applications, that represent the underlying (shared) NoSQL database connection.
    • NoSQL databases that could support ACID transaction commit/rollback of operations in a JTA transaction (via XAResource or one phase wrapper), allow JTA transaction enlistment.
      • Neo4j is an example of a NoSQL database that supports JTA transaction enlistment via their Java Transaction api.  Neo4j Session.beginTransaction() api is wrapped with one phase wrapper that is enlisted into JTA transaction, so that wrapper calls org.neo4j.driver.v1.Transaction.success() or failure(), when JTA transaction respectively commits or rolls back.
    • Connection profile allows transaction type to be specified
      • transaction type = NONE, NoSQL resources cannot be enlisted into the JTA transaction.  NONE could be specified for BASE or ACID databases.
      • transaction type = 1pc, ACID (NoSQL) resources can be enlisted into JTA transaction but BASE resources cannot be.
      • transaction type = 2pc, ACID (NoSQL) XAResource's can be enlisted into JTA transaction but BASE resources cannot not.
        • Note that if an ACID database, is later redesigned to also support mixed ACID/BASE, any use of BASE resources in the JTA transaction, may not work correctly, since the BASE use could introduce non-ACID characteristics.  While it is not likely that ACID NoSQL databases, would switch to a mixed ACID/BASE model, it could happen.

     

    What is not allowed

    • Applications are not currently allowed to close pooled NoSQL connections.
    • Users cannot use incompatible versions of the NoSQL driver libraries.

     

    MongoDB specific details

    • Compensating transactions may be used.
    • Default driver module can be used.
    • Configuration
      • Custom driver module can be selected.
      • Security
        • connection encryption
        • user authentication
    • Versions supported?

     

    Cassandra specific details

    • Compensating transactions may be used.
    • Default driver module can be used.
    • Configuration
      • Custom driver module can be selected.
      • Security
        • connection encryption
        • user authentication
    • Versions supported?

     

    Neo4j specific details

    • JNDI lookup of org.neo4j.driver.v1.Driver is allowed, however, org.neo4j.driver.v1.Session are not (future enhancement to allow TransactionScoped Session injection might work).  CDI inject of the Driver is allowed.
    • Neo4j JTA transaction enlistment will occur for (WildFly) NoSQL connection profiles defined with "transaction=1pc", the default is "transaction=none" which doesn't do JTA transaction enlistment.
    • Compensating transactions may also be used instead of JTA enlistment.
    • JTA transaction enlistment of Neo4j Session will occur when the Driver.session() call is made.
      • Attempting to call Session.beginTransaction(), will cause a RuntimeException to be thrown, as the Neo4j TX api should not be used inside of the JTA transaction, instead the application should rely on JTA for managing the transaction.
    • Default driver module can be used.
    • Configuration
      • Custom driver module can be selected.
      • Security
        • connection encryption
        • user authentication
    • Versions supported?

     

    Remote Infinispan

    • Optional
    • Protocol buffer style use (database does not have application class definitions)
    • Versions supported?
    • Configuration
      • Custom driver module can be selected.
      • Security
        • connection encryption
        • user authentication

     

    OrientDB

    • Optional
    • Versions supported?
    • Configuration
      • Custom driver module can be used.
      • Security
        • connection encryption
        • user authentication

     

    Test Plan

     

    Documentation plan

     

    Schedule