Quarkus in Google App Engine

[tfve]

Hi,

I'm developing a Quarkus app for Google App Engine (standard environment).

As soon as I add the quarkus-hibernate-orm dependency I start getting this error:

WARN [com.arj.ats.common] (main) ARJUNA048006: cannot create new instance of com.arjuna.ats.internal.arjuna.objectstore.ShadowNoFileLockStore: java.lang.reflect.InvocationTargetException ... Caused by: com.arjuna.ats.arjuna.exceptions.ObjectStoreException: ARJUNA012225: FileSystemStore::setupStore - cannot access root of object store: /srv/ObjectStore/ShadowNoFileLockStore/defaultStore/

I "solved" this error by setting the object store directory to /tmp:

-Dcom.arjuna.ats.arjuna.objectstore.objectStoreDir=/tmp

The /tmp directory, in App Engine, lives in memory and is destroyed, as you might imagine, as soon as the instance is terminated.

I don't think this is a good solution and was hoping someone here could help me. I've already asked for help in quarkus github repo and they said it would be better to ask in this forum.

[ochaloup]

hi tfve ,

sure, it's good place to ask here, just I assume there is not many experience with the Google App Engine here.

Nevertheless, the issue is about the Narayana object store needs to be saved on at a volume which is persisted over containers restarts and the app redeployments. This rule holds in general but let's elaborate a bit on it.

First, it depends if you application uses the XA transactions. If you app does not use them - aka. you use only a single database accessed via hibernate with one persistence unit you don't need to be worried about the place where the object store is saved. The 2PC of XA transaction won't be used (1PC is used) and you don't experience no data consistency issues. You can be stay with '/tmp' folder being used. But be cautious when chaning the data access pattern.

In this case the transaction manager provides the implementation of JTA calls and provides the backend for quarkus/CDI to be capable to run data in transaction.

If your application uses the 2PC of XA - you use two or more different databases or accesses a JMS broker in transactional way then you need to ensure the Narayana object store is placed in a persistent volume. The object store contains information about unfinished XA transactions. If the application crashes in the middle of processing the transaction could be finish with the wrong transaction outcome (e.g. rollback insted of commit) after the container restarts.

The application with Narayana then requires being handled as a stateful application. The object store directory has to be available at the volume accessible over the container restarts. You need to setup the Google App engine to provide such functionality.

As an example I can mention the Kubernetes as I have slight experience with it. Kubernetes provides the StatefulSet object as a way to get this. There could be defined a persistent volume which is attached to the pod and will be retained attached with the same data even after it's restarted.

Ondra

[tfve]

Thanks for your reply, ochaloup!

I was using Quarkus v0.25.0, but now I've updated my project to use 1.0.0-CR1 and the issue is gone. I don't know what happened, maybe JTA is not used by default now?

I can't say that my project won't be using 2PC or XA transactions because I'm evaluating several java frameworks and the chosen framework will be used by several projects across my company.

So, although the majority of the projects won't be using 2PC/XA transactions, some of them might have the need to use it. I need to make sure everything works flawless.

My next task will be to use distributed transactions to see how it behaves (in GAE).

[mmusgrov]

In dynamic environments where file system base storage is not guaranteed to persist the advice is to use a reliable storage medium such as a database (we have a JDBC store for this purpose). I will update the quarkus narayana-jta extension documentation to include this advice and indicate how the JDBC store should be configured.

[ochaloup]

Hi tfve ,

yes, it could be. There were changes related to jta configuration extension during the time from 0.25.0 to 1.0.0.RC1. Now I think the object store is created when it's really used. So in 0.25.0 the object store was created at startup time, even you haven't started any transaction. With 1.0 the object store directory should be created as well - so you should experience the same behaviour, I assume - but only when 2PC XA transaction is really started.
(If you want to test it you can apply Narayana setup '-Dcom.arjuna.ats.arjuna.common.CoordinatorEnvironmentBean.commitOnePhase=false' which declines usage of the 1PC optimization and the data record needs to be created for every transaction commit in the object store.)

One more point on store location: the object store is by default created at place where System.getProperty("user.dir") points to. It should be the '/srv' for Google App Engine. You can change it as you mentioned by system property to a different place with -Dcom.arjuna.ats.arjuna.objectstore.objectStoreDir

Consider, what Mike mentioned above regarding the JDBC to be used as place for the transaction storage. Still, I consider the settings really depend on the particular deployment.

On more point on other frameworks: any java framework needs to use the place where stores the transactions data if the XA is used. You will need to solve the same with any framework you choose. You need to define the location of the storage. For the Narayana it's file system or JDBC storage. For any other framework you will check.

Ondra

[marklittle]

Just a point of clarification: when Mike says "reliable storage medium" it doesn't have to be a database. Databases aren't guaranteed to be reliable (it's all probabilistic anyway). A highly available (replicated) in-memory data store could be just as reliable and make be more appropriate in some cloud environments. XA is not a requirement for transactions either, so JDBC is only one possibility.