java.util.ConcurrentModificationException in wildfly-9.0.0.Final

[mayerw01]

I'd downloaded an example application (https://www.holisticon.de/wp-content/uploads/2013/04/ticket2rock-2.0-dist.zip) which was developed for jboss-6.1.0.Final. I made some changes to get this run under Wildfly.

Most things seem to run properly now. But I've one issue. There are 2 entities (Band and Musiker) which extend the abstract entity Interpret. This is the current Band class:

@Entity

@DiscriminatorValue("B")

public class Band extends Interpret implements Serializable {

    private static final long serialVersionUID = -6677136118903105659L;

    private int gruendungsjahr;

    private Set<Musiker> musiker;

    public int getGruendungsjahr() {

        return gruendungsjahr;

    }

    public void setGruendungsjahr(int gruendungsjahr) {

        this.gruendungsjahr = gruendungsjahr;

    }

   // @ManyToMany(fetch = FetchType.EAGER, mappedBy = "bands")

    @Access(AccessType.FIELD)

    @ManyToMany (fetch = FetchType.EAGER, cascade = CascadeType.REFRESH)

    @JoinTable(name = "Interpret_Interpret",

            joinColumns = @JoinColumn(name = "BANDS_ID",

                    referencedColumnName = "ID"),

            inverseJoinColumns = @JoinColumn (name = "MUSIKER_ID",

                    referencedColumnName = "ID"))

    public Set<Musiker> getMusiker() {

        return musiker;

    }

    public void setMusiker(Set<Musiker> musiker) {

        this.musiker = musiker;

    }

}

When I try to insert a new Band entity (via em.persist(band);) an exception is thrown:

2015-07-09 13:10:28,798 WARN  [com.arjuna.ats.arjuna] (default task-6) ARJUNA012125: TwoPhaseCoordinator.beforeCompletion - failed for SynchronizationImple< 0:ffffc0a80265:187491ef:559e565c:76, org.jboss.as.txn.service.internal.tsr.JCAOrderedLastSynchronizationList@332433b9 >: javax.persistence.PersistenceException: error during managed flush

    at org.hibernate.jpa.spi.AbstractEntityManagerImpl$CallbackExceptionMapperImpl.mapManagedFlushFailure(AbstractEntityManagerImpl.java:1887)

    at org.hibernate.engine.transaction.synchronization.internal.SynchronizationCallbackCoordinatorNonTrackingImpl.beforeCompletion(SynchronizationCallbackCoordinatorNonTrackingImpl.java:119)

    at org.hibernate.engine.transaction.synchronization.internal.RegisteredSynchronization.beforeCompletion(RegisteredSynchronization.java:50)

    at org.jboss.as.txn.service.internal.tsr.JCAOrderedLastSynchronizationList.beforeCompletion(JCAOrderedLastSynchronizationList.java:116)

    at com.arjuna.ats.internal.jta.resources.arjunacore.SynchronizationImple.beforeCompletion(SynchronizationImple.java:76)

    at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.beforeCompletion(TwoPhaseCoordinator.java:371)

    at com.arjuna.ats.arjuna.coordinator.TwoPhaseCoordinator.end(TwoPhaseCoordinator.java:91)

    at com.arjuna.ats.arjuna.AtomicAction.commit(AtomicAction.java:162)

    at com.arjuna.ats.internal.jta.transaction.arjunacore.TransactionImple.commitAndDisassociate(TransactionImple.java:1200)

....

Caused by: java.util.ConcurrentModificationException

    at java.util.ArrayList$Itr.checkForComodification(ArrayList.java:859)

    at java.util.ArrayList$Itr.next(ArrayList.java:831)

    at java.util.Collections$UnmodifiableCollection$1.next(Collections.java:1067)

    at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:463)

    at org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:351)

    at org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:350)

    at org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:56)

    at org.hibernate.internal.SessionImpl.flush(SessionImpl.java:1258)

    at org.hibernate.internal.SessionImpl.managedFlush(SessionImpl.java:425)

    at org.hibernate.engine.transaction.synchronization.internal.SynchronizationCallbackCoordinatorNonTrackingImpl.beforeCompletion(SynchronizationCallbackCoordinatorNonTrackingImpl.java:114)

    ... 102 more

Surprisingly the same version runs perfectly in jboss-eap-6.4. It now looks to me that this is related to the hibernate version.

Do you have any idea what could be the reason here?

[smarlow]

Is the EntityListener doing anything when the new Band entity is persisted?  I google searched (hibernate ConcurrentModificationException ActionQueue) and found HHH-7537 which sounded similar (comments talks about an entity listener as cause).

Typically, I like to enable TRACE logging for org.jboss.as.jpa when hitting problems like this, so I have a better idea of what actually happened.  See doc here for enabling JPA TRACE.

Scott

[mayerw01]

Hi Scott,

I don't think that this is related to HHH-7537. A new Band entity is persisted when the user pushes the button (and not via EntityListener). I also don't think that this is a general coding issue since this is running fine on the JBoss versions. I've just tested this successfully under jboss-as-7.1.1.Final. So the issue seems to be related to WildFly. But my understanding so far is that there should not be big differences (from application point of view) between these 2 versions.

[smarlow]

You should enable JPA TRACE logging and view the application console output (or server.log) to see what is actually happening. 

I looked at the example code at the link you mentioned and see an entity listener (de.ejb3buch.ticket2rock.entitylistener.NewEntityListener), which is why I thought you might be using an entity listener in your testing.

[mayerw01]

Whoops!

Sorry, I've overlooked this. So this is obviously the reason. But anyway strange why this works in JBoss and not in WildFly.

And unfortunately HHH-7537 does not provide any easy solution to this.

Do you have any suggestion?

Thanks

Wolfgang

[smarlow]

Hi Wolfgang,

I would start with enabling the JPA trace logging that I mentioned.  That will give you a better idea of what the application is doing exactly.  In terms of an easy solution, you could buffer the changes being made in the entity listener (HHH-7537 mentioned that in a comment).

Scott

[mayerw01]

Yes, Scott. I'd enabled the JPA trace log and verified that the issue was caused by em.persist() in NewEntityListener.

I am not sure if the approaches mentioned in HHH-7537 are feasible.

But I've now changed this class to bypass the EntityManager and use the standard database calls instead.

So this approach seem to work as well:

public class NewEntityListener {

    private Logger logger = Logger.getRootLogger();

    @Resource(mappedName = "java:jboss/datasources/OracleXATicket2RockDS")

    DataSource driver;

    @PostPersist

    protected void newEntity(Object entity) {

        logger.info("PostPersist for entity " + entity);

        Connection conn = null;

        ResultSet rs = null;

        try {

            News news = new News(entity);

            try {

                conn = driver.getConnection("ticket2rock", "ticket2rock");

            } catch (SQLException ex) {

                java.util.logging.Logger.getLogger(NewEntityListener.class.getName()).log(Level.SEVERE, null, ex);

            }

            String stmtString = "Insert into news (id, datum, nachricht)"

                    + " values (id_seq.nextval, sysdate, 'inserted: " + entity + "')";

            PreparedStatement stmt = conn.prepareStatement(stmtString);

            rs = stmt.executeQuery();

        } catch (Exception e) {

             logger.info("Exception in NewEntityListener");

            e.printStackTrace();

        }

    }

}

[smarlow]

Wolfgang,

I like your solution.  From a JPA perspective, there is a lot going on here.  The JTA transaction is ending.  The Synchronization.beforeCompletion calls are being called (no more non-interposed synchronizations can be registered but TransactionSynchronizationRegistry.registerInterposedSynchronization can still be called).  In theory, you could try to use a different persistence unit in the entity listener.  Depending on how the persistence provider is implemented (Hibernate properly uses TransactionSynchronizationRegistry.registerInterposedSynchronization but not all persistence providers did that), a different persistence provider would have its own "action queue" and that could work.  If the persistence provider doesn't work (e.g. toplink) you would have to flush the first persistence unit before the transaction.commit is called.  Overall, I think direct use of JDBC in the entity listener is a good solution.  If you have ideas for Hibernate improvements in this area, please submit a patch for your idea via the Hibernate project (see hibernate.org for community links).  I'm not exactly sure of what the patch should do but starting with a Hibernate unit test that fails and then changes to help it pass, is a good start.

Scott

[mayerw01]

Actually I thought also about using a second persistence unit.

The JPA specification could be understood that this should work:

"In general, the lifecycle method of a portable application should not invoke EntityManager

or Query operations, access other entity instances, or modify relationships within the

same persistence context."

But maybe this is not absolutely clear. And at least my test with a 2nd PU failed.