I have a Modeshape 5.3 installation into Wildfly 10 with as binary storage a MongoDb 3.4 replica set.

The complete setup works fine I can write a file to the system and read it back without any problems.

The problems start when the primary MongoDb changes. As soon as I change the primary node the file returned from Modeshape are corrupt. As soon as we turn back to the original primary than all works fine again.

I have checked the MongoDb cluster and the collections and records are identical.

The file size from the file returned are also identical.

I have seen in the MongodbBinaryStore that is chunks the files into smaller 1024 bytes chunks stores them and than on read puts them back together. From what I can see at my returned files is that the  ChunkInputStream is not putting them back in the same order that the ChunkOutputStream has written them. From what I have seen in the code so far the class doesn't read them in any specific order. It just reads all data chunks the way MongoDb decides to return them what apparently is different depending on what replica node is requested from.

I will dive into it a bit deeper tomorrow but has anybody experienced similar issues? Is there any workaround?