Merge failed with failures

[mh-nt]

I am currently experience issues with RTGov/Elasticsearch on a test setup. Started with an empty Elasticsearch, RTGov created its indice without issues. After restarting the JBoss server, no data is written to RTGov and the following message appears in the logfile:

2017-08-23 02:10:14,318 SEVERE [org.overlord.rtgov.internal.epn.loader.fsw60.JEEEPNLoader] (ServerService Thread Pool -- 91) [] Failed to register network: org.elasticsearch.index.mapper.MergeMappingException: Merge failed with failures {[mapper [timestamp] of different type, current_type [long], merged_type [date], mapper [min] of different type, current_type [long], merged_type [integer], object mapping [requestId] can't be changed from non-nested to nested, mapper [operation] has different index values, mapper [operation] has different tokenize values, mapper [operation] has different index_analyzer, mapper [interface] has different index values, mapper [interface] has different tokenize values, mapper [interface] has different index_analyzer, mapper [average] of different type, current_type [long], merged_type [integer], object mapping [context] can't be changed from non-nested to nested, mapper [max] of different type, current_type [long], merged_type [integer], object mapping [responseId] can't be changed from non-nested to nested, mapper [serviceType] has different index values, mapper [serviceType] has different tokenize values, mapper [serviceType] has different index_analyzer]}

The mapping in the ES indice looks like the message states. I deleted RTGovs indices and restarted JBoss - then everything was working fine until the next restart.

My service is based on FSW6, RTGov is using its own Elasticsearch-Cluster like in thread 255968.

Any idea what can be wrong?

[objectiser]

Has there been any changes to the Elasticsearch version being used?

[objectiser]

Its been a while since I worked with Elasticsearch, but just as an experiment, instead of just removing the indices, could you remove the complete data folder - just to see whether it initialises the mapping correctly. You error message appears to suggest some data is being written which is creating an incorrect mapping - and then when the real schema is being applied it is failing.

[mh-nt]

Thank you very much for responding.

No changes to the Elasticsearch version, we are using 1.3.2. Already tried to remove the data folder, at the next JBoss restart the indices are created. Restarting JBoss once more then and the issue happens.

In some other environments it is running fine as by now.

[objectiser]

Sorry don't know what to suggest then - this seems very strange behaviour, as it implies that the indices are created correctly on first start, but then somehow don't exist on second start, so Elasticsearch is using a default mapping - but then it tries to create the mapping again resulting in the errors.

Is there any task running after the server shuts down that would possibly clear the indices?

[mh-nt]

For us it looks like something is modifying the mapping in some cases, but we could not find the cause.

We tried to reindex existing data with wrong mapping with a 3rd party Elasticsearch plugin into a index with a proper mapping. Works in our test environments, so we have a workaround.

Thanks you very much for your help!