Hallo,
So ich bin jetzt schon um einiges weiter. Wir haben gestern unsere CAS-Installation auf die X12 geupdated (meine Hoffnung war ja, dass damit alles wieder läuft, aber da war ich auf dem Holzweg...) und heute in der Früh haben wir uns mal die Logfiles vom Elasticsearch-Dienst vorgenommen und bin dann auf folgendes gestoßen:
Code:
[2020-11-26T03:01:14,701][WARN ][o.e.i.c.IndicesClusterStateService] [cas-searchservice-node] [[cas_ss_type_address][0]] marking and sending shard failed due to [failed recovery]
org.elasticsearch.indices.recovery.RecoveryFailedException: [cas_ss_type_address][0]: Recovery failed on {cas-searchservice-node}{UTeP3WKsSNijWCLWSDHx_A}{knXf178tT-qZZOcLLyBvuw}{127.0.0.1}{127.0.0.1:9300}{ml.machine_memory=17179303936, xpack.installed=true, ml.max_open_jobs=20, ml.enabled=true}
at org.elasticsearch.index.shard.IndexShard.lambda$startRecovery$6(IndexShard.java:2139) [elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.common.util.concurrent.ThreadContext$ContextPreservingRunnable.run(ThreadContext.java:624) [elasticsearch-6.5.4.jar:6.5.4]
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) [?:?]
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) [?:?]
at java.lang.Thread.run(Thread.java:834) [?:?]
Caused by: org.elasticsearch.index.shard.IndexShardRecoveryException: failed to recover from gateway
at org.elasticsearch.index.shard.StoreRecovery.internalRecoverFromStore(StoreRecovery.java:430) ~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.index.shard.StoreRecovery.lambda$recoverFromStore$0(StoreRecovery.java:95) ~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.index.shard.StoreRecovery.executeRecovery(StoreRecovery.java:302) ~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.index.shard.StoreRecovery.recoverFromStore(StoreRecovery.java:93) ~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.index.shard.IndexShard.recoverFromStore(IndexShard.java:1645) ~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.index.shard.IndexShard.lambda$startRecovery$6(IndexShard.java:2135) ~[elasticsearch-6.5.4.jar:6.5.4]
... 4 more
Caused by: org.elasticsearch.index.engine.EngineCreationFailureException: failed to create engine
at org.elasticsearch.index.engine.InternalEngine.<init>(InternalEngine.java:208) ~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.index.engine.InternalEngine.<init>(InternalEngine.java:168) ~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.index.engine.InternalEngineFactory.newReadWriteEngine(InternalEngineFactory.java:25) ~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.index.shard.IndexShard.newEngine(IndexShard.java:2266) ~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.index.shard.IndexShard.createNewEngine(IndexShard.java:2248) ~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.index.shard.IndexShard.innerOpenEngineAndTranslog(IndexShard.java:1413) ~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.index.shard.IndexShard.openEngineAndRecoverFromTranslog(IndexShard.java:1370) ~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.index.shard.StoreRecovery.internalRecoverFromStore(StoreRecovery.java:425) ~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.index.shard.StoreRecovery.lambda$recoverFromStore$0(StoreRecovery.java:95) ~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.index.shard.StoreRecovery.executeRecovery(StoreRecovery.java:302) ~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.index.shard.StoreRecovery.recoverFromStore(StoreRecovery.java:93) ~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.index.shard.IndexShard.recoverFromStore(IndexShard.java:1645) ~[elasticsearch-6.5.4.jar:6.5.4]
at org.elasticsearch.index.shard.IndexShard.lambda$startRecovery$6(IndexShard.java:2135) ~[elasticsearch-6.5.4.jar:6.5.4]
... 4 more
Caused by: org.elasticsearch.index.translog.TranslogCorruptedException: translog from source [C:\Program Files (x86)\Common Files\CAS-Software\Server\JavaServices\ElasticSearch\data\nodes\0\indices\IkwXGjHWQ5ixKh_RvfGOEA\0\translog\translog-3321.tlog] is corrupted, translog header corrupted
Damit war für mich eigentlich alles klar: ElasticSearch-Dienst gestoppt, den data-Ordner und den log Ordner umbenannt und dann wieder gestartet. Resultat: Diese Exception war weg, aber als ich dann den Suchdienst von CAS wieder gestartet hab, hab ich dann folgende Fehlermeldungen bekommen: Not yet initialized (you may need to run sgadmin)
Dann hab ich das nochmal gemacht, aber diesmal den Inhalt von data_template in den data-Ordner kopiert und anschließend neu gestartet, dann war auch diese Fehlermeldung weg, aber irgendwie scheints immer noch zu haken... Gibt's irgendeine Möglichkeit, dass man mal manuell in den ElasticSearch reinschauen kann, was da jetzt los ist? Also aus den Logs werd ich nicht schlauer, die schauen jetzt auf alle Fälle wieder sauber aus...
Ich hab was von einer REST-API gesehen, aber da wird immer nach SearchGuard credentials gefragt, die ich aber nicht habe...
