Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change error logging for opensearchadapter #922

Merged
merged 1 commit into from
May 8, 2024
Merged

Change error logging for opensearchadapter #922

merged 1 commit into from
May 8, 2024

Conversation

vthacker
Copy link
Collaborator

@vthacker vthacker commented May 8, 2024

Summary

A common error we see is

{"@timestamp":"2024-05-08T19:28:49.766Z","log.level":"ERROR","log.message":"Error doing map update","process.thread.name":"Timer-2","log.logger":"com.slack.astra.logstore.opensearch.OpenSearchAdapter","error_type":"java.lang.IllegalArgumentException","error_message":"can't merge a non object mapping [service] with an object mapping","error_stack_trace":"java.lang.IllegalArgumentException: can't merge a non object mapping [service] with an object mapping\n\tat org.opensearch.index.mapper.ObjectMapper.merge(ObjectMapper.java:576)\n\tat org.opensearch.index.mapper.ObjectMapper.doMerge(ObjectMapper.java:607)\n\tat org.opensearch.index.mapper.RootObjectMapper.doMerge(RootObjectMapper.java:351)\n\tat org.opensearch.index.mapper.ObjectMapper.merge(ObjectMapper.java:580)\n\tat org.opensearch.index.mapper.RootObjectMapper.merge(RootObjectMapper.java:346)\n\tat org.opensearch.index.mapper.Mapping.merge(Mapping.java:128)\n\tat org.opensearch.index.mapper.DocumentMapper.merge(DocumentMapper.java:307)\n\tat org.opensearch.index.mapper.MapperService.internalMerge(MapperService.java:482)\n\tat org.opensearch.index.mapper.MapperService.internalMerge(MapperService.java:444)\n\tat org.opensearch.index.mapper.MapperService.merge(MapperService.java:416)\n\tat com.slack.astra.logstore.opensearch.OpenSearchAdapter.tryRegisterField(OpenSearchAdapter.java:505)\n\tat com.slack.astra.logstore.opensearch.OpenSearchAdapter.reloadSchema(OpenSearchAdapter.java:228)\n\tat com.slack.astra.logstore.search.LogIndexSearcherImpl$1.afterRefresh(LogIndexSearcherImpl.java:73)\n\tat org.apache.lucene.search.ReferenceManager.notifyRefreshListenersRefreshed(ReferenceManager.java:275)\n\tat org.apache.lucene.search.ReferenceManager.doMaybeRefresh(ReferenceManager.java:182)\n\tat org.apache.lucene.search.ReferenceManager.maybeRefresh(ReferenceManager.java:213)\n\tat com.slack.astra.logstore.LuceneIndexStoreImpl.syncRefresh(LuceneIndexStoreImpl.java:227)\n\tat com.slack.astra.logstore.LuceneIndexStoreImpl.lambda$refresh$1(LuceneIndexStoreImpl.java:295)\n\tat io.micrometer.core.instrument.AbstractTimer.record(AbstractTimer.java:247)\n\tat com.slack.astra.logstore.LuceneIndexStoreImpl.refresh(LuceneIndexStoreImpl.java:291)\n\tat com.slack.astra.logstore.LuceneIndexStoreImpl$2.run(LuceneIndexStoreImpl.java:140)\n\tat java.base/java.util.TimerThread.mainLoop(Timer.java:566)\n\tat java.base/java.util.TimerThr...","error_root_cause_class_name":"java.lang.IllegalArgumentException","error_root_cause_message":"can't merge a non object mapping [service] with an object mapping","error_root_cause_stack_trace":"java.lang.IllegalArgumentException: can't merge a non object mapping [service] with an object mapping\n\tat org.opensearch.index.mapper.ObjectMapper.merge(ObjectMapper.java:576)\n\tat org.opensearch.index.mapper.ObjectMapper.doMerge(ObjectMapper.java:607)\n\tat org.opensearch.index.mapper.RootObjectMapper.doMerge(RootObjectMapper.java:351)\n\tat org.opensearch.index.mapper.ObjectMapper.merge(ObjectMapper.java:580)\n\tat org.opensearch.index.mapper.RootObjectMapper.merge(RootObjectMapper.java:346)\n\tat org.opensearch.index.mapper.Mapping.merge(Mapping.java:128)\n\tat org.opensearch.index.mapper.DocumentMapper.merge(DocumentMapper.java:307)\n\tat org.opensearch.index.mapper.MapperService.internalMerge(MapperService.java:482)\n\tat org.opensearch.index.mapper.MapperService.internalMerge(MapperService.java:444)\n\tat org.opensearch.index.mapper.MapperService.merge(MapperService.java:416)\n\tat com.slack.astra.logstore.opensearch.OpenSearchAdapter.tryRegisterField(OpenSearchAdapter.java:505)\n\tat com.slack.astra.logstore.opensearch.OpenSearchAdapter.reloadSchema(OpenSearchAdapter.java:228)\n\tat com.slack.astra.logstore.search.LogIndexSearcherImpl$1.afterRefresh(LogIndexSearcherImpl.java:73)\n\tat org.apache.lucene.search.ReferenceManager.notifyRefreshListenersRefreshed(ReferenceManager.java:275)\n\tat org.apache.lucene.search.ReferenceManager.doMaybeRefresh(ReferenceManager.java:182)\n\tat org.apache.lucene.search.ReferenceManager.maybeRefresh(ReferenceManager.java:213)\n\tat com.slack.astra.logstore.LuceneIndexStoreImpl.syncRefresh(LuceneIndexStoreImpl.java:227)\n\tat com.slack.astra.logstore.LuceneIndexStoreImpl.lambda$refresh$1(LuceneIndexStoreImpl.java:295)\n\tat io.micrometer.core.instrument.AbstractTimer.record(AbstractTimer.java:247)\n\tat com.slack.astra.logstore.LuceneIndexStoreImpl.refresh(LuceneIndexStoreImpl.java:291)\n\tat com.slack.astra.logstore.LuceneIndexStoreImpl$2.run(LuceneIndexStoreImpl.java:140)\n\tat java.base/java.util.TimerThread.mainLoop(Timer.java:566)\n\tat java.base/java.util.TimerThr..."}

I don't think this stack trace is adding any value.We see tens of thousands of these across our logging clusters . let's not print the stack trace and save our logging cluster that collects these logs

@vthacker vthacker merged commit 2d7c87f into master May 8, 2024
2 checks passed
@vthacker vthacker deleted the logging branch May 8, 2024 21:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants