Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Takes Too much Time during get and put operation using version 2.1.214 #4054

Open
chaitas90 opened this issue May 14, 2024 · 2 comments
Open

Takes Too much Time during get and put operation using version 2.1.214 #4054

chaitas90 opened this issue May 14, 2024 · 2 comments

Comments

@chaitas90
Copy link

chaitas90 commented May 14, 2024 •
edited

We are using h2 MVStore version 2.1.214

MVStore configDB = new MVStore.Builder().fileName(GlobalConstants.CURRENT_DIR +
GlobalConstants.PATH_SEPARATOR +
GlobalConstants.CONFIG_DIR +
GlobalConstants.PATH_SEPARATOR +
"????")
.encryptionKey(new String(Base64.getDecoder().decode(buffer.getBytes())).toCharArray())
.backgroundExceptionHandler((thread, exception) -> LOGGER.error(exception))
.autoCommitDisabled()
.open();

TO get data we use following logic::

configDB.openMap(collection).forEach((key, value) ->
                                {
                                    var document = new JsonObject(CommonUtil.getString(value));

                                    if (document.containsKey(query.getString(ConfigDBConstants.FIELD_NAME))
                                            && document.getValue(query.getString(ConfigDBConstants.FIELD_NAME)).equals(query.getValue(VALUE)))
                                    {
                                        items.add(transform(document));
                                    }
                                });

Over here during fetching data my thread gets stuck for almost 1-2 minutes also in thread dump i get following:

10:43:07 AM

java.lang.Thread.State: RUNNABLE
at org.h2.security.AES.decrypt(AES.java:134)
at org.h2.store.fs.encrypt.XTS.decrypt(XTS.java:83)
at org.h2.store.fs.encrypt.FileEncrypt.readInternal(FileEncrypt.java:185)
at org.h2.store.fs.encrypt.FileEncrypt.read(FileEncrypt.java:171)
at org.h2.mvstore.DataUtils.readFully(DataUtils.java:456)
at org.h2.mvstore.FileStore.readFully(FileStore.java:98)
at org.h2.mvstore.Chunk.readBufferForPage(Chunk.java:422)
at org.h2.mvstore.MVStore.readPage(MVStore.java:2569)
at org.h2.mvstore.MVMap.readPage(MVMap.java:633)
at org.h2.mvstore.Page$NonLeaf.getChildPage(Page.java:1125)
at org.h2.mvstore.Cursor.hasNext(Cursor.java:64)
at org.h2.mvstore.MVMap$2$1.hasNext(MVMap.java:746)
at java.base@17.0.6/java.util.concurrent.ConcurrentMap.forEach(Unknown Source)
at com.mindarray.db.ConfigDBServiceImpl.lambda$get$11(ConfigDBServiceImpl.java:249)
at com.mindarray.db.ConfigDBServiceImpl$$Lambda$280/0x0000000800304ad0.handle(Unknown Source)
at io.vertx.core.impl.ContextBase.lambda$null$0(ContextBase.java:137)
at io.vertx.core.impl.ContextBase$$Lambda$218/0x00000008002b9cd8.handle(Unknown Source)
at io.vertx.core.impl.ContextInternal.dispatch(ContextInternal.java:264)
at io.vertx.core.impl.ContextBase.lambda$executeBlocking$1(ContextBase.java:135)
at io.vertx.core.impl.ContextBase$$Lambda$216/0x00000008002b9470.run(Unknown Source)
at io.vertx.core.impl.TaskQueue.run(TaskQueue.java:76)
at io.vertx.core.impl.TaskQueue$$Lambda$197/0x00000008002b16e0.run(Unknown Source)
at java.base@17.0.6/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base@17.0.6/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base@17.0.6/java.lang.Thread.run(Unknown Source)

10:44:37 AM

java.lang.Thread.State: RUNNABLE
at org.h2.store.fs.encrypt.XTS.decrypt(XTS.java:73)
at org.h2.store.fs.encrypt.FileEncrypt.readInternal(FileEncrypt.java:185)
at org.h2.store.fs.encrypt.FileEncrypt.read(FileEncrypt.java:171)
at org.h2.mvstore.DataUtils.readFully(DataUtils.java:456)
at org.h2.mvstore.FileStore.readFully(FileStore.java:98)
at org.h2.mvstore.Chunk.readBufferForPage(Chunk.java:422)
at org.h2.mvstore.MVStore.readPage(MVStore.java:2569)
at org.h2.mvstore.MVMap.readPage(MVMap.java:633)
at org.h2.mvstore.Page$NonLeaf.getChildPage(Page.java:1125)
at org.h2.mvstore.Cursor.hasNext(Cursor.java:64)
at org.h2.mvstore.MVMap$2$1.hasNext(MVMap.java:746)
at java.base@17.0.6/java.util.concurrent.ConcurrentMap.forEach(Unknown Source)
at com.mindarray.db.ConfigDBServiceImpl.lambda$update$33(ConfigDBServiceImpl.java:655)
at com.mindarray.db.ConfigDBServiceImpl$$Lambda$278/0x0000000800304680.handle(Unknown Source)
at com.mindarray.db.ConfigDBServiceImpl.lambda$getOneByQuery$14(ConfigDBServiceImpl.java:340)
at com.mindarray.db.ConfigDBServiceImpl$$Lambda$279/0x00000008003048a8.handle(Unknown Source)
at com.mindarray.db.ConfigDBServiceImpl.lambda$get$12(ConfigDBServiceImpl.java:275)
at com.mindarray.db.ConfigDBServiceImpl$$Lambda$281/0x0000000800304cf8.handle(Unknown Source)
at io.vertx.core.impl.future.FutureImpl$3.onSuccess(FutureImpl.java:141)
at io.vertx.core.impl.future.FutureBase.lambda$emitSuccess$0(FutureBase.java:54)
at io.vertx.core.impl.future.FutureBase$$Lambda$224/0x00000008002e58c0.run(Unknown Source)
at io.netty.util.concurrent.AbstractEventExecutor.runTask(AbstractEventExecutor.java:174)
at io.netty.util.concurrent.AbstractEventExecutor.safeExecute(AbstractEventExecutor.java:167)
at io.netty.util.concurrent.SingleThreadEventExecutor.runAllTasks(SingleThreadEventExecutor.java:470)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:569)
at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997)
at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base@17.0.6/java.lang.Thread.run(Unknown Source)

java.lang.Thread.State: RUNNABLE
at com.fasterxml.jackson.core.json.ReaderBasedJsonParser.getText(ReaderBasedJsonParser.java:322)
at com.fasterxml.jackson.databind.deser.std.UntypedObjectDeserializerNR.deserialize(UntypedObjectDeserializerNR.java:82)
at com.fasterxml.jackson.databind.deser.std.MapDeserializer._readAndBindStringKeyMap(MapDeserializer.java:623)
at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:449)
at com.fasterxml.jackson.databind.deser.std.MapDeserializer.deserialize(MapDeserializer.java:32)
at com.fasterxml.jackson.databind.deser.DefaultDeserializationContext.readRootValue(DefaultDeserializationContext.java:323)
at com.fasterxml.jackson.databind.ObjectMapper._readValue(ObjectMapper.java:4706)
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:2879)
at io.vertx.core.json.jackson.DatabindCodec.fromParser(DatabindCodec.java:126)
at io.vertx.core.json.jackson.DatabindCodec.fromString(DatabindCodec.java:90)
at io.vertx.core.json.JsonObject.fromJson(JsonObject.java:1250)
at io.vertx.core.json.JsonObject.(JsonObject.java:51)
at com.mindarray.db.ConfigDBServiceImpl.lambda$get$9(ConfigDBServiceImpl.java:251)
at com.mindarray.db.ConfigDBServiceImpl$$Lambda$2209/0x0000000800670900.accept(Unknown Source)
at java.base@17.0.6/java.util.concurrent.ConcurrentMap.forEach(Unknown Source)
at com.mindarray.db.ConfigDBServiceImpl.lambda$get$11(ConfigDBServiceImpl.java:249)
at com.mindarray.db.ConfigDBServiceImpl$$Lambda$280/0x0000000800304ad0.handle(Unknown Source)
at io.vertx.core.impl.ContextBase.lambda$null$0(ContextBase.java:137)
at io.vertx.core.impl.ContextBase$$Lambda$218/0x00000008002b9cd8.handle(Unknown Source)
at io.vertx.core.impl.ContextInternal.dispatch(ContextInternal.java:264)
at io.vertx.core.impl.ContextBase.lambda$executeBlocking$1(ContextBase.java:135)
at io.vertx.core.impl.ContextBase$$Lambda$216/0x00000008002b9470.run(Unknown Source)
at io.vertx.core.impl.TaskQueue.run(TaskQueue.java:76)
at io.vertx.core.impl.TaskQueue$$Lambda$197/0x00000008002b16e0.run(Unknown Source)
at java.base@17.0.6/java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
at java.base@17.0.6/java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30)
at java.base@17.0.6/java.lang.Thread.run(Unknown Source)

Any help what could be the problem will be highly appreciated..
@grandinj
Copy link
Contributor

You are querying your database by looping over every single row, and then decoding a JSON blob for every row.
What did you expect? Of course it is going to be slow.

@chaitas90
Copy link
Author

chaitas90 commented May 15, 2024 •
edited

@grandinj Reason for iterating every row is that during update I don't have way to direct update with ID i have to iterate through openMap of collection fetch its id and put into it..

Is there any way i can direct update it using its ID or fetch data directly using ID??

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@grandinj @chaitas90