feat: improve index hits #2314

zpw-123 · 2023-09-17T07:04:16Z

EdgeCoreTest.java与VertexCoreTest.java是部分索引的例子！

Purpose of the PR

close #xxx

Main Changes

Verifying these changes

Trivial rework / code cleanup without any test coverage. (No Need)
Already covered by existing tests, such as (please modify tests here).
Need tests and can be verified as follows:
- xxx

Does this PR potentially affect the following parts?

Documentation Status

Doc - TODO
Doc - Done
Doc - No Need

EdgeCoreTest.java与VertexCoreTest.java是部分索引的例子！

javeme

Thank you very much for your PR.
Could you briefly describe the feature intent and design framework?

javeme · 2023-09-30T19:41:25Z

hugegraph-core/src/main/java/org/apache/hugegraph/backend/tx/GraphIndexTransaction.java


        public MatchedIndex(SchemaLabel schemaLabel,
                            Set<IndexLabel> indexLabels) {
-            this.schemaLabel = schemaLabel;
+            //this.schemaLabels = new HashSet<>();


javeme · 2023-09-30T19:41:49Z

hugegraph-core/src/main/java/org/apache/hugegraph/backend/tx/GraphTransaction.java

-        for (ConditionQuery cq : ConditionQueryFlatten.flatten(
-                                 (ConditionQuery) query, supportIn)) {
+        for (ConditionQuery cq: ConditionQueryFlatten.flatten(
+            (ConditionQuery) query, supportIn)) {


prefer to keep the old style

javeme · 2023-09-30T19:42:15Z

hugegraph-core/src/main/java/org/apache/hugegraph/backend/tx/GraphTransaction.java

@@ -1405,13 +1407,127 @@ private <R> QueryList<R> optimizeQueries(Query query,
             * 2.index-query result(ids after optimization), which may be empty.
             */
            if (q == null) {
-                queries.add(this.indexQuery(cq), this.batchSize);
+                boolean sys = cq.syspropConditions().size() != 0;


javeme · 2023-09-30T19:44:37Z

hugegraph-core/src/main/java/org/apache/hugegraph/backend/tx/AbstractTransaction.java

@@ -133,7 +133,7 @@ public Number queryNumber(Query query) {
    }

    @Watched(prefix = "tx")
-    public QueryResults<BackendEntry> query(Query query) {
+    protected QueryResults<BackendEntry> query(Query query) {


not sure why need to mark protected

javeme · 2023-09-30T19:45:42Z

hugegraph-core/src/main/java/org/apache/hugegraph/backend/tx/GraphTransaction.java

+    }
+    // Method to reorder conditions
+    private void excludeOnlyLabelQuery(ConditionQuery cq, QueryList queries) {
+        // 判断是否命中索引


can we translate all Chinese comments?

javeme · 2023-09-30T19:49:18Z

hugegraph-core/src/main/java/org/apache/hugegraph/backend/tx/GraphTransaction.java

+                .collect(Collectors.toSet());
+
+        Id label = rawQuery.condition(HugeKeys.LABEL);
+        if (label == null) {


prefer return if (label != null)

zpw-123 · 2023-10-06T08:43:51Z

前辈好我是#2314 目前好像很多测试都没有通过，我毕竟就修改170+行，为啥这么多测试都通过不了啊！然后实现的思路大概是：首先需要判断是否命中索引，如果部分命中索引就创建更加宽泛的条件查询语句，后面再用过滤操作来筛选出来最后结果！

…

------------------ 原始邮件 ------------------ 发件人: "apache/incubator-hugegraph" ***@***.***>; 发送时间: 2023年10月1日(星期天) 凌晨3:57 ***@***.***>; ***@***.******@***.***>; 主题: Re: [apache/incubator-hugegraph] 修改了上面的几个文件，来实现部分索引命中！ (PR #2314) @javeme commented on this pull request. Thank you very much for your PR. Could you briefly describe the feature intent and design framework? In hugegraph-core/src/main/java/org/apache/hugegraph/backend/tx/GraphIndexTransaction.java: > public MatchedIndex(SchemaLabel schemaLabel, Set<IndexLabel> indexLabels) { - this.schemaLabel = schemaLabel; + //this.schemaLabels = new HashSet<>(); remove it? In hugegraph-core/src/main/java/org/apache/hugegraph/backend/tx/GraphTransaction.java: > @@ -1395,8 +1397,8 @@ private <R> QueryList<R> optimizeQueries(Query query, } boolean supportIn = this.storeFeatures().supportsQueryWithInCondition(); - for (ConditionQuery cq : ConditionQueryFlatten.flatten( - (ConditionQuery) query, supportIn)) { + for (ConditionQuery cq: ConditionQueryFlatten.flatten( + (ConditionQuery) query, supportIn)) { prefer to keep the old style In hugegraph-core/src/main/java/org/apache/hugegraph/backend/tx/GraphTransaction.java: > @@ -1405,13 +1407,127 @@ private <R> QueryList<R> optimizeQueries(Query query, * 2.index-query result(ids after optimization), which may be empty. */ if (q == null) { - queries.add(this.indexQuery(cq), this.batchSize); + boolean sys = cq.syspropConditions().size() != 0; sysprop? In hugegraph-core/src/main/java/org/apache/hugegraph/backend/tx/AbstractTransaction.java: > @@ -133,7 +133,7 @@ public Number queryNumber(Query query) { } @watched(prefix = "tx") - public QueryResults<BackendEntry> query(Query query) { + protected QueryResults<BackendEntry> query(Query query) { not sure why need to mark protected In hugegraph-core/src/main/java/org/apache/hugegraph/backend/tx/GraphTransaction.java: > } else if (!q.empty()) { queries.add(q); } } return queries; } + private boolean hasOlapCondition(ConditionQuery query) { + for (Id pk : query.userpropKeys()) { + if(this.graph().propertyKey(pk).olap()) { + return true; + } + } + + return false; + } + // Method to reorder conditions + private void excludeOnlyLabelQuery(ConditionQuery cq, QueryList queries) { + // 判断是否命中索引 can we translate all Chinese comments? In hugegraph-core/src/main/java/org/apache/hugegraph/backend/tx/GraphTransaction.java: > + + // 如果条件查询中包含olap相关属性查询，暂不做补充查询 + if(this.hasOlapCondition(rawQuery)) { + return queries; + } + + Collection<VertexLabel> vertexLabels = this.graph().vertexLabels(); + Collection<EdgeLabel> edgeLabels = this.graph().edgeLabels(); + + Set<SchemaLabel> matchedLabels = + matchedIndices.stream() + .flatMap(matchedIndex -> matchedIndex.schemaLabels().stream()) + .collect(Collectors.toSet()); + + Id label = rawQuery.condition(HugeKeys.LABEL); + if (label == null) { prefer return if (label != null) — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you authored the thread.Message ID: ***@***.***>

simon824 · 2023-10-27T01:50:50Z

hi @zpw-123 , is it still in progress?

zpw-123 · 2023-10-27T12:46:15Z

是的

…

---Original--- From: "Simon ***@***.***> Date: Fri, Oct 27, 2023 09:51 AM To: ***@***.***>; Cc: ***@***.******@***.***>; Subject: Re: [apache/incubator-hugegraph] feat: improve index hits (PR #2314) hi @zpw-123 , is it still in progress? — Reply to this email directly, view it on GitHub, or unsubscribe. You are receiving this because you were mentioned.Message ID: ***@***.***>

github-actions · 2023-11-26T21:03:53Z

Due to the lack of activity, the current pr is marked as stale and will be closed after 180 days, any update will remove the stale label

javeme · 2023-11-27T03:59:49Z

前辈好我是#2314 目前好像很多测试都没有通过，我毕竟就修改170+行，为啥这么多测试都通过不了啊！然后实现的思路大概是：首先需要判断是否命中索引，如果部分命中索引就创建更加宽泛的条件查询语句，后面再用过滤操作来筛选出来最后结果！

@zpw-123 总体思路是可以的，不过涉及面较广，希望能补充一下详细设计文档。先进行文档Review没问题之后，再进行代码修改。如果有什么问题欢迎通过HugeGraph微信公众号联系我们。

github-actions · 2023-12-28T21:03:48Z

Due to the lack of activity, the current pr is marked as stale and will be closed after 180 days, any update will remove the stale label

修改了上面的几个文件，来实现部分索引命中！

a067c3f

EdgeCoreTest.java与VertexCoreTest.java是部分索引的例子！

imbajin requested review from javeme and zyxxoo September 17, 2023 08:27

javeme reviewed Sep 30, 2023

View reviewed changes

simon824 changed the title ~~修改了上面的几个文件，来实现部分索引命中！~~ feat: improve index hits Oct 27, 2023

github-actions bot added the inactive label Nov 26, 2023

javeme removed the inactive label Nov 27, 2023

imbajin marked this pull request as draft November 28, 2023 03:10

github-actions bot added the inactive label Dec 28, 2023

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat: improve index hits #2314

feat: improve index hits #2314

zpw-123 commented Sep 17, 2023

javeme left a comment

javeme Sep 30, 2023

javeme Sep 30, 2023

javeme Sep 30, 2023

javeme Sep 30, 2023

javeme Sep 30, 2023

javeme Sep 30, 2023

zpw-123 commented Oct 6, 2023 via email

simon824 commented Oct 27, 2023

zpw-123 commented Oct 27, 2023 via email

github-actions bot commented Nov 26, 2023

javeme commented Nov 27, 2023

github-actions bot commented Dec 28, 2023

feat: improve index hits #2314

Are you sure you want to change the base?

feat: improve index hits #2314

Conversation

zpw-123 commented Sep 17, 2023

Purpose of the PR

Main Changes

Verifying these changes

Does this PR potentially affect the following parts?

Documentation Status

javeme left a comment

Choose a reason for hiding this comment

javeme Sep 30, 2023

Choose a reason for hiding this comment

javeme Sep 30, 2023

Choose a reason for hiding this comment

javeme Sep 30, 2023

Choose a reason for hiding this comment

javeme Sep 30, 2023

Choose a reason for hiding this comment

javeme Sep 30, 2023

Choose a reason for hiding this comment

javeme Sep 30, 2023

Choose a reason for hiding this comment

zpw-123 commented Oct 6, 2023 via email

simon824 commented Oct 27, 2023

zpw-123 commented Oct 27, 2023 via email

github-actions bot commented Nov 26, 2023

javeme commented Nov 27, 2023

github-actions bot commented Dec 28, 2023