Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FastDiscardOfDeltas scanning all nodes and properties on delete #1955

Closed
Josipmrden opened this issue Apr 22, 2024 · 0 comments · Fixed by #2043
Closed

FastDiscardOfDeltas scanning all nodes and properties on delete #1955

Josipmrden opened this issue Apr 22, 2024 · 0 comments · Fixed by #2043
Assignees
Labels
bug bug customer customer Effort - Low Effort - Low Frequency - EveryTime Frequency - EveryTime Priority - Now Priority - Now Reach - Everyone Reach - Everyone Severity - S1 Severity - S1
Milestone

Comments

@Josipmrden
Copy link
Contributor

Josipmrden commented Apr 22, 2024

Memgraph version
v2.16

Environment
Ubuntu 22.04

Describe the bug
If any node has been deleted in the system, and the transaction is currently the only one running, it will go through the whole dataset and scan properties.

To Reproduce

  1. Create 10M nodes in Memgraph
  2. Create 10 indices on them
  3. create 5 nodes independent of those indices, and put an index on them
  4. do detach delete on those nodes

i.e.

CREATE CONSTRAINT ON (a:TestNode) ASSERT a.Id is UNIQUE;
Create Index on :TestNode;
Create Index on :TestNode(Id);

CREATE (n:TestNode{Id:1});
CREATE (n:TestNode{Id:2});
CREATE (n:TestNode{Id:3});
CREATE (n:TestNode{Id:4});
CREATE (n:TestNode{Id:5});

MATCH (n:TestNode) RETURN n;

MATCH (n:TestNode) DELETE n;
MATCH (n:TestNode) DETACH DELETE n;

ensure that you have enough number of nodes and properties in the system so you can mimick this

Expected behavior
The nodes should have been deleted fast

Additional context

image

void InMemoryStorage::InMemoryAccessor::FastDiscardOfDeltas(uint64_t oldest_active_timestamp,
                                                            std::unique_lock<std::mutex> /*gc_guard*/) {
  auto *mem_storage = static_cast<InMemoryStorage *>(storage_);

  // STEP 1 + STEP 2
  std::list<Gid> current_deleted_vertices;
  std::list<Gid> current_deleted_edges;
  GCRapidDeltaCleanup(current_deleted_vertices, current_deleted_edges);

  // STEP 3) skip_list removals
  if (!current_deleted_vertices.empty()) {
    // 3.a) clear from indexes first

THIS PART WILL GO OVER THE WHOLE DATASET
    std::stop_source dummy;
    mem_storage->indices_.RemoveObsoleteEntries(oldest_active_timestamp, dummy.get_token());
    auto *mem_unique_constraints =
        static_cast<InMemoryUniqueConstraints *>(mem_storage->constraints_.unique_constraints_.get());
    mem_unique_constraints->RemoveObsoleteEntries(oldest_active_timestamp, dummy.get_token());

    // 3.b) remove from veretex skip_list
    auto vertex_acc = mem_storage->vertices_.access();
    for (auto gid : current_deleted_vertices) {
      vertex_acc.remove(gid);
    }
  }

  if (!current_deleted_edges.empty()) {
    // 3.c) remove from edge skip_list
    auto edge_acc = mem_storage->edges_.access();
    auto edge_metadata_acc = mem_storage->edges_metadata_.access();
    for (auto gid : current_deleted_edges) {
      edge_acc.remove(gid);
      edge_metadata_acc.remove(gid);
    }
  }
}

I would also put the FastDiscardOfDeltas in a flag, so we can disable it optionally for users that are experiencing any issues. By default the fast discard would be turned on

@Josipmrden Josipmrden added bug bug Effort - Low Effort - Low Severity - S1 Severity - S1 Frequency - EveryTime Frequency - EveryTime Reach - Everyone Reach - Everyone labels Apr 22, 2024
@Josipmrden Josipmrden added this to the mg-v2.17.0 milestone Apr 22, 2024
@gitbuda gitbuda added the customer customer label Apr 22, 2024
@hal-eisen-MG hal-eisen-MG added the Priority - Now Priority - Now label Apr 28, 2024
@DavIvek DavIvek self-assigned this May 15, 2024
@Josipmrden Josipmrden modified the milestones: mg-v2.17.0, mg-v2.18.0 May 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug bug customer customer Effort - Low Effort - Low Frequency - EveryTime Frequency - EveryTime Priority - Now Priority - Now Reach - Everyone Reach - Everyone Severity - S1 Severity - S1
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants