Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update nostrdb, switch to local relay model #2121

Draft
wants to merge 116 commits into
base: master
Choose a base branch
from
Draft

Update nostrdb, switch to local relay model #2121

wants to merge 116 commits into from

Conversation

jb55
Copy link
Collaborator

@jb55 jb55 commented Apr 4, 2024

This branch updates nostrdb and switches to a local relay model

Things done and left to do:

  • Update nostrdb
  • Switch to nostrdb for note block rendering
  • Switch to local relay for querying and subscriptions

@jb55 jb55 linked an issue Apr 4, 2024 that may be closed by this pull request
@jb55
Copy link
Collaborator Author

jb55 commented Apr 5, 2024

This broke a bunch of tests which I'm fixing now

@jb55
Copy link
Collaborator Author

jb55 commented Apr 5, 2024

One thing I did here is add nostrdb's patch history to our local copy of nostrdb. This ensure we don't lose bisectability when updating nostrdb. I've also removed a lot of overlap between damus' C code and nostrdb, so most of the C code should be in nostrdb now.

jb55 and others added 22 commits April 25, 2024 15:39
When trying to build from rust

Signed-off-by: William Casarin <jb55@jb55.com>
Closes: damus-io/nostrdb#21
Signed-off-by: William Casarin <jb55@jb55.com>
Right now it's accidently hardcoded.

Fixes: 8376e5bca05c ("add "import -"")
Signed-off-by: William Casarin <jb55@jb55.com>
everything will be in here soon
rust doesn't like packed structures, so hide this from bindgen

This also buttons up the API so less things are exposed which is good.

Signed-off-by: William Casarin <jb55@jb55.com>
Signed-off-by: William Casarin <jb55@jb55.com>
This is the start of our rust library for nostrdb. Implement idiomatic
interfaces for Ndb and NdbConfig.

Changelog-Added: Add initial rust library
Signed-off-by: William Casarin <jb55@jb55.com>
otherwise build fails

Signed-off-by: William Casarin <jb55@jb55.com>
since it was overwritten when we synced with damus

Signed-off-by: William Casarin <jb55@jb55.com>
we will be applying a patch here as well
A lot of this was pulled from core-lightning. Not sure what is actually
needed or not.

Signed-off-by: William Casarin <jb55@jb55.com>
Signed-off-by: William Casarin <jb55@jb55.com>
Signed-off-by: William Casarin <jb55@jb55.com>
Signed-off-by: William Casarin <jb55@jb55.com>
since I keep overwriting it by accident

Signed-off-by: William Casarin <jb55@jb55.com>
Signed-off-by: William Casarin <jb55@jb55.com>
Signed-off-by: William Casarin <jb55@jb55.com>
Signed-off-by: William Casarin <jb55@jb55.com>
Signed-off-by: William Casarin <jb55@jb55.com>
jb55 added 25 commits April 25, 2024 15:39
This fixes a few ownership issues

Signed-off-by: William Casarin <jb55@jb55.com>
Signed-off-by: William Casarin <jb55@jb55.com>
Signed-off-by: William Casarin <jb55@jb55.com>
Signed-off-by: William Casarin <jb55@jb55.com>
This is a pretty scary looking function that realloc our large variable
filter buffer into a compact one. This saves up a bunch of memory when
we are done building the filter.

Signed-off-by: William Casarin <jb55@jb55.com>
Instead of storing exact pointers inside of our filter elements, just
store offsets. This will allow us to clone filters very easily without
having to mess around with fixing up the pointers afterwards.

Signed-off-by: William Casarin <jb55@jb55.com>
Clone filters when moving them into subscriptions. This will allow us to
fix the double free issue on the rust side.

Signed-off-by: William Casarin <jb55@jb55.com>
Signed-off-by: William Casarin <jb55@jb55.com>
Signed-off-by: William Casarin <jb55@jb55.com>
Signed-off-by: William Casarin <jb55@jb55.com>
Signed-off-by: William Casarin <jb55@jb55.com>
Signed-off-by: William Casarin <jb55@jb55.com>
This introduces the basic created_at query plan. We scan the created_at
+ id index in descending order looking for items that match a filter.
This is a very general query plan, but might not be very efficient for
anything other than local timelines.

Changelog-Added: Add general created_at query plan for timelines
Closes: damus-io/nostrdb#26
Signed-off-by: William Casarin <jb55@jb55.com>
This is less efficient for now but we don't have a small-author-list
query plan yet.

Signed-off-by: William Casarin <jb55@jb55.com>
Signed-off-by: William Casarin <jb55@jb55.com>
A small size_t/uint64 conversion issue

Signed-off-by: William Casarin <jb55@jb55.com>
Since Damus iOS is not an immediate-mode UI like android, we would
rather not poll for results. Instead we need a way to register a
callback function that is called when we get new subscription results.

This is also useful on the android side, allowing us to request a new
frame to draw when we have new results, instead of drawing every second.

Signed-off-by: William Casarin <jb55@jb55.com>
some binding dir stoppers, and configurator
doesn't need to create a copy of the id

Signed-off-by: William Casarin <jb55@jb55.com>
This is still kind of broken until queries are switched over to nostrdb.
Will do this next

Signed-off-by: William Casarin <jb55@jb55.com>
Signed-off-by: William Casarin <jb55@jb55.com>
Signed-off-by: William Casarin <jb55@jb55.com>
Signed-off-by: William Casarin <jb55@jb55.com>
Signed-off-by: William Casarin <jb55@jb55.com>
Signed-off-by: William Casarin <jb55@jb55.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
5 participants