Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

libsql wal no fixture #1332

Open
wants to merge 27 commits into
base: main
Choose a base branch
from
Open

libsql wal no fixture #1332

wants to merge 27 commits into from

Conversation

MarinPostma
Copy link
Collaborator

@MarinPostma MarinPostma commented Apr 22, 2024

same as #1279 without test fixtures

Copy link
Contributor

@LucioFranco LucioFranco left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ive made it to segment_list.rs so I will continue tomorrow from there but leaving my feedback so far.

pub enum Cipher {
// AES 256 Bit CBC - No HMAC (wxSQLite3)
#[default]
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this a new thing in rust?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

at least a year I think

libsql-sys/src/wal/mod.rs Show resolved Hide resolved
Io(#[from] std::io::Error),
#[error("error building wal index: {0}")]
IndexError(#[from] fst::Error),
/// The log has since the connection last read, and it's now trying to upgrade
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

*The log has since the connection last read
will update

let code = match self {
Error::BusySnapshot => libsql_sys::ffi::SQLITE_BUSY_SNAPSHOT,
e => {
tracing::error!("wal error: {e}");
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this the right place to log the error? I feel like that should be handled at the application level and is already namespaced from this error type.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is where we lose track of this error, it's turned into a sqlite error for the FFI, so we log it, and then propagate with loss

libsql-wal/src/fs/file.rs Show resolved Hide resolved
@@ -0,0 +1,171 @@
use std::ffi::c_int;
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't this be moved to some other type of file rather than a main file in a sub crate? We already have a tools crate/folder that we can use. This way we don't need to include extra dependencies for this when things like the server pull it in?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is temporary scafolding to be able to test, I'll go away soon enough

libsql-wal/src/registry.rs Show resolved Hide resolved
// First, we'll acquire the lock to the current transaction to make sure no one steals it from us:
let lock = shared.wal_lock.tx_id.lock();
// Make sure that we still own the transaction:
if lock.is_none() || lock.unwrap() != tx.id {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

shouldn't this return an error?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

no, swapping the log is a best effort thing, if we don't have the lock, the next one that has the lock will try to swap. This is not an error


pub fn shutdown(&self) -> Result<()> {
self.shutdown.store(true, Ordering::SeqCst);
let mut openned = self.openned.write();
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

btw its spelled opened with one n 😄

pub fn shutdown(&self) -> Result<()> {
self.shutdown.store(true, Ordering::SeqCst);
let mut openned = self.openned.write();
for (_, shared) in openned.drain() {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you explain what is going on here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we checkpoint all the wals

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

2 participants