-
Notifications
You must be signed in to change notification settings - Fork 208
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
libsql wal no fixture #1332
base: main
Are you sure you want to change the base?
libsql wal no fixture #1332
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ive made it to segment_list.rs so I will continue tomorrow from there but leaving my feedback so far.
pub enum Cipher { | ||
// AES 256 Bit CBC - No HMAC (wxSQLite3) | ||
#[default] |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is this a new thing in rust?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
at least a year I think
Io(#[from] std::io::Error), | ||
#[error("error building wal index: {0}")] | ||
IndexError(#[from] fst::Error), | ||
/// The log has since the connection last read, and it's now trying to upgrade |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
what?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
*The log has since the connection last read
will update
let code = match self { | ||
Error::BusySnapshot => libsql_sys::ffi::SQLITE_BUSY_SNAPSHOT, | ||
e => { | ||
tracing::error!("wal error: {e}"); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is this the right place to log the error? I feel like that should be handled at the application level and is already namespaced from this error type.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is where we lose track of this error, it's turned into a sqlite error for the FFI, so we log it, and then propagate with loss
@@ -0,0 +1,171 @@ | |||
use std::ffi::c_int; |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
shouldn't this be moved to some other type of file rather than a main file in a sub crate? We already have a tools crate/folder that we can use. This way we don't need to include extra dependencies for this when things like the server pull it in?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this is temporary scafolding to be able to test, I'll go away soon enough
// First, we'll acquire the lock to the current transaction to make sure no one steals it from us: | ||
let lock = shared.wal_lock.tx_id.lock(); | ||
// Make sure that we still own the transaction: | ||
if lock.is_none() || lock.unwrap() != tx.id { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
shouldn't this return an error?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
no, swapping the log is a best effort thing, if we don't have the lock, the next one that has the lock will try to swap. This is not an error
|
||
pub fn shutdown(&self) -> Result<()> { | ||
self.shutdown.store(true, Ordering::SeqCst); | ||
let mut openned = self.openned.write(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
btw its spelled opened with one n 😄
pub fn shutdown(&self) -> Result<()> { | ||
self.shutdown.store(true, Ordering::SeqCst); | ||
let mut openned = self.openned.write(); | ||
for (_, shared) in openned.drain() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you explain what is going on here?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
we checkpoint all the wals
same as #1279 without test fixtures