xargs/parallel are all fun and games until you need to include or escape quotes. What if you could use the full power of SQLite to generate, query, and store the results of shell commands?
Anything you could do by running shell commands from SQLite, you could do by piping the output of those commands to SQLite using the incredible q tool. You probably want that instead.
$ cargo build
$ sqlite3Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
sqlite> .load ./target/debug/libbasque
sqlite> create table websites(url text, body text);
sqlite> insert into websites(url) values('https://www.sqlite.org/');
sqlite> insert into websites(url) values('https://www.rust-lang.org/');
sqlite> update websites set body = basque_cmd("curl", "-L", url);
sqlite> select url, substr(body, 1, 20) from websites;
https://www.sqlite.org/|<!DOCTYPE HTML PUBLI
https://www.rust-lang.org/|<!doctype html>
<htm- Prototype to confirm
no_manglewill work and SQLite can load the Rust-built module - Generate
sqlite3ext.hAPI struct with rust-bindgen - Switch to Cargo for building
- Fix all the places I'm cheating and using
u64instead of pointer types - Make the
GLOBAL_ROUTINESstruct threadsafe (Mutex? Container crate?) - Handle
SQLITE_*error results - Actually honor
basque_cmdfunction parameters - Handle all the panicky
unwrap()s - Free Command result allocations properly
- Return
stderrtoo, probably? Or log it somehow - Implement a virtual table instead, so each line result is a row
- Actually learn Rust
You will need a copy of SQLite with module loading enabled. The default (/usr/bin) installation on macOS does not, but the Homebrew one does. To use the Homebrew one:
export PATH=/usr/local/opt/sqlite3/bin:$PATH
If module loading is disabled, you'll get this error when you try to run .load ./libbasque:
Error: unknown command or invalid arguments: "load". Enter ".help" for help
- https://www.sqlite.org/src/file/ext/misc/series.c
- https://www.sqlite.org/capi3ref.html#sqlite3_auto_extension
- https://doc.rust-lang.org/nomicon/ffi.html
- https://rust-lang.github.io/rust-bindgen/print.html
- https://doc.rust-lang.org/1.30.0/book/second-edition/ch19-01-unsafe-rust.html
- https://github.com/rust-lang/rust-bindgen/blob/master/book/src/whitelisting.md
- https://github.com/cloudmeter/sqlite/blob/master/sqlite3ext.h