How to expand a vector in parameter

[rawhuul]

Hey, there I want to perform full text search across a column for which I need to execute query like

SELECT app_name, mainfest FROM mainfests_fts WHERE mainfest MATCH ("android", "cli");

In this, android and cli are passed by user as parameters, I tried:

fn query_fts(repos: Vec<PathBuf>, query: &str) -> Result<Vec<RawData>, ScoopieError> {
    let args: Vec<&str> = query.split(" ").collect();

    let fetch_row = |row: &Row| -> Result<MainfestEntry, Error> {
        let app_name = row.get(0)?;
        let mainfest = row.get(1)?;
        Ok(MainfestEntry { app_name, mainfest })
    };

    let results: Result<Vec<Vec<RawData>>, ScoopieError> = repos
        .par_iter()
        .map(|repo| -> Result<Vec<RawData>, ScoopieError> {
            let conn = Connection::open(&repo)
                .map_err(|_| ScoopieError::Database(DatabaseError::UnableToOpen))?;

            let mut stmt = conn
                .prepare("SELECT app_name, mainfest FROM mainfests_fts WHERE mainfest MATCH (?)")
                .map_err(|_| ScoopieError::Database(DatabaseError::FailedToMkStmt))?;

            let rows = stmt
                .query_map(params![args], fetch_row)
                .map_err(|_| ScoopieError::Query(QueryError::FailedToQuery))?;

            rows.into_iter()
                .map(|row| -> Result<RawData, ScoopieError> {
                    let row =
                        row.map_err(|_| ScoopieError::Query(QueryError::FailedToRetrieveData))?;

                    let app_name = row.app_name;
                    let mainfest = row.mainfest;

                    Ok(RawData::new(&repo, app_name, mainfest))
                })
                .collect()
        })
        .collect();

    let results: Vec<Vec<RawData>> = results?;
    Ok(results.concat())
}

But on compilation it returns:

error[E0425]: cannot find value `repos` in this scope
  --> src\query.rs:67:60
   |
67 |     let results: Result<Vec<Vec<RawData>>, ScoopieError> = repos
   |                                                            ^^^^^ help: a local variable with a similar name exists: `repo`

error[E0277]: the trait bound `Vec<&str>: ToSql` is not satisfied
  --> src\query.rs:78:28
   |
78 |                 .query_map(params![args], fetch_row)
   |                            ^^^^^^^^^^^^^ the trait `ToSql` is not implemented for `Vec<&str>`
   |
   = help: the trait `ToSql` is implemented for `Vec<u8>`
   = note: required for the cast from `Vec<&str>` to the object type `dyn ToSql`
   = note: this error originates in the macro `params` (in Nightly builds, run with -Z macro-backtrace for more info)

Some errors have detailed explanations: E0277, E0425.
For more information about an error, try `rustc --explain E0277`.
error: could not compile `scoopie` (bin "scoopie") due to 2 previous errors

What could be possible solution to achieve it.

[gwenn]

https://github.com/rusqlite/rusqlite/issues?page=2&q=is%3Aissue+vec
florilèges:
#430
#345

[rawhuul]

On further investigation, I found that Value trait is implemented for String, so I changed my code accordingly and able to compile the program.

Here is code:

use std::rc::Rc;

fn query_fts(query: &str) -> Result<Vec<AppInfo>, ScoopieError> {
    let repos = Config::read()?.latest_repos()?;

    const QUERY_STMT: &'static str =
        "SELECT app_name, manifest FROM manifests_fts WHERE manifest MATCH rarray(?)";

    let args: Vec<String> = query.split(" ").map(|c| c.to_string()).collect();

    let query = Rc::new(
        args.into_iter()
            .map(rusqlite::types::Value::from)
            .collect::<Vec<rusqlite::types::Value>>(),
    );

    let fetch_row = |row: &Row| -> Result<(String, String), Error> {
        let app_name = row.get(0)?;
        let mainfest = row.get(1)?;
        Ok((app_name, mainfest))
    };

    let results: Result<Vec<Vec<Data>>, ScoopieError> = repos
        .iter()
        .map(|repo| -> Result<Vec<Data>, ScoopieError> {
            let conn = Connection::open(&repo)
                .map_err(|_| ScoopieError::Database(DatabaseError::UnableToOpen))?;

            rusqlite::vtab::array::load_module(&conn).unwrap();

            let mut stmt = conn.prepare(QUERY_STMT).unwrap();

            let rows = stmt
                .query_map([query.clone()], fetch_row)
                .map_err(|_| ScoopieError::Query(QueryError::FailedToQuery))?;

            rows.into_iter()
                .map(|row| -> Result<Data, ScoopieError> {
                    let row =
                        row.map_err(|_| ScoopieError::Query(QueryError::FailedToRetrieveData))?;

                    let app_name = row.0;
                    let mainfest = row.1;

                    Ok(Data::new(&repo, app_name, mainfest))
                })
                .collect()
        })
        .collect();

    results?
        .into_iter()
        .flatten()
        .map(|d| AppInfo::from(&d))
        .collect()
}

But now I ran into some runtime errors, which are as follows:

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: SqlInputError { error: Error { code: Unknown, extended_code: 1 }, msg: "no such function: rarray", sql: "SELECT app_name, manifest FROM manifests_fts WHERE manifest MATCH rarray(?)", offset: 66 }', src\query.rs:215:53
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
error: process didn't exit successfully: `target\debug\scoopie.exe query "android cli"` (exit code: 101)

[gwenn]

rarray is provided by rusqlite::vtab::array module like specified in the example ...

[rawhuul]

rarray is provided by rusqlite::vtab::array module like specified in the example ...

yeah and as you can see in above I load it as specified in example and still it returns error :(

[gwenn]

Because rarray is a table-valued-function:
https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=d954eabbcce57ae2e36ff4445384ad7e

[rawhuul]

Thanks @gwenn, I got it working. Is there anyway to use Arc instead of Rc, so that I can utilize par_iter from Rayon?

[rawhuul]

Hey there, I have found solution for this without needing to unpack vector in parameter.

Here it is:

fn query_fts(query: &str) -> Result<Vec<AppInfo>, ScoopieError> {
        let query = query
            .split_whitespace()
            .map(|term| format!("{term}*"))
            .collect::<Vec<String>>()
            .join(" AND ");

        let results = Query::builder(
            "SELECT app_name, mainfest FROM mainfests_fts WHERE mainfest MATCH ? COLLATE nocase",
        )?
        .run(&query)?;

        results
            .par_iter()
            .map(|raw_result| AppInfo::from(raw_result))
            .collect()
    }

pub struct Query {
    repos: Vec<PathBuf>,
    stmt: String,
}

impl Query {
    pub fn builder(stmt: &str) -> Result<Query, ScoopieError> {
        let repos = Config::read()?.latest_repos()?;
        Ok(Self {
            repos,
            stmt: stmt.into(),
        })
    }

    pub fn run(&self, params: &str) -> Result<Vec<RawData>, ScoopieError> {
        let fetch_row = |row: &Row| -> Result<(String, String), Error> {
            let app_name = row.get(0)?;
            let mainfest = row.get(1)?;
            Ok((app_name, mainfest))
        };

        let results: Result<Vec<Vec<RawData>>, ScoopieError> = self
            .repos
            .par_iter()
            .map(|repo| -> Result<Vec<RawData>, ScoopieError> {
                let conn = Connection::open(&repo)
                    .map_err(|_| ScoopieError::Database(DatabaseError::UnableToOpen))?;

                let mut stmt = conn
                    .prepare(&self.stmt)
                    .map_err(|_| ScoopieError::Database(DatabaseError::FailedToMkStmt))?;

                let rows = stmt
                    .query_map(params![params], fetch_row)
                    .map_err(|_| ScoopieError::Query(QueryError::FailedToQuery))?;

                rows.into_iter()
                    .map(|row| -> Result<RawData, ScoopieError> {
                        let row =
                            row.map_err(|_| ScoopieError::Query(QueryError::FailedToRetrieveData))?;

                        let app_name = row.0;
                        let mainfest = row.1;

                        Ok(RawData::new(&repo, app_name, mainfest))
                    })
                    .collect()
            })
            .collect();

        let results = results?;
        Ok(results.concat())
    }
}

however it doesn't gives the desired result as below query gives

SELECT app_name, mainfest FROM mainfests_fts WHERE mainfest MATCH ("android", "cli");

Until I found any good solution, I'm closing this discussion till then.