[globset]: understanding a more complex glob (idea and reality)

[chincheta0815]

Hej there,

I am trying to run a glob for filtering out music genres. There is a list of all genres:
{"synthie pop", "musicbox", "melodic hardcore", "italian rock", "soundtrack", "vallenato", "latin pop", "house", "classical", "country", "other", "glam rock", "r&b", "wagnerian rock", "cumbia", "latin house", "fastnacht", "axé", "son cubano", "eurodance", "gospel", "latin urban", "reggae", "latin reggae", "alternative rock", "rap", "swing", "guajira", "europop", "indie", "metal", "folk", "country rock", "unterhaltungsmusik", "tumbao", "unknown", "latin celtic pop", "disco", "latin heavy metal", "italian pop", "blues rock", "instrumental rock", "mpb", "new wave", "bolero", "nu metal", "criolla", "freestyle", "sertanejo", "crossover", "afro-cuban jazz", "danzón", "soft rock", "musical", "volksmusik", "rap metal", "ska-punk", "jukebox", "flamenco", "reggae fusion", "euro house", "christmas", "punk pop", "andean folk", "reggaeton", "happy hardcore", "pagode", "chanson", "schlager", "world music", "progressive pop", "rock ’n’ roll", "religiöse musik", "euro reggae", "latin celtic rock", "singer-songwriter", "rumba flamenca", "folk pop", "roots rock", "electronic", "latin jazz", "nuevo canto", "chinese hip hop", "soul", "neue deutsche härte", "latin rock", "folk rock", "ballad", "vocal", "latin folk metal", "heavy metal", "nueva trova", "psychedelic rock", "break dance", "rock & roll", "hip hop", "merengue", "rave", "celtic rock", "instrumental", "blues", "metalcore", "deutsch-rap", "heartland rock", "power metal", "pop", "britpop", "ambient music", "latin ska", "austrian pop", "punk rock", "punk", "bachata", "funk", "glam metal", "hard rock", "alternative", "funk rock", "axe", "grunge", "trance", "progressive metal", "alternative metal", "folk metal", "liedermacher", "skiffle", "contemporary folk", "pbm", "jazz", "singer", "rock", "hawaiian", "latin hip hop", "latin electro", "pop rock", "dark rock", "garage rock", "chart hits", "dance", "latin folk", "forró", "new age", "alternative hip hop", "french pop", "symphonic metal", "southern rock", "comedy", "salsa", "euro dance industrial", "bulgarian music", "progressive rock"}

I want all matches that do NOT contain "Latin" OR "Italian" "Rock" and "Metal", but all others containing "Rock" and "Metal".
The glob is done in lowercase/case_insensitive

The glob:
{[!Latin],[!Italian]}{*Rock,*Metal*}
helps me to get:
"dark rock", "progressive metal", "pop rock", "heavy metal", "folk rock", "glam metal", "power metal", "punk rock", "heartland rock", "wagnerian rock", "blues rock", "funk rock", "folk metal", "glam rock", "garage rock", "psychedelic rock", "progressive rock", "celtic rock", "country rock", "southern rock", "hard rock", "soft rock", "roots rock", "symphonic metal", "rap metal"

What I do not understand: The documentation tells me that ...[ab] matches the a or b where a and b are characters. Use [!ab] to match any except from a and b.... Why is it not excepting e.g. folk metal as the L and the l are in the except? Or is the match the word in between...

So far I get what I want, but I would really be happy to understand that as I found the solution by chance.

If you need some source code, there is the repsective function: https://github.com/CDrummond/bliss-mixer/blob/c08b79ea271d7c631d45fbece6b619d74fb144a3/src/api.rs#L188

[BurntSushi]

It's honestly a bit difficult to parse what you actually want here. Namely, 1) you've written your boolean logic in ambiguous prose, 2) the glob you've provided doesn't make any sense to me and 3) there isn't a concrete expected output that you've provided.

(2) is perhaps something worth digging into though. When trying to understand something, I recommend breaking it down into smaller parts and then trying to understand those smaller parts. So for example, why not look at [!Latin]*Rock? That glob matches folk rock. The [!Latin] just says, "match anything that isn't an L, a, t, i or n." Then the * is "match a sequence of any characters." So [!Latin] matches f since f is not L, a, t, i or n. Then the * matches olk . And then finally Rock matches Rock.

Popping up a level, globset does not support any way of saying, "matching anything except this sequence of characters." (Some globbing syntaxes do support that, and it is effectively equivalent to negative look-ahead in a regex.) So I'd suggest that you use multiple globs to implement your logic. globset does, after all, support sets of globs, and so you can do it efficiently.

[chincheta0815]

Okay, I now tried that with multiple globset and also corrected my misunderstanding.

TasK: Get all ending with "Rock", but leave out all that start with "Latin". I expect to have "Soft Rock", "Hard Rock", "Blues Rock", etc.
So far I search with one that "[!L][!a][!t][!i][!n]*Rock" which should cancel out anything beginning with the word "Latin" (at least it seems to work). Then I use one with "*Rock" to include everything else with "Rock".

Summed up:

builder.add(Glob::new("[!L][!a][!t][!i][!n]*Rock")?);
builder.add(Glob::new("*Rock")?);

Using "is_match()" shows everything ending with Rock but not starting with "Latin".

Did I understand that correctly?

[BurntSushi]

Please please please provide a complete but minimal (as much as possible) program. In this case, a simple program would demonstrate that [!L][!a][!t][!i][!n]*Rock indeed does not match Latin Rock, but *Rock does. And thus is_match would match Latin Rock:

use globset::{Glob, GlobSetBuilder};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut builder = GlobSetBuilder::new();
    builder.add(Glob::new("[!L][!a][!t][!i][!n]*Rock")?);
    builder.add(Glob::new("*Rock")?);
    let set = builder.build()?;

    dbg!(set.is_match("Latin Rock"));
    Ok(())
}

output:

$ cargo run
   Compiling globset2363 v0.1.0 (/tmp/globset2363)
    Finished dev [unoptimized + debuginfo] target(s) in 0.51s
     Running `target/debug/globset2363`
[main.rs:9] set.is_match("Latin Rock") = true

One of the problems here is that while [!L][!a][!t][!i][!n]*Rock does not match Latin Rock, it also doesn't match a lot of other things, like Folk Rock.

You can't easily build negation into globs without some kind of negative look-around, which globset very intentionally does not support. You need to implement some kind of whitelist/blacklist semantic. Here's a demonstration of this idea:

use globset::{Glob, GlobSetBuilder};

fn main() -> Result<(), Box<dyn std::error::Error>> {
    let mut builder = GlobSetBuilder::new();
    builder.add(Glob::new("*Rock")?);
    builder.add(Glob::new("Latin Rock")?);
    let set = builder.build()?;

    let matches = set.matches("Latin Rock");
    // If we matched a blacklist glob, then no match!
    if matches.contains(&1) {
        dbg!("no match, blacklisted");
        return Ok(());
    }
    // Otherwise, we need to have at least matched one whitelist glob.
    if matches.contains(&0) {
        dbg!("Match!");
        return Ok(())
    }
    // At this point, we didn't match any blacklist nor any whitelist
    // globs. So nothing matched.
    dbg!("no match");
    Ok(())
}

And the output:

$ cargo run
   Compiling globset2363 v0.1.0 (/tmp/globset2363)
    Finished dev [unoptimized + debuginfo] target(s) in 0.50s
     Running `target/debug/globset2363`
[main.rs:12] "no match, blacklisted" = "no match, blacklisted"

This is effectively how ripgrep implements whitelist/blacklist semantics with respect to gitignore files:

ripgrep/crates/ignore/src/gitignore.rs

Lines 256 to 265 in 8905d54

	for &i in matches.iter().rev() {
	let glob = &self.globs[i];
	if !glob.is_only_dir() || is_dir {
	return if glob.is_whitelist() {
	Match::Whitelist(glob)
	} else {
	Match::Ignore(glob)
	};
	}
	}

[chincheta0815]

Ahhh.
Okay thank you very much.
For completeness I will provide a code snippet - I think I did it as you did.

Only exception I have to make:
In my case I want the glob being given in a string/line Like "glob_1; glob_2; glob_3".
There won't be no whitelist/blacklist for user-friendlyness.
I think performance can be neglected here in favor for user-friendlyness.

I do not expect a lot of glob's being done (reasonably 10) on not that much genres (I assumed 200). That should be okay...

The code etc. will follow as soon as I get that from my other computer...

[BurntSushi]

You won't be able to get far without some kind of whitelist/blacklist mechanism. Globs just aren't expressive enough.