Update Filter Schemas + Query Generation

[czaloom]

Feature Type

• Adding new functionality to valor

• Changing existing functionality in valor

• Removing existing functionality in valor

Problem Description

Filter schemas still using old format.

Feature Description

Update the filter object to use the JSON schemas introduced in PR #485.

The following python client code should work .

# Q: I want overhead images from specific datasets.
f = Dataset.name.in_(["dataset1", "dataset2"])

# Q: I want to constrain to images of swimmers where a boat is also in the image. 
f &= (
    Datum.where(Label.value.in_(["swimmer", "diver"]))
    & Datum.where(Label.value.in_(["boat", "ship"]))
)

# Q: I only want images between 08:00:00 and 18:00:00.
f &= (
    (Datum.metadata["time_of_day"] > datetime.time(hour=8))
    & (Datum.metadata["time_of_day"] < datetime.time(hour=18))
)

Additional Context

No response

[czaloom]

Found edge case that is not handled.

The following query only returns annotations and labels that are present in groundtruths as the first join to groundtruth eliminates all the prediction options.

 counts = db.query(
        Query(
            func.count(distinct(models.Datum.id)),
            func.count(distinct(models.Annotation.id)),
            func.count(distinct(models.Label.id)),
        )
        .filter(groundtruth_filter)
        .any(as_subquery=False)
        .where(  # type: ignore - this will be a select statement
            or_(
                models.Annotation.model_id.is_(None),
                models.Annotation.model_id == model.id,
            )
        )
        .subquery()
    ).all()

[ntlind]

Includes "Implement ability to filter on groundtruth labels"