Create Universal Query Language

[PawelPeczek-Roboflow]

Description

In this PR I ship extension to core block in form of universal query / operation language.

The idea is the following:

• we define JSON-based language to express operations on different types of data that we support (for example, for sv.Detections we support filtering, offseting and getting detections metadata - like size or location) and evaluation of unary and binary operations that are useful for conditional logic
• definition is extensible - to add new operation we only need to create pydantic entity to host its definition, implementation of that operation and register the operation in UQL execution engine
• definitions are self-describing - entities defining ops expose JSON manifests including custom fields expressing types of operations - that together should allow creating relatively generic UI for different kinds of blocks using UQL and execute simple type inference procedure on UI side to very effectively narrow down allowed operations that users can apply in specific contexts - hopefully making the whole thing quite intuitive
• we do not make changes into workflows EE, apart from quality of life improvements - selectors embedded in blocks manifests dicts (so far we supported only singular references and lists of references, now we would like to be able to provide Dict[str, reference] as input - as this is very useful for providing parametrisation of UQL execution engine - let's say that u want to filter detections, but threshold of confidence is provided in runtime, not hardcoded in definition)
• I do not imply that all blocks should use UQL, this is optional breadth-wise extension for workflows, but I think it will be beneficial, as:
  • we would be able to have quite generic builders for tons of operations in UI
  • we would be able to quickly extend UQL
  • we would eliminate need for multiple custom glue code blocks and python-code blocks (to some extent)

Downsides of the idea:

• there needs to be some intelligence in the UI to create builders for operations and to know how to behave in context of specific blocks (for instance - condition would provide all parameters of UQL EE in the manifest definition vs detections filter would implicitly iterate over predictions in batch that is provided as predictions parameter)
• the JSON definition is not quite easy to read by human - but it is quite easy to programatically construct and parse, which is more important in that case. I do recommend not pushing into "verbose" UQL, as we will add significant amount of work on the parsing side and on the query construction side, as both sides would need to implement "parser" of some kind

Here are the examples:

• Block with detections sampling (to be used for AL)
• Block with new condition logic
• Block with transformation (filtering)

As u see, they all operate on the very similar definitions and in general this extension bring a lot of expression power.

Type of change

Please delete options that are not relevant.

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• This change requires a documentation update

How has this change been tested, please provide a testcase or example of how you tested the change?

• NOT READY TO MERGE

Any specific deployment considerations

• We need sv.Detections as main data format to proceed with this PR (requires merging Sv detections in core blocks #392)
• we need to push changes started here further - AL Data collector needs to have new sink and we need to make the operation set better and more robust

Docs

• Docs updated? What were the changes:

[grzegorz-roboflow]

If this is to handle sv.Detections then we might want to think if we want to promote operating on single detection instances. It is possible to get individual detections from sv.Detections object like this:

for i in range(len(detections)):
    yield detections[i]

Though above will create a copy of data, might be costly to execute.

[grzegorz-roboflow]

Numer -> Number

[PawelPeczek-Roboflow]

👍

[grzegorz-roboflow]

Numer -> Number

[PawelPeczek-Roboflow]

👍

[grzegorz-roboflow]

extracted_data is defined as list, so we can simplify to detections["data"] = np.array(extracted_data) if extracted_data else np.empty(0)

Also:

>>> np.array([])
array([], dtype=float64)
>>> np.empty(0)
array([], dtype=float64)

So we could further simplify to detections["data"] = np.array(extracted_data)

[PawelPeczek-Roboflow]

👍

[grzegorz-roboflow]

Can be simplified to detections["data"] = np.array(extracted_data)

[PawelPeczek-Roboflow]

👍

[grzegorz-roboflow]

Loop can be avoided: detections[PREDICTION_TYPE_KEY] = np.array(["qrcode-detection"] * len(detections))

[PawelPeczek-Roboflow]

👍

[grzegorz-roboflow]

empty

[PawelPeczek-Roboflow]

👍

[grzegorz-roboflow]

should we extract this to separate task? It seems we are already handling this by calling sv_detections_to_root_coordinates below

[PawelPeczek-Roboflow]

issue submitted