Scalable Recipe Tests

[phurwicz]

Sketch / Food for Thought

Current testing procedure
SELECTING POINTS TO LABEL

• make selection in
• [movable] enter in annotator_input
• action_view_selection() -> check selection table
• make sub-selection in selection table
• action_evict_selection() -> check old/new selection
• to be continued. Apply twice then commit? Commit twice then deduplicate? Repeated deduplication should have no effect? Neither does repeated push? Check selection sync across multiple explorers at random points? Generate filter conditions and expected outcome?

The state of a plotted recipe

• dataset/explorer consistency
  • rows for each subset
    • recovered by PUSH
    • affected by COMMIT, DEDUP
    • unchanged by VIEW, EVICT, PATCH
  • data values (including SUBSET)
    • recovered by PUSH
    • unchanged by
• dataset/selection table consistency: mutable data values
• explorer/selection table consistency
  • selected indices
    • recovered(or reset) by VIEW, PUSH
    • unchanged by PATCH
    • affected by COMMIT, DEDUP, EVICT
  • data values (including SUBSET)
    • recovered(or reset) by VIEW, PUSH
    • unchanged by EVICT
    • affected by COMMIT, DEDUP, PATCH
• widget states
  • checkbox toggles
  • input/slider values