You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In the original concept, range_config should work for {SPAN} in ExampleGen. However there is a bug in TFX < 1.4.0. As of writing this issue, the bug fix has been merged, but nightly build or build from scratch version should be working. The nightly build version should be above 1.4.0.dev20211010 (you can find out the dev versions here)
When range_config works properly, we can integrate that functionality to dynamically choose the range of spans to run the training pipeline with Resolver node. For example, initial pipeline run could depend on span-1, but when data drift is detected, the second pipeline run could use both span-1 and span-2 together. In that case, we don't need to process the ExampleGen for the span-1 once again but only the span-2. Resolver node will reuse the ExampleGen artifacts for the span-1 generated from the initial pipeline run and integrate it with the new ExampleGen for the span-2.
In the original concept,
range_config
should work for{SPAN}
inExampleGen
. However there is a bug in TFX <1.4.0
. As of writing this issue, the bug fix has been merged, but nightly build or build from scratch version should be working. The nightly build version should be above1.4.0.dev20211010
(you can find out the dev versions here)When
range_config
works properly, we can integrate that functionality to dynamically choose the range ofspans
to run thetraining pipeline
withResolver
node. For example, initial pipeline run could depend onspan-1
, but when data drift is detected, the second pipeline run could use bothspan-1
andspan-2
together. In that case, we don't need to process theExampleGen
for thespan-1
once again but only thespan-2
.Resolver
node will reuse theExampleGen
artifacts for thespan-1
generated from the initial pipeline run and integrate it with the newExampleGen
for thespan-2
.More extensive discussion about this issue can be found in the issue from the TFX official repo.
The text was updated successfully, but these errors were encountered: