You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
And I retrieve data from it using a DALIRaggedIterator:
pipe = read_decode_pipe(
files,
batch_size=batch_size,
device=device,
device_id=device_id,
num_threads=n_threads,
)
pipe.build()
it = DALIRaggedIterator(
pipe,
output_map=['snippets', 'paths'],
output_types=[DALIRaggedIterator.SPARSE_LIST_TAG, DALIRaggedIterator.SPARSE_LIST_TAG],
auto_reset=False,
last_batch_policy=LastBatchPolicy.PARTIAL
)
for data in it:
snippets = data[0]['snippets']
bytes_paths = data[0]['paths'] # <--- might not yet be filled with data
str_paths = [path.cpu().numpy().tobytes().decode() for path in bytes_paths ]
Occasionally it happens that these encoded paths still hold no value at the time of decoding. Essentially they are tensors filled with zeros and the decoded path string is useless. Interestingly, when I set a breakpoint at that location and then apply the exact same decoding operation in the debug console, the strings are properly decoded all of a sudden. Probably because enough time has passed so that the tensors got filled with the actual data. This suggests that the source_info tensors get filled with data asynchronously. This is definitely unexpected behavior. The pipeline should await the data until it gets forwarded to the for loop.
Minimum reproducible example
No response
Relevant log output
No response
Other/Misc.
No response
Check for duplicates
I have searched the open bugs/issues and have found no duplicates for this bug report
The text was updated successfully, but these errors were encountered:
Version
1.35
Describe the bug.
I am using a video reader pipeline as follows:
And I retrieve data from it using a
DALIRaggedIterator
:Occasionally it happens that these encoded paths still hold no value at the time of decoding. Essentially they are tensors filled with zeros and the decoded path string is useless. Interestingly, when I set a breakpoint at that location and then apply the exact same decoding operation in the debug console, the strings are properly decoded all of a sudden. Probably because enough time has passed so that the tensors got filled with the actual data. This suggests that the source_info tensors get filled with data asynchronously. This is definitely unexpected behavior. The pipeline should await the data until it gets forwarded to the for loop.
Minimum reproducible example
No response
Relevant log output
No response
Other/Misc.
No response
Check for duplicates
The text was updated successfully, but these errors were encountered: