You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Per Peer Review feedback, upgrade data mapping and matching processes to make the workflow more intuitive. Include further integration of data types and dual unit support. Need to improve the performance of the importing process. Look at better scaling of worker nodes.
More details:
Profile the import process (steps 1 through 6 on the UI) - some steps seem to get skipped while others take a long time to run
also profile each celery task and see which ones take the longest to run
The mapping step has room for improvement, but by far the biggest bottleneck is after you hit Confirm mappings & start matching:
Anecdotally, when uploading a file with 145,921 rows to an existing organization with 145,921 matching rows these are the celery tasks and their timing:
Per Peer Review feedback, upgrade data mapping and matching processes to make the workflow more intuitive. Include further integration of data types and dual unit support. Need to improve the performance of the importing process. Look at better scaling of worker nodes.
More details:
Profile the import process (steps 1 through 6 on the UI) - some steps seem to get skipped while others take a long time to run
Investigate if there are things that can be skipped completely. for example:
Can we improve the Progress API endpoint to be more fault tolerant:
@axelstudios
The text was updated successfully, but these errors were encountered: