You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In an effort to help alleviate the current issues we're having with larger time-based reports being generated - they will timeout and the download request will fail before the report is generated - we'd like to see if there are any performance improvements that could be made.
While this work will be for a shorter term fix while we continue discussing and planning what a long-term fix and improvement looks like, any performance gains and improvements to the processing that happens now will help the effort in the future as well.
Implementation Sketch and Acceptance Criteria
Review the current time-based report generation code
Walk through the flow of events and processing and see what's currently happening to understand the complete picture
Note any areas of the code that are retrieving data and/or looping through data - these are likely targets for the biggest performance gains
Is there a better way of retrieving data to process it in memory more efficiently?
Make adjustments to the processing where possible and test/benchmark for changes
Testing is a known challenge - we need a way to simulate processing hundreds, if not thousands, of rows
I did some profiling locally after doing 50 one-off messages:
the downloads (if the job is not in the cache) take anywhere from 250 milliseconds to 1400 and average out at about 400, which is 90% of the time the report needs to generate.
So there is no optimization available in the code itself. I think the options might be:
increase the cache time to 7 days (hmmm)
increase the cache time to 1 day and make a one day report with phone numbers, then remove phone numbers from the 7 day report
just remove the phone numbers from the 7 day report
In an effort to help alleviate the current issues we're having with larger time-based reports being generated - they will timeout and the download request will fail before the report is generated - we'd like to see if there are any performance improvements that could be made.
While this work will be for a shorter term fix while we continue discussing and planning what a long-term fix and improvement looks like, any performance gains and improvements to the processing that happens now will help the effort in the future as well.
Implementation Sketch and Acceptance Criteria
Security Considerations
The text was updated successfully, but these errors were encountered: