Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Data Export Times Out on Production #137

Open
rylanb opened this issue Sep 27, 2017 · 1 comment
Open

Data Export Times Out on Production #137

rylanb opened this issue Sep 27, 2017 · 1 comment

Comments

@rylanb
Copy link
Collaborator

rylanb commented Sep 27, 2017

Expected Behavior

Data Export at https://robot.boulderfoodrescue.org/logs/export.csv?utf8=%E2%9C%93&start_date%5Bmonth%5D=8&start_date%5Bday%5D=1&start_date%5Byear%5D=2017&stop_date%5Bmonth%5D=9&stop_date%5Bday%5D=27&stop_date%5Byear%5D=2017&commit=Export

does NOT time-out.

Actual Behavior

It times out!

Steps to Recreate

Log in as a super admin or a region admin and select a large data range and try to export. It will work fine locally, but on web server Heroku will kill it after 30 seconds.

Look for N+1 queries and ways to clean up / generate more cleanly.

@Dania02525
Copy link

Dania02525 commented Oct 10, 2017

Considered moving to async exports and storing them on s3? I've got some classes that can be used to upload exports into an s3 bucket with object expiry, then listing the recent exports scoped to user or anything really for download

Edit: I see that sidekiq is not currently a dependency. I'll create a PR to address the sql injection issues in the log controller + speed up the queries, then on top of that one, one to bring in sidekiq + s3 async reporting so we can weigh whether it will be worth doing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants