Deleting certain data sets takes an inordinate amount of time

[hackdna]

• Specific code commit: v1.6.8
• Version of the web browser and OS: any
• Environment where the error occurred: any

Steps to reproduce

1. Set up a dev/test instance of SCC using a recent RDS snapshot
2. Open /admin/core/dataset/319 (all.gene.FPKM) in a web browser
3. Delete the data set

Observed behavior

• After a few minutes web browser showed HTTP error 504
• Data set deleted after about 1 hour 25 minutes
• CPU utilization stayed at ~50% (on a t2.medium instance)
• Memory utilization went up to ~3GB:
  • Django: ~1.5GB of RAM (~40% of total system memory)
  • Solr: ~0.5GB

Expected behavior

Data set should be deleted in seconds without excessive CPU or memory utilization

Notes

See also: #463, #713, #2394, #2645

PostgreSQL dump size:

• before this data set was removed: ~268MB
• after this data set was removed: ~190MB