Improve docker build times for ludwig-ray and ludwig-ray-gpu

[arnavgarg1]

Old docker build times: a707840 (https://github.com/ludwig-ai/ludwig/actions/runs/8104368079/job/22150838886)
New docker build times: e0f112c (https://github.com/ludwig-ai/ludwig/actions/runs/8191677680/job/22401422091)

Seems like ludwig-ray and ludwig-ray-gpu build times have increased by nearly 4x - 6x (1.5 hours as opposed to 15-20 min). We should figure out how to speed it back up.

Seems like it might just need us to do some pinning of packages like accelerate.