This is an SMS bot and OCR tool to help Shipt shoppers keep track of their pay more easily. Shoppers can text the bot screenshots of their pay history, and then export parsed data as a CSV file, compare their pay to others who have reported pay in their metro area, and get basic statistics on their pay.
It was used in a 2020 report that found that 41% of shoppers that contributed data have made less after Shipt changed their payment algorithm.
- note that this is a research repository. The core, dirty OCR script that lives at
shipt/receipts.pycan be used as a base for other services. It's not perfect, but it got the job done for a few hundred workers in the summer-fall of 2020.
This project uses the following external services:
Once you enable Twilio and Firebase, you need to define the following env variables. You should make
two files with these vars defined in the root of the repo: .env for development, and .env.prod for production.
# service account key needed for running firebase in-container
GOOGLE_APPLICATION_CREDENTIALS=/app/service_account.json
# used for generating temporary links
SECRET_KEY=dev
# password you give to anyone who should be able to export all the data
EXPORT_PASSWORD=mysecretpass
# SID & account token & phone for twilio
TWILIO_SID=twiliosid
TWILIO_TOKEN=twiliotoken
TWILIO_NUMBER=+5555555555
# need this for production flask deployment
FLASK_APP=shipt:create_app
If you already have a twilio account + number, and a firebase account, fill in the env variables and download your google service account key, and build + run the docker container in development mode:
docker build -t "shipt-calculator-dev" -f Dockerfile.dev .
docker run -e CONFIG='development' --env-file .env -p 5000:5000 -v $PWD:/app shipt-calculator-dev:latestMake sure to set your SMS endpoint in twilio to wherever you have the container running:
Then, you should be able to interact over SMS:
If you submit a screenshot of the Shipt pay screen, you should see something like the following:
To run the service locally on your development machine, you really just need to get twilio set up. This means making an account with a phone number you can text.
Once twilio is set up, you can assign the endpoint to your SMS service. I use ngrok to tunnel requests from my dev environment to twilio, but you can use whatever you want. The endpoint for receiving sms is http://your-hostname/sms
This project uses the Twilio messaging API to send and receive text messages. To run and test the code, you need to sign up for a twilio account, and create a messaging service. I recommend making two, each with a different phone number attached: one for development and one for deployment.
If you are developing locally, I recommend using ngrok to get a public URL you
can use to send twilio web requests. For example, if my docker container is running at
localhost:5000, I create a tunnel with ngrok http 5000 --subdomain shipt-calculator, and set
shipt-calculator.ngrok.io/sms to my Twilio endpoint.
This project uses firebase to store user data.
To get up and running, create a firestore
instance, and download the service account
key. Generate a private key for the
service account with access to the firestore, and save it as shipt-calculator/service_account.json.
First, make sure that you have a .env file (for development) and a .env.prod file (for production), as discussed above.
To run in development, and have local changes appear in the container:
docker build -t "shipt-calculator-dev" -f Dockerfile.dev .
docker run -e CONFIG='development' --env-file .env -p 5000:5000 -v $PWD:/app shipt-calculator-dev:latestAnd to run in production:
docker run -e CONFIG='production' --env-file .env.prod -p 8080:8080 shipt-calculator:latest
To test this project, I use a bunch of test images that workers have shared. Because these images include actual pay data and order numbers, I am not making them public, and the test code will remain in a private repository. If you're interested in testing changes you've made to this repo, please contact me and I can share access.
