Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Setup docker for project #3

Merged
merged 1 commit into from
May 4, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
3 changes: 1 addition & 2 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -349,5 +349,4 @@ pip-selfcheck.json
media
logs


# End of https://www.gitignore.io/api/vim,linux,macos,dotenv,django,python,virtualenv,intellij+all,visualstudiocode
# End of https://www.gitignore.io/api/vim,linux,macos,dotenv,django,python,virtualenv,intellij+all,visualstudiocode
28 changes: 28 additions & 0 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,28 @@
# The build image
FROM python:3.8.0-slim as builder
RUN apt-get update \
&& apt-get install gcc python-dev libpq-dev -y \
&& apt-get clean

COPY ./requirements.txt /app/requirements.txt
WORKDIR app
RUN pip install --upgrade pip \
&& pip install --user -r requirements.txt

COPY . /app

# The app image
FROM python:3.8.0-slim as app
COPY --from=builder /root/.local /root/.local
COPY --from=builder /app /app
WORKDIR app
RUN apt-get update && apt-get install libpq-dev -y
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I had to repeat this libpq-dev package because apparently psycopg2 needs it both to install and to run and I couldn't find a way to copy the packages from the build image to the app image. All in all, I was able to bring the final image down to 275MB from around 599MB. I'll keep doing more research concerning this libpq-dev package and definitely raise an patch pr to fix it when I get a fix.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Weird but cool @ascii-dev i.e. would have assumed dev packages are only needed to build binaries & hence we should have been able to just copy the built binaries into app image... but we're short on time right now so 300MB reduction is good enough. 🚀

ENV PATH=/usr/pgsql-9.1/bin/:/root/.local/bin:$PATH
COPY ./contrib/docker/start.sh /start.sh
COPY ./contrib/docker/entrypoint.sh /entrypoint.sh

RUN chmod +x /entrypoint.sh && \
chmod +x /start.sh

ENTRYPOINT ["/entrypoint.sh"]
CMD ["/start.sh"]
24 changes: 24 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,2 +1,26 @@
# DebunkBot
A bot that debunks claims shared on social media by sharing a fact check. Powered by Google Sheets and the rains in Africa. Accessible at https://debunkbot.codeforafrica.org/

## Development

Gitignore is standardized for this project using [gitignore.io](https://www.gitignore.io/) to support various development platforms.
To get the project up and running:

- Clone this repo

### Virtual environment

- Use virtualenv to create your virtual environment; `virtualenv venv`
- Activate the virtual environment; `source venv/bin/activate`
- Install the requirements; `pip install -r requirements.txt`
- Create a debunkbot database
- Add database connection details to `.env` file, using `.env.sample` as a template
- Migrate the database: `python manage.py migrate`
- Run the server: `python manage.py runserver`

### Docker

Using docker compose:

- Build the project; `docker-compose build`
- Run the project; `docker-compose up -d`
17 changes: 17 additions & 0 deletions contrib/docker/entrypoint.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,17 @@
#!/bin/sh

if [ "$DATABASE" = "postgres" ]
then
echo "Waiting for postgres..."

while ! nc -z $SQL_HOST $SQL_PORT; do
sleep 0.1
done

echo "PostgreSQL started"
fi

python manage.py flush --no-input
python manage.py migrate

exec "$@"
20 changes: 20 additions & 0 deletions contrib/docker/start.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
#!/bin/sh
python manage.py migrate --noinput # Apply database migrations
python manage.py collectstatic --clear --noinput # Collect static files

# Prepare log files and start outputting logs to stdout
touch /src/logs/gunicorn.log
touch /src/logs/access.log
tail -n 0 -f /src/logs/*.log &

# Start Gunicorn processes
echo Starting Gunicorn.
exec gunicorn \
--bind 0.0.0.0:8000 \
--workers 3 \
--worker-class gevent \
--log-level=info \
--log-file=/src/logs/gunicorn.log \
--access-logfile=/src/logs/access.log \
--name debunkbot --reload debunkbot.wsgi:application \
--chdir debunkbot/
26 changes: 26 additions & 0 deletions docker-compose.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
version: '3.3'

services:
db:
image: "postgres:12"
hostname: postgres
volumes:
- postgres_data:/var/lib/postgresql/data/
environment:
- POSTGRES_USER=debunkbot
- POSTGRES_PASSWORD=debunkbot
- POSTGRES_DB=debunkbot
web:
build:
context: .
volumes:
- ./:/src
ports:
- 8000:8000
env_file:
- ./.env
depends_on:
- db

volumes:
postgres_data:
5 changes: 4 additions & 1 deletion requirements.txt
Original file line number Diff line number Diff line change
Expand Up @@ -4,14 +4,17 @@ certifi==2020.4.5.1
chardet==3.0.4
dj-database-url==0.5.0
Django==2.2
gevent==20.4.0
google-auth==1.14.1
google-auth-oauthlib==0.4.1
greenlet==0.4.15
gspread==3.5.0
gunicorn==20.0.4
httplib2==0.17.3
idna==2.9
oauth2client==4.1.3
oauthlib==3.1.0
psycopg2==2.8.5
psycopg2==2.8.3
pyasn1==0.4.8
pyasn1-modules==0.2.8
python-dotenv==0.13.0
Expand Down
8 changes: 4 additions & 4 deletions utils/gsheet/helper.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
class GoogleSheetHelper(object):
"""Helper class for getting data from google sheet"""

def __init__(self):
def __init__(self) -> None:
"""Instance method to initialize Google Drive API
:param self:
:return: None
Expand All @@ -18,12 +18,12 @@ def __init__(self):
'https://spreadsheets.google.com/feeds',
'https://www.googleapis.com/auth/drive'
]

google_credentials = json.loads(
os.getenv('GOOGLE_CREDENTIALS'), strict=False)
eval(os.getenv('GOOGLE_CREDENTIALS')), strict=False)
self.credentials = ServiceAccountCredentials.from_json_keyfile_dict(
google_credentials, scopes=self.scope)
self.client = gspread.authorize(self.credentials)
self.sheet_name = google_credentials['sheet_name']

def open_sheet(self) -> Optional[dict]:
"""Instance method to open a workbook and get the data
Expand All @@ -32,7 +32,7 @@ def open_sheet(self) -> Optional[dict]:
:return: Sheet Record as dict or None
"""
try:
sheet = self.client.open(os.getenv('GOOGLE_SHEET_NAME')).sheet1
sheet = self.client.open(self.sheet_name).sheet1
return sheet.get_all_records()

except gspread.exceptions.SpreadsheetNotFound as e:
Expand Down