This application sends push notifications for new announcements on Immobilienscount based on at least one search query.
The application uses free services or free usage plans of a variety of online services. The services used are:
- Heroku - Serverless hosting platform with free plan available
- mongoDB Atlas - Free mongoDB hosting. Used to store the already seen announcements.
- Telegram - Cross-platform messaging application. Used to receive push notifications.
Suggested, but not required:
- Pipedream - Used to trigger the application based on a defined schedule
After registering in Heroku, Atlas and Telegram, one can start setting up the application and deploy it.
First install on your local machine Heroku CLI and login.
heroku login
Afterwards, select an application name and store it in an environment variable:
export APP_NAME=<YOUR_HEROKU_APP_NAME>
Create a free account at Mongo Cloud and set up a user for accessing the database. Provide the database credentials, database name and collection names as environment variables:
export DB_USERNAME=<YOUR_DB_USERNAME>
export DB_PASSWORD=<YOUR_DB_PASSEWORD>
export DB_NAME=<YOUR_DB_NAME>
export BH_COLLECTION_NAME=<YOUR_BH_COLLECTION_NAME>
export IMMO_COLLECTION_NAME=<YOUR_IMMO_COLLECTION_NAME>
To keep the secrets safe, you can use Heroku's secret injection feature. It provides secure storage. After the deployment, the values are make available as environment variables.
Use BotFather to create a bot in telegram. Follow the guide here.
After you have created the bot, you should receive a token that allows you to communicate with the bot, something like 110201543:AAHdqTcvCH1vGWJxfSeofSAs0K5PALDsaw.
$ export BOT_TOKEN=<TELEGRAM_BOT_TOKEN>
From Telegram, it is also needed a chat id, i.e. the id of your chat with the bot. After writing a couple of dummy messages to the bot, run this command to get the id:
$ curl https://api.telegram.org/bot<YourBOTToken>/getUpdates
The chat object in the message looks something along the lines:
{
"update_id": 8393,
"message": {
"message_id": 3,
"from": {"id": 7474, "first_name": "AAA"},
"chat": {"id": "<TELEGRAM_BOT_CHAT_ID>", "title": ""},
"date": 25497,
"new_chat_participant": {
"id": 71, "first_name": "NAME", "username": "YOUR_BOT_NAME"
}
}
}
After finding the chat id, export it:
$ export CHAT_ID=<TELEGRAM_BOT_CHAT_ID>
If the response does not look like that, write a couple of more messages to the bot.
Finally, one can set up the configuration variables:
heroku config:set -a $APP_NAME DB_NAME=$DB_NAME
heroku config:set -a $APP_NAME DB_USERNAME=$DB_USERNAME
heroku config:set -a $APP_NAME DB_PASSWORD=$DB_PASSWORD
heroku config:set -a $APP_NAME BH_COLLECTION_NAME=$BH_COLLECTION_NAME
heroku config:set -a $APP_NAME IMMO_COLLECTION_NAME=$IMMO_COLLECTION_NAME
heroku config:set -a $APP_NAME BOT_TOKEN=$BOT_TOKEN
heroku config:set -a $APP_NAME CHAT_ID=$CHAT_ID
heroku config:set -a $APP_NAME SECRET=<SECRET>
The variable IMMO_SEARCH_URL is one ImmobilienScout24 URL or a list of them split by ;.
The scrapping process monitors these URLs. For the application to work properly, you should make sure the URLs have the following properties:
- Apartments are in list view (not map view)
- Sorted by date the newest first. This is important as the application only checks the first page of the results.
After getting the URLs from the browser, run in the terminal:
export IMMO_SEARCH_URL=<YOUR_IMMO_URL_1>;<YOUR_IMMO_URL_2>
heroku config:set -a $APP_NAME IMMO_SEARCH_URL=<IMMOSEARCH_URL>
Application deployment gets invoked on every git push, via GitHub Integration.
The application offers a single GET endpoint /checkimmobilienscout.
It returns the unseen announcements as a json list, and sends a notification for each of them via Telegram.