Skip to content

IDR(Integrated Data Repository) Server is a service that houses metadata for/and receives extracted data from IDR Clients.

License

Notifications You must be signed in to change notification settings

savannahghi/idr-server

Repository files navigation

IDR Server

IDR(Integrated Data Repository) Server is a service that houses extract metadata for/and receives extracted data from IDR Clients. The service is built mainly using Django and Django Rest Framework.

Coverage Status pre-commit Code style: black

Project overview

The server has three main entry points:

  • api:

    The server uses RESTful approach;
    this section gives a brief description of the available endpoints.

      -> {base_url}/api
          This is the entry point for all the API requests to the server.
    
      -> .../sql_database_sources:
          This serves all operations related to adding, updating, listing and deleting sources of data.
          The sources are mainly database vendors e.g MySQL, PostgreSQL etc. This url provides input for
          data source versions.
    
      -> .../data_source_versions:
          Helps in adding version labels to the database sources created in the previous endpoint.
    
      -> .../sql_extract_metadata:
          Serves valid sql queries that clients can use to extract data from local databases. The metadata
          includes proper labels that should be tagged along with the data that it extracts.
    
      -> .../sql_upload_metadata:
          This serves a summary information about the data that the client needs to upload. Data is
          uploaded in form of chunks; every chunk has to be related to a upload_metadata.
    
      -> .../sql_upload_chunks:
          This serves the actual data that is uploaded by the client. A client shoud first request for
          the creation of upload_metadata before sending chunks to the server.
    
  • home:

    This the default view/landing page of the base url. It displays the dashboards that has been added using admin tool.

  • admin:

    Admin acts as a supplement for what cannot be handled by api endpoints. It uses django's built-in tool that provides a web interface for managing objects in the database. One can do all the basic CRUD(Create, Update, Delete) and List operations using the tool.

Getting started

To run the project locally, first ensure you have python >=3.9 and PostgreSQL installed. Create a database for the app and proceed in with these steps:

  1. Ensure you have set the following variables. (You can use a simple file_name.sh file that can be sourced)

    Variable Example value Description
    DJANGO_ADMIN_URL admin/ url to access admin view
    DJANGO_ALLOWED_HOSTS "localhost,127.0.0.1" list of allowed hosts
    DJANGO_DEBUG false whether or not to run the app in debug mode
    DJANGO_SECRET_KEY "django_project_secret_key.." django secret key
    DJANGO_SETTINGS_MODULE "config.settings.local" the configuration file containing your settings
    POSTGRES_DB "db_name" the database name in your local env
    POSTGRES_PASSWORD "db_pwd" the password to the database
    POSTGRES_PORT 5432 the port on which the db is running
    POSTGRES_HOST "127.0.0.1" host address on which the db is running
    POSTGRES_USER "db_user" the database user name
    GOOGLE_CLOUD_PROJECT "cloud_prj_id" gcloud project ID
    GOOGLE_APPLICATION_CREDENTIALS "google_app_credentials" the path to google app credentials

    The values for the test variables are the same as above

    Test Variables Example value Description
    TEST_POSTGRES_DB "test_db_name" your test database name
    TEST_POSTGRES_HOST "test host for the db" host address on which the db is running
    TEST_POSTGRES_PASSWORD "test password" test database password
    TEST_POSTGRES_PORT 5432 the port on which your db is running
    TEST_POSTGRES_USER "test user" test db user name
  2. Clone the repo and CD into the root of the directory of the application(directory containing manage.py file)

  3. Create a virtual environment for the server. You may first have to install virtualenvwrapper

    [machine@user idr-server]$ mkvirtualenv idr_venv
    
    [machine@user idr-server]$ workon idr_venv
    
  4. Run pip install to install the required packages and run migrate to create tables.

    (idr_venv) [machine@user idr-server]$ pip install -r requirements.txt
    
    (idr_venv) [machine@user idr-server]$ ./manage.py migrate
    
  5. Create the superuser by running and following command and filling the resulting prompts.

    (idr_venv) [machine@user idr-server]$ ./manage.py createsuperuser
    
  6. Run the project by;

    (idr_venv)[machine@user idr-server]$ ./manage.py runserver
    

Running Tests

(idr_venv)[machine@user idr-server]$ pytest .

Licence

MIT License

Copyright (c) 2022, Savannah Informatics Global Health Institute