edX Analytics API Server

DEPRECATION NOTICE

The Insights product and associated repositories are in the process of being deprecated and removed from service. Details on the deprecation status and process can be found in the relevant Github issue <https://github.com/openedx/public-engineering/issues/221>_.

This repository may be archived and moved to the openedx-unsupported Github organization at any time.

The following sections are for historical purposes only.

---

This repository includes the Django server for the API as well as the API package itself. The client is hosted at https://github.com/openedx/edx-analytics-data-api-client.

The enterprise client is hosted at https://github.com/openedx/frontend-app-admin-portal.

License

The code in this repository is licensed under version 3 of the AGPL unless otherwise noted.

Please see LICENSE.txt for details.

Getting Started With Devstack

The best way to run this service is with edX Devstack: https://github.com/openedx/devstack.

See the Devstack Readme for information on how to install and run devstack.

With devstack running and this repo checked-out locally, you'll want to do the following from your devstack directory:

$ make dev.pull.analyticsapi
$ make dev.up.analyticsapi
$ make dev.provision.analyticsapi     # this will provision the DB user, run migrations, etc.
$ docker-compose restart analyticsapi # maybe necessary to reset the DB connection
$ make analyticsapi-logs              # make sure the app is running without errors

Getting Started Standalone (deprecated, prefer running in devstack)

1. Create a virtual environment and activate it.

2. Install the requirements:

   $ make develop

3. Setup the databases:

   $ make migrate-all

   The learner API endpoints require elasticsearch with a mapping defined on this wiki page. The connection to elasticsearch can be configured by the ELASTICSEARCH_LEARNERS_HOST and ELASTICSEARCH_LEARNERS_INDEX django settings. To run tests locally use:

   $ make test

   This will launch an Elasticsearch Docker container for the tests. To simply run the container without starting the tests, run:

   $ make test.run_elasticsearch

   To stop an Elasticsearch Docker container, run:

   $ make test.stop_elasticsearch

4. Create a user and authentication token. Note that the user will be created if one does not exist.

   $ ./manage.py set_api_key <username> <token>

5. Run the server:

   $ ./manage.py runserver

Development with edx-enterprise-data

If you need to make changes to edx-enterprise-data and have them reflected when you run the edx-analytics-data-api server, you can follow these steps. If you do not intend to make changes to edx-enterprise-data, you can skip this section.

Devstack Development

#. Clone the edx-enterprise-data repo into the src folder alongside your devstack workspace on the host system. #. The modified edx-enterprise-data repository will now be available to devstack containers at /edx/src/edx-enterprise-data. #. Inside the edx-data-analytics-api container run pip install -e /edx/src/edx-enterprise-dataStandalone Development ~~~~~~~~~~~~~~~~~~~~~~ #. Recommended: Install this repo into a subfolder of your working directory. Within that subfolder create ansrcfolder. #. Clone the `edx-enterprise-data <https://github.com/openedx/edx-enterprise-data>`_ repo into thesrcfolder. #.cdinto youredx-data-analytics-apifolder and activate your virtualenv. #. Runpip install -e ./src/edx-enterprise-data. #. Run the server as per instructions above. Changes toedx-enterprise-datashould be picked up by the server. Loading Data ------------ The fixtures directory contains demo data and thegenerate_fake_enrollment_datamanagement command can generate enrollment data. Run the command below to load/generate this data in the database. :: $ make loaddata Additional management commands for creating data can be found in `edx-enterprise-data <https://github.com/openedx/edx-enterprise-data>`_ Loading Video Data ~~~~~~~~~~~~~~~~~~ The above command should work fine on its own, but you may see warnings about video ids: :: WARNING:analyticsdataserver.clients:Course Blocks API failed to return video ids (401). See README for instructions on how to authenticate the API with your local LMS. In order to generate video data, the API has to be authenticated with your local LMS so that it can access the video ids for each course. Instead of adding a whole OAuth client to the API for this one procedure, we will piggyback off of the Insights OAuth client by taking the OAuth token it generates and using it here. 1. Start your local LMS server. (e.g. in devstack, run `paver devstack --fast lms`). 2. If your local LMS server is running on any address other than the default of `http://localhost:18000/`, make sure to add this setting to `analyticsdataserver/settings/local.py` with the correct URL. (you will likely not need to do this): :: # Don't forget to add the trailing forward slash LMS_BASE_URL = 'http://example.com:18000/' 3. Sign into your local Insights server making sure to use your local LMS for authentication. This will generate a new OAuth access token if you do not already have one that isn't expired. The user you sign in with must have staff access to the courses for which you want generated video data. 4. Visit your local LMS server's admin site (by default, this is at `http://localhost:18000/admin`). 5. Sign in with a superuser account. Don't have one? Make one with this command in your devstack as the `edxapp` user: :: $ edxapp@precise64:~/edx-platform$ ./manage.py lms createsuperuser Enter a username and password that you will remember. 6. On the admin site, find the "Django OAuth Toolkit" section and click the link "Access tokens". The breadcrumbs should show "Home > Django OAuth Toolkit > Access tokens". Copy the string in the "Token" column for the first row in the table. Also, make sure the "User" of the first row is the same user that you signed in with in step 3. 7. Paste the string as a new setting in `analyticsdataserver/settings/local.py`: :: COURSE_BLOCK_API_AUTH_TOKEN = '<paste access token here>' 8. Run `make loaddata` again and ensure that you see the following log message in the output: :: INFO:analyticsdataserver.clients:Successfully authenticated with the Course Blocks API. 9. Check if you now have video data in the API. Either by querying the API in the swagger docs at `/docs/#!/api/Videos_List_GET`, or visiting the Insights `engagement/videos/` page for a course. Note: the access tokens expire in one year so you should only have to follow the above steps once a year. Running Tests ------------- Runmake validate`` install the requirements, run the tests, and run lint.