Skip to content

Backend for automated evaluation of programming tasks in higher education

License

Notifications You must be signed in to change notification settings

danilobecke/codemaze

Repository files navigation

Codemaze (aka backend)

Test Coverage Maintainability Tests Linter (Pylint) Type-Check (MyPy)

Codemaze is an advanced platform specifically designed to enhance the programming education experience in higher education. Providing a virtual environment for code execution, Codemaze allows users to practice, test, and evaluate their programs across various programming languages. Furthermore, the software integrates powerful features including plagiarism detection reports, detailed logging capabilities, and Swagger documentation for easy reference. With support for adding programming languages, customizable configurations, and flexible architecture built on Docker containers, Codemaze empowers both instructors and higher education students to collaboratively and efficiently explore the realm of programming.

This is the backend project. The suggested frontend is Codemaze-Web.

Table of contents

  1. Prerequisites
  2. Deploy
    1. Environment
    2. Configuration
    3. Run deploy
  3. Plugins
    1. Plagiarism report
    2. Logs
  4. Swagger
  5. Contributing
    1. Running as debug
    2. Testing
    3. Adding Supported Programming Languages
    4. Modelling
      1. ERD
      2. DSD
  6. Stopping
  7. TODO
  8. License
  9. Technologies
  10. Contact Info

Prerequisites

You must have installed in the host machine before deploying or in your local machine before running:

  1. Docker - used to manage the database, the deploy-container, and sandbox environments for the runners.

Deploy

Clone this repository in the host machine.

Environment

Optionally, you can create an environment file on the root directory named .env.deploy with keys for the supported plugins (plagiarism report and logs):

MOSS_USER_ID="999999999"
PAPERTRAIL_ADDRESS="logsN.papertrailapp.com:XXXXX"

Configuration

The config.toml file contains admin-level settings:

Key Description Kind
managers-mail-list [Can be empty] - if not empty, list of emails allowed to register as manager. Strings array
allowed-origins Allowed origins (CORS). Strings array - REGEX supported
session-duration Duration of the session (minutes). Integer
task-max-size-mb Max size allowed for each task description (MB). Float
test-max-size-mb Max size allowed for each test case (MB). Float
code-max-size-mb Max size allowed for each source code (MB). Float
timeout Timeout for the student's code run (seconds). Float
max-memory-mb Max memory allowed to be used for each container running students' code (MB). Integer
gcc-parameters Compilation flags for the GCC (C compiler). String

Run deploy:

make build
make deploy

By doing this, Codemaze will be running in the :8080 port. You can customize the port by updating the compose.deploy.yaml.

Plugins

The following plugins are optional but improve the Codemaze experience.

Plagiarism Report

Codemaze uses Stanford's Moss as a plagiarism report tool. This is optional, but if you want it to work, the following steps must be followed:

Get a user ID:

Send an email to moss@moss.stanford.edu with:

Subject: New Account

Message:

registeruser
mail email@mail.com

replacing email@mail.com with your email.

From the response, search for $userid= and save your user ID.

NOTE: as per Moss's suggestion: if you are having trouble registering, please try using Gmail to send the registration message.

Edit the .env file

Add your user ID as the environment var MOSS_USER_ID in your .env file(s). See .env.deploy, .env.test, and .env.debug.

Logs

Codemaze will keep logs from the past 10 days in the /logs directory. Optionally, you can have your logs on Papertrail. To do so, the following steps must be followed:

Get a destination address:

  1. Create an account on https://papertrailapp.com/signup;
  2. Go to Settings and Log Destinations;
  3. Copy your address. Format: logsN.papertrailapp.com:XXXXX;

Add the env var

Add your destination address as the environment var PAPERTRAIL_ADDRESS in your .env file(s). See .env.deploy, .env.test, and .env.debug.

Swagger - v1

The Swagger documentation is hosted at the /api/v1/docs URL.

Contributing

You can run Codemaze locally for debugging and testing purposes.

To install the dependencies, navigate to the root folder and run:

make setup

As debug (localhost:8080)

Navigate to the root folder and run:

make build-debug
make debug

Optionally, you can have a .env.debug file in the same way as the .env.deploy one.

Tests

Navigate to the root folder and run:

make test # or make smoke-test

Optionally, you can have a .env.test file in the same way as the .env.deploy one.

Adding Supported Programming Languages

In order to add support for another programming language, you must implement a class inheriting from the Runner class and add an object of this class in the runner's list of the RunnerService. Each runner must have its own Docker Container and run the student's code inside of it. See CRunner and this commit for reference.

Modelling

Entity-Relationship Diagram (ERD)

entity-relationship

Data-Structure diagram (DSD)

data-structure

Stopping

To stop and remove the Docker containers, you can run:

make stop-deploy # if deploy
make stop-debug # if debugging
make stop-test # if testing

TODO

Endpoints

  • WIP - POST /session/reset_password
  • DELETE /tasks/:id: - authorization: manager
  • DELETE /groups/:id: - authorization: manager

Improvements

  • Return the number of errors for the test with more failures, updating the Report model
  • Add support for supporting files when creating tasks (required updating the MossService)

License

Introduced by Danilo Cleber Becke in 2023, the BSD 2-Clause License outlines the terms under which the software can be utilized and distributed. Users are free to modify the software, as long as they adhere to two main conditions: 1) they must include the original copyright notice and a list of conditions, and 2) if they distribute the software in binary form, they must reproduce the copyright notice and conditions in the documentation or accompanying materials. The copyright holder and contributors are not liable for any direct, indirect, incidental, consequential, or other damages arising from the use of the software, regardless of the legal theory applied.

Technologies

Developed using Python 3.11.

Contact Info

You can reach me at danilobecke@gmail.com and http://linkedin.com/in/danilobecke/.