Skip to content

Commit

Permalink
Merge pull request #6 from qualified/server-side-embed
Browse files Browse the repository at this point in the history
[QUAL-408] Add server validation example
  • Loading branch information
ggorlen committed Oct 26, 2023
2 parents 45f2017 + 1f4545f commit 223e3e4
Show file tree
Hide file tree
Showing 11 changed files with 986 additions and 0 deletions.
2 changes: 2 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,3 +4,5 @@
modules.xml
.idea/misc.xml
*.ipr

node_modules
2 changes: 2 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -45,3 +45,5 @@ This shows how you can use Embedded Qualified to set up an experimental challeng
### [Written Response](written-response)
Embed supports challenges that offer testable written response fields using markdown.

### [Server Validation](server-validation)
This demonstration illustrates how to run Embed on the server programmatically with the browser automation library Playwright. This allows you to securely run your candidate's code, validate solutions and persist the results on your server without using Qualified to manage assessments or assessment results.
6 changes: 6 additions & 0 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

1 change: 1 addition & 0 deletions server-validation/.node-version
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
v18
30 changes: 30 additions & 0 deletions server-validation/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# Qualified Embed Server Validation

This demo app shows how you can run Qualified's [Embed](https://www.qualified.io/embed) feature on a server, allowing you to validate candidate (or student) solutions securely and persist the results.

Normally, Embed is run in a stateless [challenge](https://www.qualified.io/embed/api-docs/tutorial-challenges.html) mode. Solutions are sent to Qualified's Code Runner service which runs the test suite and responds directly to the client with the result. Since the Code Runner doesn't track the results, there's no way to trust that the result sent to the client hasn't been tampered with.

Embed can also run in a stateful [assessment](https://www.qualified.io/embed/api-docs/tutorial-assessments.html) mode, eliminating the trust issue. Adding an auth token to the challenge mode is another way to save results securely. However, results must be stored on Qualified, and creating and managing assessments and invitations on Qualified may not fit all use cases.

This demo app uses a browser automation service, [Playwright](playwright.dev/), to run Embed on your server, effectively proxying requests to the Code Runner in a way that lets you record the results in a trustworthy manner, yet continue to use Embed's lightweight challenge mode. The only input from the candidate would typically be the solution code, which the candidate can test and develop before submission using a regular Embed instance such as the [index.html](public/index.html) in this project.

Using Embed this way, you can even create custom content outside of Qualified and inject arbitrary testing and starter code into Embed, effectively allowing you to manage your own content and results. This minimalist approach uses Qualified as a stateless code runner and, optionally, a web code editor.

Note that Playwright supports many popular languages, and other browser automation solutions exist, so there's nothing special about the particular stack used in this proof of concept.

As as a proof of concept, this app is not completely production-ready out of the box, so adaptation to your specific needs is expected.


## Usage

```
git clone https://github.com/qualified/embed-demos.git
cd embed-demos/server-validation
npm i
npx playwright install
npm start
```

With the server running, you can navigate to <http://localhost:3003>, open the browser developer tools console and use the Submit button to run your custom Embed server-side validation. `node test.js` can also be used to test the backend service directly, without a browser client.

[server.html](server.html) is the code Playwright uses to run Embed on the server and won't be seen by the candidate. You can open this file in a browser directly to help debug your server code. A `testRunEmbed()` function is provided to test your server Embed logic without Playwright.

0 comments on commit 223e3e4

Please sign in to comment.