Skip to content

Commit

Permalink
Merge pull request #315 from Exabyte-io/hotfix/tests
Browse files Browse the repository at this point in the history
Fix: TS + README
  • Loading branch information
k0stik committed Apr 8, 2024
2 parents 0e38ae5 + 730df3a commit f940b16
Show file tree
Hide file tree
Showing 8 changed files with 18 additions and 14 deletions.
14 changes: 8 additions & 6 deletions README.md
Expand Up @@ -12,7 +12,6 @@ The latest variants of schemas and examples are available at [schemas.mat3ra.com

ESSE has a dual-nature as both a Python and a Node.js package.


## 1. Installation

### 1.1. Python
Expand Down Expand Up @@ -41,7 +40,6 @@ pip install -e PATH_TO_ESSE_REPOSITORY
npm install @mat3ra/esse
```


## 2. Usage

ESSE contains separate but equivalent interfaces for Python and Javascript.
Expand All @@ -65,7 +63,6 @@ const helper = new ESSE();
const schema = helper.getSchemaById("material");
```


## 3. Directory Structure

ESSE contains 3 main directories, [schema](schema), [example](example) and [src](src) outlined below.
Expand All @@ -88,12 +85,11 @@ This directory contains the examples formed according to the schemas and impleme

This directory contains Python and Javascript interfaces implementing the functionality to access and validate schemas and examples.


## 4. Conventions

### 4.1. Generative vs Non-generative keys
Generative keys are the fields which allow for user input prior to calculation of the final property values. A flag is included in the schema comments on the fields in [property schemas](schema/properties_directory): `isGenerative:true` marks which fields to use as subschemas in the generation of a user input schema. On properties allowing user inputs, additional fields may be tagged, as in [the `file_content` property](schema/properties_directory/non-scalar/file_content.json)

Generative keys are the fields which allow for user input prior to calculation of the final property values. A flag is included in the schema comments on the fields in [property schemas](schema/properties_directory): `isGenerative:true` marks which fields to use as subschemas in the generation of a user input schema. On properties allowing user inputs, additional fields may be tagged, as in [the `file_content` property](schema/properties_directory/non-scalar/file_content.json)

## 5. Development

Expand All @@ -118,13 +114,19 @@ NOTE: The PY and JS modules are built from the same JSON sources, but using diff
When developing in python the following should be taken into account:

1. The modules containing the schemas and examples are generated using the [build-schemas.py](./build_schemas.py) script. There is a setup for it to be run automatically on every commit, but it is recommended to run it manually before committing to make sure that the changes are reflected in the modules. This can be done with `pre-commit run --all-files`. The pre-commit package can be installed with `pip install pre-commit`. To rebuild schemas manually, run (note `-e` in install):

```bash
virtualenv .venv
source .venv/bin/activate
pip install -e ."[tests]"
pip install pre-commit
pre-commit --install
git config --unset-all core.hooksPath
python build_schemas.py
```

2. Tests can be run using the following commands:

```bash
virtualenv .venv
source .venv/bin/activate
Expand All @@ -135,6 +137,7 @@ When developing in python the following should be taken into account:
### 5.2. Development in Javascript/Typescript

See [package.json](package.json) for the list of available npm commands. The JS modules are generated using the [build_schema.js](./build_schema.js) script. There is a setup for it to be run automatically when the package is installed (see "transpile" directive). To rebuild schemas manually, run:

```bash
npm install
npm run transpile
Expand All @@ -149,7 +152,6 @@ Other suggestions:
- Use unique IDs for schemas
- Do not use circular references in the schemas, instead leave the type as object and add explanation to description.


## Links

[^1]: [Data-centric online ecosystem for digital materials science](https://arxiv.org/pdf/1902.10838.pdf)
Expand Down
2 changes: 1 addition & 1 deletion dist/js/schema/3pse/db/nist_jarvis/2024.3.13/db_entry.json
@@ -1,7 +1,7 @@
{
"$id": "3pse/db/nist-jarvis/2024.3.13/db-entry",
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "NIST JARVIS Atoms schema",
"title": "NIST JARVIS db entry schema",
"description": "NIST J.A.R.V.I.S. db entry `atoms` key schema. Based on https://figshare.com/articles/dataset/Monolayer_data_for_heterostructure/22344571",
"type": "object",
"additionalProperties": true,
Expand Down
2 changes: 1 addition & 1 deletion dist/js/schemas.json

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion dist/js/types.ts
Expand Up @@ -48,7 +48,7 @@ export interface NISTJARVISAtomsSchema {
/**
* NIST J.A.R.V.I.S. db entry `atoms` key schema. Based on https://figshare.com/articles/dataset/Monolayer_data_for_heterostructure/22344571
*/
export interface NISTJARVISAtomsSchema {
export interface NISTJARVISDbEntrySchema {
/**
* NIST J.A.R.V.I.S. db entry `atoms` key schema. Based on https://figshare.com/articles/dataset/Monolayer_data_for_heterostructure/22344571
*/
Expand Down
2 changes: 1 addition & 1 deletion package.json
Expand Up @@ -7,7 +7,7 @@
"build-schemas": "BUILD_ASSETS=true BUILD_PYTHON_MODULES=true BUILD_PATH='./dist/js' ts-node build_schemas.ts",
"compile-types": "ts-node compile_ts.ts",
"transpile-and-build-assets": "npm run transpile && npm run build-schemas && npm run compile-types",
"test": "npm run build-schemas && nyc --reporter=text mocha --recursive --bail tests/js",
"test": "npm run build-schemas && nyc --reporter=text mocha --recursive --timeout 15000 --bail tests/js",
"test-only": "nyc --reporter=text mocha --recursive --bail tests/js",
"lint": "eslint src/js && prettier --write src/js",
"lint:fix": "eslint --fix --cache src/js && prettier --write src/js",
Expand Down
2 changes: 1 addition & 1 deletion schema/3pse/db/nist_jarvis/2024.3.13/db_entry.json
@@ -1,7 +1,7 @@
{
"$id": "3pse/db/nist-jarvis/2024.3.13/db-entry",
"$schema": "http://json-schema.org/draft-07/schema#",
"title": "NIST JARVIS Atoms schema",
"title": "NIST JARVIS db entry schema",
"description": "NIST J.A.R.V.I.S. db entry `atoms` key schema. Based on https://figshare.com/articles/dataset/Monolayer_data_for_heterostructure/22344571",
"additionalProperties": true,
"type": "object",
Expand Down
2 changes: 1 addition & 1 deletion src/py/mat3ra/esse/data/schemas.py

Large diffs are not rendered by default.

6 changes: 4 additions & 2 deletions tests/js/validate.ts
Expand Up @@ -37,8 +37,10 @@ describe("validate all examples", () => {

describe("schema titles must be unique or empty", () => {
JSONSchemasInterface.setSchemaFolder(schemasPath);
const schemas = JSONSchemasInterface.schemasCache.values();
const repeatedSchemaTitles = Object.entries(groupBy(schemas, "title"))

const repeatedSchemaTitles = Object.entries(
groupBy(Object.fromEntries(JSONSchemasInterface.schemasCache), "title"),
)
.filter(([title, groupedValues]) => title !== "undefined" && groupedValues.length > 1)
// @ts-ignore
.map(([title, groupedValues]) => [title, groupedValues.map(({ $id }) => $id)]);
Expand Down

0 comments on commit f940b16

Please sign in to comment.