diff --git a/content/developers/api-reference/_index.md b/content/developers/api-reference/_index.md index 017818ed2..38b8cf74f 100644 --- a/content/developers/api-reference/_index.md +++ b/content/developers/api-reference/_index.md @@ -19,14 +19,12 @@ aliases: Assets API →
Attachments API →
Blobs API →
- Blockchain API (v1alpha2) →
Compliance API →
Events API →
IAM Policies API →
IAM Subjects API →
Locations API →
Public Assets API →
- System API →
Tenancies API →
Tenancy Caps API →

diff --git a/content/developers/api-reference/assets-api/index.md b/content/developers/api-reference/assets-api/index.md index dc0746570..c04a623ee 100644 --- a/content/developers/api-reference/assets-api/index.md +++ b/content/developers/api-reference/assets-api/index.md @@ -252,7 +252,7 @@ To fetch all Assets that use a specific Proof Mechanism, `GET` the Assets resour ```bash curl -g -v -X GET \ -H "@$HOME/.datatrails/bearer-token.txt" \ - "https://app.datatrails.ai/archivist/v2/assets?attributes.proof_mechanism=simple_hash" + "https://app.datatrails.ai/archivist/v2/assets?proof_mechanism=MERKLE_LOG" ``` #### Fetch Events Ordered for SIMPLEHASHV1 Schema diff --git a/content/developers/api-reference/system-api/index.md b/content/developers/api-reference/system-api/index.md deleted file mode 100644 index e11ed830d..000000000 --- a/content/developers/api-reference/system-api/index.md +++ /dev/null @@ -1,89 +0,0 @@ ---- -title: "System API" -description: "System API Reference" -lead: "System API Reference" -date: 2021-06-09T13:49:35+01:00 -lastmod: 2021-06-09T13:49:35+01:00 -draft: false -images: [] -menu: - developers: - parent: "api-reference" -weight: 113 -toc: true -aliases: - - /docs/api-reference/system-api/ ---- -{{< note >}} -**Note:** This page is primarily intended for developers who will be writing applications that will use DataTrails for provenance. -If you are looking for a simple way to test our API you might prefer our [Postman collection](https://www.postman.com/datatrails-inc/workspace/datatrails-public/overview), the [YAML runner](/developers/yaml-reference/story-runner-components/) or the [Developers](https://app.datatrails.ai) section of the web UI. - -Additional YAML examples can be found in the articles in the [Overview](/platform/overview/introduction/) section. -{{< /note >}} -## System API Examples - -Create the [bearer_token](/developers/developer-patterns/getting-access-tokens-using-app-registrations) and store in a file in a secure local directory with 0600 permissions. - -### Querying Blockchain Status - -The `archivistnode` endpoint reports on the status of the blockchain. - -Query the endpoint: - -```bash -curl -v -X GET \ - -H "@$HOME/.datatrails/bearer-token.txt" \ - https://app.datatrails.ai/archivist/v1/archivistnode -``` - -The response is: - -```json -{ - "identity": "quorum", - "blockchain_nodes": [ - { - "validator_pubkey": { - "kty": "EC", - "crv": "P-256K", - "x": "VBKHictTWJC-3sqknXCb8MI4IxTc3c_My7lnem2C74E=", - "y": "ItNeb5d-6vEHkvtUOcDYrEADxsZXeOCJm18pQWntenE=", - "d": "" - }, - "block_height": "38773", - "connection_status": "REACHABLE" - "genesis_hash":"0x1b526bd9c7f9bf7c43ba91ad07e5530eb7ceedf390396f9fbfeb68722e097e95", - "state_root":"0x9606fc44a382938703678ac90581ab1260c9efd20ea8c7f90c87852bc982f3a7", - "timestamp_committed": "2019-01-02T01:03:07Z", - "timestamp_created": "2019-01-01T12:00:27Z", - "syncing": null, - "peers": [ - { - "validator_pubkey": { - "kty": "EC", - "crv": "P-256K", - "x": "o0uZ8ix5DE42srPCw1o22wYibkHGkvyCuLVqwcVAxb0=", - "y": "W43sUjWg-ociR2x3CcAlWeOqc6oDkYui1JLup1q-ojU=", - "d": "" - }, - "connection_status": "REACHABLE" - }, - { - "validator_pubkey": { - "kty": "EC", - "crv": "P-256K", - "x": "5HcU1PJgTe0LGyGxKFrIPFZWdTbxPySfi6bKxdQeO8A=", - "y": "dEpMURyTwEGzpgIgLdm4Csl1BgF6H39tb1Kf8wLLhVI=", - "d": "" - }, - "connection_status": "REACHABLE" - } - ] - } - ] -} -``` - -## System OpenAPI Docs - -{{< openapi url="https://raw.githubusercontent.com/datatrails/datatrails-openapi/master/doc/archivistnodev1.swagger.json" >}} diff --git a/content/developers/developer-patterns/verifying-with-simple-hash/AuditSearch.png b/content/developers/developer-patterns/verifying-with-simple-hash/AuditSearch.png deleted file mode 100644 index 4528470f2..000000000 Binary files a/content/developers/developer-patterns/verifying-with-simple-hash/AuditSearch.png and /dev/null differ diff --git a/content/developers/developer-patterns/verifying-with-simple-hash/SimpleHashDetails.png b/content/developers/developer-patterns/verifying-with-simple-hash/SimpleHashDetails.png deleted file mode 100644 index e7e797616..000000000 Binary files a/content/developers/developer-patterns/verifying-with-simple-hash/SimpleHashDetails.png and /dev/null differ diff --git a/content/developers/developer-patterns/verifying-with-simple-hash/index.md b/content/developers/developer-patterns/verifying-with-simple-hash/index.md deleted file mode 100644 index 11cf202a8..000000000 --- a/content/developers/developer-patterns/verifying-with-simple-hash/index.md +++ /dev/null @@ -1,147 +0,0 @@ ---- - title: "Archived: Verifying Assets and Events with Simple Hash" - description: "Ensure Asset and Event Data Has Not Changed" - lead: "Ensure Asset and Event Data Has Not Changed" - date: 2021-05-18T14:52:25+01:00 - lastmod: 2021-05-18T14:52:25+01:00 - draft: false - images: [] - menu: - developers: - parent: "developer-patterns" - weight: 34 - toc: true - aliases: - - /docs/beyond-the-basics/verifying-with-simple-hash/ ---- - -Verifying your Simple Hash events provides an additional layer of assurance to your data, so you can ensure that the information you have in your hand at a given time has not changed. - -This allows you to check for yourself, and prove to others, that any attestations that you made have not been changed since they were recorded in DataTrails. - -To verify your data, you may use the [DataTrails Simple Hash tool](https://github.com/datatrails/datatrails-simplehash-python), available on GitHub. - -Please note that with Simple Hash, Events are committed to the DataTrails blockchain as a batch. Events with the blue tick have been committed to the blockchain as part of a batch, and will have a `Transaction ID`. With the free tier of DataTrails, Simple Hash batched commits happen every 30 days by default. For Public Assets, batched commits happen each day. If the tick mark is grey, your event has been confirmed in the system but not yet committed to the blockchain. **Your event(s) must have a blue tick for transaction details to be available for data verification.** - -## Step-by-Step Guide for Using the Simple Hash Tool - -1. Retrieve your transaction information. This will give you the inputs you need in later steps to check the hash for that batch of Events. - - {{< note >}} - For Public Assets, retrieve the transaction information from the public view of the Asset and Events, or from the [Public Assets Endpoint](/developers/api-reference/public-assets-api/). - {{< / note >}} - - {{< tabs name="retrieve-transaction-info" >}} - {{< tab name="UI" >}} - Select `Audit/Filters` from the sidebar and select a `Transaction` from the Events Overview List. - {{< img src="AuditSearch.png" alt="Rectangle" caption="Audit/Search" class="border-0" >}} - - Copy the `start time` and `end time` from the Simple Hash Details. These will be used as inputs to the [DataTrails Simple Hash tool](https://github.com/datatrails/datatrails-simplehash-python). - - {{< img src="SimpleHashDetails.png" alt="Rectangle" caption="Simple Hash Details" class="border-0" >}} - {{< /tab >}} - {{< tab name="Command Line" >}} - The [Blockchain API](/developers/api-reference/blockchain-api/) allows you to fetch transactions for an Event. See instructions for [creating your `BEARER_TOKEN_FILE`](/developers/developer-patterns/getting-access-tokens-using-app-registrations/) here. - - Using the Event ID as a parameter, run the following command: - - ```bash - curl -v -X GET \ - -H "@$HOME/.datatrails/bearer-token.txt" \ - https://app.datatrails.ai/archivist/v1alpha2/blockchain/assets//events/ - ``` - - This will return a list of matching blockchain transactions, as well as the `simple_hash_details`. Copy the `start_time` and `end_time` values to be used as inputs to the [DataTrails Simple Hash tool](https://github.com/datatrails/datatrails-simplehash-python). - - {{< /tab >}} - {{< /tabs >}} - -1. Use the [DataTrails Simple Hash tool](https://github.com/datatrails/datatrails-simplehash-python) to generate the hash of your Events. - - {{< tabs name="simple-hash-script" >}} - {{< tab name="Python" >}} - Use Python pip utility to install the `datatrails-simplehash` package. This package is supported for Python versions 3.7, 3.8, 3.9, and 3.10. - - ```bash - python3 -m pip install datatrails-simplehash - ``` - - You may then use the code to recreate the hash, using your [`BEARER_TOKEN_FILE`](/developers/developer-patterns/getting-access-tokens-using-app-registrations/) as the `auth_token` and the `start_date` and `end_date` copied in the last step: - - ```python - from datatrails_simplehash.v1 import ( - anchor_events, - SimpleHashError, - ) - - with open("credentials/token", mode="r", encoding="utf-8") as tokenfile: - auth_token = tokenfile.read().strip() - - try: - simplehash = anchor_events( - "2022-10-07T07:01:34Z", - "2022-10-16T13:14:56Z", - "app.datatrails.ai", - auth_token, - ) - except SimpleHashError as ex: - print("Error", ex) - - else: - print("simplehash=", simplehash) - ``` - - Run your Python file to return the hash value. - - {{< note >}} - **Note:** SimpleHashClientAuthError is raised if the auth token is invalid or expired. - {{< /note >}} - - {{< /tab >}} - {{< tab name="Command Line" >}} - Enter the query information you copied in the last step and run the command. See instructions for [creating your `BEARER_TOKEN_FILE`](/developers/developer-patterns/getting-access-tokens-using-app-registrations/) here. - - Commands can be executed anywhere using a virtual environment and published wheel. Credentials are stored in files within the credentials directory. - - Using an [`auth token`](/developers/developer-patterns/getting-access-tokens-using-app-registrations/) directly: - - ```bash - python3 -m venv simplehash-venv - source simplehash-venv/bin/activate - python3 -m pip install -q datatrails_simplehash - - datatrails_simplehashv1 \ - --auth-token-file "credentials/token" \ - --start-time "2022-11-16T00:00:00Z" \ - --end-time "2022-11-17T00:00:00Z" - - deactivate - rm -rf simplehash-venv - ``` - - Using a `Client ID` and `Client Secret`: - - ```bash - python3 -m venv simplehash-venv - source simplehash-venv/bin/activate - python3 -m pip install -q datatrails_simplehash - - CLIENT_ID=$(cat credentials/client_id) - datatrails_simplehashv1 \ - --client-id "${CLIENT_ID}" \ - --client-secret-file "credentials/client_secret" \ - --start-time "2022-11-16T00:00:00Z" \ - --end-time "2022-11-17T00:00:00Z" - - deactivate - rm -rf simplehash-venv - ``` - - {{< note >}} - **Note:** If you are using an environment other than `app.datatrails.ai`, add the URL with the `--fqdn` option. For example, `--fqdn "app.datatrails-poc.ai"`. - {{< /note >}} - - {{< /tab >}} - {{< /tabs >}} - -1. Compare the hash from your `Transaction Details` to the hash generated by the tool. If they match, your Event history has not changed. diff --git a/content/glossary/common-datatrails-terms/index.md b/content/glossary/common-datatrails-terms/index.md index 5c6dc634b..f611120f7 100644 --- a/content/glossary/common-datatrails-terms/index.md +++ b/content/glossary/common-datatrails-terms/index.md @@ -23,7 +23,6 @@ Select a term for more information. | [access policy](/platform/overview/core-concepts/#access-policies) | grants chosen Asset and Event access to stakeholders | | [actor](/platform/overview/creating-an-asset/)| person/machine/software integration that created a particular entry on the provenance record| | [administrator](/developers/developer-patterns/getting-access-tokens-using-app-registrations/#creating-an-app-registration) | user with permission to see all Asset and Event information within a Tenancy, and to grant access to other users | -| [anchored](/developers/developer-patterns/verifying-with-simple-hash/) | Simple Hash events are committed to the blockchain by hashing them in batches. The hash recorded on the chain is called the anchor | | [asset](/platform/overview/core-concepts/#assets) | a DataTrails Asset is an entry in your tenancy, which has a collection of attributes that describes its current state and a complete life history of Events | | [asset attributes](/platform/overview/creating-an-asset/#creating-an-asset) | key-value pairs that represent information about an Asset | | [asset ID](/platform/overview/creating-an-asset/)| the permanent unique identifier for an Asset, under which all provenance information (Events) can be found| @@ -46,17 +45,16 @@ Select a term for more information. | [organization](/platform/administration/verified-domain/)| any entity with a distinct DataTrails account who publishes or verifies provenance information on the platform| | [principal_accepted](/platform/overview/advanced-concepts/#user-principals-on-events)| the actual user principal information belonging to the credential used to access the DataTrails REST interface| | [principal_declared](/platform/overview/advanced-concepts/#user-principals-on-events)| an optional user-supplied value that tells who performed an Event| -| [proof mechanism](/platform/overview/core-concepts/#proof-mechanisms) | method by which information on the DataTrails blockchain can be verified; selected when an Asset is created | +| [proof mechanism](/platform/overview/core-concepts/#proving-provenance) | method by which information to the DataTrails distributed ledger can be verified | | [provenance](https://en.wiktionary.org/wiki/provenance) | the version and ownership history of a piece of data. With DataTrails this is an immutable audit trail to prove Who Did What When to any piece of data | | [public asset](/platform/overview/public-attestation/) | Assets that can be used to publicly assert data, accessible by URL without the need for a DataTrails account | | [selector](/platform/overview/creating-an-asset/#creating-an-asset) | identifying attribute the Yaml Runner will use to check if your Asset exists already before attempting to create it | -| [simple hash](/platform/overview/advanced-concepts/#simple-hash) | Proof Mechanism that commits information to the DataTrails blockchain in batches; value can confirm that information in the batch has not changed | | [tenancy](/platform/overview/core-concepts/#tenancies) | an organization’s private area within DataTrails, containing Asset and Event data | | [tenant display name](/platform/administration/identity-and-access-management/#tenant-display-name) | displayed only within own Tenancy for easy identification and switching | | [tenant_accepted](/platform/overview/advanced-concepts/#timestamps-on-events) | the time an event was actually received on the DataTrails REST interface | | [tenant_committed](/platform/overview/advanced-concepts/#timestamps-on-events) | the time an event was confirmed distributed to all DLT nodes in the value chain | | [tenant_declared](/platform/overview/advanced-concepts/#timestamps-on-events) | an optional user-supplied value that tells when an Event happened | -| [transaction](/developers/developer-patterns/verifying-with-simple-hash/)| final commitment of data to the Distributed Ledger Technology so that it is sealed and cannot be modifed, tampered or erased| +| [transaction](/developers/developer-patterns/verifying-with-simple-hash/)| final commitment of data to the Distributed Ledger Technology so that it is sealed and cannot be modified, tampered or erased| | [unlinked folder](/platform/administration/dropbox-integration/#editing-the-list-of-linked-folders) | a folder that has not been selected to be linked to DataTrails during the configuration or reconfiguration of an Integration | | [verified domain](/platform/administration/verified-domain/) | tenancy name visible to others in place of the tenancy ID when viewing the Asset Overview of a public Asset or a shared private Asset. Must be verified by the DataTrails team | | [verified organization](/platform/administration/verified-domain/)| an organization which has paid to have their domain verified and displayed in place of their tenancy ID in Instaproof results and in the Asset Overview | diff --git a/content/platform/overview/core-concepts/index.md b/content/platform/overview/core-concepts/index.md index b5d22bab5..19a445e99 100644 --- a/content/platform/overview/core-concepts/index.md +++ b/content/platform/overview/core-concepts/index.md @@ -37,28 +37,29 @@ Events are things that happen during an Asset's lifecycle. Each Event Record con Events can never be deleted or modified. Events provide details on Asset attributes, such as updating the weight of a shipment, and/or details about the event itself, such as a recording a new document version. -## Proof Mechanisms +## Proving Provenance -Assets and Events are core to the DataTrails platform, and being able to quickly demonstrate proof that these artifacts have not been tampered is key to being able to use them. - -When [creating an Asset](/platform/overview/creating-an-asset/), DataTrails uses a proof mechanism for that Asset and its Events. This determines how your data is recorded on the DataTrails blockchain. +Artifacts and Events are core to the DataTrails platform, and being able to quickly demonstrate proof that these artifacts have not been tampered is key to knowing the information is secure and trustworthy. DataTrails attestations are committed to immutable storage that is underpinned by cryptographically verifiable Merkle Mountain Range data structures for long term verifiability, even when offline. -{{< img src="merkleflow.png" alt="Rectangle" caption="" class="border-0" >}} - **Four Increasing Trust Levels** -In the customer's environment, data can be tampered, shredded, backdated... +At DataTrails we believe in holding ourselves to the same levels of accountability as our customers, and the Merkle Log proof mechanism provides the robustness, integrity and availability guarantees needed to ensure data authenticity in any digital or data supply chain. And you don't have to just take our word for it: you can check. + +Here's how it works: + +{{< img src="merkleflow.png" alt="Rectangle" caption="" class="border-0" >}} -Once `STORED` in DataTrails and shared with partners, no party to the transaction can tamper, back-date or shred evidence. However there could be a suspicion that DataTrails (or a hacker in our systems, or Microsoft under subpœna) could tamper with the data, or make it unavailable or something. +Without an Immutable Audit Trail, there is always the risk - or at least the suspicion - that data can be shredded, backdated or otherwise tampered with. -Once `COMMITTED` in the customer's Tenancy Merkle tree in public blob storage, customers can prove their Events to 3rd parties, AND any tampering by DataTrails is detectable (as long people check). Because this data is public, anyone can keep and maintain a copy just in case DataTrails' version disappears. If this checking is weak, and/or copies are not made, then in principle Data Trails could create forks. +Once `STORED` in DataTrails and shared with partners, no party to the transaction can tamper, back-date or shred evidence. However while the security and integrity of our customers' data is our top priority, there could still be a suspicion that DataTrails (or a hacker in our systems, or our cloud service provider under subpœna) could tamper with the data, or make it unavailable. -By adding all Tenancies to the Merkle Mountain Range (MMR) and signing the root, Events move to the `CONFIRMED` stage. The signature on to root holds DataTrails to account and prevents forks, and also prevents a kind of sybil attack that could otherwise be mounted by 3rd party verifiers. Even so, a tiny chance of tampering remains: DataTrails could possibly sign multiple MMRs and maintain multiple split histories, then present whichever version of the history is most advantageous. +Once `COMMITTED` in the customer’s Tenancy Merkle tree in public blob storage, customers can prove their Events to 3rd parties, AND any tampering by DataTrails is detectable. Because this data is public, anyone can keep and maintain a copy just in case DataTrails’ version disappears. These copies are great for availability and holding DataTrails accountable, but there is a risk that a kind of Sybil attack could be mounted where the community creates forks and then tries to accuse the DataTrails version of being wrong. -To make the whole history and individual events `UNEQUIVOCAL`, the root hash of the Committed MMR is periodically broadcast to single, well known location outside of DataTrails control (such as a smart contract address on Ethereum, or an official X account). +By adding all Tenancies to the Merkle Mountain Range (MMR) and signing the root, Events move to the `CONFIRMED` stage. The signature on the root at once holds DataTrails to account *and* prevents forks and the Sybil attack mentioned above. Even so, a tiny chance of tampering remains: in principle, multiple MMRs could be signed, creating multiple versions of history. +To make the whole history and individual events `UNEQUIVOCAL`, the root hash of the Committed MMR is periodically broadcast so that it is clear that there is one, and only one, version of history to underpin your data authenticity. ## Access Policies diff --git a/content/platform/overview/instaproof/index.md b/content/platform/overview/instaproof/index.md index 294f82f6b..9d7d48d33 100644 --- a/content/platform/overview/instaproof/index.md +++ b/content/platform/overview/instaproof/index.md @@ -56,7 +56,7 @@ The **Other Results** results are those from from unverified DataTrails accounts While they may seem less 'official' than verified account results, they may still be useful to you. The identity of all users making attestations in DataTrails is checked, recorded, and immutable, even if they are not (yet) associated with a verified domain name. -### What Do the Instaproof Results Mean +### What Do the Instaproof Results Mean? #### Immutable Audit Trail @@ -95,7 +95,7 @@ The **Overview** information about the Event **Asset Identity** - the ID of the parent Asset for this Event. -**Transaction** - This link contains the details of the blockchain transaction. +**Transaction** - This link contains the details of the Event transaction. {{< img src="InstaproofTransaction.png" alt="Rectangle" caption="Transaction Details" class="border-0" >}} **Type** - For Document Profile Events this will always be 'Publish' diff --git a/content/platform/overview/registering-a-document-profile-asset/index.md b/content/platform/overview/registering-a-document-profile-asset/index.md index 22c187154..7d1e9a661 100644 --- a/content/platform/overview/registering-a-document-profile-asset/index.md +++ b/content/platform/overview/registering-a-document-profile-asset/index.md @@ -132,9 +132,9 @@ In the file you created earlier, begin adding metadata for your Asset: 1. The `Advanced Options` tab is where you enter the Asset Attributes that are required for all asset types and also the optional document profile asset attributes. * `Document Type` - This is the class of the object; while it is arbitrary, it is best to have consistency amongst the type of Documents you use i.e. if it is a purchase order, the type could be `Purchase Order`, which will then be pre-populated for future Documents to use as their own types. - * `Proof Mechanism` - The method used to commit the blockchain transaction. + * `Proof Mechanism` - The method used to record the transaction to the DataTrails distributed ledger. - Please see our [Advanced Concepts](/platform/overview/core-concepts/#proof-mechanisms) section for more information on the Proof Mechanism for your Document + Please see our [Core Concepts](/platform/overview/core-concepts/#proving-provenance) section for more information on the Proof Mechanism for your Document {{< tabs name="add_asset_details_min" >}} {{{< tab name="UI" >}} {{< img src="RegDocAdvancedOptions.png" alt="Rectangle" caption="Advanced Options" class="border-0" >}}