Skip to content

Latest commit

 

History

History
467 lines (325 loc) · 16.3 KB

building-modules.md

File metadata and controls

467 lines (325 loc) · 16.3 KB
Raw

Building Modules

A Station Module is a long-running process that's performing jobs like network probes, content delivery, and computation.

Zinnia provides a JavaScript runtime with a set of platform APIs allowing modules to interact with the outside world.

In the long run, we want Zinnia to be aligned with the Web APIs as much as feasible.

For the shorter term, we are going to take shortcuts to deliver a useful platform quickly.

Getting Started

If you haven't done so, then install zinnia CLI per our instructions.

Using your favourite text editor, create a file called module.js with the following content:

console.log("Hello universe!");

Open the terminal and run the module by using zinnia run command:

$ zinnia run module.js
Hello universe!

See example modules for more advanced examples.

Table of Contents

Importing JavaScript Modules

Zinnia supports ES Modules (also known as JavaScript Modules).

Sandboxing in Filecoin Station

Filecoin Station limits module imports to files in the root directory of the Zinnia module being executed.

This limitation DOES NOT apply when running your code using zinnia run.

Consider the following directory layout:

src
  my-module
    util
      log.js
      helpers.js
    main.js
    lib.js
  other
    code.js

When you execute zinniad src/my-module/main.js:

  • In main.js, you can import any JavaScript file inside src/my-module directory and its subdirectories (e.g. src/my-module/util).
  • The same restriction applies transitively to other imported files too.

Example:

// These imports are allowed in `main.js`
import { processJob } from "./lib.js";
import { log } from "./util/log.js";

// This will be rejected at runtime
import * as code from "../other/code.js";

// This will work in `util/log.js`
import { format } from "../lib.js";

// This will be rejected
import * as code from "../../other/code.js";

Platform APIs

Standard JavaScript APIs

Zinnia provides all standard JavaScript APIs, you can find the full list in MDN web docs.

Web APIs

The following entities are defined in the global scope (globalThis).

Console Standard

Zinnia implements most of the console Web APIs like console.log. You can find the full list of supported methods in Deno docs and more details about individual methods in MDN web docs

Note: Messaged logged using Console APIs will not show in Station UI, they purpose is to help Station Module authors to troubleshoot issues. See Zinnia.activity.info and `Zinnia.activity.error APIs for reporting information to be shown to Station users in the Station Desktop UI.

DOM Standard

Encoding Standard

Fetch Standard

HTML Standard

Performance & User Timing

Streams Standard

URL Standard

Web Cryptography API

WebSockets Standard (partial support)

Web IDL Standard

Unsupported Web APIs

File API

Tracking issue: n/a

Service Workers & Web Workers

Tracking issue: n/a

WebSockets Standard

Tracking issue: n/a

Other

  • XMLHttpRequest Standard

libp2p

Zinnia comes with a built-in libp2p node based on rust-libp2p. The node is shared by all Station Modules running on Zinnia. This way we can keep the number of open connections lower and avoid duplicate entries in routing tables.

The initial version comes with a limited subset of features. We will be adding more features based on feedback from our users (Station Module builders).

Networking stack

  • Transport: tcp using system DNS resolver
  • Multistream-select V1
  • Authentication: noise with XX handshake pattern using X25519 DH keys
  • Stream multiplexing: both yamux and mplex

Zinnia.peerId

Type: string

Return the peer id of Zinnia's built-in libp2p peer. The peer id is ephemeral, Zinnia generates a new peer id every time it starts.

Zinnia.requestProtocol(remoteAddress, protocolName, requestPayload)

requestProtocol(
  remoteAddress: string,
  protocolName: string,
  requestPayload: Uint8Array,
): Promise<PeerResponse>;

Dial a remote peer identified by the remoteAddress and open a new substream for the protocol identified by protocolName. Send requestPayload and read the response payload.

The function returns a promise that resolves with a readable-stream-like object. At the moment, this object implements async iterable protocol only, it's not a full readable stream. This is enough to allow you to receive response in chunks, where each chunk is an Uint8Array instance.

Notes:

  • The peer address must include both the network address and peer id.
  • The response size is limited to 10MB. Larger responses will be rejected with an error.
  • We will implement stream-based API supporting unlimited request & response sizes in the near future, see zinnia#56 and zinnia#57.

Example

const response = await Zinnia.requestProtocol(
  "/dns/example.com/tcp/3030/p2p/12D3okowHR71QRJe5vrPm6zZXoH4K7z5mDsWWtxXpRIG9Dk8hqxk",
  "/ipfs/ping/1.0.0",
  new Uint8Array(32),
);

for await (const chunk of response) {
  console.log(chunk);
}

Integration with Filecoin Station

Zinnia.stationId

The associated Station Core's unique identifier (public key)

The value is hard-coded to 88 0 characters when running the module via zinnia CLI.

Zinnia.walletAddress

The wallet address where to send rewards. When running inside the Station Desktop, this API will return the address of the Station's built-in wallet.

The value is hard-coded to the Ethereum (FEVM) address 0x000000000000000000000000000000000000dEaD when running the module via zinnia CLI.

Zinnia.activity.info(message)

Add a new Activity Log item informing the Station user when things proceed as expected.

Example messages:

Saturn Node will try to connect to the Saturn Orchestrator...
Saturn Node is online and connected to 9 peers.

Zinnia.activity.error(message)

Add a new Activity Log informing the Station user about an error state.

Example messages:

Saturn Node is not able to connect to the network.

Zinnia.jobCompleted()

Report that a single job was completed.

Call this function every time your module completes a job. It's ok to call it frequently.

IPFS Retrieval Client

Zinnia provides a built-in IPFS retrieval client making it easy to fetch content-addressed data from IPFS and Filecoin networks. You can retrieve data for a given CID using the web platform API fetch together with the URL scheme ipfs://.

Example:

const response = await fetch("ipfs://bafybeib36krhffuh3cupjml4re2wfxldredkir5wti3dttulyemre7xkni");
assert(response.ok);
const data = await response.arrayBuffer();
// data contains binary data in the CAR format

Note: At the moment, Zinnia does not provide any tools for interpreting the returned CAR data. We are discussing support for reading UnixFS data in zinnia#245.

Under the hood, Zinnia handles ipfs://bafy... requests by calling Lassie's HTTP API. You can learn more about supported parameters (request headers, query string arguments), response headers and possible error status codes in Lassie's HTTP Specification. The format of CAR data returned by the retrieval client is described in Lassie's Returned CAR Specification.

Timeouts

The IPFS retrieval client is configured to time out after one day. When this happens, the response body stream is terminated in a way that triggers a reading error.

We strongly recommend to configure a client-side timeout using AbortController or AbortSignal.timeout().

Example:

const requestUrl = "ipfs://bafybeib36krhffuh3cupjml4re2wfxldredkir5wti3dttulyemre7xkni";
const response = await fetch(requestUrl, {
  signal: AbortSignal.timeout(500), // abort after 500ms
});
// etc.

Miscelaneous APIs

Zinnia.versions.zinna

The version of Zinnia runtime, e.g. "0.11.0".

Zinnia.versions.v8

The version of V8 engine, e.g. "11.5.150.2".

Testing Guide

Zinnia provides lightweight tooling for writing and running automated tests.

Test Runner

The built-in test runner is intentionally minimalistic for now. Let us know what features you would like us to add!

Example test file (e.g. test/smoke.test.js):

import { test } from "zinnia:test";

test("a sync test", () => {
  // run your test
  // throw an error when an assertion fails
});

test("a test can be async too", async () => {
  // run some async code
  // throw an error when an assertion fails
});

Notes:

  • Calling test() DOES NOT run the test immediately. It adds the test to the queue.
  • Therefore, you should never await the value returned by test() .
  • The tests are executed sequentially in the order in which they were registered via test() calls.

You can run the tests using zinnia run:

❯ zinnia run test/smoke.test.js

To run a test suite consisting of multiple test files, create a top-level test suite file and import individual test files.

For example, you can create test-all.js in your project root:

import "./test/smoke.test.js";
import "./test/user.test.js";
// and so on

Assertions

You can use most assertion libraries that are compatible with browsers and Deno, for example Chai.

Zinnia provides a built-in assertion library based on Deno's std/assert.

Example usage:

import { assertEquals } from "zinnia:assert";
assertEquals(true, false);
// ^ throws an error

You can find the API documentation at deno.land website: https://deno.land/std@0.214.0/assert