Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Initial setup of AblyD #1

Open
wants to merge 19 commits into
base: basic-branch
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 14 commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
20 changes: 20 additions & 0 deletions .gitignore
@@ -0,0 +1,20 @@
# Binaries for programs and plugins
*.exe
*.exe~
*.dll
*.so
*.dylib

# Test binary, build with `go test -c`
*.test

# Output of the go coverage tool, specifically when used with LiteIDE
*.out

# Ignore env files
.env

.Ds_Store

# NodeJS
node_modules
65 changes: 65 additions & 0 deletions README.md
@@ -0,0 +1,65 @@
# AblyD

This is a basic demo which takes [websocketd](https://github.com/joewalnes/websocketd), which makes command-line commands on a device available to other devices via WebSockets, and provides this functionality through [Ably](https://www.ably.com).

## Setup

Firstly you need to get an Ably API key. You can sign up for an account with [Ably](https://www.ably.com/) and access your API key from the [app dashboard](https://www.ably.com/accounts/any/apps/any/app_keys).

Now all you need to do is run the main go file, passing in the command line command you want to be run as a parameter! For example, to use a bash file `count.sh`, which is included in this repo's `examples` folder, just run:

```bash
~ $ ./ablyD --apikey=YOUR_API_KEY ./examples/bash/count.sh
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I recommend expecting the API key to be passed as an environment variable rather than a command line parameter, since command line parameters leak more easily.

```

The program is now running, waiting for a message in the `command` channel in Ably to be sent. The message's data field should match the structure, with the value `start` for the message's name:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I sent a message to the command channel and nothing happened, after digging a little bit it seems the channel needs to be ablyd:command.

I'd recommend giving a full example of the message that should be published, for example using cURL:

curl -X POST https://rest.ably.io/channels/ablyd:command/messages \
  -u "${API_KEY}" \
  -H 'Content-Type: application/json' \
  --data \
  '{
    "name": "start",
    "data": {
      "MessageID": "unique string value",
      "Args": [ "some", "additional", "args", "for", "the", "programs" ]
    },
    "format":"json"
  }'


```json
{
"MessageID": "unique string value",
"Args": [ "some", "additional", "args", "for", "the", "programs" ]
}
```

Once the server receives a message in the `command` channel, it will start up an instance of the program, using the `Args` you specify in the message as well. Once the program has started up, the server will send a message onto the `command` channel of structure:

```json
{
"MessageID": "unique string value matching the requesting MessageID",
"Pid": "process_id",
"Namespace": "ablyd",
"ChannelPrefix": "ablyd:server_id"
}
```

The client can identify the process which has started for them by the `MessageID`, then use the `Prefix` to connect to an input and an output channel for the process. These will be of structure `{Prefix}{Pid}:serverinput` and `{Prefix}{Pid}:serveroutput`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It would be useful if the command message included the names of the channels explicitly, for example:

{
  "MessageID": "unique string value matching the requesting MessageID",
  "Stdin": "ably:c42jl786n88h66cnsu70:5328:serverinput",
  "Stdout": "ably:c42jl786n88h66cnsu70:5328:serveroutput"
}


Subscribing to the `serveroutput` channel will allow the client to receive any stdout messages from the server. The client can also publish messages into the `serverinput` channel which will be passed into the stdin of the process.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This design is a little racy, since I only get to know the channels to subscribe to after the process has started, and so in my testing I sometimes miss the first bit of output.

It would be better if I could create the command, get back an identifier, subscribe to the channels, and then start the command, this way I'll always be subscribed before anything gets published to the output channel.

I guess this would need ablyD to generate random identifiers for commands rather than using the PID (since the PID won't be known before starting the command), but I don't think that presents a problem.


This will continue until the program naturally terminates, resulting in the process dying, or the client submits a message to the `serverinput` with data `KILL`.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should consider adding a third signal channel the client can use to send OS signals to the process rather than trying to re-purpose the stdin channel.

That being said, we should also consider just using a single channel with typed messages, rather than using different channels for stdin / stdout / signals. For example, there could be a single "data" channel with messages like {"type":"STDIN","data":"..."} and {"type":"SIGNAL","data":"15"}. This is pretty much what the SSH protocol does (see https://datatracker.ietf.org/doc/html/rfc4254#section-5), but is probably out of scope for what you're trying to achieve here 🙂


## Checking Current State of Processes and an AblyD Instance

AblyD makes use of Ably Presence to identify what AblyD instances exist, and what processes are running on each instance. If you check the presence set of the `command` channel, you'll see each currently active process present with the following attached data:

```json
{
"ServerID": "my-server-id",
"Namespace": "ablyd",
"MaxProcesses": 20,
"Processes": {
"3490348": "Running",
"Another PID": "Running"
}
}
```

This indicates the most processes you can have, and the currently active processes. The server will also enter the presence set of any `serverinput` channels to indicate it is actively listening to them.

## Interacting with an AblyD Instance

To simplify the process of interacting with an AblyD instance, there is currently a client available for NodeJS on [GitHub](https://github.com/ably-labs/Ablyd-client) and on [npm](https://www.npmjs.com/package/ablyd-client).

## Testing

You can use the `/examples/bash/count.html` file to easily test this out. Replace the `INSERT_API_KEY_HERE` with the same API key used in your main Go function, and load the webpage. If you press the publish button on that page, you should see counting coming from the server!
Binary file added ablyD
Binary file not shown.