Courier

Courier provides an in-memory pub/sub service with an HTTP, JSON interface. There are three primary objects that apps using Courier interact with messages, topics, and subscriptions. The basic flow is that apps publish messages to a given topic while subscribers read messages from the topic to which they are subscribed.

I am a full-stack software engineer whose language of choice is Rust. I am interested in pursuing new opportunities. See my resume and please contact me with potential openings.

Install

Grab the latest release. The x86_64-unknown-linux-musl is 100% statically linked and should run on any x86, unix-like system. Currently, Courier is not built for other architectures.

Setup

Run courier -h to see commands and options. If you execute courier run, the services will be bound to host 0.0.0.0 on port 3140. Run courier ui to open up your default web browser to the management page or navigate your browser to http://0.0.0.0:3140/ui

You can interact with Courier through the web interface or programmatically through the HTTP, JSON API. For examples see the C++, Go, Python, and Rust clients.

HTTP JSON API

Table of Contents

• Topic End Points
  • Create
  • Update
  • Delete
  • Get
  • List
  • Subscriptions
  • Publish
• Subscription End Points
  • Create
  • Update
  • Delete
  • Get
  • List
  • Pull
  • Ack

All messages require the following HTTP headers to be set:

Headers	Value
Content-Type	application/json

Types

Topic

{
  "name": "string", // The name of the topic
  "message_ttl": "i64", // The time to live (ttl) applied to all messages, use 0 for no ttl (seconds)
  "ttl": "i64", // The time to live (ttl) of the topic, use 0 for no ttl (seconds)
  "created": "string", // When the topic was created as an ISO 8601 datetime string (UTC)
  "updated": "string" // // When the topic was last updated as an ISO 8601 datetime string (UTC)
}

TopicList

{
  "topics": "Topic[]"
}

Subscription

{
  "name": "string", // The name of the subscriptions
  "topic": "string", // The name of the topic to subscribe to
  "ack_deadline": "i64", // The amount of time given to ack a message before it is resent (seconds)
  "ttl": "i64", // The time to live (ttl) of the subscription, use 0 for no ttl (seconds)
  "created": "string", // When the subscription was created as an ISO 8601 datetime string (UTC)
  "updated": "string" // // When the subscription was last updated as an ISO 8601 datetime string (UTC)
}

SubscriptionList

{
  "subscriptions": "Subscription[]"
}

SubscriptionNameList

{
  "subscription_names": "string[]"
}

RawMessage

{
  "data": "string" // The messages contents as a string blob
}

Message

{
  "id": "string", // The unique id of the message
  "time": "string", // When the messages was published as an ISO 8601 datetime string (UTC)
  "tries": "u32", // The number of times the message has been pulled
  "data": "string" // The messages contents as a string blob
}

MessageList

{
  "messages": "Message[]"
}

MessageIdList

{
  "message_ids": "string[]"
}

Topic End Points

Create - (PUT) /api/v1/topics/<topic>

Create a new topic.

Request

{
  "message_ttl": "u32",
  "ttl": "u32"
}

Parameter	Description	Units	Format	Required
topic	The unique name of the topic, a random name will be generated if empty	n/a	path	false
message_ttl	The time to live (ttl) applied to all messages, use 0 for no ttl	seconds	body	false
ttl	The time to live (ttl) of the topic, use 0 for no ttl	seconds	body	false

Response

Status Code	Response Body	Description
201 (Created)	Topic	Successfully created a new topic
409 (Conflict)	<empty>	Could not create a topic because a topic with the specified name already exists

Update - (PATCH) /api/v1/topics/<topic>

Update a topic. Updates the topic's updated field regardless of if a value is actually updated.

Request

{
  "message_ttl": "u32",
  "ttl": "u32"
}

Parameter	Description	Units	Format	Required
topic	The name of the topic	n/a	path	true
message_ttl	The time to live (ttl) applied to all messages, use 0 for no ttl	seconds	body	false
ttl	The time to live (ttl) of the topic, use 0 for no ttl	seconds	body	false

Response

Status Code	Response Body	Description
200 (Ok)	Topic	Successfully updated the topic
404 (Not Found)	<empty>	A topic with the specified name could not be found

Delete - (DELETE) /api/v1/topics/<topic>

Delete a topic. This will also delete all the subscriptions subscribed to this topic.

Request

Parameter	Description	Units	Format	Required
topic	The name of the topic	n/a	path	true

Response

Status Code	Response Body	Description
200 (Ok)	<empty>	Successfully deleted the topic
404 (Not Found)	<empty>	A topic with the specified name could not be found

Get - (GET) /api/v1/topics/<topic>

Get a topic.

Request

Parameter	Description	Units	Format	Required
topic	The name of the topic	n/a	path	true

Response

Status Code	Response Body	Description
200 (Ok)	Topic	Successfully retrieved the topic
404 (Not Found)	<empty>	A topic with the specified name could not be found

List - (GET) /api/v1/topics

List all of the topics.

Request

Parameter	Description	Units	Format	Required

Response

Status Code	Response Body	Description
200 (Ok)	TopicList	Successfully retrieved the topic list

Subscriptions - (GET) /api/v1/topics/<topic>/subscriptions

List all of the subscription names which are subscribed to this topic.

Request

Parameter	Description	Units	Format	Required
topic	The name of the topic	n/a	path	true

Response

Status Code	Response Body	Description
200 (Ok)	SubscriptionNameList	Successfully retrieved the subscription name list
404 (Not Found)	<empty>	A topic with the specified name could not be found

Publish - (POST) /api/v1/topics/<topic>/publish

Add messages to a topic. Updates the topics updated fields

{
  "raw_messages": "RawMessage[]"
}

Request

Parameter	Description	Units	Format	Required
topic	The name of the topic	n/a	path	true
messages	The list of raw messages	n/a	body	true

Response

Status Code	Response Body	Description
200 (Ok)	MessageIdList	Successfully published the messages
404 (Not Found)	<empty>	A topic with the specified name could not be found

Subscription End Points

Create - (PUT) /api/v1/subscriptions/<subscription>

Create a new subscription.

Request

{
  "topic": "string",
  "ack_deadline": "u32",
  "ttl": "u32",
  "historical": "bool"
}

Parameter	Description	Units	Format	Required
subscription	The unique name of the subscription, a random name will be generated if empty		path	false
topic	The name of the topic to subscribe		body	true
ack_deadline	The amount of time given to ack a message before it is resent	seconds	body	false
ttl	The time to live (ttl) of the subscription, use 0 for no ttl	seconds	body	false
historical	Should this subscription start pulling from the first message that is part of the subscribed topic, otherwise it will only pull messages added after the subscription is created		body	false

Response

Status Code	Response Body	Description
201 (Created)	Subscription	Successfully created a new subscription
409 (Conflict)	<empty>	Could not create a subscription because a subscription with the specified name already exists

Update - (PATCH) /api/v1/subscriptions/<subscription>

Update a subscription. Update the subscriptions updated field regardless of if a value is actually updated.

Request

{
  "ack_deadline": "u32",
  "ttl": "u32"
}

Parameter	Description	Units	Format	Required
subscription	The name of the subscription	n/a	path	true
ack_deadline	The amount of time given to ack a message before it is resent	seconds	body	false
ttl	The time to live (ttl) of the subscription, use 0 for no ttl	seconds	body	false

Response

Status Code	Response Body	Description
200 (Ok)	Subscription	Successfully updated the subscription
404 (Not Found)	<empty>	A subscription with the specified name could not be found

Delete - (DELETE) /api/v1/subscriptions/<subscription>

Delete a subscription.

Request

Parameter	Description	Units	Format	Required
subscription	The name of the subscription	n/a	path	true

Response

Status Code	Response Body	Description
200 (Ok)	<empty>	Successfully deleted the subscription
404 (Not Found)	<empty>	A subscription with the specified name could not be found

Get - (GET) /api/v1/subscriptions/<subscription>

Get a subscription.

Request

Parameter	Description	Units	Format	Required
subscription	The name of the subscription	n/a	path	true

Response

Status Code	Response Body	Description
200 (Ok)	Subscription	Successfully retrieved the subscription
404 (Not Found)	<empty>	A subscription with the specified name could not be found

List - (GET) /api/v1/subscriptions

List all of the subscriptions.

Request

Parameter	Description	Units	Format	Required

Response

Status Code	Response Body	Description
200 (Ok)	SubscriptionList	Successfully retrieved the subscription list

Pull - (POST) /api/v1/subscriptions/<subscription>/pull

Pull messages from a subscription. Updates the subscriptions updated field.

{
  "max_messages": "u32"
}

Request

Parameter	Description	Units	Format	Required
subscription	The name of the subscription	n/a	path	true
max_messages	The max number of messages to retrieve	n/a	body	false

Response

Status Code	Response Body	Description
200 (Ok)	MessageList	Successfully retrieved the messages
404 (Not Found)	<empty>	A subscription with the specified name could not be found

Ack - (POST) /api/v1/subscriptions/<subscription>/ack

Acknowledged that messages have been processed. Updates the subscriptions updated field. Returns only the ids which where successfully acked.

{
  "message_ids": "string[]"
}

Request

Parameter	Description	Units	Format	Required
subscription	The name of the subscription	n/a	path	true
message_ids	The ids of the messaged to acknowledge	n/a	body	true

Response

Status Code	Response Body	Description
200 (Ok)	MessageIdList	Successfully acknowledged the messages
404 (Not Found)	<empty>	A subscription with the specified name could not be found

Develop

This project makes heavy use of the rust ecosystem. It is highly recommended to use rustup and cargo when working on Courier.

Courier also depends on:

• rustfmt - for code formatting
• clippy - for linting
• tarpaulin - for code coverage
• cross - for cross compilation

These can be installed with:

> rustup component add rustfmt-preview
> cargo +nightly install clippy
> cargo install cargo-tarpaulin
> cargo install cross

Run the application

> cargo run

Run the test suite

> cargo test

Check test coverage

> cargo tarpaulin --ignore-tests --line --no-count

Check the code formating

> cargo fmt --all -- --check

Lint with clippy

> cargo +nightly clippy --lib --bins

Perform a cross release build with the musl target

> cross build --release --target=x86_64-unknown-linux-musl

Web Development

Courier uses yarn for web development, but the commands should work equally well with npm.

Install dependencies

> yarn install

Start the development server

> yarn start

Create a production build

> yarn build

Clean

> yarn clean

Example Commands

Create a topic and subscription. Publish a message to the topic and then pull and ack the message.

> curl -X PUT -H "Content-Type: application/json" -d '{"message_ttl": 600}' http://localhost:3140/api/v1/topics/topic0 && echo
> curl -X PUT -H "Content-Type: application/json" -d '{"topic": "topic0", "ack_deadline": 60}' http://localhost:3140/api/v1/subscriptions/sub0 && echo
> curl -X POST -H "content-Type: application/json" -d '{"messages": [{"data": "testing 123"}]}' http://localhost:3140/api/v1/topics/topic0/publish && echo
> curl -X POST -H "Content-Type: application/json" -d '{}' http://localhost:3140/api/v1/subscriptions/sub0/pull && echo
> curl -X POST -H "Content-Type: application/json" -d '{"message_ids": ["<some id>"]}' http://localhost:3140/api/v1/subscriptions/sub0/ack && echo