This is the SDK to access Devo directly from client code in browsers. It can be used to query Devo, and to manage deferred tasks.
A modern Node.js installation (version 14 or later) is required.
These steps will allow you to use the SDK right away. First clone the repo:
git clone https://github.com/devoinc/browser-sdk
Install all dependencies:
cd browser-sdk
npm ci
Run the webpack task:
$ npm run webpack:prod
Now include the generated file dist/devo-bundle.js directly in your HTML pages.
You can also minify it or embed it in your software.
From your code invoke devo as a global:
const client = devo.client(credentials);
// do something with the client
There are several ways to use the Devo SDK in the browser. All of them require a working set of credentials. You can obtain your API key and API secret from Devo: go to the "Administration/Credentials" section, and then to "Access Keys". Alternatively you can get an API token instead. Note: API tokens are only supported on Devo 5.3 or later.
An object containing access credentials must be passed to the constructor. Example:
const devo = require('@devoinc/browser-sdk')
const client = devo.client({
url: 'https://api-us.logtrust.com/search',
apiKey: 'your-api-key',
apiSecret: 'your-api-secret',
})The credentials parameter to devo.client() will have the following attributes.
Parameter url is the Devo endpoint you want to connect to.
We currently have the following endpoints:
API key, obtained from Devo.
API secret, obtained from Devo.
An alternative to API key and secret, API tokens are a simple way of authenticating. They are also obtained from Devo.
The demo pages show the capabilities of the Devo SDK. To generate them first clone the repo:
git clone https://github.com/devoinc/browser-sdk
Install all dependencies:
cd browser-sdk
npm ci
Place your Devo credentials in the root of the project
credentials.json.
It should look like this:
{
"url": "https://api-us.logtrust.com/search",
"apiKey": "your-api-key",
"apiSecret": "your-api-secret"
}Then run the task:
$ npm run webpack:dev
This will generate the examples with the right credentials: Open this page in a browser in localhost to try out the capabilities in action.
To install the SDK use npm:
$ npm i @devoinc/browser-sdk
Place this code including your
credentials
in a file called e.g. credentials.json:
const devo = require('@devoinc/browser-sdk')
// create object with credentials
const client = devo.client(credentials)
// do something with the clientsnpm run webpack:prodAnd finally include the generated file in dist/ in your web page:
<script src="mybundle.js"></script>
API usage follows.
The client can be used to send queries to Devo, either in regular or streaming mode.
Use the function client.query(options, callback) to send a query.
It will accept an object with options specifying how the query is performed
(see below),
and will return a promise that will resolve to the query results.
Example:
const devo = require('@devoinc/browser-sdk')
const client = devo.client(credentials)
client.query({
query: 'from demo.ecommerce.data select eventdate,protocol,statusCode,method',
dateFrom: new Date(Date.now() - 60 * 1000),
dateTo: new Date()
}).then(result => console.log('Received %s', result))
.catch(error => console.error('Query failed: %s', error))The result object will have the following attributes:
msg: an optional message.status: a status code, 0 means success.object: an array with one element per data row.
Example result object:
{
"msg": "",
"status": 0,
"object": [{
"eventdate": 1519339261018,
"protocol": "HTTP 1.1",
"statusCode": 404,
"method": "POST"
},{
"eventdate": 1519339261089,
"protocol": "HTTP 1.1",
"statusCode": 200,
"method": "GET"
},{
"eventdate": 1519339261161,
"protocol": "HTTP 1.1",
"statusCode": 200,
"method": "GET"
}]
}Instead of receiving all results in the promise, they can be streamed back to the client.
- Use the function
client.streamFetch(options, callbacks)to stream back query results in array filled with multiple data objects, parsed in chunks.
It will accept an options parameter (see below) and a callbacks parameter that will contain four callbacks:
data: callback to receive data rows.meta: optional callback to receive a custom object with field definitions.error: optional callback to receive any errors.done: optional callback to invoke once the streaming has finished.progress: optional callback to invoke every time a progress event is sent.abort: optional callback to invoke when query is aborted from client.
The first data callback will recieve an object with the data from a row with several fields.
Example data:
{
"eventdate": 1519339261018,
"protocol": "HTTP 1.1",
"statusCode": 404,
"method": "POST"
}The meta callback will receive an object with field definitions.
Each field will have a type and an index specifying its position.
Example:
{
"eventdate":{"type":"timestamp","index":0},
"protocol":{"type":"str","index":1},
"statusCode":{"type":"int8","index":2},
"method":{"type":"str","index":3}
}where type can be one of the following:
timestamp: the number of milliseconds since 1970-01-01T00:00:00Z.str: a string.int8: a byte-sized integer.
Full example:
const devo = require('@devoinc/browser-sdk')
const client = devo.client(credentials)
client.streamFetch({
query: 'from demo.ecommerce.data select eventdate,protocol,statusCode,method',
dateFrom: new Date(),
dateTo: -1
}, {
meta: data => console.log('Received metadata: %s', data),
data: data => console.log('Received data: %s', data),
error: error => console.error('Query failed: %s', error),
done: () => console.log('Finished receiving query results'),
progress: () => console.log('Progress event is sent'),
abort: () => console.log('Abort action made')
})Streaming is mandatory when the ending date is -1,
which means that new results will be sent in near real time to the client.
The utility function client.download(options)
directly sends the requested results as a file to the browser.
It also returns a promise to catch for errors.
Example:
const devo = require('@devoinc/browser-sdk')
const client = devo.client(credentials)
client.download({
query: 'from demo.ecommerce.data select eventdate,protocol,statusCode,method',
dateFrom: new Date(Date.now() - 60 * 1000),
dateTo: new Date(),
format: 'csv',
}).catch(error => console.error('Download failed: %s', error))With client.table(tableName) method you be able to obtain a table structure (field name and its type).
It also returns a promise to catch for errors.
Example:
const devo = require('@devoinc/browser-sdk')
const client = devo.client(credentials)
client.table('demo.ecommerce.data')
.then(result => console.log(result.object))
.catch(error => console.error('Request failed: %s', error))All query functions have the following attributes in the options parameter.
String with query to send, in linq format. Example:
from demo.ecommerce.data select eventdate,protocol,statusCode,method
Alternatively identifies a particular query in the server.
Response format, can be one of:
json: JSON, one object per row of data.json/compact: a JSON with a header row and an array per row of data.json/simple: one JSON per row of data separated by newlines. Spec.json/simple/compact: one JSON for the header and one with an array per row, same spec.msgpack: MessagePack.xls: Microsoft Excel format.csv: comma-separated values.tsv: tab-separated values.
Default is json.
When streaming the format is always json/compact.
Starting date for the query. Can be either a JavaScript Date or a string in ISO-8601 format. If not present will use the current date.
Ending date for the query.
Can be either a JavaScript
Date
or a string in ISO-8601 format, default is current date.
If not present, or set to the special value -1,
it will start a never-ending query that returns results
as they are generated (streaming only).
Number of registers to skip from the beginning of the query. Optional parameter.
Max number of registers to return. Optional parameter.
Optional destination for the data:
an object with a given string type
and a params array.
The special donothing type is only used for tests.
Example:
client.query({
query: 'from demo.ecommerce.data select eventdate,protocol,statusCode,method',
destination: {
type: 'donothing',
params: {
param0: '1',
},
},
})
When destination is present
the server will not return the results in the response.
Instead a task will be created,
the results will be sent to the desired destination
and the response will contain the task ID.
The client can then be used to check the state of the task.
For instance, you may want to stream the results to the Amazon object storage, AWS S3. In this case you will send a query with a destination for S3 and the required parameters to authenticate. The server will reply with the task ID, and you can query with this ID for completion. See below for details on task management.
(only for streaming)
Optional parameter to avoid that events will mapped with its columns and send to onData callback the raw event. Default value: true
Example:
mapMetadata: true
meta:
{ colA: { index: 0, type: 'int8' }}
rawData:
[[0], [1]]
onData(ev):
ev -> { colA: 0 }
ev -> { colA: 1 }
ev -> { colA: 2}
mapMetadata: false
meta:
{ colA: { index: 0, type: 'int8' }}
rawData:
[[0], [1]]
onData(ev):
ev -> [0]
ev -> [1]
ev -> [2]
The client is also used for task management.
Tasks are created by sending queries with a destination.
Get a list of outstanding tasks from the server. This includes stopped and removed tasks.
The function returns a promise that will resolve to a list of tasks.
Get a list of outstanding tasks of the given type. Returns a promise that will resolve to a list of tasks.
Get info for the given task.
- Parameter
taskIdidentifies the task, and is returned by the query functions when adestinationis present. - Returns a promise for the task info including its status.
Starts the task, if stopped. If the task was already running will have no effect. Removed tasks cannot be started.
- Parameter
taskIdidentifies the task, and is returned by the query functions when adestinationis present. - Returns a promise for the status of the task.
Stops the task, if running. If the task was already stopped will have no effect. Removed tasks cannot be stopped.
- Parameter
taskIdidentifies the task, and is returned by the query functions when adestinationis present. - Returns a promise for the status of the task.
Delete the given task.
- Parameter
taskIdidentifies the task, and is returned by the query functions when adestinationis present. - Returns a promise for the status of the task.
Each task has a status at any given point,
which determines what it is doing.
A task starts its life as created,
and is changed to running when it starts collecting data.
It is then changed to stopped when stopped,
and can be changed back to running if restarted.
If it is removed then it changes to removed.
The SDK requires ES2015. It can be used with Babel and other transpilers.
Clone the repo:
git clone https://github.com/devoinc/browser-sdk
Install all dependencies:
cd browser-sdk
npm ci
Make sure that everything runs fine:
npm t
To run manual tests against our integration server run:
npm run manualtest
Note: you will need to place your credentials in the file
/credentials.json:
just a JSON file that contains the same attributes as the
initialization parameter.
If needed you can also use environment variables with the file $HOME/.devo.json. In the case you have the file credentials.json in the root of your project, this will replace the enviroment variables.
See the Node.js project for more details.
To publish to npmjs.com, simply create a tag
git tag vX.X.X
git push --tagsand from the GH interface, create a release associated to that tag.
And start playing! Pull requests are welcome ☺