This repository contains workers for the ACT platform.
The source code the workers are available on github.
To use the workers, install from PyPi:
sudo pip3 install act-workers
This will install scripts for all workers:
- act-argus-case
- act-attack
- act-country-regions
- act-misp-feeds
- act-mnemonic-pdns
- act-origin
- act-scio
- act-search-graph
- act-shadowserver-asn
- act-uploader
- act-url-shorter-unpack
- act-veris
- act-vt
All workers support the optional arguments --origin-name and --origin-id. If they are not specified, facts will be added with the origin of the user performing the upload to the platform.
For managing origins, use the tool act-origin
which can list, add and delete origins.
$ act-origin --act-baseurl <BASEURL> --user-id <USERID> --add
Origin name: myorigin
Origin description: My Test Origin
Origin trust (float 0.0-1.0. Default=0.8):
Origin organization (UUID):
[2019-11-11 10:46:22] app=origin-client level=INFO msg=Created origin: myorigin
Origin added:
Origin(name='myorigin', id='e5a9792e-78c7-4190-9275-27616be47ca8', organization=Organization(), description='My Test Origin', trust=0.8)
$ act-origin --act-baseurl <BASEURL> --user-id <USERID> --list
Origin(name='Test origin', id='5da8b157-5129-4f2f-9b90-6d624d62eebe', organization=Organization(), description='My test origin', trust=0.5)
$ act-origin --act-baseurl <BASEURL> --user-id <USERID> --delete --origin-id e5a9792e-78c7-4190-9275-27616be47ca8
Origin deleted: e5a9792e-78c7-4190-9275-27616be47ca8
To print facts to stdout:
$ act-country-regions
{"type": "memberOf", "value": "", "accessMode": "Public", "sourceObject": {"type": "country", "value": "Afghanistan"}, "destinationObject": {"type": "subRegion", "value": "Southern Asia"}, "bidirectionalBinding": false}
{"type": "memberOf", "value": "", "accessMode": "Public", "sourceObject": {"type": "subRegion", "value": "Southern Asia"}, "destinationObject": {"type": "region", "value": "Asia"}, "bidirectionalBinding": false}
(...)
Or print facts as text representation:
$ act-country-regions --output-format str
(country/Afghanistan) -[memberOf]-> (subRegion/Southern Asia)
(subRegion/Southern Asia) -[memberOf]-> (region/Asia)
(...)
To add facts directly to the platform, include the act-baseurl and user-id options:
$ act-country-regions --act-baseurl http://localhost:8888 --user-id 1
All workers support options specified as command line arguments, environment variables and in a configuration file.
A utility to show and start with a default ini file is also included:
act-worker-config --help
usage: ACT worker config [-h] {show,user,system}
positional arguments:
{show,user,system}
optional arguments:
-h, --help show this help message and exit
show - Print default config
user - Copy default config to /home/fredrikb/.config/actworkers/actworkers.ini
system - Copy default config to /etc/actworkers.ini
You can see the default options in act/workers/etc/actworkers.ini.
The configuration presedence are (from lowest to highest):
- Defaults (shown in --help for each worker)
- INI file
- Environment variable
- Command line argument
Arguments are parsed in two phases. First, it will look for the argument --config argument which can be used to specify an alternative location for the ini file. If not --config argument is given it will look for an ini file in the following locations:
/etc/<CONFIG_FILE_NAME>
~/.config/<CONFIG_ID>/<CONFIG_FILE_NAME> (or directory specified by $XDG_CONFIG_HOME)
The ini file contains a "[DEFAULT]" section that will be used for all workers. In addition there are separate sections for each worker which you can use to configure worker-specific options, and override default options.
The configuration step will also look for environment variables in uppercase and with "-" replaced with "_". For the example for the option "cert-file" it will look for the enviornment variable "$CERT_FILE".
All workers requires python version >= 3.5 and the act-api library:
In addition some of the libraries might have additional requirements. See requirements.txt for a full list of all requirements.
Workers will honor the proxy-string
option on the command line when connecting to external APIs. However, if you need to
use the proxy to connect to the ACT platform (--act-baseurl), you will need to add the "--proxy-platform" switch:
echo -n www.mnemonic.no | act-vt --proxy-string <PROXY> --user-id <USER-ID> --act-baseurl <ACT-HOST> --proxy-platform
Use pip to install in local development mode. act-workers (and act-api) uses namespacing, so it is not compatible with using setup.py install
or setup.py develop
.
In repository, run:
pip3 install --user -e .
A worker to run graph queries is also included. A sample search config is inscluded in etc/searc_jobs.ini
:
act-search-graph etc/search_jobs.ini