passive-data-kit-client

This library is still under construction. Do not use in stable contexts as the implementation is likely to change.

This library implements a Python client library for accessing a Passive Data Kit (PDK) server and querying data into a local context with all of the relevant logging and auditing in place server-side. It uses Django queries as an inspiration and functions very similarly.

Usage

import json
from pdk_client import PDKClient

# Create a client object.
client = PDKClient(site_url=SITE_URL, token=TOKEN)

# Create a data point query object
query = client.query_data_points(page_size=PAGE_SIZE)

# Return the total number of data points on the server.
query.count()

# Create a new query object that constrains it to a specific data source.
new_query = query.filter(source='source-id')
new_query.count()

# Get the first observed data point for a specific generator.
new_query = query.filter(source='source-id', generator_identifier='pdk-device-battery').order_by('created')
first_battery_point = new_query.first()

# Get the most recently observed point, instead.
last_battery_point = new_query.last()

# Iterate over all matching items in query.
for point in new_query:
    print(json.dumps(point, indent=2))

# Get the data points for a specific generator, excluding specific source.
exclude_query = query.filter(generator_identifier='pdk-device-battery').exclude(source='source-id').order_by('created')

# Create a data source query object
query = client.query_data_sources(page_size=PAGE_SIZE)

# Return the total number of data sources on the server.
query.count()

# Filter sources
filtered_query = query.filter(source__startswith='1')
filtered_query.filter(source__startswith='1').count()

# Iterate over all matching items in query.
for source in filtered_query:
    print(json.dumps(source, indent=2))

Notes

Behind the scenes, this library obtains a time-limited token for querying the PDK server on client creation. Parameters:

• site_url: URL prefix of the PDK installation. (Example: https://mysite/data/)
• token: API token, visible on your user profile within the PDK dashboard.

The state of the token may be checked while in use:

from pdk_client import PDKClient

# Create a client object.
client = PDKClient(site_url=SITE_URL, token=TOKEN)

# Returns True if the token has expired and a new client should be created, False otherwise.
is_expired = client.expired()

# Returns datetime.datetime object encoding the expiration date.
when_expires = client.expires

# Returns True if the client is authorized to communicate with the server, False otherwise.
# May change when the token expires.
is_connected = client.connected()

When a query object is obtained, the page_size parameter may be passed to control the number and size of queries:

query = client.query_data_points(page_size=PAGE_SIZE)

The library uses paging internally to provide a Python list-like interface for accessing the data from the server. Since the size of the available data may exceed memory and bandwidth resources, this allows the necessary work to be decomposed into workable chunks. Internally, the library also adds an additional constraint where the data returned will be the data present on the server at the time of the PDKClient.query_data_points call, eliminating any unexpected behavior where data may be added to the server while the query is in use.

When constraining the query using filter or excludes functions, these functions are mapped onto their Django equivalents on the PDK server. Arguments on corresponding server Data Point objects are supported, as well as any special arguments supported by the remote server, such as JSONField queries on modern Postgres servers:

query.filter(properties__application__icontains='youtube').count()

Installation

The package is intended to be installed with pip:

pip install git+https://github.com/audaciouscode/PassiveDataKit-Client-Python.git

When the library reaches maturity, it will be made available on PyPi as passive-data-kit-client.

Major Outstanding Items

The following items are on the roadmap for support:

• Support for querying other Passive Data Kit types: data sources, alerts, exports.
• Support for Q-object equivalents, supporting more flexible Boolean parameters.
• Support for renewing tokens.
• Full support for slices in querys. Currently items may be accessed by index, but not range.

If you encounter any bugs or other issues, please add an issue.