-
Notifications
You must be signed in to change notification settings - Fork 7
Selecting the evaluation services to apply
- Getting started lets you find the right series of documentation pages based on what you're looking to get out of DataFAQs.
- Selecting the datasets to analyze, the other half of configuring an evaluation epoch.
The evaluation services to apply are selected using the following Core Services. Details for each are provided at DataFAQs Core Services.
- select-datasets/identity
- select-datasets/via-sparql-query
- select-datasets/by-ckan-group
- select-datasets/by-ckan-installation
- select-datasets/by-ckan-tag
This page describes how to write an epoch configuration, which specifies evaluation services to apply and [datasets to analyze](Selecting the datasets to analyze) during an evaluation epoch. Here, we focus on selecting the evaluation services.
An epoch configuration prescribes the process that should be used to select evaluation services and datasets for an evaluation epoch. The configuration is described in RDF using the PROV-O vocabulary, among others. All FAqT Service Selectors follow the same simple pattern. First, they accept an RDF graph via HTTP post. Next, they return an RDF description of the FAqT Services to use in the evaluation epoch.
config
graph goes in
selector invocation
graph comes out
The structure of the RDF that goes to each FAqT Service Selector can vary depending on what the selector service does, but all of the services' output is structured in the exact same way. For example, the RDF POSTed to select-datasets/identity requires descriptions of the FAqT Service to select, while the RDF POSTed to select-datasets/via-sparql-query requires a description of a SPARQL query to execute and the endpoint to query. Meanwhile, both services return the same RDF structure (using the dcterms:hasPart property to list the FAqT services).
Using this simple pattern allows the core DataFAQs engine to generically perform the evaluation epoch. Any newly implemented FAqT Service Selector that follows this pattern can be used without changing the core engine -- simply mention it in the epoch configuration and it will be used!
To illustrate the common output, we'll show two configurations that use two different FAqT Service Selectors to obtain a list of FAqT Services. After determining the evaluation services, the datasets to analyze will also be determined, but that is described [elsewhere](Selecting the datasets to analyze).
- There are at least a dozen real examples of epoch configurations at doc/examples/*/faqt-brick/epoch.ttl.
- Selecting the datasets to analyze, the other half of configuring an evaluation epoch.
- DataFAQs Core Services provides details for each core services' implementation.