-
Notifications
You must be signed in to change notification settings - Fork 7
df find
Tim L edited this page Mar 28, 2015
·
12 revisions
- This utility helps working with FAqT Bricks.
This pages describes how to use df-find.sh to list different portions of a FAqT Brick epoch directory structure, to determine the status of completion of an epoch, or to see how many descriptions or reports were invalid. The directory convention that df-find works against is illustrated in doc/faqt-brick-directory-convention.graffle.pdf.
df-find.sh --help
df-find.sh in <epoch> datasets
e.g. __PIVOT_epoch/2014-04-07/__PIVOT_dataset/datahub.io/dataset/aemet/dataset.ttl
df-find.sh in <epoch> invalid dataset descriptions
e.g. __PIVOT_epoch/2014-04-07/__PIVOT_dataset/datahub.io/dataset/pokepedia-fr/augmentation-1
df-find.sh in <epoch> dataset evaluations
e.g. __PIVOT_faqt/lodcloud.tw.rpi.edu/sadi-services/named-graphs/__PIVOT_dataset/datahub.io/dataset/aemet/__PIVOT_epoch/2014-04-07
df-find.sh in <epoch> dataset evaluation requests
e.g. __PIVOT_faqt/lodcloud.tw.rpi.edu/sadi-services/named-graphs/__PIVOT_dataset/datahub.io/dataset/aemet/__PIVOT_epoch/2014-04-07/request.sh
df-find.sh in <epoch> datasets evaluated
e.g. __PIVOT_epoch/2014-04-07/__PIVOT_dataset/datahub.io/dataset/aemet/dataset.ttl
df-find.sh in <epoch> incomplete evaluations
e.g. __PIVOT_faqt/lodcloud.tw.rpi.edu/sadi-services/named-graphs/__PIVOT_dataset/datahub.io/dataset/eea/__PIVOT_epoch/2014-04-07
df-find.sh in <epoch> valid evaluations
e.g. __PIVOT_faqt/lodcloud.tw.rpi.edu/sadi-services/named-graphs/__PIVOT_dataset/datahub.io/dataset/eagle-i-utep/__PIVOT_epoch/2014-04-07/evaluation.rdf
df-find.sh in <epoch> invalid evaluations [and CLEAR]
e.g. __PIVOT_faqt/lodcloud.tw.rpi.edu/sadi-services/named-graphs/__PIVOT_dataset/datahub.io/dataset/radatana/__PIVOT_epoch/2014-04-07/evaluation
[and CLEAR] - remove ALL files within the evaluation directory (i.e. request.sh, evaluation*).
at lodcloud@lodcloud:~/prizms/lodcloud/data/source/datahub-io/lodcloud-group/version/faqt-brick
$ df-find.sh in __PIVOT_epoch/2014-07-05 datasets
__PIVOT_epoch/2014-07-05/__PIVOT_dataset/datahub.io/dataset/rkb-explorer-roma/dataset.ttl
__PIVOT_epoch/2014-07-05/__PIVOT_dataset/datahub.io/dataset/southampton-ecs-eprints/dataset.ttl
__PIVOT_epoch/2014-07-05/__PIVOT_dataset/datahub.io/dataset/rkb-explorer-ecs/dataset.ttl
$ df-find.sh in __PIVOT_epoch/2014-07-05 invalid dataset descriptions
__PIVOT_epoch/2014-07-05/__PIVOT_dataset/datahub.io/dataset/rkb-explorer-roma/augmentation-1
__PIVOT_epoch/2014-07-05/__PIVOT_dataset/datahub.io/dataset/linked-open-data-of-ecology/reference-1
__PIVOT_epoch/2014-07-05/__PIVOT_dataset/datahub.io/dataset/pokepedia-fr/augmentation-1
And, see that one of them is invalid:
$ rapper -g -c __PIVOT_epoch/2014-07-05/__PIVOT_dataset/datahub.io/dataset/rkb-explorer-roma/augmentation-1
rapper: Parsing URI file:///home/lodcloud/prizms/lodcloud/data/source/datahub-io/lodcloud-group/version/faqt-brick/__PIVOT_epoch/2014-07-05/__PIVOT_dataset/datahub.io/dataset/rkb-explorer-roma/augmentation-1 with parser guess
rapper: Guessed parser name 'rdfxml'
rapper: Error - - XML parser error: Input is not proper UTF-8, indicate encoding !
Bytes: 0xE0 0x20 0x64 0x65
rapper: Failed to parse file __PIVOT_epoch/2014-07-05/__PIVOT_dataset/datahub.io/dataset/rkb-explorer-roma/augmentation-1 guess content
rapper: Parsing returned 103 triples
$ df-find.sh in __PIVOT_epoch/2014-07-05 dataset evaluations
__PIVOT_faqt/aquarius.tw.rpi.edu/projects/datafaqstest/sadi-services/named-graphs/__PIVOT_dataset/datahub.io/dataset/rkb-explorer-roma/__PIVOT_epoch/2014-07-05
__PIVOT_faqt/aquarius.tw.rpi.edu/projects/datafaqstest/sadi-services/named-graphs/__PIVOT_dataset/datahub.io/dataset/southampton-ecs-eprints/__PIVOT_epoch/2014-07-05
__PIVOT_faqt/aquarius.tw.rpi.edu/projects/datafaqstest/sadi-services/named-graphs/__PIVOT_dataset/datahub.io/dataset/rkb-explorer-ecs/__PIVOT_epoch/2014-07-05
and see what is in one of the directories:
$ ls __PIVOT_faqt/aquarius.tw.rpi.edu/projects/datafaqstest/sadi-services/named-graph
s/__PIVOT_dataset/datahub.io/dataset/rkb-explorer-roma/__PIVOT_epoch/2014-07-05
evaluation.meta.rdf evaluation.rdf evaluation.rdf.sd_name request.sh