Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat!: add archive representation to DSP-API (DEV-17) #1926

Merged
merged 61 commits into from Nov 25, 2021
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
61 commits
Select commit Hold shift + click to select a range
5c1714e
feat: add bundle representation to knora-base ontology and constants
BalduinLandolt Oct 28, 2021
ec25aa1
fix: overlook in knora-base ontology
BalduinLandolt Oct 28, 2021
46d722e
test: try to break stuff that should break
BalduinLandolt Nov 1, 2021
38a88e4
Merge branch 'main' into wip/DEV-17-bundle-representation
BalduinLandolt Nov 1, 2021
965240a
Merge branch 'main' into wip/DEV-17-bundle-representation
BalduinLandolt Nov 2, 2021
8233897
feat: un-break tests as good as possible
BalduinLandolt Nov 2, 2021
f4f806e
refactor: move some json-ld generation to model
BalduinLandolt Nov 2, 2021
bfbc09f
Merge branch 'main' into wip/DEV-17-bundle-representation
BalduinLandolt Nov 4, 2021
41ccd4f
refactor: tidy up a bit
BalduinLandolt Nov 4, 2021
bcdee11
refactor: add ChangeFileRequest to FileModels
BalduinLandolt Nov 4, 2021
3115db8
chore: update test data according to new knora-base ontology
BalduinLandolt Nov 4, 2021
b52544e
refactor: get rid of some code smells
BalduinLandolt Nov 4, 2021
9ce8f57
refactor: remove code smell
BalduinLandolt Nov 4, 2021
e096a3e
refactor: move more json-ld generation into FileModels
BalduinLandolt Nov 4, 2021
2066c0e
temporarily disable all sipi integration tests
BalduinLandolt Nov 8, 2021
3afbb8c
fix: error in books ontology
BalduinLandolt Nov 8, 2021
1d6b735
Merge branch 'main' into wip/DEV-17-bundle-representation
BalduinLandolt Nov 8, 2021
b32e390
bring back sipi inntegration test
BalduinLandolt Nov 8, 2021
647746c
ignore failing tests
BalduinLandolt Nov 8, 2021
73706f7
test: move more json-ld generation to FileModels
BalduinLandolt Nov 8, 2021
9b767c7
refactor: further simplify generating json-ld through FileModels
BalduinLandolt Nov 9, 2021
4ed755f
refactor: improve file model
BalduinLandolt Nov 9, 2021
97365ea
feat: add bundle to lua scripts
BalduinLandolt Nov 9, 2021
3ae2cbb
test: temporarily ignore some tests
BalduinLandolt Nov 9, 2021
31323ec
Merge branch 'main' into wip/DEV-17-bundle-representation
BalduinLandolt Nov 11, 2021
98453ad
feat: forbid bundles in document file type
BalduinLandolt Nov 11, 2021
0454cd7
test: add test for uploading a BundleRepresentation
BalduinLandolt Nov 11, 2021
560ed2f
test: add test for changing BundleRepresentation
BalduinLandolt Nov 11, 2021
5578693
refactor: improve file models
BalduinLandolt Nov 11, 2021
47966af
docs: update documentation
BalduinLandolt Nov 11, 2021
cf04f0e
docs: update documentation
BalduinLandolt Nov 11, 2021
76e7370
docs: add documentation and auto-reformat some markdown
BalduinLandolt Nov 11, 2021
0b8e793
test: add unit test for creating a DocumentRepresentation
BalduinLandolt Nov 11, 2021
04f869d
refactor: start using models for unit tests
BalduinLandolt Nov 15, 2021
84a79e2
refactor: generalize file message model
BalduinLandolt Nov 15, 2021
cbc4c28
refactor: further generalize file message model
BalduinLandolt Nov 15, 2021
6aa7b0b
refactor: expand file message models
BalduinLandolt Nov 15, 2021
1c1faf9
test: add unit test for creating a bundle representation
BalduinLandolt Nov 15, 2021
2cf8db0
refactor: tidy up books ontology
BalduinLandolt Nov 15, 2021
97f3724
Merge branch 'main' into wip/DEV-17-bundle-representation
BalduinLandolt Nov 15, 2021
5e5f2fa
docs: update documentation
BalduinLandolt Nov 15, 2021
81f5ccc
refactor: move some string literals into constants
BalduinLandolt Nov 16, 2021
191ee59
docs: add scaladoc comments to FileModels
BalduinLandolt Nov 16, 2021
ee18893
refactor: final clean up
BalduinLandolt Nov 16, 2021
49a9c53
docs: update docs according to review
BalduinLandolt Nov 18, 2021
6183213
fix: remove wrong properties from ArchiveRepresentation
BalduinLandolt Nov 18, 2021
45c1ecc
refactor: rename bundle to archive
BalduinLandolt Nov 18, 2021
43b6f9c
test: update test data
BalduinLandolt Nov 18, 2021
221bdb9
refactor: change file models
BalduinLandolt Nov 18, 2021
ab443ea
refactor: change file models
BalduinLandolt Nov 18, 2021
2d32a33
refactor: rework file models
BalduinLandolt Nov 18, 2021
d51203d
Merge branch 'main' into wip/DEV-17-bundle-representation
BalduinLandolt Nov 18, 2021
e8de2c9
Merge branch 'main' into wip/DEV-17-bundle-representation
BalduinLandolt Nov 22, 2021
40fa156
fix: oversight in json-ld generation in file models
BalduinLandolt Nov 22, 2021
2fa5c6a
fix: use wrong IRI in FileModels toMessage function
BalduinLandolt Nov 22, 2021
a78634b
fix: expect correct permission for archive representation
BalduinLandolt Nov 22, 2021
6bc916f
Merge branch 'main' into wip/DEV-17-bundle-representation
BalduinLandolt Nov 22, 2021
02406c5
test: add test for file models
BalduinLandolt Nov 23, 2021
f749d05
test: add more file model tests
BalduinLandolt Nov 23, 2021
24a1075
test: improve file model tests
BalduinLandolt Nov 23, 2021
b2d785a
Merge branch 'main' into wip/DEV-17-bundle-representation
BalduinLandolt Nov 25, 2021
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
1,372 changes: 517 additions & 855 deletions docs/02-knora-ontologies/knora-base.md

Large diffs are not rendered by default.

5 changes: 5 additions & 0 deletions docs/03-apis/api-v2/editing-values.md
Expand Up @@ -222,6 +222,7 @@ DSP-API v2 currently supports using Sipi to store the following types of files:
* Audio: MPEG, MP4, or Waveform audio file format (.wav, .x-wav, .vnd.wave)
* Text files: TXT, XML, or CSV
* Video files: MP4
* Archive files: ZIP, TAR, GZIP

Support for other types of files will be added in the future.

Expand Down Expand Up @@ -328,6 +329,10 @@ For a text file, use `knora-api:TextRepresentation`, which has the property
`knora-api:hasTextFileValue`, pointing to a
`knora-api:TextFileValue`.

For an archive like zip, use `knora-api:ArchiveRepresentation`, which has the property
`knora-api:hasArchiveFileValue`, pointing to a
`knora-api:ArchiveFileValue`.

## Updating a Value

To update a value, use this route:
Expand Down
85 changes: 66 additions & 19 deletions knora-ontologies/knora-base.ttl
Expand Up @@ -371,22 +371,45 @@

:hasDocumentFileValue rdf:type owl:ObjectProperty ;

rdfs:label "hat Dokument"@de ,
"has document"@en ,
"a document"@fr ,
"ha documento"@it ;
rdfs:label "hat Dokument"@de ,
"has document"@en ,
"a document"@fr ,
"ha documento"@it ;

rdfs:comment "Connects a Representation to a document"@en ;
rdfs:comment "Connects a Representation to a document"@en ;

:objectClassConstraint :DocumentFileValue ;
:objectClassConstraint :DocumentFileValue ;

:subjectClassConstraint :DocumentRepresentation ;
:subjectClassConstraint :DocumentRepresentation ;

:isEditable true ;
:isEditable true ;

salsah-gui:guiElement salsah-gui:Fileupload ;
salsah-gui:guiElement salsah-gui:Fileupload ;

rdfs:subPropertyOf :hasFileValue .
rdfs:subPropertyOf :hasFileValue .



### http://www.knora.org/ontology/knora-base#hasArchiveFileValue

:hasArchiveFileValue rdf:type owl:ObjectProperty ;

rdfs:label "hat Zip"@de ,
"has zip"@en ,
"a zip"@fr ,
"ha zip"@it ;

rdfs:comment "Connects a Representation to a zip archive"@en ;

:objectClassConstraint :ArchiveFileValue ;

:subjectClassConstraint :ArchiveRepresentation ;

:isEditable true ;

salsah-gui:guiElement salsah-gui:Fileupload ;

rdfs:subPropertyOf :hasFileValue .



Expand Down Expand Up @@ -1041,8 +1064,8 @@
# No :subjectClassConstraint, because this can be used with :Resource or :Value.

:objectClassConstraint knora-admin:User .


#################################################################
#
# Datatype properties
Expand Down Expand Up @@ -1797,15 +1820,39 @@
:DocumentRepresentation rdf:type owl:Class ;

rdfs:label "Repräsentation (Dokument)"@de ,
"Representation (Document)"@en ,
"Répresentation (Document)"@fr ,
"Rappresentazione (Documento)"@it ;
"Representation (Document)"@en ,
"Répresentation (Document)"@fr ,
"Rappresentazione (Documento)"@it ;

rdfs:subClassOf :Representation ,
[ rdf:type owl:Restriction ;
owl:onProperty :hasDocumentFileValue ;
owl:cardinality "1"^^xsd:nonNegativeInteger
] .
[ rdf:type owl:Restriction ;
owl:onProperty :hasDocumentFileValue ;
owl:cardinality "1"^^xsd:nonNegativeInteger
] .



### http://www.knora.org/ontology/knora-base#ArchiveFileValue

:ArchiveFileValue rdf:type owl:Class ;

rdfs:subClassOf :FileValue .


### http://www.knora.org/ontology/knora-base#ArchiveRepresentation

:ArchiveRepresentation rdf:type owl:Class ;

rdfs:label "Repräsentation (Zip)"@de ,
"Representation (Zip)"@en ,
"Répresentation (Zip)"@fr ,
"Rappresentazione (Zip)"@it ;

rdfs:subClassOf :Representation ,
[ rdf:type owl:Restriction ;
owl:onProperty :hasArchiveFileValue ;
owl:cardinality "1"^^xsd:nonNegativeInteger
] .



Expand Down
34 changes: 27 additions & 7 deletions sipi/scripts/file_info.lua
Expand Up @@ -11,6 +11,7 @@ IMAGE = "image"
DOCUMENT = "document"
AUDIO = "audio"
VIDEO = "video"
ARCHIVE = "archive"

-------------------------------------------------------------------------------
-- Mimetype constants
Expand Down Expand Up @@ -67,10 +68,7 @@ local text_mime_types = {

local document_mime_types = {
APPLICATION_PDF,
APPLICATION_TAR,
APPLICATION_ZIP,
APPLICATION_ISO,
APPLICATION_GZIP,
APPLICATION_DOC,
APPLICATION_DOCX,
APPLICATION_XLS,
Expand All @@ -79,6 +77,12 @@ local document_mime_types = {
APPLICATION_PPTX
}

local archive_mime_types = {
APPLICATION_TAR,
APPLICATION_ZIP,
APPLICATION_GZIP
}

local video_mime_types = {
VIDEO_MP4
}
Expand All @@ -99,10 +103,7 @@ local text_extensions = {

local document_extensions = {
"pdf",
"zip",
"tar",
"iso",
"gz",
"doc",
"docx",
"xls",
Expand All @@ -111,6 +112,12 @@ local document_extensions = {
"pptx"
}

local archive_extensions = {
"zip",
"tar",
"gz"
}

local video_extensions = {
"mp4"
}
Expand Down Expand Up @@ -162,6 +169,17 @@ function make_document_file_info(extension)
end
end

function make_archive_file_info(extension)
if not table.contains(archive_extensions, extension) then
return nil
else
return {
media_type = ARCHIVE,
extension = extension
}
end
end

-------------------------------------------------------------------------------
-- Determines the media type and file extension of a file.
-- Parameters:
Expand All @@ -174,7 +192,7 @@ end
function get_file_info(filename, mimetype)

local extension = filename:match("^.+%.([^.]+)$")

if extension == nil then
return nil
elseif table.contains(image_mime_types, mimetype) then
Expand All @@ -187,6 +205,8 @@ function get_file_info(filename, mimetype)
return make_text_file_info(extension)
elseif table.contains(document_mime_types, mimetype) then
return make_document_file_info(extension)
elseif table.contains(archive_mime_types, mimetype) then
return make_archive_file_info(extension)
else
-- no supported mediatype could be determined
return nil
Expand Down
4 changes: 2 additions & 2 deletions test_data/ontologies/books-onto.ttl
Expand Up @@ -41,7 +41,7 @@

:Book
rdf:type owl:Class ;
rdfs:subClassOf knora-base:Resource;
rdfs:subClassOf knora-base:Resource ;
knora-base:resourceIcon "book.png" ;
rdfs:label "Buch"@de, "Book"@en ;
rdfs:comment """A resource describing a book."""@en .
Expand All @@ -58,7 +58,7 @@

:Page
rdf:type owl:Class ;
rdfs:subClassOf knora-base:Resource;
rdfs:subClassOf knora-base:Resource ;
knora-base:resourceIcon "page.png" ;
rdfs:label "Seite"@de, "Page"@en ;
rdfs:comment """A resource describing a page in a book."""@en .