Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Store and return in the API iris of source files when copying #4618

Draft
wants to merge 6 commits into
base: master
Choose a base branch
from

Conversation

dantb
Copy link
Contributor

@dantb dantb commented Dec 22, 2023

Fixes #4611

The iris include rev/tag information. However, if the source file is latest at the time of copy and is then subsequently updated, the revision would be lost I think?

Need to wait for #4615 before adding an integration test case for this

@@ -25,7 +25,8 @@
"_storage": {
"@id": "https://bluebrain.github.io/nexus/vocabulary/storage",
"@type": "@id"
}
},
"_sourceFile": "https://bluebrain.github.io/nexus/vocabulary/fileId"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It should be sourceFile as a suffix.
Can you check the expanded json-ld/n-triples output to see if it is ok ?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done in 69176fc. The expansion works fine, see below

JSON LD with sourceFile expanded
{
  "@id": "http://delta:8080/v1/resources/d266ifuc81w8t2q/tcqjlehnys34k1b/_/2283170b-6f60-4928-8b51-c5255ce10284",
  "@type": [
    "https://bluebrain.github.io/nexus/vocabulary/File"
  ],
  "https://bluebrain.github.io/nexus/vocabulary/bytes": [
    {
      "@value": 16
    }
  ],
  "https://bluebrain.github.io/nexus/vocabulary/constrainedBy": [
    {
      "@id": "https://bluebrain.github.io/nexus/schemas/files.json"
    }
  ],
  "https://bluebrain.github.io/nexus/vocabulary/createdAt": [
    {
      "@type": "http://www.w3.org/2001/XMLSchema#dateTime",
      "@value": "2024-01-08T12:18:54.503877Z"
    }
  ],
  "https://bluebrain.github.io/nexus/vocabulary/createdBy": [
    {
      "@id": "http://delta:8080/v1/realms/test-kvxjtpxtbuttvjfb/users/digfeugjngcpfclh"
    }
  ],
  "https://bluebrain.github.io/nexus/vocabulary/deprecated": [
    {
      "@value": false
    }
  ],
  "https://bluebrain.github.io/nexus/vocabulary/digest": [
    {
      "https://bluebrain.github.io/nexus/vocabulary/algorithm": [
        {
          "@value": "SHA-256"
        }
      ],
      "https://bluebrain.github.io/nexus/vocabulary/value": [
        {
          "@value": "0afef4502ba93acba18d3fa81eb6ae04867ecd3aa886eb5e4df252b86ceaba7f"
        }
      ]
    }
  ],
  "https://bluebrain.github.io/nexus/vocabulary/filename": [
    {
      "@value": "yschpokcsvrmeuad"
    }
  ],
  "https://bluebrain.github.io/nexus/vocabulary/incoming": [
    {
      "@id": "http://delta:8080/v1/files/d266ifuc81w8t2q/tcqjlehnys34k1b/http:%2F%2Fdelta:8080%2Fv1%2Fresources%2Fd266ifuc81w8t2q%2Ftcqjlehnys34k1b%2F_%2F2283170b-6f60-4928-8b51-c5255ce10284/incoming"
    }
  ],
  "https://bluebrain.github.io/nexus/vocabulary/mediaType": [
    {
      "@value": "text/plain; charset=UTF-8"
    }
  ],
  "https://bluebrain.github.io/nexus/vocabulary/origin": [
    {
      "@id": "Client"
    }
  ],
  "https://bluebrain.github.io/nexus/vocabulary/outgoing": [
    {
      "@id": "http://delta:8080/v1/files/d266ifuc81w8t2q/tcqjlehnys34k1b/http:%2F%2Fdelta:8080%2Fv1%2Fresources%2Fd266ifuc81w8t2q%2Ftcqjlehnys34k1b%2F_%2F2283170b-6f60-4928-8b51-c5255ce10284/outgoing"
    }
  ],
  "https://bluebrain.github.io/nexus/vocabulary/project": [
    {
      "@id": "http://delta:8080/v1/projects/d266ifuc81w8t2q/tcqjlehnys34k1b"
    }
  ],
  "https://bluebrain.github.io/nexus/vocabulary/rev": [
    {
      "@value": 1
    }
  ],
  "https://bluebrain.github.io/nexus/vocabulary/self": [
    {
      "@id": "http://delta:8080/v1/files/d266ifuc81w8t2q/tcqjlehnys34k1b/http:%2F%2Fdelta:8080%2Fv1%2Fresources%2Fd266ifuc81w8t2q%2Ftcqjlehnys34k1b%2F_%2F2283170b-6f60-4928-8b51-c5255ce10284"
    }
  ],
  "https://bluebrain.github.io/nexus/vocabulary/sourceFile": [
    {
      "@value": "http://delta:8080/v1/resources/dt3kxjohvk4ldv3/jmnip0odairhgju/_/dhlxiaok32i1w2h"
    }
  ],
  "https://bluebrain.github.io/nexus/vocabulary/storage": [
    {
      "@id": "https://bluebrain.github.io/nexus/vocabulary/dhq25ddx8cdauda",
      "@type": [
        "https://bluebrain.github.io/nexus/vocabulary/DiskStorage"
      ],
      "https://bluebrain.github.io/nexus/vocabulary/rev": [
        {
          "@value": 1
        }
      ]
    }
  ],
  "https://bluebrain.github.io/nexus/vocabulary/updatedAt": [
    {
      "@type": "http://www.w3.org/2001/XMLSchema#dateTime",
      "@value": "2024-01-08T12:18:54.503877Z"
    }
  ],
  "https://bluebrain.github.io/nexus/vocabulary/updatedBy": [
    {
      "@id": "http://delta:8080/v1/realms/test-kvxjtpxtbuttvjfb/users/digfeugjngcpfclh"
    }
  ],
  "https://bluebrain.github.io/nexus/vocabulary/uuid": [
    {
      "@value": "6324ceaf-e153-48c4-b653-4ce408393073"
    }
  ]
}

@@ -94,7 +94,8 @@ object FileEvent {
rev: Int,
instant: Instant,
subject: Subject,
tag: Option[UserTag]
tag: Option[UserTag],
sourceFile: Option[ResourceRef]
Copy link
Contributor

@imsdu imsdu Jan 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for missing it earlier but we also need the original source project to be able to properly identify the original file

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm the field contains source project in the integration test 🤔

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ahh ok, it's because the Iri passed to ResourceRef is already expanded here - is this a bad idea?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@imsdu the purpose of this method seems to be exactly what I'm using it for - creating a resource ref that contains the full Iri with project information. It works as expected. I'm not sure I see anything wrong with it?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is slightly different, this is meant for expanding segments in the url.
For example when someone creates a resource:
/resources/org/proj/prefix:suffix/
It relies on the configuration of the org/proj project to expand the iri by replacing the prefix thanks to the api mappings / the base defined in the project.
It is only meant to be used for segments, not in payloads where we can properly use json-ld contexts if we need to expand.
And it does not give information on where the resource lives as it is only the id.
In my example, prefix:suffix can point in a schema in another project for example.

@dantb dantb marked this pull request as draft January 19, 2024 11:50
@@ -112,14 +114,23 @@ object BatchCopy {
private def notEnoughSpace(totalSize: Long, spaceLeft: Long, destStorage: Iri) =
IO.raiseError(TotalCopySizeTooLarge(totalSize, spaceLeft, destStorage))

private def fetchFileAndValidateStorage(id: FileId)(implicit c: Caller): IO[(File, Storage)] = {
private def fetchFileAndValidateStorage(id: FileId)(implicit c: Caller): IO[(File, Storage)] =
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should be able to use ResourceRef directly here, I see that FileId use IdSegmentRef
Those are meant to be used for segment in the api enpoint where the Iri can be compacted.
The source file being in a payload, you should be able to use ResourceRef directly and make things simpler

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Store / return source file reference in metadata for copied files
3 participants