Releases: BlueBrain/nexus
Releases · BlueBrain/nexus
v1.10.0-M11
What's Changed
S3 support
- Configure a global path prefix used when saving S3 files by @dantb in #4923
- Make S3 file prefix optional in type by @dantb in #4929
- Deal with S3 download failures by streaming files by @shinyhappydan in #4928
- Make S3 file location relative within bucket + path independent of the prefix by @dantb in #4936
- Make sha256 default for s3 storages by @olivergrabinski in #4950
- Handle S3 access error gracefully with 403 by @dantb in #4954
- Get file size from S3 after upload + fix upload by @imsdu in #4952
Nexus ship
- Patch target projects for views within an aggregated ES view by @dantb in #4930
- Remove empty ids from source when importing resources by @imsdu in #4931
- Add S3 multipart copy and use it in Nexus ship by @olivergrabinski in #4933
- Add progress logging, allow to skip file events by @imsdu in #4932
- Update multi part copy operation with checksum by @olivergrabinski in #4938
- In Nexus Ship, ignore files bigger than 5gb (temporary) by @olivergrabinski in #4939
- Use readFileMultipart in ship event streamer by @olivergrabinski in #4947
- Push files to a different location in import batch + add files suffix by @imsdu in #4951
- Pass down custom metadata for file creation and update events by @imsdu in #4953
- Temporarily ignore file with empty filenames in Nexus Ship by @olivergrabinski in #4959
- Ignore views registered as resource events in nexus ship by @olivergrabinski in #4965
- Ignore resource types by config in nexus-ship by @olivergrabinski in #4973
- Patch files with empty names + add tests in ship for files by @imsdu in #4964
- Handle FileAttributesUpdated in import batch by @imsdu in #4980
- Handle gracefully when a file can not be found in a bucket by @imsdu in #4982
- Fix bug while importing files with special characters in name by @imsdu in #4983
- Log dropped events in database by @imsdu in #4984
- Filter out directories from import batch by @imsdu in #4985
- Setting content type when copying over the file in import batch by @imsdu in #4987
- Support for s3 multipart checksums by @olivergrabinski in #4991
- Enable copying of large files (>5GB) in Nexus ship by @olivergrabinski in #4992
Maintenance
- Run actions on ubuntu-latest by @olivergrabinski in #4937
- Update Nexus dependencies by @imsdu in #4934
- Update Elasticsearch and Localstack versions by @imsdu in #4935
- Update to Scala 2.13.14 by @imsdu in #4946
- Update to SBT 1.10.0 by @imsdu in #4958
- Update to postgres 15.7 by @imsdu in #4960
- Add slow query logger based on Doobie by @imsdu in #4993
Bug fixes
- Replay the tagged state from the main node to avoid replication delays by @imsdu in #4949
- Fallback to default value when the provided filename is empty by @imsdu in #4957
- Add test to ensure filenames are encoded correctly in S3 by @shinyhappydan in #4955
- Add index on types for the tombstone table by @imsdu in #4988
Search
- Add MEModel example for tests by @crisely09 in #4990
Full Changelog: v1.10.0-M10...v1.10.0-M11
v1.10.0-M10
What's Changed
Schema validation
- Add dedicated error when there is no targeted node during schema validation by @imsdu in #4920
- Add test to validate a brain region against an ontology by @imsdu in #4918
S3
- S3 content type user supplied by @shinyhappydan in #4895
- Add endpoint to point delta S3 file to different physical file by @dantb in #4902
- Allow to use default AWS credential provider via config by @olivergrabinski in #4906
- Add configured default bucket on S3 storages by @dantb in #4917
Nexus Ship
- Patch target projects for views within an aggregated blazegraph view by @dantb in #4907
- Read events file from S3 using multipart request by @dantb in #4911
- Rewire the nexus ship resources by @olivergrabinski in #4898
- Build snapshot on ship changes by @olivergrabinski in #4899
- Fix config loading in nexus ship by @olivergrabinski in #4900
- Use correct ship config for transactors and inputbucket by @olivergrabinski in #4901
- Use update registered file in nexus ship; log import report by @olivergrabinski in #4903
- Allow disabling of resource validation in nexus ship by @olivergrabinski in #4908
- Use readFileMultipart in S3 fetch operation by @olivergrabinski in #4916
Fixes
- Limit query in export event function by @olivergrabinski in #4922
Maintenance
Full Changelog: v1.10.0-M9...v1.10.0-M10
v1.10.0-M9
What's Changed
Bug fixes
S3 Support
- Replace alpakka S3 with fs2-aws and minio with localstack by @dantb in #4852
- Remove endpoint and region from the S3 Storage instances by @shinyhappydan in #4859
- Reduce library S3 client usage by @dantb in #4865
- Refactor file operations behind new interfaces by @dantb in #4871
- Use S3 to generate checksum by @shinyhappydan in #4873
- Register S3 files in-place by @dantb in #4878
- Remove s3 md5 by @shinyhappydan in #4882
Nexus Ship
- Add logback.xml file for nexus ship by @olivergrabinski in #4858
- Add ability to read import file/directory from S3 in nexus ship by @olivergrabinski in #4863
- Include nexus-ship in CI by @olivergrabinski in #4864
- When running nexus ship with s3 flag, read config from s3 by @olivergrabinski in #4866
- View initialization is nexus ship by @olivergrabinski in #4868
- Refactor ship before introducing save summary feature by @imsdu in #4869
- Create S3 storage on project creation during nexus ship import by @olivergrabinski in #4870
- Save summary at the end of the batch by @imsdu in #4872
- Patch cross-project resolvers when a project is renamed by @imsdu in #4874
- Patch distributions in import batch by @imsdu in #4880
- Process files in nexus ship by @olivergrabinski in #4884
Maintenance
- Update dependencies for next milestone M9 by @imsdu in #4861
- Improve message for 'ResourceShaclEngineRejection' by @imsdu in #4862
- Remove StoragesStatistics from Files by @olivergrabinski in #4879
Full Changelog: v1.10.0-M8...v1.10.0-M9
v1.10.0-M8
What's Changed
S3 Support
- Use localstack for S3 integration tests by @dantb in #4789
- Remove the linking operation for S3 storages by @dantb in #4797
- Rework S3 storage access by @dantb in #4807
Nexus Ship
- Add
ResourceProcessor
,SchemaProcessor
; add E2E tests for ship by @olivergrabinski in #4803 SchemaProcessor
implementation; ES & BG view processors; E2E testing by @olivergrabinski in #4804- Add
CompositeViewProcessor
by @olivergrabinski in #4806 - Remove ship dependency on acls by @olivergrabinski in #4811
- Allow new resolvers with same priority as deprecated resolver by @olivergrabinski in #4805
- Add incremental import option for Nexus ship by @olivergrabinski in #4813
- Add tests for transfer of resolvers by @shinyhappydan in #4817
- Allow mapping of org/project during ship import by @olivergrabinski in #4816
- Do not validate schemas when importing with nexus ship by @olivergrabinski in #4826
- Fix resource wiring in ship by @olivergrabinski in #4828
- Fix RemoteContextResolution wiring for ValidateResource in nexus ship by @olivergrabinski in #4831
- Wire more contexts in nexus ship; use
source
instead ofvalue
in BG and ES processor by @olivergrabinski in #4836 - Allow directory import in nexus ship by @olivergrabinski in #4848
Bug fixes
- Use the StreamingQuery approach to handle long exports by @imsdu in #4820
- Allow to create schemas on the resource endpoint for legacy purposes by @imsdu in #4851
Documentation
- Bumped forge config file to commit hash 638047f in mooc notebooks by @MFSY in #4818
- Update forge.yml to use local file-to-resource-mapping.json by @MFSY in #4821
- Update docs nnotebooks to point to d9f6cb8 forge config by @MFSY in #4823
- Remove references to tagging resolvers by @shinyhappydan in #4824
- Fix HTTP method type in example by @olivergrabinski in #4827
Maintenance
- Use GitHub cloud runners for more workflows by @olivergrabinski in #4792
- Plugin tests on cloud runner by @olivergrabinski in #4793
- Don't start an Elasticsearch container when running MainSuite by @imsdu in #4794
- Update dependencies by @imsdu in #4795 #4809
- Fix hostname validation on remote sources for composite views by @imsdu in #4798
- Run integration pipeine in the cloud by @olivergrabinski in #4800
- Add AWS config file by @shinyhappydan in #4802
- Break circular dependencies between resources and schemas by @olivergrabinski in #4799
- Abstract away the fetching of ACLs from the AclCheck trait by @olivergrabinski in #4810
- Attempt to fix flaky test in
ElasticSearchClientSpec
by @olivergrabinski in #4812 - Use Docker compose v2 command in github actions by @imsdu in #4822
- Fix external doc link validation workflow by @olivergrabinski in #4829
- Make
ValidatePriority
a trait by @olivergrabinski in #4832 - Update sbt-paradox to 0.10.7 by @olivergrabinski in #4833
- Use Jena and Topbraid versions by @imsdu in #4837
- Use Elasticsearch 8.13 in integration tests by @imsdu in #4843
- Add kamon spans for shacl validation by @imsdu in #4845
- Improve error logging when a resource can't be parsed back in composite views by @imsdu in #4849
- Add another test using the dataset shape by @imsdu in #4850
Full Changelog: v1.10.0-M7...v1.10.0-M8
v1.10.0-M7
What's Changed
Features
- Add custom header for file metadata by @olivergrabinski in #4764
- Add method & endpoint to update custom file metadata by @olivergrabinski in #4775
- Allow tagging during file custom metadata updates by @olivergrabinski in #4778
- Improve fulltext search experience in search by @imsdu in #4782
Nexus Ship
- Refactor FetchContext + add resolver to import batch by @imsdu in #4756
- Wire resolvers and add integration tests by @imsdu in #4765
- Add basic reporting for import batch by @imsdu in #4773
Fixes
Maintenance
- Update SBT to 1.9.9 / Scala to 2.13.13 by @imsdu in #4760
- Remove wrapped rejections for Json-LD errors by @imsdu in #4776
- Update dependencies, reenable kamon-jdbc by @imsdu in #4777
Full Changelog: v1.10.0-M6...v1.10.0-M7
v1.10.0-M6
What's Changed
Features
- Add annotate param to fetching original payload via resolvers by @imsdu in #4748
- Allow custom metadata when creating/linking a file by @shinyhappydan @olivergrabinski in #4724, #4758
- Add type hierarchy route by @olivergrabinski in #4729
Nexus Ship
- Introduce Nexus ship to load events by @imsdu in #4739
- Init org provider + project processor by @imsdu in #4742
Fixes
- Fix annotated source format by @imsdu in #4747
- Return 404 when an operation on a resource is unknown by @imsdu in #4740
- Custom binary files are not compressed when fetched by the API by @imsdu in #4751
- Do not create an event when updating attributes and digest is not computed by @imsdu in #4745
- Create a BNODE depending on the resource id for emodels by @imsdu in #4741
Maintenance
- Update dependencies by @imsdu in #4749
- Update postgresql driver, temporarily remove kamon-jdbc by @imsdu in #4757
- Update tests and docs for Elasticsearch 8.12 by @imsdu in #4746
Full Changelog: v1.10.0-M5...v1.10.0-M6
v1.10.0-M5
What's Changed
Breaking changes
Bug fixes
- Allow unconstrained resources to be updated even if schema are enforced by @imsdu in #4712
- Fix error handling in search route by @imsdu in #4709
Features
- Add keywords to files during creation by @shinyhappydan in #4625
- Add type hierarchy model by @olivergrabinski in #4718
- Allow to disable the creation of a default storage at project creation by @imsdu in #4713
- Allow linking with copy/delete in storage service by @dantb in #4728
Maintenance
- Add cede to mark rdf expensive operations by @imsdu in #4710
- Do not store the endpoint for remote storages and use the one in the configuration by @imsdu in #4715
- Update dependencies by @imsdu in #4725
- Update to Postgres 15.6 by @imsdu in #4735
Full Changelog: v1.10.0-M4...v1.10.0-M5
v1.9.1
v1.10.0-M4
What's Changed
Breaking changes
- Remove the ability to tag storages by @dantb in #4675
- Remove remaining references to tags in storages docs by @dantb in #4692
- Deprecate resolver tagging by @olivergrabinski in #4673
- Remove ability to tag SPARQL views by @dantb in #4691
Bug fixes
- Fix computing diff on large resources during updates by @imsdu in #4684
- Retrieve tagged value in a previous transaction, better error handling by @imsdu in #4690
Features
- Add the ability to export events for a subset of projects and from a starting offset by @imsdu in #4701
- Add an API mapping for search by @imsdu in #4702
Maintenance
- Update typelevel dependencies by @imsdu in #4685
- Update Keycloak image in integration tests by @imsdu in #4689
- Update github issue template by @shinyhappydan in #4706
- Refactor fetch context to remove wrapped rejection by @imsdu in #4707
Full Changelog: v1.10.0-M3...v1.10.0-M4
v1.10.0-M3
What's Changed
Full Changelog: v1.10.0-M2...v1.10.0-M3