Releases: nasa/cumulus
v16.1.5
Release v16.1.5
[v16.1.5] 2024-05-09 [BACKPORT]
Please note changes in 16.1.5 may not yet be released in future versions, as this
is a backport/patch release on the 16.x series of releases. Updates that are
included in the future will have a corresponding CHANGELOG entry in future releases.
Fixed
CUMULUS-3721
-
Update lambda:GetFunctionConfiguration policy statement to fix error related to resource naming.
CUMULUS-3527 -
Added support for additional kex algorithms in the sftp-client.
v18.2.1
Release v18.2.1
[v18.2.1] 2024-05-08
Please note changes in 18.2.1 may not yet be released in future versions, as this
is a backport/patch release on the 18.2.x series of releases. Updates that are
included in the future will have a corresponding CHANGELOG entry in future releases.
Fixed
- CUMULUS-3721
- Update lambda:GetFunctionConfiguration policy statement to fix error related to resource naming
- CUMULUS-3701
- Updated
@cumulus/api
to no longer improperly pass PATCH/PUT null values to Eventbridge rules
- Updated
v18.2.0-3721
Hotfix v18.2.0-3721
Note: This release is a hotfix intended to address the issues noted in CUMULUS-3721 as a direct patch to release 18.2.0. Users are advised to wait for a formal release on the 18.2.x release series and forward versions unless this issue directly impacts them.
- CUMULUS-3721
- Update lambda:GetFunctionConfiguration policy statement to fix error related to resource naming
v16.1.4
[v16.1.4] 2024-4-23
Migration Notes
For the v16.1 release series, Cumulus Core will be tested against PostgreSQL v13. Users
should migrate their datastores to Aurora PostgreSQL 13.12+ compatible data
stores as soon as possible after upgrading to this release.
IMPORTANT -- When upgrading from v16.1.x users should update to a release
following 18.2.0/the first forward release supporting Postgres v13, as versions
between 16.1.x and 18.2.x+ are unsupported on Aurora Postgres v13.
Database Upgrade
Users utilizing the cumulus-rds-tf
module should reference cumulus-rds-tf
upgrade
instructions.
Changed
- CUMULUS-3564
- Update webpack configuration to explicitly disable chunking
- CUMULUS-3444
- Update
cumulus-rds-tf
module to take additional parameters in support of
migration from Aurora PostgreSQl v11 to v13. See Migration Notes for more details.
- Update
- CUMULUS-3547
- Updated ECS Cluster
/dev/xvdcz
EBS volumes so they're encrypted.
- Updated ECS Cluster
Fixed
- CUMULUS-3673
- Fixes Granules API so that paths containing a granule and/or collection ID properly URI encode the ID.
v18.2.0
[v18.2.0] 2024-02-02
Migration Notes
From this release forward, Cumulus Core will be tested against PostgreSQL v13. Users
should migrate their datastores to Aurora PostgreSQL 13.9+ compatible data
stores as soon as possible after upgrading to this release.
Database Upgrade
Users utilizing the cumulus-rds-tf
module should reference cumulus-rds-tf upgrade instructions.
Changed
- CUMULUS-3444
- Update
cumulus-rds-tf
module to take additional parameters in support of
migration from Aurora PostgreSQl v11 to v13. See Migration Notes for more details.
- Update
- CUMULUS-3564
- Update webpack configuration to explicitly disable chunking
- CUMULUS-2895
- Updated KMS code to aws sdk v3
- CUMULUS-2888
- Update CloudWatch Events code to AWS SDK v3
- CUMULUS-2893
- Updated Kinesis code to AWS SDK v3
- CUMULUS-3555
- Revert 3540, un-stubbing cmr facing tests
- Raise memory_size of ftpPopulateTestLambda to 512MB
- CUMULUS-2887
- Updated CloudFormation code to aws sdk v3
- CUMULUS-2899
- Updated SNS code to aws sdk v3
- CUMULUS_3499
- Update AWS-SDK dependency pin to "2.1490" to prevent SQS issue. Dependency
pin expected to be changed with the resolution to CUMULUS-2900
- Update AWS-SDK dependency pin to "2.1490" to prevent SQS issue. Dependency
- CUMULUS-2894
- Update Lambda code to AWS SDK v3
- CUMULUS-3432
- Update
cumulus-rds-tf
engine_version
to13.9
- Update
cumulus-rds-tf
parameter_group_family
toaurora-postgresql13
- Update development/local stack postgres image version to postgres:13.9-alpine
- Update
- CUMULUS-2900
- Update SQS code to AWS SDK v3
- CUMULUS-3352
- Update example project to use CMA v2.0.3 for integration testing
- Update example deployment to deploy cnmResponse lambda version
2.1.1-aplha.2-SNAPSHOT - Update example deployment to deploy cnmToGranule lambda
version 1.7.0-alpha.2-SNAPSHOT
- CUMULUS-3501
- Updated CreateReconciliationReport lambda to save report record to Elasticsearch.
- Created docker image cumuluss/async-operation:48 from v16.1.2, and used it as default async_operation_image.
- CUMULUS-3502
- Upgraded localstack to v3.0.0 to support recent aws-sdk releases and update unit tests.
- CUMULUS-3540
- stubbed cmr interfaces in integration tests allow integration tests to pass
- needed while cmr is failing to continue needed releases and progress
- this change should be reverted ASAP when cmr is working as needed again
Fixed
- CUMULUS-3177
- changed
_removeGranuleFromCmr
function for granulebulkDelete
to not throw an error and instead catch the error when the granule is not found in CMR
- changed
- CUMULUS-3293
- Process Dead Letter Archive is fixed to properly copy objects from
/sqs/
to/failed-sqs/
location
- Process Dead Letter Archive is fixed to properly copy objects from
- CUMULUS-3467
- Added
childWorkflowMeta
toQueueWorkflow
task configuration
- Added
- CUMULUS-3474
- Fixed overridden changes to
rules.buildPayload' to restore changes from ticket
CUMULUS-2969which limited the definition object to
nameand
arn` to
account for AWS character limits.
- Fixed overridden changes to
- CUMULUS-3479
- Fixed typo in s3-replicator resource declaration where
var.lambda_memory_size
is supposed to bevar.lambda_memory_sizes
- Fixed typo in s3-replicator resource declaration where
- CUMULUS-3510
- Fixed
@cumulus/api
validateAndUpdateSqsRule
method to allow 0 retries and 0 visibilityTimeout
in rule's meta. This fix from CUMULUS-2863 was not in release 16 and later.
- Fixed
- CUMULUS-3562
- updated crypto-js to 4.2.0
- updated aws-sdk/client-api-gateway to 3.499 to avoid older crypto-js dependency
v16.1.3
Release v16.1.3
[v16.1.3] 2024-1-15
Please note changes in 16.1.3 may not yet be released in future versions, as this
is a backport/patch release on the 16.x series of releases. Updates that are
included in the future will have a corresponding CHANGELOG entry in future releases.
Changed
- CUMULUS_3499
- Update AWS-SDK dependency pin to "2.1490" to prevent SQS issue. Dependency
pin expected to be changed with the resolution to CUMULUS-2900
- Update AWS-SDK dependency pin to "2.1490" to prevent SQS issue. Dependency
Fixed
- CUMULUS-3474
- Fixed overriden changes to
rules.buildPayload' to restore changes from ticket
CUMULUS-2969which limited the definition object to
nameand
arn` to
account for AWS character limits.
- Fixed overriden changes to
- CUMULUS-3501
- Updated CreateReconciliationReport lambda to save report record to Elasticsearch.
- Created docker image cumuluss/async-operation:48 from v16.1.2, and used it as default async_operation_image.
- CUMULUS-3510
- Fixed
@cumulus/api
validateAndUpdateSqsRule
method to allow 0 retries and 0 visibilityTimeout
in rule's meta. This fix from CUMULUS-2863 was not in release 16 and later.
- Fixed
- CUMULUS-3540
- stubbed cmr interfaces in integration tests allow integration tests to pass
- needed while cmr is failing to continue needed releases and progress
- this change should be reverted ASAP when cmr is working as needed again
v18.2.0-alpha.0
Release v18.2.0-alpha.0
v16.1.2
Release v16.1.2
Please note changes in 16.1.2 may not yet be released in future versions, as this
is a backport/patch release on the 16.x series of releases. Updates that are
included in the future will have a corresponding CHANGELOG entry in future releases.
Dashboard Requirement
Cumulus Dashboard v12.0.1 https://github.com/nasa/cumulus-dashboard/releases/tag/v12.0.1
Added
- CUMULUS-3218
- Added optional
maxDownloadTime
field toprovider
schema - Added
max_download_time
column to PostgreSQLproviders
table - Updated
@cumulus/ingest/lock
to check expired locks based onprovider.maxDownloadTime
- Added optional
Fixed
- @AWS-SDK upgrade
- Fixed TS compilation error on aws-client package caused by @aws-sdk/client-dynamodb 3.433.0 upgrade
- Updated mapping for collection Elasticsearch records to prevent dynamic field for keys under
meta
.
- CUMULUS-3286
- Fixed
@cumulus/cmrjs/cmr-utils/getGranuleTemporalInfo
and@cumulus/message/Granules/getGranuleCmrTemporalInfo
to handle non-existing cmr file. - Updated mapping for granule and deletedgranule Elasticsearch records to prevent dynamic field for keys under
queryFields
.
- Fixed
- CUMULUS-3293
- Process Dead Letter Archive is fixed to properly copy objects from
/sqs/
to/failed-sqs/
location
- Process Dead Letter Archive is fixed to properly copy objects from
- CUMULUS-3393
- Fixed
PUT
collection endpoint to update collection configuration in S3.
- Fixed
- CUMULUS-3467
- Added
childWorkflowMeta
toQueueWorkflow
task configuration
- Added
v18.1.0
Dashboard Requirement
Note that this release contains breaking changes for the Cumulus Dashboard. You must use the Cumulus Dashboard v12.1.0 or greater
MIGRATION notes
Rules API Endpoint Versioning
As part of the work on CUMULUS-3095, we have added a required header for the
rules PUT/PATCH endpoints -- to ensure that older clients/utilities do not
unexpectedly make destructive use of those endpoints, a validation check of a
header value against supported versions has been implemented.
Moving forward, if a breaking change is made to an existing endpoint that
requires user updates, as part of that update we will set the current version of
the core API and require a header that confirms the client is compatible with
the version required or greater.
In this instance, the rules PUT/PATCH
endpoints will require a Cumulus-API-Version
value of at least 2
.
curl --request PUT https://example.com/rules/repeat_test\
--header 'Cumulus-API-Version: 2'\
--header 'Content-Type: application/json'\
--header 'Authorization: Bearer ReplaceWithToken'\
--data ...
Users/clients that do not make use of these endpoints will not be impacted.
Breaking Changes
- CUMULUS-3427
- Changed the naming conventions for memory size and timeouts configuration to simply the lambda name
Notable Changes
- CUMULUS-3095
- Added
PATCH
rules endpoint to update rule which works as the existingPUT
endpoint. - Updated
PUT
rules endpoint to replace rule.
- Added
Added
- CUMULUS-3218
- Added optional
maxDownloadTime
field toprovider
schema - Added
max_download_time
column to PostgreSQLproviders
table - Updated
@cumulus/ingest/lock
to check expired locks based onprovider.maxDownloadTime
- Added optional
Changed
- CUMULUS-3095
- Updated
@cumulus/api-client/rules
to havereplaceRule
andupdateRule
methods. - Updated mapping for rule Elasticsearch records to prevent dynamic field for keys under
meta
andpayload
, and fixedrule
field mapping.
- Updated
- CUMULUS-3351
- Updated
constructOnlineAccessUrls()
to group CMR online access URLs by link type.
- Updated
- CUMULUS-3377
- Added configuration option to cumulus-tf/terraform.tfvars to include sns:Subscribe access policy for
executions, granules, collections, and PDRs report topics.
- Added configuration option to cumulus-tf/terraform.tfvars to include sns:Subscribe access policy for
- CUMULUS-3392
- Modify cloudwatch rule by deleting
custom
- Modify cloudwatch rule by deleting
- CUMULUS-3434
- Updated
@cumulus/orca-recovery-adapter
task to output both input granules and recovery output. - Updated
example/cumulus-tf/orca.tf
to use v9.0.0.
- Updated
Fixed
- CUMULUS-3095
- Added back
rule
schema validation which is missing after RDS phase 3. - Fixed a bug for creating rule with tags.
- Added back
- CUMULUS-3286
- Fixed
@cumulus/cmrjs/cmr-utils/getGranuleTemporalInfo
and@cumulus/message/Granules/getGranuleCmrTemporalInfo
to handle non-existing cmr file. - Updated mapping for granule and deletedgranule Elasticsearch records to prevent dynamic field for keys under
queryFields
. - Updated mapping for collection Elasticsearch records to prevent dynamic field for keys under
meta
.
- Fixed
- CUMULUS-3393
- Fixed
PUT
collection endpoint to update collection configuration in S3.
- Fixed
- CUMULUS-3427
- Fixed issue where some lambda and task memory sizes and timeouts were not configurable
- @AWS-SDK upgrade
- Fixed TS compilation error on aws-client package caused by @aws-sdk/client-dynamodb 3.433.0 upgrade
v18.0.0
Release v18.0.0
Notable Changes
-
CUMULUS-3259
- Updated Terraform version from 0.13.6 to 1.5.3. Please see the instructions to upgrade your deployments.
-
CUMULUS-3270
- update python lambdas to use python3.10
- update dependencies to use python3.10 including cumulus-message-adapter, cumulus-message-adapter-python and cumulus-process-py
Changes
- CUMULUS-3366
- Added logging to the
collectionRuleMatcher
Rules Helper, which is used by the sqs-message-consumer and message-consumer Lambdas,
to report when an incoming message's collection does not match any rules.
- Added logging to the