Sitespeed.io compatibility to send html report to Ceph S3 Storage ? #4117
Comments
|
Hi @valgurau thanks for creating the issue. I think this could be that we need to upgrade to v3? I did a test a year ago or something and if I remember there was a new parameter that we needed so it was not backward compatible so I didn't upgrade at that time. If I do some changes, are you able to test from a branch in the GitHub repo? If so I would first try to upgrade to latest v2 and you can try that, and then if that do not work, I can make a branch with v3 (I still need to upgrade that in the future). |
|
Hi @soulgalore, thank you for responding so quickly. Yes I will be able to test from a branch. |
|
Great! Can you try with https://github.com/sitespeedio/sitespeed.io/tree/aws-2.1582.0 ? That updates to latest v2. Hopefully tomorrow I can have branch with v3. I remember before the updates/upgrades doc for v3 wasn't super helpful but that was long time ago. |
|
I checked the new code. In v2 you don't need to set region (it defaulted to one of the US regions, don't remember which). In v3 you need to set region, and that will be a breaking change for users. I can probably push a branch tonight or early tomorrow so you at least can try with v3. |
|
In the meantime while trying with version 2 aws-sdk (current version used by the sitespeedio s3 plugin) I realized that there is an option to force the path-style for the bucket in the options --s3.options.s3ForcePathStyle=true and with that it works in my case (with Ceph S3). In addition I had an error in the bucket name, the use of the s3:// prefix also caused problems. And as you mentioned I don't need to set the region parameter. So the right configuration that works for me: --s3.endpoint "https://a-radosgw-endpoint/" --s3.bucketname "bucket-name" --s3.key "xxxx" --s3.secret "xxxx" --s3.options.s3ForcePathStyle=true I tested with the branch you created for the latest version 2 and it also works. Also I will try with the branch for version 3 of the aws sdk to see if it's ok. |
|
Ok! Here's a version using v3: https://github.com/sitespeedio/sitespeed.io/tree/aws-s3-v3 I need to tune it but I could verify I could upload as long as I set the region. |
|
Hi @soulgalore, |
|
Hi @valgurau thank you for trying it out. Endpoint config was moved so I missed adding it. I updated the branch now so it can be set. For |
|
Aws documentation : https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/migrating/notable-changes/
|
|
Ok cool, thank you. Were you able to retry with my endpoint fix? Hoping setting the endpoint fix forcePathStyle. |
|
Yes, I retried yesterday, everything seems ok with the endpoint and the forcePathStyle option. But as you mentioned before there is a problem with the region on uploading files. Even if I set a region, it still says that the region is missing on each uploaded file. Example: |
|
Ok, so for Ceph S3 using AWS v3 it do not work if you set the region to an empty string (like in the example here)? |
Your question
Hello,
We use a self-managed Ceph S3 cluster inside our company, so we want to store the sitespeed HTML reports there.
Unlike Amazon S3 there is no region configured for the bucket, so I tried to set --s3.region with an empty string ("").
S3 parameters in sitespeed.io command : --s3.endpoint "https://a-radosgw-endpoint" --s3.bucketname "s3://bucket-name" --s3.key "" --s3.secret "***" --s3.region ""
And I have this error :
ERROR: [sitespeedio.plugin.s3] Could not upload to S3 SignatureDoesNotMatch: null at Request.extractError (/usr/src/app/node_modules/aws-sdk/lib/services/s3.js:711:35) at Request.callListeners (/usr/src/app/node_modules/aws-sdk/lib/sequential_executor.js:106:20) at Request.emit (/usr/src/app/node_modules/aws-sdk/lib/sequential_executor.js:78:10) at Request.emit (/usr/src/app/node_modules/aws-sdk/lib/request.js:686:14) at Request.transition (/usr/src/app/node_modules/aws-sdk/lib/request.js:22:10) at AcceptorStateMachine.runTo (/usr/src/app/node_modules/aws-sdk/lib/state_machine.js:14:12) at /usr/src/app/node_modules/aws-sdk/lib/state_machine.js:26:10 at Request.<anonymous> (/usr/src/app/node_modules/aws-sdk/lib/request.js:38:9) at Request.<anonymous> (/usr/src/app/node_modules/aws-sdk/lib/request.js:688:12) at Request.callListeners (/usr/src/app/node_modules/aws-sdk/lib/sequential_executor.js:116:18) at Request.emit (/usr/src/app/node_modules/aws-sdk/lib/sequential_executor.js:78:10) at Request.emit (/usr/src/app/node_modules/aws-sdk/lib/request.js:686:14) at Request.transition (/usr/src/app/node_modules/aws-sdk/lib/request.js:22:10) at AcceptorStateMachine.runTo (/usr/src/app/node_modules/aws-sdk/lib/state_machine.js:14:12) at /usr/src/app/node_modules/aws-sdk/lib/state_machine.js:26:10 at Request.<anonymous> (/usr/src/app/node_modules/aws-sdk/lib/request.js:38:9) Errors while running: SignatureDoesNotMatch: null Error: Errors while running: SignatureDoesNotMatch: null at start (file:///usr/src/app/bin/sitespeed.js:178:15) at processTicksAndRejections (node:internal/process/task_queues:95:5) at runNextTicks (node:internal/process/task_queues:64:3) at process.processImmediate (node:internal/timers:449:9) at async file:///usr/src/app/bin/sitespeed.js:189:1Using an other tool s3cmd everything works with the same configuration.
So, I wonder if there is an "issue" with Ceph S3 and the "aws-sdk" (v2) used by sitespeedio ?
The text was updated successfully, but these errors were encountered: