Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Transfer Manager Metrics #2305

Merged
merged 16 commits into from
Sep 26, 2023
Merged

feat: Transfer Manager Metrics #2305

merged 16 commits into from
Sep 26, 2023

Conversation

danielbankhead
Copy link
Member

🦕

@danielbankhead danielbankhead requested review from a team as code owners September 14, 2023 21:53
@product-auto-label product-auto-label bot added size: m Pull request size is medium. api: storage Issues related to the googleapis/nodejs-storage API. labels Sep 14, 2023
@danielbankhead
Copy link
Member Author

First commit is feature-only (no tests) to show it works w/o breaking anything. Working on tests next.

@@ -537,7 +539,9 @@ export class Util {
body: writeStream,
},
],
} as {} as r.OptionsWithUri & GCCL_GCS_CMD;
} as {} as r.OptionsWithUri & {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is clever!

*
* E.g the `V` in `X-Goog-API-Client: gccl-gcs-cmd/V`
**/
export const GCCL_GCS_CMD_KEY = Symbol('GCCL_GCS_CMD');
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Had to go look this up, didn't know Symbol was a thing.

Copy link

@danielduhh danielduhh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approach LGTM

@product-auto-label product-auto-label bot added size: l Pull request size is large. and removed size: m Pull request size is medium. labels Sep 22, 2023
Comment on lines +511 to +513
transferManager.bucket.storage.authClient = new GoogleAuth({
authClient: new TestAuthClient(),
});
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes it more of an integration test.

Comment on lines +303 to +309
directory = await fsp.mkdtemp(
path.join(tmpdir(), 'tm-uploadFileInChunks-')
);

filePath = path.join(directory, 't.txt');

await fsp.writeFile(filePath, 'hello');
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sharing a real, temp file to make it more robust. We can refactor the other tests if we want to use the real XML tool w/o exporting it.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yea this looks better than the hacky way I had it.

},
idempotencyStrategy: IdempotencyStrategy.RetryConditional,
};
const STORAGE = sinon.stub(
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, couldn't figure out the correct way to do this.

}

// If the header isn't present, add it
if (!headerFound) {

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'd have to remember to update this line every time we add something new to headers. Is there a better way to do this?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can have a centralized place for this, however it would be cleaner to first complete the gaxios migration.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is also already matching the pattern in resumable uploads.

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

here's an example in #2296.

Yeah, thats fine. Please create a TODO & ticket to follow up

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@danielbankhead
Copy link
Member Author

PR's ready for review

@danielbankhead danielbankhead merged commit 9be3b6a into main Sep 26, 2023
19 checks passed
@danielbankhead danielbankhead deleted the tm-metrics branch September 26, 2023 18:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: storage Issues related to the googleapis/nodejs-storage API. size: l Pull request size is large.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

3 participants