Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Amend File API design document to address file.Close constraints #3328

Open
wants to merge 1 commit into
base: master
Choose a base branch
from
Open

Amend File API design document to address file.Close constraints #3328

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Jump to
Jump to file
Failed to load files.
Diff view
Diff view
13 changes: 12 additions & 1 deletion docs/design/019-file-api.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -189,11 +189,12 @@ function ab2str(buf) {

The proposed API is made feasible through the following implementation aspects:

- A file's content is loaded into memory only once:
- A file's content is loaded into memory only once:
- When a file is opened, its content is buffered at the module's root in a dedicated registry, returning a handle with a pointer to that buffer.
- Each VU receives a copy of the file handle, enabling them to interact with files using the unique memory area linked to the handle, instead of each receiving a full copy of the buffer.
- As each invocation of `open*` receives a unique file handle linked to the same memory area, they each have unique offsets. This setup allows each VU to process file data independently without conflict or race conditions.


### Possible future improvements

#### Introduce stream support
Expand All @@ -206,6 +207,16 @@ See [#3020](https://github.com/grafana/k6/issues/3020)

This is currently unachievable because we must anticipate which files will be opened. While some quick fixes might appear feasible (e.g., parsing the AST before execution to identify files), they quickly fall apart: What if the filename resides in a variable? A plausible solution would involve requiring users to declare necessary resources (files/folders) ahead of time. This approach would ensure these resources are captured and included in the archive for future VU code access.

#### Scoped file closing:

Based on the current implementation of k6, file opening is confined to the init context. Given this, we operate under the presumption that the lifespan of a file handle aligns with that of the init context. Furthermore, we recognize the possibility that the init context may undergo multiple evaluations throughout a script's execution.

As a result, the `File` interface's `close` method is similarly bound to the init context's scope. That is, it can either be invoked outside the VU stage (in the `handleSummary` function for instance) or be automatically executed, implicitly, at the conclusion of the test, harnessing [the k6 internal event system](https://github.com/grafana/k6/pull/3112).

A refinement could consist in:
- subsequent versions, [once we enable file opening within the VU context](https://github.com/grafana/k6/issues/3020), would then permit and advocate for files to be closed within the VU stage as well.
- presuming the design of the open function remains consistent with the current proposal, our underlying mechanism would pivot to a reference counting system. This would ensure that a file is only closed when all VUs have concluded their interactions with it.

## Conclusion

We believe the [proof of concept](https://github.com/grafana/k6/tree/poc/experimental/fs-module/js/modules/k6/experimental/fs) developed with this proposal illustrates the feasibility and benefits of developing such an API.