Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure private directory exists for custom targets #13196

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

appden
Copy link

@appden appden commented May 8, 2024

Some custom target commands will expect the @PRIVATE_DIR@ to already exist, such as with make -C @PRIVATE_DIR@ ...

@appden appden requested a review from jpakkane as a code owner May 8, 2024 17:23
@tristan957
Copy link
Contributor

Can you add a test for this?

Comment on lines 868 to 872
try:
if isinstance(target, build.BuildTarget):
if isinstance(target, (build.BuildTarget, build.CustomTarget)):
os.makedirs(self.get_target_private_dir_abs(target))
except FileExistsError:
pass
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While you're here... this could be simplified to:

if isinstance(target, (build.BuildTarget, build.CustomTarget)):
    os.makedirs(self.get_target_private_dir_abs(target), exist_ok=True)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Glad you suggested that I do that, just pushed up that change.

@appden
Copy link
Author

appden commented May 9, 2024

Added a test case that fails without this change.

@@ -0,0 +1,14 @@
project('277 custom target private dir')

python = find_program('python3', required: true)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

true is the default.

project('277 custom target private dir')

python = find_program('python3', required: true)
args = [
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would just inline this.

custom_target(
'check-private-dir',
command: [python, args, '@PRIVATE_DIR@'],
output: 'check-private-dir',
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might have your little script actually create this file just for completeness

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sounds good, updated the test case with feedback above along with a print and capture: true so this file actually gets created.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it should create the file directly instead of using capture: true since that kwarg will cause the command line for the custom target to be saved into a second (python) script, which meson internally generates to handle the capturing part.

It means the test case runs two python programs instead of one, and contributes a bit more to the time that it takes to complete the test run on systems with slow forking, such as windows

Admittedly the overhead of adding an entire new test case directory is larger than forking an extra python copy, and so is the overhead of detecting MSVC (which this case doesn't do but many other unittests do a little too frequently). Still, I'm not sure there's a good solution at the moment for those.

Copy link
Author

@appden appden May 9, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test case truly does not need to the create the output file to test the functionality of this fix. I feel like if I add more code to this to write out that file (instead of just using capture, which itself isn't even necessary for the test) it would detract from focusing on the actual reduction of what this is meant to test.

If you disagree though, please let me know and I'll be happy to update with writing to @OUTPUT@ to hopefully get this merged.

Edit: Updated to use @OUTPUT@ 😀

@eli-schwartz
Copy link
Member

For some reason I thought that ninja was already supposed to create these...

Does this mean that every single custom_target will also get a custom_target.p/ directory created? We do NOT want that unless it's actually getting used.

Comment on lines 867 to 869
def generate_target(self, target):
try:
if isinstance(target, build.BuildTarget):
os.makedirs(self.get_target_private_dir_abs(target))
except FileExistsError:
pass
if isinstance(target, (build.BuildTarget, build.CustomTarget)):
os.makedirs(self.get_target_private_dir_abs(target), exist_ok=True)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, please do not unconditionally do this for all targets, it would be better to only create the directory if @PRIVATE_DIR@ appears in its command: kwarg.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, took a different approach where it's only created when replacing @PRIVATE_DIR@. This has the added advantage of working across all backends.

@eli-schwartz
Copy link
Member

Can you squash this down into two commits -- one for the main change and one for the exist_ok cleanup?

@appden appden force-pushed the create-custom-target-private-dir branch from 0067b59 to 69253c3 Compare May 9, 2024 19:32
@appden
Copy link
Author

appden commented May 9, 2024

Force pushed squashed commits

appden added 2 commits May 9, 2024 21:11
Some custom target commands will expect the `@PRIVATE_DIR@` to already exist, such as with `make -C @PRIVATE_DIR@ ...`
@appden appden force-pushed the create-custom-target-private-dir branch from 69253c3 to a1902f7 Compare May 10, 2024 04:11
@tristan957
Copy link
Contributor

Nice work on this.

@appden
Copy link
Author

appden commented May 13, 2024

@eli-schwartz kindly requesting another review, thanks!

@appden
Copy link
Author

appden commented May 23, 2024

Gentle ping for another review, thanks @eli-schwartz @tristan957

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

4 participants