Migrate `run-make/pgo-branch-weights` to `rmake` #125165

Oneirical · 2024-05-15T20:01:22Z

Part of #121876 and the associated Google Summer of Code project.

This is a scary one and I expect things to break. Set as draft, because this isn't ready.

There is this comment here, which suggests the test is excluded from the testing process due to a platform specific issue? I can't see anything here that would cause this test to not run...

// FIXME(mati865): MinGW GCC miscompiles compiler-rt profiling library but with Clang it works
// properly. Since we only have GCC on the CI ignore the test for now."

EDIT: This is specific to Windows-gnu.

The Makefile has this line:

ifneq (,$(findstring x86,$(TARGET)))
COMMON_FLAGS=-Clink-args=-fuse-ld=gold

I honestly can't tell whether this is checking if the target IS x86, or IS NOT. EDIT: It's checking if it IS x86.

I don't know why the Makefile was trying to pass an argument directly in the Makefile instead of setting that "aaaaaaaaaaaa2bbbbbbbbbbbb2bbbbbbbbbbbbbbbbcc" input as a variable in the Rust program directly. I changed that, let me know if that was wrong.
Trying to rewrite cat "$(TMPDIR)/interesting.ll" | "$(LLVM_FILECHECK)" filecheck-patterns.txt resulted in some butchery. For starters, in tools.mk, LLVM_FILECHECK corrects its own backslashes on Windows distributions, but there is no further mention of it, so I assume this is a preset environment variable... but is it really? Then, the command itself uses a Standard Input and a passed input file as an argument simultaneously, according to the documentation.

try-job: aarch64-gnu

rustbot · 2024-05-15T20:01:30Z

r? @jieyouxu

rustbot has assigned @jieyouxu.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

lqd · 2024-05-15T20:15:29Z

There is this comment here, which suggests the test is excluded from the testing process due to a platform specific issue? I can't see anything here that would cause this test to not run...

// FIXME(mati865): MinGW GCC miscompiles compiler-rt profiling library but with Clang it works
// properly. Since we only have GCC on the CI ignore the test for now."

it's the # ignore-windows-gnu in the makefile, which is where this mingw gcc bug is expected to happen, according to the comment

jieyouxu · 2024-05-15T20:19:16Z

Yeah, run-make Makefiles have # <directive-name> directives. What's funny about them like the old-format // <directive-name> directive in Rust code is that you can sneak in a directive in a comment block, e.g.

# This is a comment block that has a starting line,
# ignore-windows in the middle of the sentence,
# and hello world in the end of the sentence.

Here # ignore-windows is a (Makefile) directive.

Oneirical · 2024-05-15T20:23:54Z

Ah, I was wondering if that comment meant the test did not run at all... I am equally soothed that my work on this one was not wasted, and worried that this means it will have to be fully understood to proceed...

It does use features found in other tests, though, so this should make the other tests using LLVM instrumentation much easier to port.

EDIT: That does mean that the LLVM_FILECHECK part that fixes Windows path to use slashes won't have to be ported over in this test, since Windows is already getting ignored. Not sure if that mattered in the first place.

EDIT 2: Not Windows, Windows-gnu.

lqd · 2024-05-15T20:35:18Z

since Windows is already getting ignored

To clarify, windows-gnu is ignored, not windows-msvc (the main windows target), or the other windows targets.

tests/run-make/pgo-branch-weights/rmake.rs

tests/run-make/pgo-branch-weights/main.rs

jieyouxu · 2024-05-16T19:43:55Z

I'll take a closer took tomorrow

src/tools/run-make-support/src/run.rs

tests/run-make/pgo-branch-weights/rmake.rs

jieyouxu · 2024-05-18T00:04:27Z

Feel free to ping me if you need any PR CI runs (i.e. try jobs) for various platforms.

Oneirical · 2024-05-21T16:44:21Z

Feel free to ping me if you need any PR CI runs (i.e. try jobs) for various platforms.

@jieyouxu I might need some try jobs for windows-gnu (once CI is green). The description has been changed already.

Oneirical · 2024-06-04T18:44:09Z

The fact that there were two try jobs simultaneously (to the point where two job fail notifications were printed at once) is suspicious, not to mention how strange the error message is. I am restarting the try job just in case it was only caused by bors being silly.

@bors try

bors · 2024-06-04T18:45:24Z

⌛ Trying commit ba842c4 with merge d43c70b...

Migrate `run-make/pgo-branch-weights` to `rmake` Part of rust-lang#121876 and the associated [Google Summer of Code project](https://blog.rust-lang.org/2024/05/01/gsoc-2024-selected-projects.html). This is a scary one and I expect things to break. Set as draft, because this isn't ready. - [x] There is this comment here, which suggests the test is excluded from the testing process due to a platform specific issue? I can't see anything here that would cause this test to not run... > // FIXME(mati865): MinGW GCC miscompiles compiler-rt profiling library but with Clang it works // properly. Since we only have GCC on the CI ignore the test for now." EDIT: This is specific to Windows-gnu. - [x] The Makefile has this line: ``` ifneq (,$(findstring x86,$(TARGET))) COMMON_FLAGS=-Clink-args=-fuse-ld=gold ``` I honestly can't tell whether this is checking if the target IS x86, or IS NOT. EDIT: It's checking if it IS x86. - [x] I don't know why the Makefile was trying to pass an argument directly in the Makefile instead of setting that "aaaaaaaaaaaa2bbbbbbbbbbbb2bbbbbbbbbbbbbbbbcc" input as a variable in the Rust program directly. I changed that, let me know if that was wrong. - [x] Trying to rewrite `cat "$(TMPDIR)/interesting.ll" | "$(LLVM_FILECHECK)" filecheck-patterns.txt` resulted in some butchery. For starters, in `tools.mk`, LLVM_FILECHECK corrects its own backslashes on Windows distributions, but there is no further mention of it, so I assume this is a preset environment variable... but is it really? Then, the command itself uses a Standard Input and a passed input file as an argument simultaneously, according to the [documentation](https://llvm.org/docs/CommandGuide/FileCheck.html#synopsis). try-job: aarch64-gnu

bors · 2024-06-04T23:00:24Z

💥 Test timed out

bors · 2024-06-04T23:51:08Z

☔ The latest upstream changes (presumably #125989) made this pull request unmergeable. Please resolve the merge conflicts.

jieyouxu · 2024-06-05T04:51:26Z

This PR's associated CI runs are kinda cursed

jieyouxu · 2024-06-05T04:52:13Z

We could give it another try after fixing the merge conflict.
@rustbot author

Oneirical · 2024-06-05T14:10:00Z

@rustbot review

Conflict resolved, though, while running around and removing conflicts, I noticed that there used to be a silent fail here:

let llvm_bin_dir = env::var("LLVM_BIN_DIR");

This is really strange, since I would have expected this to cause a compilation error from the unexpected Result type then getting passed around.

This could explain why the test job is getting stuck. Maybe we'll get a different error message this time.

Oneirical · 2024-06-05T15:18:58Z

@bors try

Migrate `run-make/pgo-branch-weights` to `rmake` Part of rust-lang#121876 and the associated [Google Summer of Code project](https://blog.rust-lang.org/2024/05/01/gsoc-2024-selected-projects.html). This is a scary one and I expect things to break. Set as draft, because this isn't ready. - [x] There is this comment here, which suggests the test is excluded from the testing process due to a platform specific issue? I can't see anything here that would cause this test to not run... > // FIXME(mati865): MinGW GCC miscompiles compiler-rt profiling library but with Clang it works // properly. Since we only have GCC on the CI ignore the test for now." EDIT: This is specific to Windows-gnu. - [x] The Makefile has this line: ``` ifneq (,$(findstring x86,$(TARGET))) COMMON_FLAGS=-Clink-args=-fuse-ld=gold ``` I honestly can't tell whether this is checking if the target IS x86, or IS NOT. EDIT: It's checking if it IS x86. - [x] I don't know why the Makefile was trying to pass an argument directly in the Makefile instead of setting that "aaaaaaaaaaaa2bbbbbbbbbbbb2bbbbbbbbbbbbbbbbcc" input as a variable in the Rust program directly. I changed that, let me know if that was wrong. - [x] Trying to rewrite `cat "$(TMPDIR)/interesting.ll" | "$(LLVM_FILECHECK)" filecheck-patterns.txt` resulted in some butchery. For starters, in `tools.mk`, LLVM_FILECHECK corrects its own backslashes on Windows distributions, but there is no further mention of it, so I assume this is a preset environment variable... but is it really? Then, the command itself uses a Standard Input and a passed input file as an argument simultaneously, according to the [documentation](https://llvm.org/docs/CommandGuide/FileCheck.html#synopsis). try-job: aarch64-gnu

bors · 2024-06-05T15:20:10Z

⌛ Trying commit fdd3967 with merge 30cb27a...

rust-log-analyzer · 2024-06-05T16:24:22Z

The job aarch64-gnu failed! Check out the build log: (web) (plain)

Click to see the possible cause of the failure (guessed by this bot)

file:.git/config remote.origin.url=https://github.com/rust-lang-ci/rust
file:.git/config remote.origin.fetch=+refs/heads/*:refs/remotes/origin/*
file:.git/config gc.auto=0
file:.git/config http.https://github.com/.extraheader=AUTHORIZATION: basic ***
file:.git/config branch.try.remote=origin
file:.git/config branch.try.merge=refs/heads/try
file:.git/config submodule.library/backtrace.url=https://github.com/rust-lang/backtrace-rs.git
file:.git/config submodule.library/stdarch.active=true
file:.git/config submodule.library/stdarch.url=https://github.com/rust-lang/stdarch.git
file:.git/config submodule.src/doc/book.active=true
---
failures:

---- [run-make] tests/run-make/pgo-branch-weights stdout ----

error: run-make test failed: could not build `rmake.rs` recipe
status: exit status: 1
command: env -u CARGO_MAKEFLAGS -u MAKEFLAGS -u MFLAGS HOST_RPATH_DIR="/checkout/obj/build/aarch64-unknown-linux-gnu/stage2/lib" LD_LIBRARY_PATH="/checkout/obj/build/aarch64-unknown-linux-gnu/stage2/lib:/checkout/obj/build/aarch64-unknown-linux-gnu/stage0-bootstrap-tools/aarch64-unknown-linux-gnu/release/deps:/checkout/obj/build/aarch64-unknown-linux-gnu/stage0/lib" LD_LIB_PATH_ENVVAR="LD_LIBRARY_PATH" LLVM_COMPONENTS="aarch64 aarch64asmparser aarch64codegen aarch64desc aarch64disassembler aarch64info aarch64utils aggressiveinstcombine all all-targets analysis arm armasmparser armcodegen armdesc armdisassembler arminfo armutils asmparser asmprinter avr avrasmparser avrcodegen avrdesc avrdisassembler avrinfo binaryformat bitreader bitstreamreader bitwriter bpf bpfasmparser bpfcodegen bpfdesc bpfdisassembler bpfinfo cfguard codegen codegentypes core coroutines coverage csky cskyasmparser cskycodegen cskydesc cskydisassembler cskyinfo debuginfobtf debuginfocodeview debuginfodwarf debuginfogsym debuginfologicalview debuginfomsf debuginfopdb demangle dlltooldriver dwarflinker dwarflinkerclassic dwarflinkerparallel dwp engine executionengine extensions filecheck frontenddriver frontendhlsl frontendoffloading frontendopenacc frontendopenmp fuzzercli fuzzmutate globalisel hexagon hexagonasmparser hexagoncodegen hexagondesc hexagondisassembler hexagoninfo hipstdpar instcombine instrumentation interfacestub interpreter ipo irprinter irreader jitlink libdriver lineeditor linker loongarch loongarchasmparser loongarchcodegen loongarchdesc loongarchdisassembler loongarchinfo lto m68k m68kasmparser m68kcodegen m68kdesc m68kdisassembler m68kinfo mc mca mcdisassembler mcjit mcparser mips mipsasmparser mipscodegen mipsdesc mipsdisassembler mipsinfo mirparser msp430 msp430asmparser msp430codegen msp430desc msp430disassembler msp430info native nativecodegen nvptx nvptxcodegen nvptxdesc nvptxinfo objcarcopts objcopy object objectyaml option orcdebugging orcjit orcshared orctargetprocess passes powerpc powerpcasmparser powerpccodegen powerpcdesc powerpcdisassembler powerpcinfo profiledata remarks riscv riscvasmparser riscvcodegen riscvdesc riscvdisassembler riscvinfo riscvtargetmca runtimedyld scalaropts selectiondag sparc sparcasmparser sparccodegen sparcdesc sparcdisassembler sparcinfo support symbolize systemz systemzasmparser systemzcodegen systemzdesc systemzdisassembler systemzinfo tablegen target targetparser textapi textapibinaryreader transformutils vectorize webassembly webassemblyasmparser webassemblycodegen webassemblydesc webassemblydisassembler webassemblyinfo webassemblyutils windowsdriver windowsmanifest x86 x86asmparser x86codegen x86desc x86disassembler x86info x86targetmca xray" PYTHON="/usr/bin/python3" RUSTC="/checkout/obj/build/aarch64-unknown-linux-gnu/stage2/bin/rustc" RUST_BUILD_STAGE="stage2-aarch64-unknown-linux-gnu" S="/checkout" TARGET="aarch64-unknown-linux-gnu" TARGET_RPATH_DIR="/checkout/obj/build/aarch64-unknown-linux-gnu/stage2/lib/rustlib/aarch64-unknown-linux-gnu/lib" TMPDIR="/checkout/obj/build/aarch64-unknown-linux-gnu/test/run-make/pgo-branch-weights/pgo-branch-weights/rmake_out" "/checkout/obj/build/aarch64-unknown-linux-gnu/stage2/bin/rustc" "-o" "/checkout/obj/build/aarch64-unknown-linux-gnu/test/run-make/pgo-branch-weights/pgo-branch-weights/rmake" "-Ldependency=/checkout/obj/build/aarch64-unknown-linux-gnu/stage2-tools-bin" "-Ldependency=/checkout/obj/build/aarch64-unknown-linux-gnu/stage2-tools/aarch64-unknown-linux-gnu/release/deps" "-Ldependency=/checkout/obj/build/aarch64-unknown-linux-gnu/stage2-tools/release/deps" "--extern" "run_make_support=/checkout/obj/build/aarch64-unknown-linux-gnu/stage2-tools-bin/librun_make_support.rlib" "--edition=2021" "/checkout/tests/run-make/pgo-branch-weights/rmake.rs"
--- stderr -------------------------------
warning: unused imports: `rustdoc` and `target`
##[warning]  --> /checkout/tests/run-make/pgo-branch-weights/rmake.rs:17:58
   |
---

error[E0599]: no method named `run` found for mutable reference `&mut LlvmProfdata` in the current scope
##[error]  --> /checkout/tests/run-make/pgo-branch-weights/rmake.rs:34:85
   |
34 |     llvm_profdata().merge().output(&path_merged_profdata).input(path_prof_data_dir).run();
   |                                                                                     ^^^ method not found in `&mut LlvmProfdata`
error[E0599]: no method named `run` found for mutable reference `&mut LlvmFilecheck` in the current scope
##[error]  --> /checkout/tests/run-make/pgo-branch-weights/rmake.rs:43:79
   |
   |
43 |     llvm_filecheck().patterns("filecheck-patterns.txt").stdin(interesting_ll).run();
   |                                                                               ^^^ method not found in `&mut LlvmFilecheck`
error: aborting due to 2 previous errors; 1 warning emitted

For more information about this error, try `rustc --explain E0599`.
------------------------------------------

bors · 2024-06-05T16:24:42Z

💔 Test failed - checks-actions

rustbot assigned jieyouxu May 15, 2024

rustbot added A-testsuite Area: The testsuite used to check the correctness of rustc S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-bootstrap Relevant to the bootstrap subteam: Rust's build system (x.py and src/bootstrap) labels May 15, 2024