Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

how do you use benchmark::RegisterBenchmark but with a fixtured benchmark that was previously defined using BENCHMARK_DEFINE_F(FixtureClass, MethodName)[BUG] #1694

Open
ahmednofal opened this issue Nov 7, 2023 · 5 comments

Comments

@ahmednofal
Copy link

Describe the bug
how do you use benchmark::RegisterBenchmark but with a fixtured benchmark that was previously defined using BENCHMARK_DEFINE_F(FixtureClass, MethodName)
whenever I use the MethodName in benchmark::RegisterBenchmark I get a compilation error
System
Which OS, compiler, and compiler version are you using:

  • OS: linux
  • Compiler and version: gcc 9.4

To reproduce
Steps to reproduce the behavior:

  1. define fixture benchmark using : BENCHMARK_DEFINE_F(FixtureClass, MethodName)
  2. use the MethodName in benchmark::RegisterBenchmark

Expected behavior
No compilation error
it gives me : undefined reference error to MethodName

@ahmednofal ahmednofal changed the title [BUG] how do you use benchmark::RegisterBenchmark but with a fixtured benchmark that was previously defined using BENCHMARK_DEFINE_F(FixtureClass, MethodName)[BUG] Nov 7, 2023
@dmah42
Copy link
Member

dmah42 commented Nov 7, 2023

benchmarks using fixtures expect to be registered using BENCHMARK_REGISTER_F.

you could probably go through the macros and figure out what the callable ends up being. ie, BENCHMARK_REGISTER_F calls RegisterBenchmarkInternal(new FixtureClass_MethodName_Method()) essentially, so something like RegisterBenchmark("my_bm", FixtureClass_MethodName_Method, args) might get you close... but it's probably easier to stick with the macro registration.

@ahmednofal
Copy link
Author

I wish I could do it using the macro, but I need to take in arguments from the user and configure the Args of the benchmark with those inputs programmatically. I tried your approach, but I am not sure I am doing it correctly, can you please give a full code example, if it is not too much trouble.

Thank you very much for response.

@dmah42
Copy link
Member

dmah42 commented Nov 8, 2023

can you share what you've tried?

@ahmednofal
Copy link
Author

ahmednofal commented Nov 8, 2023

Of course, sorry for not posting it at first.

BENCHMARK_DEFINE_F(FixtureClassName, BM_Func)
(::benchmark::State& state) {
  for (auto _ : state) {
     [..... bench logic here .....]
  }
}

int main(int argc, char** argv){
  // Some arg parsing logic here (verified to work just fine)
  benchmark::Initialize(&rem_argv_len, rem_argv_c.data());
  benchmark::RegisterBenchmark("SomeArbitraryName", FixtureClassName_BM_Func_Benchmark)->Arg(0); // checked what the macro resolved to and it puts "_Benchmark" at the end and passing here dummy 0 in Arg but it represents what I would like to do
  benchmark::RunSpecifiedBenchmarks();
  benchmark::Shutdown();
}

This does not compile.
Thank you greatly for responding :)

@dmah42
Copy link
Member

dmah42 commented Nov 8, 2023

working on this as i get bits of time. i'll post progress here as maybe you can figure it out before i do.

the preprocessor output includes

class FixtureClass : public benchmark::Fixture {
 public:
  void SetUp(const ::benchmark::State& state) override {}
  void TearDown(const ::benchmark::State& state) override {}
};

class FixtureClass_BM_Func_Benchmark : public FixtureClass {
 public:
  FixtureClass_BM_Func_Benchmark() { this->SetName("FixtureClass" "/" "BM_Func"); }
 protected:
  void BenchmarkCase(::benchmark::State&)         override; };
  void FixtureClass_BM_Func_Benchmark::BenchmarkCase(::benchmark::State& state) {
    int i = 0;
    for (auto _ : state) {
      benchmark::DoNotOptimize(i += 1);
    }
  }

static ::benchmark::internal::Benchmark* benchmark_uniq_2FixtureClass_BM_Func_Benchmark __attribute__((unused)) =
  (::benchmark::internal::RegisterBenchmarkInternal(new FixtureClass_BM_Func_Benchmark()))->Arg(0);

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants