Example benchmark #96

dantleech · 2016-09-24T09:38:28Z

Just an example benchmark using PHPBench to get things started.

$ phpbench report --uuid=latest --report=hoa
benchmark: RulerBench, subject: benchAssert
+---------------------------------------------------------------+------+-----+------------+---------+---------+---------+--------+
| params                                                        | revs | its | mem_peak   | best    | mean    | mode    | rstdev |
+---------------------------------------------------------------+------+-----+------------+---------+---------+---------+--------+
| {"rule":"group in [\"customer\", \"guest\"] and points > 30"} | 10   | 10  | 2,249,760b | 3.786ms | 4.724ms | 4.123ms | 15.20% |
| {"rule":"points > 30"}                                        | 10   | 10  | 2,200,408b | 1.386ms | 1.836ms | 1.909ms | 17.00% |
| {"rule":"group in [\"customer\", \"guest\"]"}                 | 10   | 10  | 2,216,400b | 2.240ms | 2.781ms | 2.715ms | 15.34% |
+---------------------------------------------------------------+------+-----+------------+---------+---------+---------+--------+``

dantleech · 2016-09-24T09:38:58Z

composer.json

@@ -29,7 +29,8 @@
        "hoa/visitor"    : "~2.0"
    },
    "require-dev": {
-        "hoa/test": "~2.0"
+        "hoa/test": "~2.0",
+        "phpbench/phpbench": "^0.12.0"


This would be moved to hoa/test I suppose.

Totally agree. And we should also introduce a thin layer between PHPBench and Hoa, like we did for atoum, as Hoa\Test\Benchmark\Suite and this class will extend it. Thoughts?

I think that makes sense (you could define f.e. setUp etc there too).

Hywan

Excellent start, thank you! This is good basis for a discussion.

Hywan · 2016-09-26T09:28:00Z

Test/Benchmark/RulerBench.php

+ * @Iterations(10)
+ * @OutputTimeUnit("milliseconds")
+ */
+class RulerBench


I think we can rename RulerBench to Ruler since this is an Benchmark namespace.

Historically this was a mandatory suffix (like Test in PHPUnit), but not since 0.12 so, can do.

Hywan · 2016-09-26T09:32:17Z

Test/Benchmark/RulerBench.php

+     * Executes the method once (warmup) before measuring time.
+     *
+     * @Warmup(1)
+     * @BeforeMethods({"setUpAssert"})


Can you have a default setUp method?

You can have an abstract class which defines a setUp method (and its annotation)

Hywan · 2016-09-26T09:33:00Z

Test/Benchmark/RulerBench.php

+     * @BeforeMethods({"setUpAssert"})
+     * @ParamProviders({"provideRules"})
+     */
+    public function benchAssert($params)


Can we rename the method, like case_assert? How do you detect a method representing a benchmark?

You can, but then you need to use the @Bench annotation.

So there is no way to say: “All methods matching this pattern are of kind bench”?

See phpbench/phpbench#449

It sounds pretty cool!

Hywan · 2016-09-26T09:34:10Z

composer.json

@@ -29,7 +29,8 @@
        "hoa/visitor"    : "~2.0"
    },
    "require-dev": {
-        "hoa/test": "~2.0"
+        "hoa/test": "~2.0",
+        "phpbench/phpbench": "^0.12.0"


Totally agree. And we should also introduce a thin layer between PHPBench and Hoa, like we did for atoum, as Hoa\Test\Benchmark\Suite and this class will extend it. Thoughts?

Hywan · 2016-09-26T09:36:50Z

phpbench.json

@@ -0,0 +1,11 @@
+{


Can we move this file in a single place, like in Hoa\Test, it will avoid to have it inside each library?
Note that we will probably create the hoa test:bench or update the hoa test:run command, and thus I suppose we could link this phpbench.json file with an argument. We already do this with hoa test:run where we already compute the atoum command (https://github.com/hoaproject/Test/blob/f80f083a0d1cc7a044fdbca10ea46cdcdbef4c38/Bin/Run.php#L291-L326) with all the bootstrap file, configuration file etc.

The issue would be if you wanted to define f.e. a custom report.

However, most, if not all, of the configuration options can be specified at the CLI, so if you wrap it in hoa test:bench then we can ommit the phpbench.json file and include it only when the library has specific requirements.

We could also add a Hoa phpbench extension which programatically adds reports if required.

Hmm, sounds interesting. Can you tell me more about it please?

The Extension instances for PHPBench are like modules for the DI container. It should be possible to add new configurations and ~~overwrite existing ones~~ actually this is disallowed currently.

The extensions can be enabled via. the command line or in phpbench.json as with the example

Would you like to give it a try? Please, feel free to say no!

dantleech · 2016-09-26T13:04:32Z

Not currently, sounds like something that could easily be configured
though (i.e. assert_ instead of bench)

On Mon, Sep 26, 2016 at 05:20:18AM -0700, Ivan Enderlin wrote:

@Hywan commented on this pull request.

In [1]Test/Benchmark/RulerBench.php:
       ],
       [
           'rule' => 'points > 30'
       ],
   ];
}
/**
\* Benchmark parsing with rules of different complexity.
\* Executes the method once (warmup) before measuring time.
*
\* @Warmup(1)
\* @BeforeMethods({"setUpAssert"})
\* @ParamProviders({"provideRules"})
*/
public function benchAssert($params)
So there is no way to say: “All methods matching this pattern are of kind
bench”?

—
You are receiving this because you authored the thread.
Reply to this email directly, [2]view it on GitHub, or [3]mute the thread.

Reverse link: [4]unknown

References

Visible links

Example benchmark #96

Example benchmark #96

https://github.com/notifications/unsubscribe-auth/AAgZcQIOyBIeqWioLjZfmtK-jRt2hSjuks5qt7iCgaJpZM4KFmfz

Example benchmark #96

Hywan · 2016-09-26T13:05:21Z

It could be a good idea and a good feature!

Hywan · 2016-10-14T07:41:56Z

@dantleech Am I a despot if I ask you to update this PR based on PHPBench 0.12.2?

dantleech · 2016-10-14T08:40:38Z

@Hywan no problem, updated :)

Example benchmark

e19cf84

dantleech commented Sep 24, 2016

View reviewed changes

dantleech mentioned this pull request Sep 24, 2016

PHPBench integration hoaproject/Test#61

Open

Hywan reviewed Sep 26, 2016

View reviewed changes

Hywan added enhancement in progress difficulty: hard labels Sep 26, 2016

Hywan self-assigned this Sep 26, 2016

Updated to PHPBench 0.12.2

9c32bca

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Example benchmark #96

Example benchmark #96

dantleech commented Sep 24, 2016 •

edited

dantleech Sep 24, 2016

Hywan Sep 26, 2016

dantleech Sep 26, 2016 •

edited

Hywan left a comment

Hywan Sep 26, 2016

dantleech Sep 26, 2016

Hywan Sep 26, 2016

dantleech Sep 26, 2016

Hywan Sep 26, 2016

dantleech Sep 26, 2016

Hywan Sep 26, 2016

dantleech Oct 2, 2016

Hywan Oct 3, 2016

Hywan Sep 26, 2016

Hywan Sep 26, 2016

dantleech Sep 26, 2016

dantleech Sep 26, 2016

Hywan Sep 26, 2016

dantleech Sep 26, 2016

Hywan Sep 26, 2016 •

edited

dantleech commented Sep 26, 2016

Hywan commented Sep 26, 2016

Hywan commented Oct 14, 2016

dantleech commented Oct 14, 2016

Example benchmark #96

Are you sure you want to change the base?

Example benchmark #96

Conversation

dantleech commented Sep 24, 2016 • edited

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dantleech Sep 26, 2016 • edited

Choose a reason for hiding this comment

Hywan left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Hywan Sep 26, 2016 • edited

Choose a reason for hiding this comment

dantleech commented Sep 26, 2016

Hywan commented Sep 26, 2016

Hywan commented Oct 14, 2016

dantleech commented Oct 14, 2016

dantleech commented Sep 24, 2016 •

edited

dantleech Sep 26, 2016 •

edited

Hywan Sep 26, 2016 •

edited