/json benchmark is overly simplistic

[RobertHenry6bev]

The json returned from /JSON test is an encoding of a single dictionary, with a single string-valued key and value. That's not a lot of work to do, so most of the cycles will be spent with headers and meta data, much like many other of the workloads. There's no chance to show off clever json scanners that use vector instructions.

I would like to see the test both read and write more representative JSON blobs, including:

• long lists
• dictionaries with many keys
• nested dictionaries
• utf-8 dictionary keys and values
• Many numeric values.

1. The numbers should be floats that convert without rounding to small integers;
2. to larger integers < 2^53
3. to a wide range of floats that exercise denorms, negative 0, and values > 2^53. (Of course, legacy JSON doesn't let you directly encode NaN and Inf, and it would be nice if those could be there too.)

[franz1981]

Although I am great fan of vaectorized (and SWAR) approaches, I am not sure it is in the spirit of the benchmark: most frameworks relies on the same or similar JSON encoders and if the cost of the encoding become dominant, the framework must be very inefficient in order to still matter...i
Which mean the test to become a rather poor framework benchmark, but more a JSON encoder micro-benchmark in disguise...
That's based on my personal interpretation of what is the spirit of techempower, meaning that I can be very wrong too!