Output of failing property is confused by comments between signature and definition

[brprice]

With certain comment placement, the output of a failing test can be rather confusing, with failures and annotations ascribed to the wrong source lines.

The bug

Consider the file

import Hedgehog
import Hedgehog.Main

prop_bug :: Property
-- This comment moves all the test output (annotations and diffs and failure indication) down by a line.
-- (Well, two now.) The line numbers in the output are attached to the wrong code, but the right output.
-- (Ok, three.) Note that this comment does not appear in the source printout on error
prop_bug = property $ do
-- Comments here (Line 9) are not problematic, and show up in the source printout on error
  annotate "Line 10"
  -- Here (Line 11) is also fine
  i <- forAll $ pure 0
  i === 1
  success
  success

main :: IO ()
main = defaultMain [check prop_bug]

which has comments between prop_bug :: Property and prop_bug = ....

The output of this failing test attaches annotations and failures to the wrong source lines. It also attaches the line numbers incorrectly (the annotations etc are attached to the correct line number, but the wrong source code is printed).

The output is (commentary below)

  ✗ <interactive> failed at Main.hs:13:3
    after 1 test.
  
       ┏━━ Main.hs ━━━
     4 ┃ prop_bug :: Property
     5 ┃ prop_bug = property $ do
     6 ┃ -- Comments here (Line 9) are not problematic, and show up in the source printout on error
     7 ┃   annotate "Line 10"
     8 ┃   -- Here (Line 11) is also fine
     9 ┃   i <- forAll $ pure 0
    10 ┃   i === 1
       ┃   │ Line 10
    11 ┃   success
    12 ┃   success
       ┃   │ 0
  
    ━━━ Failed (- lhs) (+ rhs) ━━━
    - 0
    + 1
  
    This failure can be reproduced by running:
    > recheck (Size 0) (Seed 13577958291790835144 10236192217858061691) <property>

Commentary

In the output we see the source code of the failing property, with attached line numbers.
It starts fine, with the first line correctly numbered

  ✗ <interactive> failed at Main.hs:13:3
    after 1 test.
  
       ┏━━ Main.hs ━━━
     4 ┃ prop_bug :: Property

But after that the line numbers do not correspond to the source code printed:

     5 ┃ prop_bug = property $ do

is the next line of output. We have skipped the three comment lines, but not updated the line numbering correctly. So far this is a minor annoyance, but things are made worse because only the source code is shifted - other output such an annotations are not. Thus we continue

     6 ┃ -- Comments here (Line 9) are not problematic, and show up in the source printout on error
     7 ┃   annotate "Line 10"
     8 ┃   -- Here (Line 11) is also fine

Notice that hedgehog has not (yet) displayed anything for this annotation on line (output number)7/(source line)10, as it is attached to output line number 10: 3 lines further down!
Continuing, we see the same shifted output problem for the forAll

     9 ┃   i <- forAll $ pure 0
    10 ┃   i === 1
       ┃   │ Line 10

Now the annotate output shows up, confusingly delayed. The test i === 1 fails, but this has not been reported yet.

    11 ┃   success
    12 ┃   success
       ┃   │ 0
  
    ━━━ Failed (- lhs) (+ rhs) ━━━
    - 0
    + 1

Finally, we drop off the end and report the failure, which is not ascribed to any particular line (because it has been "shifted off the end").

For more confusion, if we had an extra success line at the end, we would see

    13 ┃   success
       ┃   ^^^^^^^
       ┃   │ ━━━ Failed (- lhs) (+ rhs) ━━━
       ┃   │ - 0
       ┃   │ + 1

which apparently blames a trivial test for our earlier failure.

Workaround

Since this bug is triggered by comments between signatures and definitions:

prop_bug :: Property
-- I.e. comments here
prop_bug = property $ do

it is easy to avoid by writing such comments elsewhere

-- Here is fine
prop_bug :: Property
prop_bug = property $ do
-- Here is also fine