Tests with many snapshots are sorted 1, 10, 11, ..., 2, 20, ..., 3, 4, 5, ..., not numeric order

[huonw]

Describe the bug

Snapshots are stored sorted in the .ambr, but this seems to be doing a string comparison on the index, meaning 10 sorts before 2, etc.

To reproduce

# test_many.py
def test_many(snapshot):
    for i in range(25):
        assert i == snapshot

Run pytest --snapshot-update test_many.py:

Resulting __snapshots__/test_many.ambr:

# name: test_many
  0
# ---
# name: test_many.1
  1
# ---
# name: test_many.10
  10
# ---
# name: test_many.11
  11
# ---
# name: test_many.12
  12
# ---
# name: test_many.13
  13
# ---
# name: test_many.14
  14
# ---
# name: test_many.15
  15
# ---
# name: test_many.16
  16
# ---
# name: test_many.17
  17
# ---
# name: test_many.18
  18
# ---
# name: test_many.19
  19
# ---
# name: test_many.2
  2
# ---
# name: test_many.20
  20
# ---
# name: test_many.21
  21
# ---
# name: test_many.22
  22
# ---
# name: test_many.23
  23
# ---
# name: test_many.24
  24
# ---
# name: test_many.3
  3
# ---
# name: test_many.4
  4
# ---
# name: test_many.5
  5
# ---
# name: test_many.6
  6
# ---
# name: test_many.7
  7
# ---
# name: test_many.8
  8
# ---
# name: test_many.9
  9
# ---

Expected behavior

Snapshots should be sorted in numeric order 1, 2, ..., 9, 10, ..., 20, ..., 24.

Some options:

• formatting the number with leading zeros. If it's possible the leading zeros could be computed dynamically based on how many snapshots are actually used in each test, otherwise could be hard-coded (e.g. 01, 02, ... 09, 10, ... would ensure natural sorting up to 100 snapshots in one test)
• not doing string comparison, e.g. Store snapshot assertion index in snapshot file  #569 might be related.

Screenshots

N/A

Environment (please complete the following information):

• OS: macOS
• Syrupy Version: 3.0.5
• Python Version: 3.9

Additional context

Thank you for syrupy!

[noahnu]

I like the idea of implementing #569 first, removing the index from the name field and then sorting by (name, index).

[noahnu]

I fixed this in the v4 PR here (not released yet): 4ca0716 however my solution is causing performance issues. I suspect because of the nested tuples in my solution. Will need to improve the performance or come up with another approach before releasing syrupy v4.

I suppose we could try sort only on the text after the last .

[noahnu]

So as not to hold up the v4 release, I'm not making this the default behaviour, however it can be accomplished via a custom extension with a couple lines of code: https://github.com/tophat/syrupy/blob/f416e9a408bc252fdeab1888cb8b7f0cd8fdef92/tests/syrupy/extensions/amber_sorted/test_amber_sort.py

[tophat-opensource-bot]

🎉 This issue has been resolved in version 4.0.0 🎉

The release is available on:

• v4.0.0
• GitHub release

Your semantic-release bot 📦🚀

[huonw]

A neat solution 👍 Thank you!