Replies: 5 comments
-
Hi @bnewbold! That's a brilliant idea! To be honest, I invented something similar for my unit tests. I have a data collector: https://github.com/MarshalX/atproto/blob/main/tests/models/fetch_test_data.py As you can see, the most problematic edge cases are about custom lexicon and parsing. For example parsing of literals and Union types. It will be awesome to have only one database with the whole test data! The test data for the CAR file will be useful for my python-libipld project. Some bugs that I have with parsing and it's covered by unit tests now:
Also, I am confused about this and idk is it a reproducible thing. Any Union type could be any object with $type and fields: Code ref: https://github.com/bluesky-social/atproto/blob/b01e47b61730d05a780f7a42667b91ccaa192e8e/packages/lex-cli/src/codegen/lex-gen.ts#L325. It will be awesome to cover this case with the test data too. |
Beta Was this translation helpful? Give feedback.
-
Handling unspecified extensions to existing Lexicons is not very flushed out, both in our implementations and in the specifications. It should all come together at the protocol level, but how to expose things well in other languages, especially strongly typed, isn't totally clear. And there are sharp edges around decoding/recording records. Likewise what it looks like to work with records (or other data, in JSON or DAG-CBOR) for which a Lexicon is not available. There are restrictions on what that data can look like: records are always supposed to have We do want it to be possible to parse through data with an unknown schema and identity things like blobs. Moderation tooling can use this to do things like extract and label blobs even if the application lexicon isn't known. When we added self-labels recently, we also did that in a way to ensure that the For unions, yes, open unions are very flexible. Implementations are only really expected to handle the enumerated (known) types, and just "not fail" when they encounter an unknown type in that position. This is distinct from closed unions, where an unknown type would be an error. IIRC there might be ambiguity about whether literals are allowed in unions, because there is no |
Beta Was this translation helpful? Give feedback.
-
Had not seen Thanks for these notes, will review and fold this in to our test vectors. |
Beta Was this translation helpful? Give feedback.
-
@bnewbold thank you for your explanation!
Pure Python implementation exists but is abandoned and slow as hell. That's why I moved to Rust. Pls check the recent performance boost update: https://github.com/MarshalX/atproto/releases/tag/v0.0.26 |
Beta Was this translation helpful? Give feedback.
-
|
Beta Was this translation helpful? Give feedback.
-
Hey @MarshalX (and others contributing to this repo)!
Wanted to share a resource and see if you had any feedback. We have started collecting some cross-language test vectors at: https://github.com/bluesky-social/atproto/tree/main/interop-test-files
For example, long lists of valid and invalid identifiers, trying to hit corner cases. These are intended to be easy to copy directly in to other implementations. So far they are used in both the typescript code and golang implementation (indigo).
Would be curious what other test files would be helpful for ensuring inter-operation between implementations, particularly if there have been any sharp edges that you have run in to.
We will probably add:
app.bsky
record Lexicons, both valid and invalidBeta Was this translation helpful? Give feedback.
All reactions