-
Notifications
You must be signed in to change notification settings - Fork 56
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[DRAFT]: Add initial types and unit tests for coercion of tricky inputs using typebox #772
base: devel
Are you sure you want to change the base?
Conversation
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## devel #772 +/- ##
==========================================
+ Coverage 93.30% 93.84% +0.54%
==========================================
Files 27 31 +4
Lines 732 878 +146
Branches 247 286 +39
==========================================
+ Hits 683 824 +141
- Misses 49 54 +5 ☔ View full report in Codecov by Sentry. |
This all looks great, @eddie-atkinson! General comments:
Minor nitpick:
Thanks again, this is all great. Looking forward to following along and very excited to write new code using this in the future... the current system works well but can be quite tedious constantly rebuilding the schema to test things or forgetting to compile/commit on changes `:) So yeah, I'm excited, and big thanks again for all your efforts here 🙏 |
Hi @gadicc
Done.
So my understanding is that the I've added a couple more commits which add a There were also a couple of test cases I dropped from the old Looking forward to hearing more feedback, and I'll start hacking on the perf tests in the meantime |
All looks great as usual, @eddie-atkinson; thanks again!
Ah yes, right you are... didn't even notice that 😅 If it's not included in the npm package there's really not much we can do, but as you say, I guess this was their intention anyway. Definitely fine for now, and if we notice any bugs, we now have the commit hash (thanks) to report upstream or rebase upstream fixes (by hand).
Awesome, thanks! Absolutely fine with the new error reporting. All we did before was use ajv's format... no API promise here, just something the end user can understand - within reason. My only note is that - as you'll see in the existing
The non-obvious reason for this (which I should have mentioned in a comment) is for library usage by non-TS users. That's the only reason why we still need to validate at runtime. And I guess that - unfortunately - means we'll need to use typebox for each module's options too. Sorry 🙈 But step by step... the most important thing is to get the foundations up - which you've done an excellent job of. Both systems can exist side by side for a while and we can convert everything gradually. So, thanks again. I continue to follow with interest and appreciation :) |
await expect(rwo({ invalid: true })).rejects.toThrow(InvalidOptionsError); | ||
}); | ||
|
||
it("accepts empty queryOptions", async () => { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This test is slightly weird because no other module other than search
seems to support this. But I'm not really sure why
src/lib/validateAndCoerceTypes.ts
Outdated
handleResultError(e, options); | ||
break; | ||
} | ||
// TODO: The existing implementation of 'validate' assumes that this parameter may not be provided |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is another weird thing I found. The implementation in validate
uses a bare else
to cover the condition that type !== 'result'
instead of checking for type === 'options'
. I'm guessing this is a catch all case for when someone ignores the TS and passes through completely garbage options.
I have two thoughts on this:
- I feel like this case shouldn't really exist as this function should only be called internally within the library where we can assume TS.
- If we do want to be defensive around this I think it would make more sense to hoist the handling of this out of the
validateAndCoerce
function. Ideally we should be able to do that in the modules themselves and then we can have a clean interface for internal consumption, and catch weird errors where they originate rather than deep in the library.
Just my $0.02. I concede there is probably something going on here that I don't quite grok.
Hi @gadicc, Me again. A few updates on where I'm at. I've added some more functionality for using typebox to validate options and responses, notably For the naming of the typebox files I have adopted In terms of the mechanical process of converting typescript interfaces to Typebox, that part is pretty easy. You can basically copy the entire file wholesale into the tool that the maintainer of Typebox created and copy out the result with two important caveats:
I had a bit of a play around with a performance test without getting very far, generally it seems like the example files in the repo aren't large enough for a meaningful performance difference to manifest. Keen to discuss what the steps are from here, I'm not sure whether we want to merge this PR or split it up a bit to make reviewing more tractable |
Amazing work as usual @eddie-atkinson - big thanks! 🙏 Not worried at all about including both ajv and typebox for now... the small difference in bundle size won't make a big difference outside of the browser. In any event, it won't be long lived. So yeah, current parallel staging looks great. I have some thoughts on best next steps, but I'm just in the middle of some travel, so will need to get back. The code I've reviewed so far all looks great but I'm a few commits behind. Need to get back to you on the code comment above too (you're right, in theory, if it's only used internally, no need to validate at runtime - but I'll double check). Yeah, all the tschema scripts etc, although they're all stable and work great (if you don't change anything), as you saw, it can create become a hassle, so yeah, another reason why I was excited form your proposal and to move to something both clearer and more maintainable :) Thanks also for appeasing the coverage gods! No easy feat :D In short, thanks again, and will be in touch soon! |
@eddie-atkinson, thanks again for your patience. I'm still abroad and have had a lot less time online than anticipated. To that end, I'm very happy where this is all going, and in light of my limited time now, happy for you to take the lead on how best to implement this. My original thoughts were to rebase and squash the commits down into 1) all preliminary support (packages, coersion, etc), 2) commit(s) for module(s). For the latter, I had originally thought it would be nice to deal with a single file, so the diff would show the exact changes to switch from the old way to the new way. But that's not mutually exclusive to the parallel track we've taken here... we can have both the original and So, just let me know how you feel about everything and if there's still more immediate work you'd like to do or if things are ready to merge now and to continue on from there. And most importantly, thanks again for all your exceptional work on this and patience too! 🙏 |
Hi @gadicc,
Enjoy it! There's always more time to sit in front of a computer. In this profession time away from the computer is an essential thing to keeping the fire burning over the long-term.
I'm happy to work it this way. My plan for this PR was to try out a few modules and see how it all hangs together. After having done a few of the more hairy modules I'm confident we can handle all of them. The rough plan I was thinking of was to:
I was keen to make a separate PR for each module to make reviews more tractable as some of the modules are incredibly large. For the smaller ones I'm also happy to combine a few together, I just know
We might be able to have our cake and eat it here. Just move files like so Anyways, happy to work the PR flow however it's going to make it easiest to review. I am conscious that this shouldn't result in breaking changes, but it's hard to be certain the TS interface won't change |
Just a WIP PR to show where I'm at with exploring the use of typebox as a replacement for
ajv
.So far I'm reasonably happy with where this work is at. Typebox seems like it can fit the needs of the project and is well documented and supported.
So far what I've done is:
YahooFinanceNumber
,YahooFinanceDate
,DateInMs
, andTwoNumberRange
Outstanding TODOs:
ajv
validationvalidateAndParse
replacement using the Typebox APIs. This should be pretty straightforward, only potential wrinkle is whether error formatting is sufficiently readable to not require any significant manipulation. I think this will be OK as Typebox appears to just pipe through JSON Schema validation errors which are pretty readable IMO