Extract test infrastructure into a separate package?

[gvanrossum]

A friend of mine was trying to contribute to mypy and discovered our testing infrastructure, in particular the rather nice DataDrivenTestCase and support around it (esp. diffing expected output).

Maybe it would be easy to extract this into a separate 3rd party package, so others can benefit from our gradual refinement of this code?

[warsaw]

I'm that friend, and thanks @gvanrossum - this would be very cool. I can imagine it would make things like importlib_resources and importlib_metadata test suites much nicer.

[msullivan]

This is a good idea in principle, but the implementation of DataDrivenTestCase and friends is all pretty ad-hoc and specialized to mypy and would need some substantial rework in order to generalize it.

But it's not that much code, so a "substantial" rework might not be that hard.

[JukkaL]

I like this idea. The extracted package could be useful to third party stub authors and mypy plugin authors as well -- and this is not the first request of this kind. I guess it's mostly a matter of finding the time to do this.

[brandtbucher]

I’ve been working on extracting this as cleanly as possible (in order to preserve the commit history), and playing around with what should stay in mypy and what should be vendored through the new package.

I should have something up for review in a few weeks!