Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unit Testing framework #7474

Open
vpenades opened this issue Mar 2, 2021 · 7 comments
Open

Unit Testing framework #7474

vpenades opened this issue Mar 2, 2021 · 7 comments
Labels
Help Wanted Testing Related to unit tests or needs unit tests added

Comments

@vpenades
Copy link
Contributor

vpenades commented Mar 2, 2021

Right now, MonoGame has some projects intended for unit testing the framework. These projects contain utility classes
to help test different aspects of MonoGame.

I think it could be useful to move these utility classes to its own project, so we could use it to create our own monogame based
unit tests.

For example, all the "runner" architecture is very handy to test if a given code has been able to render a specific screenshot.

So, a project called MonoGame.Framework.NUnit or something like that would be handy.

What version of MonoGame does the bug occur on:

  • MonoGame 3.8

What operating system are you using:

  • Windows

What MonoGame platform are you using:

  • DesktopGL
@Jjagg Jjagg added Testing Related to unit tests or needs unit tests added Help Wanted labels Mar 2, 2021
@Mindfulplays
Copy link
Contributor

Second this, great idea.
I also found that tests including MonoGame Game class as is usually gets invoked from a background thread which causes many exceptions in SDLGameWindow as well as any UI thread based work (new Texture2D is the best example).

I ended up adding a full mock (wrapper/proxy) around Game as well as many MonoGame classes but a proper Monogame-based framework will really help decouple the classes (and avoid any static dependencies such as Platform classes as well as unavoidable inheritance hierarchy such as Game).

@softwareantics
Copy link

100% a unit testing framework would be great for MonoGame!

@Mindfulplays
Copy link
Contributor

@harry-cpp @mrhelmut I have a PR that does this exactly. Some kind soul had already set this up for iOS.

My proposal:

Tests/Interactive/iOS-> Move to Tests/iOS/Common and use a .shproj
Tests/Interactive/TestRunners/ -> New test runners that use the .shproj with the existing tests.

  • Add Tests/Interactive/TestRunners/DesktopGL/ and Tests/Interactive/TestRunners/iOS/ for now.
  • Add Android next.
  • Console: not sure... due to private SDK access

This is how I have been testing things like this PR

@Mindfulplays
Copy link
Contributor

Sent PR #8110

@vpenades
Copy link
Contributor Author

Actually I split the MonoGame's test runner years ago to run our own unit tests.

It is located here: https://github.com/OfCourseMyHorse/NUnit.Utilities/tree/main/src/MonoGame.NUnitRunner

It's been used in tests, so it's confirmed it works.

@Mindfulplays feel free to compare the PR and grab whatever you need from that project.

@Mindfulplays
Copy link
Contributor

Actually I split the MonoGame's test runner years ago to run our own unit tests.

It is located here: https://github.com/OfCourseMyHorse/NUnit.Utilities/tree/main/src/MonoGame.NUnitRunner

It's been used in tests, so it's confirmed it works.

@Mindfulplays feel free to compare the PR and grab whatever you need from that project.

Thanks! I will take a look. There are some interesting things definitely perhaps for a future PR. The iOS interactive tests have a different mechanism/goal: show an actual UI (both iOS ui and a MG UI that this person had coded up!) and allows discovering/running tests one at a time. It still has the drawback of MG with the GDM/Game being a singleton but in certain platforms it's easy to overcome as the notion of a view is cleanly separated.

I would think of this current test framework as a cross between Showcase/Samples and an Automated test.

@Mindfulplays
Copy link
Contributor

This PR is now ready for you folks, please take a look.

I have more coming up that adds RenderTarget/etc tests to this and also adds WindowsDX/Android runners etc. it's been very useful to understand/verify behaviors as I have made Metal work with this 100%.

Not sure how I magically summon reviewers but here we go: @SimonDarksideJ @harry-cpp @tomspilman :)
Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Help Wanted Testing Related to unit tests or needs unit tests added
Projects
None yet
Development

No branches or pull requests

4 participants