Highlight clearly why mocking is problematic

[aszenz]

I often see developers using mocks for entities to simplify their creation for testing.

While this is easy enough it creates a lot of coupling to the implementation details.

I would like to see a clear warning that mocks shouldn't be used especially for data classes like entities which have no side effects

[sarven]

Yeah, I also have experienced using mocks for almost everything, especially when using phpspec 🙈. Did you see this section: https://github.com/sarven/unit-testing-tips#always-prefer-own-test-double-classes-than-those-provided-by-a-framework?

I have an ambitious plan for this year to extend this guide and add an explanation for things like resistance to refactoring which may be not clear to everyone. However, it's not an easy task to explain a lot of things and keep this guide concise.

[aszenz]

Yes creating own test doubles is indeed a nicer way.

I personally think mocks are not useful for testing anything valuable, they provide a false sense of security and almost never catch any real bugs.

Most guides just repeat the same lingo about mocks vs stubs vs fakes, I think all of that is less important in output based testing.

Some things which I find most guides don't cover:

1. How to write tests for complex data structures, like real world entities with multiple dependencies on other classes.

2. How to simplify the arrange part of testing, it is the most complicated aspect when reproducing a bug i.e. how to get the system under test in the right state?

3. How to make powerful assertions in tests which can actually prevent bugs?

[sarven]

I will try to cover these topics in the future, now just quick explanations:

1. It's probably the problem when TDD is not used. TDD helps us write testable code. Highly complicated objects with lots of dependencies don't seem like a good design.
2. Could you provide any examples? Reproducing a bug should be quite simple, write a red test and then adjust the implementation to make this test green.
3. Have you heard about Mutation testing? https://github.com/sarven/unit-testing-tips#100-test-coverage-shouldnt-be-the-goal
   There is a link to a separate article: https://sarvendev.com/2019/06/mutation-testing-we-are-testing-tests/

[sargath]

I think that worth noting is also an aspect related to legacy app architecture, where we'd like to create a test before we change something. It would be really laborious to write test only with basic tools without mocking ability.

Interesting take about the topic https://www.youtube.com/watch?v=uVHGt2qbjXI

[aszenz]

Thanks for the video, what i get from the video is that mocking is useful for fleshing out modular designs when doing TDD. One objects interaction with other objects is tested via mocks.

I can understand it helping in the design process but I don't find it a good argument to keep such tests after the design process is over. Why should I commit and run tests tests which are just useful for design?

Output based tests actually verify the design gives the right results which is important to test continuously.

This is also why i don't understand how legacy apps can benefit from mocking, they are already not modular, and their design is set in stone. The only thing we care about is whether refactoring such code doesn't break anything. This is a great case for writing broad integration tests not unit tests with mocks.

[aszenz]

1. It's probably the problem when TDD is not used. TDD helps us write testable code. Highly complicated objects with lots of dependencies don't seem like a good design.

I really wish guides would separate out TDD and testing in general, majority of the projects don't use TDD, but would still like to verify their code works. Highly complicated objects are found across business code and still need to be tested even if we agree their design is bad.

2. Could you provide any examples? Reproducing a bug should be quite simple, write a red test and then adjust the implementation to make this test green.

Most bugs i face are edge cases, they don't happen always but only when the system is under a particular state. To get the system under the right state requires creating multiple objects with fake data and calling multiple methods in sequence to trigger the bug. Unless the object has been tested before it is usually a lot of work to set up. But it is necessary work which must be learned.

3. Have you heard about Mutation testing? https://github.com/sarven/unit-testing-tips#100-test-coverage-shouldnt-be-the-goal
   There is a link to a separate article: https://sarvendev.com/2019/06/mutation-testing-we-are-testing-tests/

Yes I have come across mutation testing, it is certainly useful to give us an idea of how much we can rely on our tests to detect broken code.

[sarven]

The only thing we care about is whether refactoring such code doesn't break anything. This is a great case for writing broad integration tests not unit tests with mocks.

@aszenz Totally I agree with that. Writing unit tests with mocking for legacy code gives us no confidence when we want to refactor something. So I usually prefer writing an integration test when I have legacy code and I have to do some refactoring.

I will try to cover this topic during the next iteration of developing this guide. It's getting bigger, testing seems to be a broad topic so it would be great to prepare a comprehensive ebook with all this knowledge.

[sargath]

I agree with both of you, although as usual, it all depends. Sometimes legacy !== legacy.

Mocking is required when the decomposition strategy has failed.

For instance, I would mock some side effects made by I/O in a particular scenario rather than re-create everything from scratch or build a heavy integration test.

I have worked with an app with the enormous cost of bootstrapping the container and test environment.

[sarven]

@aszenz

I really wish guides would separate out TDD and testing in general, majority of the projects don't use TDD, but would still like to verify their code works.

We should encourage everyone to use TDD, it's an important technique to achieve better solutions. I think that tests and TDD are inherent topics.

Most bugs i face are edge cases, they don't happen always but only when the system is under a particular state. To get the system under the right state requires creating multiple objects with fake data and calling multiple methods in sequence to trigger the bug. Unless the object has been tested before it is usually a lot of work to set up. But it is necessary work which must be learned.

There are practices that can help with that:

• Fail fast -> the sooner we find the bug the better
• and one more time TDD -> we face the issue on PROD -> try to reproduce and create the failing test for this edge case and then make this test green

In legacy systems with bad architecture, no modularity, and so on, certainly creating good tests will be hard. But I think that it's worth investing some time and trying to work out a habit of always securing our change with broad integration tests and then we can change the logic under the hood more safely.

I highly recommend this presentation: https://www.youtube.com/watch?v=2vEoL3Irgiw

Mutation testing is very helpful to verify our unit tests. Using mutations for slower integration tests isn't viable.

[sarven]

@sargath

I agree with both of you, although as usual, it all depends. Sometimes legacy !== legacy.

Mocking is required when the decomposition strategy has failed.

For instance, I would mock some side effects made by I/O in a particular scenario rather than re-create everything from scratch or build a heavy integration test.

I have worked with an app with the enormous cost of bootstrapping the container and test environment.

Yeah, of course, legacy systems are very different. However, I think that broad integration tests are usually the best for legacy systems with a bad design because we still have the possibility to change the logic underneath. It's often better to just allocate more resources, create the environment to execute tests in parallel, and so on, and just have the possibility to refactor that code.