Eliminate checking factory items in our tests

[cielf]

Summary

Eliminate any dependency on factory items in the test code - step 1 - find them all.

Why?

More robust testing

Details

There shouldn't be any tests that rely on factory items having specific values or that check against factory items.
Note: Look especially at any tests that are dependent on sorting or pagination.

Review the tests with an eye to eliminating any checks against factory items. If there are very few, you may go ahead and submit
PRs to fix them, but let's assume you'll need to compile a list

That list should include, for each affected test:
source file, test description (i.e. context, describe, etc) and approximate line number

If you want to review some, and give us a list including what has been reviewed, so some one can pick things up afterwards, that's fine too.

Criteria for completion

• lists of affected tests, as described above

Bonus round

• submit PRs to fix some or all of the tests

[veezus]

Does this mean we'd be looking for tests that are asserting against default factory values? Like, a validation spec that assumes a factory creates a valid object?

In the pagination example, we'd want factoried objects with spec-local values to make sure we control like which items appear in the list, as opposed to relying on values in the factory file?

[elasticspoon]

Pretty much yes, for example:

#4099 (comment)
#4099 (comment)

[dorner]

Yeah. My concerns with using factory values are:

1. It's brittle. If the factory ever changes, all the specs break and have to be changed as well.
2. It's confusing. If the value "FOOSPAZ" is given as a default in the factory, and the spec checks that a particular value is "FOOSPAZ", we have to intuit that that value came from the factory and now have two files contributing to that spec (the factory and the test itself).
3. Relying on factory values puts too much power into the factory, and makes us want to use it more. This is how we get to where we are now, where we auto-create a whole bunch of stuff that half the time we don't need.

Specifying exactly the values we care about is a bit more typing, but makes it much easier to follow and change the tests. Factories (IMO) are basically there to fill in values that are required (due to validations etc.) but we don't actually care about or want to check in the current test.