Running tests in parallel means multiple tests may be using, modifying, creating, and destroying the same resources. That includes...
- Files
- Directories
- Databases
- Service Accounts
For example, test 1 expects some.file to not exist. Test 2 creates some.file. When run one at a time, this is fine. When run together, they will interfere with each other.
Generally, you need to ensure each test is isolated from the others. Usually you do this by giving it its own unique resources. These are also useful for non-parallel testing as it guards against tests leaving their testing environment a mess if the code or test does not work as expected.
Make no assumptions about the state of your shared resources. Tests should not assume that resources (files, directories, databases, services) do or do not exist. They should not assume they contain certain values. They should begin by setting up a clean slate.
Here's some techniques.
Use temporary files and directories
Hard coded filenames and directories like test.txt or test/tmp/test.file mean parallel tests will collide with each other. Instead, always use temporary files with generated filenames. If you need specific filenames, make a temporary directory and write all your files there.
Temporary files and directories are generally automatically deleted when the process completes, or when the temp object goes out of scope. This can make it difficult to debug failing tests, so you may want to keep your temp files around if the test fails.
No hard coded names for shared resources
Often tests will have a particular username or filename or table name or magic number the test author prefers to use for testing. Avoid them. For example, maybe a test always makes a user named Shaq Overflow. If username are unique, parallel tests may fail.
Instead, these resource names must be randomly generated. This could be fully random, like an md5 hash, or you can just add a random number onto the end of the name each time. There are also libraries out there to create valid testing data such as Faker.
The more robust technique is to combine a faker library with test factories, see below.
Copy test data for each test
If your test relies on existing data, place that data in a central location and have each test make its own copy to work with. This can be as simple as having a file with known contents which your tests copy into their own temp directory to modify.
Start with an empty database and do each test in a transaction
While it would be best if each test created and populated its own SQL database for each test, this can get expensive and slow. Instead, make a database just for testing and populate it with empty tables. Then each test runs in its own transaction, isolating it from the other tests. At the end of each test, rollback your changes.
Some frameworks, such as Ruby On Rails, do this for you automatically.
Use test factories, not fixtures
Test fixtures are static data used for testing. Static data is the bane of parallel testing because it risks two tests using the same resource (same filename, same username) at the same time.
When you need to create complicated objects for testing, use a test factory to create consistent but semi-random data. For example, factory_bot has been ported to many languages. Combine your test factories with a library to generate fake data.