What a regression is
A regression is a defect where something that used to work stops working. The word is used for two related situations: a brand-new break in a feature that was previously fine, and an old bug that was fixed and has reappeared. Both mean that the game moved backward, and both are common.
Regression testing is the practice of re-checking existing behavior after a change to confirm that it still works. It differs from testing a new feature, where the question is whether the feature works at all. Here the question is whether the change disturbed anything else.
Why old bugs return
It is tempting to think that a fixed bug is gone for good. In practice, there are several ordinary reasons it may not be.
Parts of a game depend on each other. Games are made of many systems that share data and assumptions. A change in the system that handles jumping can affect animation, collision detection and sound at once. The developer fixing one part may not know about every other part that relies on it.
A fix can be undone by accident. If two people edit the same area, one person’s changes can overwrite the other’s. Version control systems are designed to reduce this, but they cannot decide which of two conflicting edits is correct. When the merge goes wrong, an old problem can reappear without anyone intending it.
The fix addressed a symptom. Suppose a character occasionally falls through the floor, and the developer adds a rule that pushes it back up. The visible symptom goes away, but the underlying cause remains and may show up in a new form later.
Old content comes back. Teams sometimes reuse earlier files, restore an older version of an asset or copy a level from a previous project. If the old copy contained the bug, the bug returns with it.
Conditions change. A defect that was hidden because of a slow frame rate can appear when performance improves, since the timing that masked it has changed. The bug never left, but it became visible.
Different branches. Large projects often keep several lines of work running together, such as one for a release and one for the next version. A fix made on one line has to be carried across to the other, and sometimes it is not.
What a regression test looks like
Regression tests are usually things the team already knows how to check, gathered into a list. They tend to include:
- Checks for every serious bug fixed in the past, written so the exact situation can be repeated.
- Checks for the most important features, such as starting a game, saving, loading and completing the first level.
- Checks for areas known to be fragile.
One useful habit is that when a bug is fixed, the steps from its report are added to the regression list. Good bug reports make this easy because the steps are already written down, so the same text becomes a repeatable test.
Smoke tests and full passes
Teams often distinguish between a quick check and a thorough one.
A smoke test is a short run, perhaps ten or fifteen minutes, that confirms the build is basically alive: it starts, menus work, a level loads and a character can move. If a smoke test fails, the build is usually returned to the developers before anyone spends time on deeper testing. The name comes from the idea of switching a new device on and seeing whether smoke comes out.
A full regression pass goes through a much larger list and can take days for a big project. Because it is expensive, it is typically reserved for key moments such as before a milestone or a release. At other times, teams run a smaller selection chosen for the change at hand.
Choosing what to retest
Retesting everything after every change is rarely possible, so teams make choices. A reasonable approach is to look at what the change touched and what depends on it. A change to the save system obviously calls for loading old saves, but it may also affect menus that display save information. Developers who understand the code are valuable here because they can say what else might be connected.
This is a judgment, and judgments can be wrong. A hidden dependency, by definition, is one nobody expected, so some regressions will always slip through. The goal is to catch the likely ones and to catch the rest as early as possible.
Where automation helps
Machines are well suited to repeating the same checks without getting bored or skipping steps. Automated tests can start the game, load each level, check that a function returns the expected value or compare screenshots against stored images. Teams often run these automatically whenever code is changed, so a break is found within hours rather than weeks. Small checks on game logic can be quick to write, and they run reliably.
Automation has limits. Tests need to be written and maintained, and a test that fails for the wrong reasons wastes time. Judgments such as whether a jump feels right still need a person. Most teams use both.
Keeping regressions down
No process removes the risk, but several habits reduce it:
- Make small changes rather than large ones, so that when something breaks the cause is easier to find.
- Record every fix with a way to confirm it.
- Test soon after a change, while the details are fresh.
- Keep notes on fragile areas.
- Be cautious about late changes near a release, when there is little time to retest.
For players, the lesson is that a returning bug is rarely carelessness alone. Games are large, interconnected systems, and a fix is itself a change that needs checking. Reporting that a problem has returned, with the version number, is useful information for the team.