Why guessing fails
When a game runs poorly, almost everyone has a theory. The lighting is too heavy, there are too many enemies, the code is badly written. Sometimes the theory is right. Often it is not, and a team can spend days optimizing something that accounts for a small slice of the problem. Performance testing replaces theories with measurements, so the effort goes where the time is actually being spent.
The method is simple to state: measure, change one thing, measure again. Doing it well takes some care.
Frame rate and frame time
Frame rate is how many images the game draws per second. Frame time is the other side of the same number: how many milliseconds each image takes. The two are related by simple division. At 60 frames per second, each frame has a budget of 1000 / 60, which is about 16.7 milliseconds. At 30 frames per second the budget is about 33.3 milliseconds, and at 120 it is about 8.3.
Developers tend to prefer frame time because it adds up. If the game logic takes 4 milliseconds, rendering takes 9 and audio takes 1, the total is 14 milliseconds, which fits inside a 60 frames per second budget with some room to spare. If rendering grows to 14, the total becomes 19, which is over budget, and the frame rate falls. Thinking in milliseconds makes it clear which part is responsible.
The main loop that runs input, updates and drawing every frame is the thing being timed, so each part of that cycle is a candidate for a slowdown.
Averages hide stutter
A single average frame rate can mislead. Imagine a hypothetical test of 10 seconds where 9 seconds run at a steady 60 frames per second and 1 second is spent freezing in a few long frames. The average could still look respectable, yet the freeze is exactly what a player would notice.
For that reason, testers often look at the slowest frames as well as the average. One common approach is to report the slowest 1 percent of frames, which describes the bad moments rather than the typical one. Consistency also matters: a steady 30 frames per second can feel smoother than a game that swings between 40 and 60.
Profiling: finding where time goes
A profiler is a tool that records how long each part of the game takes during a frame. Most game engines include one, and graphics hardware makers supply others. A profiler typically shows:
- How long the processor spends on game logic, physics and animation.
- How long the graphics hardware spends drawing.
- Which individual functions or draw operations are the most expensive.
- Memory use over time.
Performance profiling is the activity of reading these results to find the bottleneck, the one part that limits the whole. This matters because speeding up something that is not the bottleneck changes little. If the graphics hardware is the limit, simplifying the code will not help, and the reverse is also true.
Typical findings include too many objects drawn at once, an expensive effect covering the whole screen, a calculation run every frame that could be done occasionally, collision detection checks between too many objects, or character pathfinding recalculated more often than needed.
Making tests repeatable
A measurement is only useful if it can be compared. A tester playing freely will produce different numbers each run. To get comparable data, teams often use:
- A fixed route or recorded input that follows the same path each time.
- The same build settings, hardware and graphics settings every run.
- Several runs, with the numbers compared rather than a single one trusted.
- Repeated test locations known to be demanding, such as a crowded area or one with heavy effects.
Variation is normal. Background programs, temperature and other factors can move results slightly, so a small difference between two runs does not prove that a change helped.
Teams should also test on the weakest hardware they intend to support, since a game that is smooth on a powerful development machine may struggle elsewhere. This connects to compatibility testing, where devices differ so widely.
Load times
Load time is the wait between asking for something and being able to play it. Measuring it means timing from a clear start event, such as selecting a level, to a clear end event, such as control being returned. The tools often break the wait into parts: reading files from storage, decompressing data, building the scene and starting systems.
Storage speed has a large effect, so it is wise to test on a mechanical drive, a solid-state drive and the console’s own storage if those are in scope. The first load after startup may differ from later ones because data can be cached. Testers therefore note whether a load was “cold” or “warm”, and compare like with like.
Memory and heat
Frame rate and load time are the visible symptoms, but memory use and temperature can be the cause. A game that slowly uses more memory during a long session may eventually slow down or crash, so testers run long sessions rather than only short ones. On phones and handheld devices, heat can force the hardware to slow itself, which means a game can run well for five minutes and badly after thirty.
Setting targets
Before testing, a team usually decides what good means: a target frame rate on each platform, a longest acceptable load and a memory limit. Targets give the measurements something to be compared against. Without them, results are just numbers.