Nenamark3 is a benchmark that measures the 3d performance of a system by gradually increasing the complexity of a scene until the frame rate drops below 60.

Purpose

Nenamark3 is run to compare different systems against each other from a 3d graphics perspective. A 3d graphics benchmark tries to show how well a system will perform in games or or apps that make use of 3d graphics.

3D Graphics Apps

A 3d app, such as a game, will typically have individual settings for different tiers of devices. A lo-end system will have less features and look less amazing than a hi-end system. If hi-end settings are used on a lo-end system it will not be able to maintain its frame rate of 60 fps, causing it to drop frames and “stutter”.

What a 3D benchmark should do

This means that a graphics benchmark should define a set of scenes of increasing complexity and find how far a system can go while maintaining 60 fps. The common way of running static scenes and measure the fps have several problems. First, it is not relevant to run at any other frame rate than 60. It is not certain that lower fps are comparable to each other. All we can say for certain is - did it make 60 or not? Second, once 60 is reached where do you go? While it is possible to do off screen render and let the fps go as high as it can this is also irrelevant. If you have more power you would probably want more advanced features rather than more of the lesser features (perhaps you’d want a more correct lighting model rather than say more geometry).

Go Slow

There is a point to scaling the complexity gradually over time. Especially embedded systems have quite sophisticated clock governors that speeds up the clocks when needed, even for the GPU. These sample the load at certain intervals and adapt so it makes sense to slowly build complexity. In a game there would be a few slow frames before the system could react which would be annoying but not critical.

Native Resolution

The power of the GPU should match the display connected to the system. A system with a hi-resolution display needs to have a more powerful GPU than a lo-resolution system to reach the same level. A system builder should not compromise on the GPU-side when mounting a hi-end display. Doing so is like mounting a Prius-engine in a truck. Unfortunately this happens a lot and this is why cheaper systems sometimes performs better than more expensive ones.

Future Improvements

• 4k off-screen rendering. In the name of science it might be interesting to see what a system performs like on a normalised resolution surface. It is not certain all devices with the same GPU performs equal since manufacturers optimises drivers differently.
• Different FPS. Since there are systems that have different update frequencies than 60 it might be interesting to test against other locked frequencies. A gaming rig might have 120 Hz for VR or better flow.