[programming] Higher refresh rates on input and game update

[Ahigh]

One thing that some people fail to realize is that a game can actually update the game state at a rate faster than the display. If you don't understand how it can still be important to update your inputs faster than your visuals, it's an interesting exercise to understand why.

The game that is actually occurring is happening inside your brain. The computer gives you cues through both audio and visual and you form a mental image for where things are and how things get updated based on your inputs.

As someone who writes games, I can tell you that the games that I write generally update at 240hz and above no matter what the display rate is. And when it is available, I read the mouse input through USB HID device data not even the normal windows mouse routines. When you get mouse input through the HID messages that come through and all is well, you don't get a steady stream of messages for each and every millisecond. You get data when data is generated by movement with a limit of 1000 times per second. So it doesn't really tax your system at all when the mouse is not moving.

But if you imagine the raw data being a line drawn in 2d with a brush the difference between a very smooth arc and a triangulated shape where you only get samples every 16-17 milliseconds is quite different. And the difference in gameplay is also quite different with twitch movements to aim, shoot, and kill. First off, just clicking the mouse button to fire is so radically different with 16x as many possible moments when you fire with 1000hz, especially if the game programmer is doing variable time-step updated based on when your input data arrived.

It's just amazing when you understand all of this .. that you can actually program variable framerate gameplay that is so much more accurate, fluid, and fun. It does all make a difference. Every game I work on I ensure I do all of this right. And I know others in the game industry who also understand this stuff. It's sort of the black art of hard-core game programming -- the inputs -- and the update of the game state. Doing them right.

Some of these realizations were made by myself and others that I worked with. One of those others was my college roommate who went on to be the lead of COD MW2 and now TitanFall. I am certain he knows how to do update loops and input samples right based on our conversations back in the mid 90's about such things as iteration rates. And I'm sure that other games also make these realizations and program the game update loop and input samples the right way using a FIFO for the inputs and updating at intervals according to inputs or otherwise using the sample arrival time of the inputs as part of the gameplay.

And while I don't know 100% for sure for every good game what they do, I do know that you're smart to get a mouse that does 1000hz, and if you're in a good enough game, the game itself may figure out how to read the data from the mouse independently of your windows settings. Yet you should ALWAYS set your mouse rate at 1000 hz if it's an option, just in my opinion, that's how it should be.

DPI matters too for your preference, and you can ideally scale up the DPI and scale things back in the game application to compensate to have more precise data for smaller movements. But that's not even as important as the update rate in my view, more of a preference.

I hope this is a useful perspective from a game programmer. But definitely get a good logitech or razer mouse and crank up the rate is pretty much the bottom line. If you have the skill for it to matter of course. If you get 1 kill a day with the best hardware and settings, maybe you should have saved your money, though. And why not go ahead and post a message how it doesn't make a difference while you're at it to make yourself feel better.

[flood]

but what can you do if your game is graphically heavy and cannot update faster than the refresh rate?
Would it be posssible to have two loops (two threads maybe) where one runs the game logic every ms and the other does the drawing?

[Chief Blur Buster]

but what can you do if your game is graphically heavy and cannot update faster than the refresh rate?
Would it be posssible to have two loops (two threads maybe) where one runs the game logic every ms and the other does the drawing?

The ideal situation (but not always practical) for modern games, is to design the game so that input reads are processed immediately before rendering. This results in the absolute minimum input lag for that specific frame.

But not all game engines do this, and sometimes the game engine essentially needs a global metronome (e.g. online games like Quake Live) that ticks on all computers simultaneously at the same rate. This can level the playing field, but definitely can produce unintended side effects, such as aliasing between the input reads and game engine, that creates amplified microstutters or random variances in input lag.

Input read timing (in game engines) relative to frame rendering timing, is definitely an additional hidden input lag chain issue (the time between input reads and the frame rendering) that is often never measured by reviewers using today's input lag measurement tools. One method of measuring this input lag is the 1000fps+ high speed video, by AnandTech:

phpBB [video]

...Observe how long it takes from keypress to gun fire reaction, in this high speed video...

One potential test of input lag measurement accuracy is to do several passes of this, for various different game engines. You will find that games do it predictably (e.g. during 1000fps video, almost exactly 35 high speed video frames = 35ms between keypress and gun fire). Other games will jitter randomly (e.g. varies between 20 and 45 high speed video frames = variance of 20-45ms between keypress and gun fire). The high speed video benchmarking tool, severely underused by reviewers, is an excellent tool that can potentially measure game engine consistency/accuracy in input latencies. More high speed video testing by reviewers is needed, to show the problem of input lag variances (jitter) in various different game engines and various different modes (VSYNC ON, VSYNC OFF and GSYNC).

Blur Busters may plan to begin writing articles in this direction sometime in early 2014...

[flood]

The ideal situation (but not always practical) for modern games, is to design the game so that input reads are processed immediately before rendering.

Yes that's ideal for input lag. I've always thought that that would be the obvious thing to do. Unfortunately some game engines update the inputs when the buffer swap is called, and if there is vsync, that adds an additional frame of input latency.

[Ahigh]

Game update loops should, in general, operate at independent rates. It is only necessary to operate in a thread if you have real-time feedback. On San Francisco Rush 2049 coin-op, for example, the input from the steering wheel operated on a 1ms interrupt at 1000hz. The vehicle simulation operated at 240hz (both in real time) and the graphics operated at a maximum of 60hz. Rush 2049 was the first video game to use what NVidia calls Adaptive Sync. You can see it in action on Track 5 in ghost mode after the first left turn. You will see a tear creep down from the top of the screen go back up then then the game is sync'd to verical refresh again.

To the best of my knowledge, I invented Adaptive Sync as I know of no other games release before Rush 2049 in August 1999 that did this. But I believe other programmers has independently come up with this method. I know I saw it in the Jak3 engine when I worked at Naughty Dog, but I didn't narrow it down to who implemented that or when, but that was in 2004, a couple years after I did it in Rush 2049. I'm pretty sure other games have done this too.

But anyway, when you are updating your game update loop in non-real-time, you queue up the inputs and you do each iteration of the game update loop with the latest state of the inputs for the current time, and you increment the time value by whatever delta time each iteration of the game update loop.

There's not much too it, and you don't need threads at all. You don't even have to have a constant delta-time in your game update loop. You can just update with a dt between input samples, for example, if you game integrates well enough this way.

This may or may not be clear, but if you do have a constant delta-time for your game update rate, you generally have to interpolate between the previous two game update states for objects' positions and suffer a slight delay assuming that your game update rate is not a divisor of you graphical update rate. In other words, if your game update delta-time is 4.16ms, then on average you will have an extra 2.08ms of delay added to the visuals due to interpolation between the previous two game update states to get a smooth visual result for the graphical update rate which could be 60hz, 59hz, 30hz, 24hz or whatever...

But to be clear, if the graphical update rate changes how the game plays, in my humble opinion, YOU ARE DOING IT WRONG!

[flood]

hm interesting

but what can you do if the rendering takes a long time (10ms), which is totally normal for modern 3d games? how can you update the game loop without threads or something

[Ahigh]

hm interesting

but what can you do if the rendering takes a long time (10ms), which is totally normal for modern 3d games? how can you update the game loop without threads or something

You do it more than once per visual frame.

pseudocode:

Code: Select all

unsigned int msec( void )
{
    return real_time_clock_in_milliseconds;
}

void update( void )
{
    const unsigned int game_update_dtms = 4;
    unsigned int ms = msec();

    while( game_update_ms < ms ) // iterates multiple times per visual frame
    {
        ms = msec();
        // state of input can be different on each iteration
        grab_latest_input_state( game_update_ms );
        game_update( game_update_dtms );
        save_position_of_all_game_objects( game_update_ms );
        game_update_ms += game_update_dtms;
    }

    // interpolate from previous two game state as visuals are on average game_update_dtms/2 older than game state
    assert( ms <= game_update_ms );
    interpolate_visually_displayed_position_of_all_game_objects_for_visuals( ms );

    draw_everything(); // always drawing position of game objects at or behind their most current game state at 4ms boundaries
}

You typically also have a maximum number of iterations and some skew for when the game gets paused and what-not but that's beyond the scope of this discussion.

[Chief Blur Buster]

Rush 2049 was the first video game to use what NVidia calls Adaptive Sync. You can see it in action on Track 5 in ghost mode after the first left turn. You will see a tear creep down from the top of the screen go back up then then the game is sync'd to verical refresh again.

To the best of my knowledge, I invented Adaptive Sync as I know of no other games release before Rush 2049 in August 1999 that did this.

Damn, that's impressive.
You helped create San Francisco Rush 2049?

Thoughts: Hmmm, between you and flood, plus myself, I wonder if I should create a separate "Game Programming" discussion forum here at Blur Busters. I'm not a game programmer by trade -- though I created several games and demos for the C64 (old page), and worked with raster interrupts on C64. This provides the genesis of my knowledge of interaction between computer and display scanout behaviour. And I worked in the home theater industry, so I know more about the inner workings of displays than most game programmers do! I've set many game programmers straight on their misunderstandings of display behaviour. Even though I'm more amateur of a game programmer than some of you are.

[Ahigh]

Rush 2049 was the first video game to use what NVidia calls Adaptive Sync. You can see it in action on Track 5 in ghost mode after the first left turn. You will see a tear creep down from the top of the screen go back up then then the game is sync'd to verical refresh again.

To the best of my knowledge, I invented Adaptive Sync as I know of no other games release before Rush 2049 in August 1999 that did this.

Damn, that's impressive.
You helped create San Francisco Rush 2049?

Thoughts: Hmmm, between you and flood, plus myself, I wonder if I should create a separate "Game Programming" discussion forum here at Blur Busters. I'm not a game programmer by trade -- though I created several games and demos for the C64 (old page), and worked with raster interrupts on C64. This provides the genesis of my knowledge of interaction between computer and display scanout behaviour. And I worked in the home theater industry, so I know more about the inner workings of displays than most game programmers do! I've set many game programmers straight on their misunderstandings of display behaviour. Even though I'm more amateur of a game programmer than some of you are.

I was the lead programmer on Rush/2049 coin-op.

I even got sole inventor for "arcade keypad" which was critical for adding player accounting to an arcade driving game.

My boss' boss Mark Stephen Pierce said "this will be the first 3d Atari Driving Game with 60hz graphics since Pole Position." Of course pole position was actually a Namco game, so go figure. I put a WHOLE LOT of effort into getting solid 60hz in Solo mode with phantoms.

http://www.google.com/patents/US6572477

I also added some "gambling-like" features into the patented "ghost car" where you beat the free game car and win a free game.

http://www.google.com/patents/WO2001005475A1

NFL Blitz had player accounting before Rush, but I think this was the first arcade driver with player accounting.

I am currently building a team to create games here in Las Vegas, but it's very early stages.

http://finance.yahoo.com/news/nanotech- ... 00354.html

High framerate, low latency, hard-core gameplay is my favorite stuff to work on. I'm looking forward to doing some more of it here in Vegas. Thanks for the interest in the details, guys.

[flood]

pseudocode:

Code: Select all

unsigned int msec( void )
{
    return real_time_clock_in_milliseconds;
}

void update( void )
{
    const unsigned int game_update_dtms = 4;
    unsigned int ms = msec();

    while( game_update_ms < ms ) // iterates multiple times per visual frame
    {
        ms = msec();
        // state of input can be different on each iteration
        grab_latest_input_state( game_update_ms );
        game_update( game_update_dtms );
        save_position_of_all_game_objects( game_update_ms );
        game_update_ms += game_update_dtms;
    }

    // interpolate from previous two game state as visuals are on average game_update_dtms/2 older than game state
    assert( ms <= game_update_ms );
    interpolate_visually_displayed_position_of_all_game_objects_for_visuals( ms );

    draw_everything(); // always drawing position of game objects at or behind their most current game state at 4ms boundaries
}

You typically also have a maximum number of iterations and some skew for when the game gets paused and what-not but that's beyond the scope of this discussion.

Not quite understanding this. How does the program know when to leave the game update while loop?
it seems what's wanted is something like
(g for game update)
g-g-g-draw frame 1-g-g-draw frame 2-g-g-g-draw frame 3 etc...