The Netcode, Spectator, and the Future: Part 1
Posted: Tue Apr 28, 2015 8:44 am
The so called Frostbite netcode has been a huge topic of debate over the past couple of years.
Since release, various 'unoptimized' portions of the networking system in Frostbite games have been plaguing the Battlefield community, with high in-game response times from client to client, hit detection issues, slow replication of movement updates, and so on...
Most people first 'discovered' this topic with the release of Battlefield 4, where some of the previously mentioned issues were significant, and directly affected the enjoyment and playability of the game, but the truth is that even earlier games are plagued by these issues, and in our case, Battlefield 3.
After numerous requests from our community, we decided it was finally time to take a closer look at the so called netcode, perform some research on it, and see what we could do about it.
The first thing we did is to gather some statistical data on how the game performed.
For all of our tests, we spawned two VU clients connected to a dedicated server, all running on the same machine, and therefore with virtually no network latency overhead.
Then, we fired 38 shots (from both game instances), and we measured the time it took for every shot to register its damage on either client.
All of our measurements were accurate down to the microsecond, but for the purposes of this presentation, all times were converted to milliseconds.
In the above graph, you can see the time distribution between different shots, along with the low, high, and average latency values in ms.
To our surprise, updates averaged to a staggering 90ms, which means that from the moment someone shot you on his screen, it would take approximately 90ms (+ network latency) for you to see that reflected on your screen.
The other thing that we could deduct from this data is that updates appeared to occur within a 30ms interval from each other.
From that data we came to the conclusion that the incoming frequency for clients was approximately 15Hz, and the outgoing frequency was approximately 30Hz (and vice-versa for the server).
Out of curiosity, we performed another test, but this time, with network smoothing set to 100% (in the first test it was set to 0%).
We think the results speak for themselves...
You might notice that after the 19th shot, updates are shifted upwards by approximately 30ms.
This appears to happen because of the simulation latency difference between the two clients, since the last 19 shots were shot from the second client.
Long story short, unless you're filming a cinematic (or something similar), you should avoid having network smoothing on.
After we were done with our initial tests, it was time for research and experimentation.
Our goal wasn't to optimize the internal workings of the netcode, as doing so would require countless hours of work and dedication, with probably little to no payoff.
What we set out to do was to figure out whether it was possible or not to up the replication frequency whilst maintaining a fully functional game, and if in doing so we could improve the update latency.
After approximately two weeks of constant research, tests, failures, and more tests, we came to the following conclusion:
It was definitely possible!
That's why we're extremely excited to announce the High Frequency update for Venice Unleashed!
Starting with build 8312, the Venice Unleashed client can optionally function in High Frequency mode, allowing you to connect to High Frequency servers and enjoy low latency updates.
We could go on and on babbling about how much more better the game works with our High Frequency code, but instead we'll let our test results speak for themselves:
From what you can see in the above graph, the average latency has been decreased by almost 3 times, to an impressive 35ms, a value which makes our client compete with some of the most optimized competitive games out there.
There is still a slight deviation of ~12ms, but that's to be expected and shouldn't be significant when it comes to gameplay.
For those of you that don't really like graphs, we've also recorded a comparison video, comparing the regular frequency netcode against our high frequency netcode:
The original footage was recorded at 60fps and then slowed down in post.
Now, all these changes might sound good and dreamy, but they do come at a cost.
A server (or client) running in High Frequency mode will consume approximately four times (x4) more bandwidth (both incoming and outgoing), and will see a slight increase in CPU usage.
Additionally, there are a couple of issues that are currently present, the two most significant of them being wobbly first person models, and weird dirt-bike physics.
We are of course currently in the process of fixing the aforementioned issues, and we hope to have a fully functional high frequency client as soon as possible.
Interestingly, another side-effect we observed was that loading times were slightly decreased when running in high frequency mode, but we don't think that's something that requires fixing.
Unfortunately, because some of our code changes are being performed at a very deep level in the engine, switching to (or from) high frequency mode requires a client restart.
When connecting to a high frequency server from a regular frequency client (or vice-versa), the client will automatically prompt you to restart your game.
Alternatively, you can always run your game in high frequency mode by providing the -highfreq parameter when launching vu.exe (also usable for running a server in high frequency mode).
Test Server #001 is already running in High Frequency mode, so you can already jump in and test it for yourself!
This post is the first of a two-part post.
Stay tuned for the next part, in which we will discuss the latest changes in the Spectator mode, and the future of the project.
Since release, various 'unoptimized' portions of the networking system in Frostbite games have been plaguing the Battlefield community, with high in-game response times from client to client, hit detection issues, slow replication of movement updates, and so on...
Most people first 'discovered' this topic with the release of Battlefield 4, where some of the previously mentioned issues were significant, and directly affected the enjoyment and playability of the game, but the truth is that even earlier games are plagued by these issues, and in our case, Battlefield 3.
After numerous requests from our community, we decided it was finally time to take a closer look at the so called netcode, perform some research on it, and see what we could do about it.
The first thing we did is to gather some statistical data on how the game performed.
For all of our tests, we spawned two VU clients connected to a dedicated server, all running on the same machine, and therefore with virtually no network latency overhead.
Then, we fired 38 shots (from both game instances), and we measured the time it took for every shot to register its damage on either client.
All of our measurements were accurate down to the microsecond, but for the purposes of this presentation, all times were converted to milliseconds.
In the above graph, you can see the time distribution between different shots, along with the low, high, and average latency values in ms.
To our surprise, updates averaged to a staggering 90ms, which means that from the moment someone shot you on his screen, it would take approximately 90ms (+ network latency) for you to see that reflected on your screen.
The other thing that we could deduct from this data is that updates appeared to occur within a 30ms interval from each other.
From that data we came to the conclusion that the incoming frequency for clients was approximately 15Hz, and the outgoing frequency was approximately 30Hz (and vice-versa for the server).
Out of curiosity, we performed another test, but this time, with network smoothing set to 100% (in the first test it was set to 0%).
We think the results speak for themselves...
You might notice that after the 19th shot, updates are shifted upwards by approximately 30ms.
This appears to happen because of the simulation latency difference between the two clients, since the last 19 shots were shot from the second client.
Long story short, unless you're filming a cinematic (or something similar), you should avoid having network smoothing on.
After we were done with our initial tests, it was time for research and experimentation.
Our goal wasn't to optimize the internal workings of the netcode, as doing so would require countless hours of work and dedication, with probably little to no payoff.
What we set out to do was to figure out whether it was possible or not to up the replication frequency whilst maintaining a fully functional game, and if in doing so we could improve the update latency.
After approximately two weeks of constant research, tests, failures, and more tests, we came to the following conclusion:
It was definitely possible!
That's why we're extremely excited to announce the High Frequency update for Venice Unleashed!
Starting with build 8312, the Venice Unleashed client can optionally function in High Frequency mode, allowing you to connect to High Frequency servers and enjoy low latency updates.
We could go on and on babbling about how much more better the game works with our High Frequency code, but instead we'll let our test results speak for themselves:
From what you can see in the above graph, the average latency has been decreased by almost 3 times, to an impressive 35ms, a value which makes our client compete with some of the most optimized competitive games out there.
There is still a slight deviation of ~12ms, but that's to be expected and shouldn't be significant when it comes to gameplay.
For those of you that don't really like graphs, we've also recorded a comparison video, comparing the regular frequency netcode against our high frequency netcode:
The original footage was recorded at 60fps and then slowed down in post.
Now, all these changes might sound good and dreamy, but they do come at a cost.
A server (or client) running in High Frequency mode will consume approximately four times (x4) more bandwidth (both incoming and outgoing), and will see a slight increase in CPU usage.
Additionally, there are a couple of issues that are currently present, the two most significant of them being wobbly first person models, and weird dirt-bike physics.
We are of course currently in the process of fixing the aforementioned issues, and we hope to have a fully functional high frequency client as soon as possible.
Interestingly, another side-effect we observed was that loading times were slightly decreased when running in high frequency mode, but we don't think that's something that requires fixing.
Unfortunately, because some of our code changes are being performed at a very deep level in the engine, switching to (or from) high frequency mode requires a client restart.
When connecting to a high frequency server from a regular frequency client (or vice-versa), the client will automatically prompt you to restart your game.
Alternatively, you can always run your game in high frequency mode by providing the -highfreq parameter when launching vu.exe (also usable for running a server in high frequency mode).
Test Server #001 is already running in High Frequency mode, so you can already jump in and test it for yourself!
This post is the first of a two-part post.
Stay tuned for the next part, in which we will discuss the latest changes in the Spectator mode, and the future of the project.
Hope to get keys to try it out ! Hopefully SOOON 