Internet all the Things

In this new, connected age many of us could not do anything without our link to the world wide web. Shopping, driving, TV and games are but a tiny ingredient that forms the modern IOT constructed landscape that stretches out before us. But a change to the method in which we access this has been slowly changing over time, having to buy a huge PC or a set-top box that connects to your TV are dead. Smart phones became smart TV’s and our devices have become smaller and smaller, current consumer mobile phones now hold more computational power than a desktop computer from 10 to 15 years ago, something that can play modern games such as Fornite in a device small enough to fit in your pocket is a revolution all of its own.

You may be surprised that many of the features and functions these tethered devices deliver so seamlessly are not being computed within the device at all, merely a host for the result that is beamed from and to your hand over the wireless connection. They form a very basic node within a “farm” topology and as such it is designed to be extensible as required. Meaning that new connections, balancers, processors and clients (you) can be added and removed with ease and continuity. This is how your TV can have access to all the various channels, on-demand content and applications from your remote control even how some TV’s process speech recognition for your commands, many of these handle the intelligent part within this vast span of CPU’s, switches, routers within a few ms. Popular devices and apps such as Siri, Alexa and even Google maps all work this way, taking your input, compressing down to as few bytes as possible and firing it upstream. The data is analysed, processed and the results fired downstream, routing your destination or answering your history homework to how many wives did Henry the VIII actually have? And of course, this does mean that all the companies have access to this data just as you do, this is the information age after all.

 All the power in our hands

The subscription model has become vital to much of what we enjoy. From cars, phones, power, insurance, television and even Internet, you are likely using to read this based on this persistence, locked down model. Just like these 2 areas have transformed how we consume Television and Film so too they are and will transform how we consume our games and it may have been sneaking up on us all the time. Sony have placed their boot in this camp a long time ago, acquiring the Streaming company Gaikai (Guy-Ki) in 2012 alongside Onlive and then closing this down, competition is always easier to handle when you own them, one of the joys of PSVita was remote play. It also is the key to the PSNow subscription service that has been running for almost as many years. You can check out my video covering this area in more technical depth including some input latency tests on remote versus local play on my channel.

   Why do I care?

Why this brief history lesson then? Microsoft are now poised to enter this arena and once again, the signs have been around for a few years. The rumoured collection of Xbox(s) in the plural that Phil Spencer talked about at E3 this year covers 2 distinct models with the first one to launch being a standard, consumer funded machine that will contain a Ryzen CPU, more powerful GPU combined within an APU, memory, storage, optical drive, in essence a standard Next gen machine from them as expected, nothing new here other than what I covered in my PS5 and Xbox2 article earlier in the year. All the speculation is wrapping around the other member to the Xbox family and may launch a little later, a light weight streaming device that will not house any of the higher spec controlling internals of the newer machine and likely less than what we have in the current XboxOne YET will still be able to play all the next gen games as its bigger  brother and, sit down as this is the kicker, they will look & sound identical, How?

Short answer is the thin client set-up I just covered above, the fun is in the longer answer below.

Before we get into the possible methods and infrastructure that will sit behind this let’s look at the benefits.

1.  Cost:-

•  By having all the costly hardware handled within the server infrastructure rather than the box under your TV means we pay much less for it, rumoured $99 for a taste of this generation ending leap, with a subscription plan making this “free” imagine a phone contract.
• Much of the core pipeline and server environment that underpins this already exists for Microsoft.

2.  Expandability/Flexible:-

• For us, the client, we can choose to access this service cheaply and easily. From a new device such as the Scarlet collection and almost certainly our current ones in the Xbox One and X via a subscription model (see that history lesson paying off already).
• In addition, other devices could just as easily be added to this AND if you do not want to subscribe you could rent a game just like you do a Netflix episode or the latest summer blockbuster.
• As hardware and software improves so will your games, Microsoft can add more servers into the farm or better hardware which is invisible to us. We simply see more choice, better looking games and faster performance.

3.   Eco-system & guaranteed revenue:-

• This last one is the main attraction for MS, Sony and any other firm wanting in on this new fibre fed gaming landscape, revenue. The current spiralling  costs to deliver the latest AAA blockbuster is something that cannot be ignored, and it is not as Risk on investment is core. Netflix, Amazon et al have all helped create many series or films that would have never been greenlit for a large-scale cinema release, via to this subscription model. Having a fixed, constant and more importantly, certain revenue stream allows these risks to be mitigated.

   How will it work?

Now we have the theory, we need to look at the reality. On paper this all sounds like a dream for films or TV this is much simpler to achieve. You can buffer the video feed seconds or even minutes ahead, have a small 1-2GB storage space on the local device and then add to this in the background as we watch the feed oblivious to this delay or latency from the source to the screen. We have no control, so it does not affect us at all, games on the other hand are vastly different.

Xbox Scarlet and Streaming Games

Getting a high quality 4k feed to our screen is one thing, allowing you to play and feel in control of that is another thing entirely. the challenges are 2- fold, image quality and the biggest, latency, the time it takes for us to click the fire button, move left or right or even open the menu screen. Locally this is much easier, checkout my latency test video to learn more on this, short version is the best you can expect from a game is something like Call of Duty or fighting titles with around 40-50 ms player. But what what you see is not always what you get and this is the infamous ‘LAG’ comment we have all heard shouted across our headsets. You press fire, yet the kill cam showed you did not, or you ran behind cover and then die only to see the replay of you 10ft further back, LAG or latency from what the host or server processed and what your last input was is the case.

Now getting an input latency around that 50ms level should be out of our expectations at this point and to be honest it is much lower than the majority of games. Taking an average 30fps console  title such as Forza Horizon or Uncharted you can expect around double that, 100ms still very fast for a remotely hosted application or Cloud if you like PR talk, but more realistic if still more tough. With a baseline on the 40Mbps synchronous internet connection, you can expect latency to vary depending on distance to the server etc. Using current Azure remote servers here in the UK we see a best expectation of 31ms to a UK server, roughly the same as a 30fps game displays a new frame. Leaping to America the best is almost 5 times that at 142ms, this of course varies greatly with the average being much higher, and the other countries will to. This time  taken to connect to a server is never going to be consistent with spikes happening all the time, through Wi-Fi and this will be even bigger.

Shortening the Line

Xbox Scarlet and Streaming Games

The lion’s share of the bandwidth is used to feed the image back to us, using a 1080p image as a guide we would need 5Mbps to get a 30Hz feed of Netflix quality, the best IQ we can expect, 60Hz will require approximately 2x that. Full 4K at these levels leaps to 50Mpbs+ a decent ISP speed for homes. Based on what you have at your house this pushes the limit you reach just in image quality alone not to mention the inconsistency, e.g. if you have a 25Mbps connection then running a 4K feed @30fps will almost certainly have dips and compression artefacts such as macroblocking creeping in, remember buffering the image here is not really an option without adding to that latency.

To summerise for those at the back, this essentially means that the new Scarlet device or current machines will be doing little more than streaming a movie of the game you are playing. The only difference is the game is being rendered and being run miles away on a collection of servers. The resulting rendered image occuring remotely alongside your local input will be processed by the cloud sic!) and resulting frame is beamed back to your TV via your streaming box, like an incredibly long HDMI cable. This repeats every 30 or 60Hz as needed but the remote processing and rendering time would significantly faster than this.

If our aim is to have 100-120ms of latency (this excludes your TV which will add its own depending on how you set this up) then this is possible from current solutions even if the image quality can be noticeably worse than playing a 1080 game at home. How can Microsoft improve this then?

1)    Reduce distance to servers: –

• By adding more server farms and load balancing clients across more of these the current delay from your home to the nearest server can be significantly reduced. Desktop applications do not require the low latency tat games do, this will help when maximising the VM or LPAR’s allocations (see my video on VM’s).

2)   Render the game faster:-

• One of the best ways to improve all factors of the experience is to simply run the game at faster rates. Very simple when you are running PS4 games on servers that are many factors more capable than the host platform and even forthcoming machines.

3)   Process input, decompression, encryption collision locally:-

• The planned new device will likely have a CPU and GPU that is more capable than modern phones. With some storage and processing power input can be calculated locally, data compressed and decompressed, encryption and other elements via hardware thus further reducing latency, bandwidth and quality.

The result of these could mean latency to the server could be 15ms, the game itself that ran at 33ms on your console, now runs at 8ms or more. Your input would still be limited to the distance and latency but you could achieve results that come close to local 30fps games via remote option. I doubt at the start we will see a huge chunk of 60hz games that run as well as they would on dedicated hardware, but this solution is not for us the hardcore but the far more vital audience. Dom Perignon is still one of the best champions money can buy, but more people will drink Moyet or Chardonny and enjoy it all the same.

These are of course just high level areas with many more than the teams will be working on that i could not possibly be aware of. Looking forward it will expand, such as balancing the workloads from logic, physics, rendering etc with both local and remote calculations which is a standard element in online games. The physics based battler I covered at launch from Ready at Dawn ran almost all of its collision and physics calculations on the servers and not on the client machine, this enabled much higher levels of simulation and poll rates, completely oblivious to the player. The Division synced the world time on the servers so everyone has the same experience and environments, forthcoming Crackdown was going to be a shining example of cloud based physics and destruction beyond what the Xbox one could deliver. Since Epic purchased the Edinburgh based Cloudgine, the company behind the tech, this could be one possible reason for the delay. Unreal engine will certainly have this incorporated into its base options soon enough and that will enable enhanced AI options, physics, simulation even lighting can all be factored in.

For now we can look forward to what the next generation of consoles can and will deliver, and for those still after the core, fast and cleanest gaming experience the pure local high powered hardware is the only way. For the masses not concerned about bleeding edge responsiveness, the sharpest pixels then the next round of consoles may not be fought in our homes, but in the clouds.