The value of the triple-A first-party smash hit can’t be downplayed – an incredible quantity of time, cash and effort is focused into developing experiences that press console hardware to its limitations. And in turn, the technological developments discovered in these titles are frequently shown the advancement neighborhood, enhancing the technical quality of titles throughout the board. It’s an increasing tide that drifts all boats, if you like. So it is with Gears 5 from the Coalition – an exceptional release that redefines expectations from Xbox One hardware, while at the very same time providing what we (and lots of others!) think is among the best PC ports of the generation.
In this deep dive technical interview with The Coalition’s technical art director Colin Penty and studio technical director Mike Rayner, we go over how the group even more repeated and personalized essential functions of the Unreal Engine 4 innovation while at the very same time incorporating the most recent and biggest developments from Epic back into their video game. We likewise find how the studio’s intent in doubling frame-rate from 30fps to 60fps produced a variety of modifications that didn’t jeopardize visual quality as such, however rather enhanced it over Gears of War 4.
And then there’s the conversation on the value of native rendering resolution up versus vibrant resolution and image restoration strategies. Alongside the sort of work we’re seeing in titles like the upcoming Call of Duty: Modern Warfare from Infinity Ward, The Coalition executes a versatile technique to rendering resolution to attain 60 frames per 2nd, developing and enhancing from the work performed in Gears of War 4. But how is this done – and are we nearing completion of the pixel-counting period in regards to utilizing a set figure (or series of figures) to get some concept of total image quality?
We’re getting in an amazing time: completion of one console generation and the shift to the next. These end-or-era first-party titles – whether they’re from Sony or Microsoft – not just reveal today’s innovation pressed to its limitations, however likewise assist to set the phase for the generation to come. So get a coffee, take a seat and prepare yourself for a unique insight into the advancement of among Xbox One’s crowning technological accomplishments.
Digital Foundry: Gears of War 4 was a best title using much of the high-end making functions of Unreal Engine 4 at its time of release. UE4 has given that developed throughout Gears 5’s advancement including a volumetric fog, range field shadows/GI, a much faster depth of field, parallax occlusion mapping, higher GPU tessellation assistance, etc. What more recent UE4 functions has The Coalition incorporated for Gears 5 and where are they finest noticeable?
The Coalition: Gears 5 is using a variety of UE4’s most current rendering functions which offers us access to innovation that has actually come online given that Gears of War 4 such as temporal upscaling, volumetric fog, range fields, calculate ray traced shadows, and bent normals for specular occlusion.
We utilize calculate ray traced shadows in the video game beyond our waterfall shadow maps to enable us to have totally vibrant shadows in our video game which aids with our LOD quality, texture memory use, and disk area, while likewise obviously enabling portable shadows. The ray traced shadows makes use of the range field representation of each fixed mesh.
All of our main characters have bent typical and AO maps to enable them to light in an extremely ‘airtight’ method and have no light leakages. It truly assists for a 3rd individual video game with a character utilizing metal armour (glossy)! We utilize volumetric fog throughout our project, versus maps – and our Escape mode leverages it with the ‘venom’ impact. The brand-new and better depth of field is utilized throughout all of our cinematics and front end. It looks exceptional to the DOF that Gears of War 4 utilized on PC’s outrageous quality and is 4x quicker enabling us to run it on Xbox.
Alongside results and strategies packaged with UE4, Gears of War 4 significantly had much of its custom results – like character shadow pills, its own volumetric lighting option, screen-space shadows, virtual point lights from reflective shadow maps for vibrant GI from choose area lights/flashlights, etc. What are some examples of techniques/effects included by The Coalition now for Gears 5 and which ones have returned?
With the relocate to real-time waterfalls for Gears 5 (Gears of War 4 utilized per-object shadows on Xbox One S), we wished to minimize our dependence on screen-space shadows and utilize waterfalls anywhere we could.
We made a huge push for GPU based particles on Gears 5, so we incorporated some GPU spawning innovation from Rare which assisted alleviate the particle problem on the CPU and assisted us attain 60fps on Xbox One X. Our Escape venom impact was an exceptionally complicated impact that streams through a procedural created map in Escape at 60fps on Xbox One. We established a custom-made system that permitted us to dynamically move the volume fog throughout the map, however likewise needed to establish a brand-new particle system that permitted us to run countless ‘venom’ particles in the air and keep efficiency. So, we established a brand-new particle system utilizing vertex shaders that sustained absolutely no CPU efficiency overhead called Swift Particles which we not just utilized for Venom however utilized for all of our ecological VFX.
We additional take advantage of the Swift Particles concept and produced a brand-new system called Swift Destruction which enables us to damage items utilizing a vertex shader. We had Houdini Engine tools we would run in Maya that would pre-fracture a mesh – we would need to view the quantity of fracture points we had on a things – if we went expensive we would truly strike our GPU. Overall though this would develop really thick damage minutes that would sustain no CPU efficiency overhead.
We established a volume fog painting system that would enable us to ‘paint’ volume fog throughout our levels – this permitted us to bypass the basic volume fog injection expense you get on the GPU per light source/material – we would just pay an injection expense as soon as. We likewise would dynamically produce a height map for all of our levels that we would feed into the volume fog painting system, enabling us to call in or out the quantity of ground fog. Sometimes we would utilize this height map for phony GI too in our shaders.
We composed a custom-made calculate tessellation shader which permitted us to produce really high frequency tessellated surface areas and after that run the tessellation async to assist soak up the expense at 60fps. We selected to refrain from doing parallax occlusion mapping due to high expense, and rather executed unwinded cone action mapping (GPU Gems 3) which in fact produces much better quality than POM (no stepping) and runs much quicker. It likewise slotted in well with our customized product masking system (MMS for brief – provided at Siggraph 2017) which needs to bake out textures anyhow per-material so we would likewise pre-compute the cone action map too.
We retained our custom geocache system, originally developed for Halo 5 and likewise used in Gears of War 4. This technology allows us to create massive complex offline destruction simulations in tools like Houdini and then play them back in real-time on the GPU. You will see this in use in a number of the real-time large-scale set-pieces destruction moments in the game. We enhanced and improved our material masking system from Gears of War 4 adding in much more aggressive texture packing and shader optimisations which in-turn allowed us to create more complex layered materials with some headroom for tessellation or Cone Step Mapping.
Digital Foundry: Gears of War 4 had excellent character models with realistic skin, hair (Delmont Walker particularly), and animation deformation – Gears 5 looks noticeably improved in this area particularly in regards to the faces themselves. What technical/artistic changes are reflected here?
The Coalition: We made a huge up-front investment in our characters for Gears 5 and we are really happy with the results. It was a combination of asset improvements and new technology that really helped up make the noticeable leap. First off, we decided to increase our hero character triangle counts by 50 per cent over Gears of War 4, then went about cleaning up all of our face-shapes, wrinkles, and blood flow regions making them much higher quality in the process. We then went through and improved our skin shader by adding dual lobe specular and improving the backscattering component.
We also greatly enhanced our eye shader with dynamic iris caustics, proper sub-surface scattering, AO maps, and other micro-geometry additions and we switched over to using Faceware for our facial animation capture on Gears 5 – capturing our actors’ facial animation, not just on the mo-cap stage but also in the audio booth. This, of course, was then hand tuned by an animator to bring the performance to life. We improved our Hair (including Del’s!) by improving the artist tools for authoring, material tuning, and baking out hair AO maps. The core hair shader is unchanged from Gears of War 4.
Digital Foundry: With many more biomes, weather types and locations, what technical changes or art pipeline changes had to be made? Watching trailers and looking at the PC benchmark, snow/sand deformation looks present and foliage looks rather dynamic.
The Coalition: The real content challenges on Gears 5 has been scale and variety, for sure. Our large overworld maps have a level of scale in them we never had to deal with before in Gears of War – this forced us to generate new workflows and tech to deal with the size (fully dynamic shadows is an example). Having snow and sand in this game was challenging from a content creation point of view, forcing us to create high quality sand and snow materials but also deform them dynamically.
We created a customized deformation system that would allow us to deform large areas of the map and not have the deformation disappear for a long time. We would run a tessellation shader on this deformation allowing there to be enough resolution around the player’s feet so it looks realistic, as well as do some product blending at the deformation point. Far in the distance we cannot afford to run tessellation on those triangles, so we dynamically generate a normal map in the material and blend it in based on distance.
For things like water ripples and vegetation displacing around the character we were inspired by Uncharted 4’s approach to this – so we implemented a similar approach of spawning particles into a buffer which allowed us to influence the materials dynamically. This works best for temporary effects.
Digital Foundry: Gears 5 does not adhere to a strict realism in its graphical presentation, having fantastical materials and proportions, does that alter the authoring of assets/textures and materials in comparison to a more photorealistic presentation?
The Coalition: It does to a certain extent. Our materials tend to be mostly grounded in reality but our objects are not. So we can’t rely on photogrammetry to generate assets for us like props – we tend to concept or gather some rough example assets then ‘Gearsify’ them. This usually involves making them chunkier, dirtier, worn, with metal trim pieces – though there’s an art bible with much more nuanced definition than that.
For our materials we wanted our objects to feel more believable than Gears of War 4, so we moved the content teams to a shared material library for the first time – so this way we didn’t get 25 permutations of ‘steel’ in the database and instead we would just mask and layer our pre-built steel material to generate our assets. This allowed much more consistent and high quality assets, especially when you consider going to scale with our outsource vendors. We found the up-front creation time of an asset increased slightly but the amount of asset iterations we went through decreased dramatically, saving us time.
We likewise leaned into detail maps on our materials much more on Gears 5 – knowing that this game would be an Xbox One X title running at 4k we knew we had to increase our texel density from Gears of War 4 to hold-up and I’m really happy with the results.
Digital Foundry: Alongside the custom layered material model which bakes layers together for performance reasons as used in Gears of War 4, there appears to be a dynamic layer system of sorts now with blood, mud, and other materials dynamically coating character models. How does this exactly work?
The Coalition: We created a slot system for our character materials that can do various things. We have a single ‘dynamic’ slot which allows for temporary material effects such as blood splatter or swarm goo. This was great in that we only have to pay for whatever temporary effect we are showing at the time and not all possible permutations at once. We have a wetness layer as well we activate for our rain levels. We also have additional slots for character power-ups in multiplayer as well as numerous other effects.
Overall, it’s really tricky to balance new gameplay systems and character GPU performance. Every time a designer would add a new weapon or effect (eg an ice cannon that can freeze characters) the tech art team would have to discuss how to cheaply integrate that new effect, leveraging as much of the existing framework as possible.
Digital Foundry: Dynamic resolution was key to Gears of War 4’s performance on Xbox One previously and an amazing tool on PC – how has this changed in Gears 5? Is it taking advantage of temporal reconstruction? What are the set up differences between Xbox One base systems and Xbox One X?
The Coalition: Utilising Epic’s temporal upscaling technology was a huge visual improvement from the standard upscaling that Gears of War 4 used. We render up to native 4K Xbox One X (1080p on Xbox One/S) resolution and dynamically scale based on the load on the GPU to ensure we remain locked at 60fps. We always output and render the post-process chain at native resolution (4K Xbox One X and 1080p on Xbox One/S) and only scale anything that happens before the post-process chain. It allows the frame to remain sharp even while resolution is scaling internally.
Philosophically, it’s much harder to see resolution scaling on Xbox One X due to the higher resolution at play – so we tended to be a bit more sensitive to resolution scaling swings on Xbox One as that can be a bit more apparent. Ultimately, this technique enables us to increase the overall visual fidelity of the game. It allows us to maintain a higher visual baseline across the board and absorb scene-to-scene fluctuations and even extreme scenarios (eg a heavy VFX battle sequence) without dropping frames.
On PC, we now support temporal reconstruction. We have added a setting called minimum FPS. It allows the user to choose the minimum framerate they want the game to render at (30, 60, 90fps or none). If the game detects it is dropping below that framerate it will dynamically scale before the post-process chain and use temporal upscaling to reconstruct the final image at a higher quality than the dynamic resolution feature in Gears of War 4. The scaling range is determined by the game and based on the screen resolution.
Digital Foundry: Asynchronous compute was featured as an option on PC and used on Xbox One X and is key to extracting performance on certain GPU architectures, has this changed in Gears 5?
The Coalition: On Gears of War 4, we ran only a few passes async such as reflection capture actors. We made a large investment on Gears 5 to run our entire post-process chain on async compute. This made a lot of sense given it is always running at native resolution due to the temporal upscale. This gave us a few more milliseconds to work with and achieve 60fps on Xbox One X, as well as in all MP modes on Xbox One S. There was a non-trivial amount of engineering effort to convert every post-process pass we used on Gears 5 to compute so we could then run it async. We also added a deferred present where the base pass of the next frame is started while the GPU completes the async post process pass of the current frame. These same asynchronous compute optimisations are also supported on PC.
Digital Foundry: Gears of War 4 offered some unique effects to the PC version with the insane quality option, like glossy screen-space reflections. What do the insane quality settings this time do on PC for the various effects they affect?
The Coalition: Gears of War 4 supported an insane setting for screen-space reflections and depth of field. On Gears 5, we’ve refactored depth of field to a higher quality and removed the insane setting in the process. We’ve maintained insane screen-space reflections and added it to dynamic shadows and volumetric fog. Insane screen-space reflections are still glossy, but we’ve made some adjustments to make the setting scale better at 4K. Insane volumetric fog is one of our favourites which runs volume fog at such a high quality level it’s indistinguishable from screen-space light shafts.
Our shadows are more scalable than Gears of War 4’s pre-baked shadows, so the Insane shadow quality settings look amazing on Gears 5, with extremely high resolution cascades and spotlight shadows as well as high quality ray traced shadows in the distance. Most noticeable will be the extra detail seen in the distance at the insane setting, with additional cascades and extra distance on the spot lights.
Digital Foundry: DirectX 12 supports ray tracing on PC now through DXR and the Project Scarlett has announced hardware ray tracing support, UE4 has likewise added support for various ray traced effects. Where do you see the usage of ray tracing in Gears 5 or in future titles from The Coalition?
The Coalition: On Gears 5 we are using GPU compute-based ray traced distance field shadows which works across all platforms and does not depend on DXR or dedicated ray tracing hardware. We are excited with the possibilities of DXR and Project Scarlett’s native hardware support for accelerated ray tracing. At this point, we are not ready to talk about our future plans, however we are following closely the emerging techniques and innovations in this exciting space.
Digital Foundry: With image reconstruction techniques becoming more prevalent, what is your opinion of native resolution rendering on systems with set hardware like in consoles?
The Coalition: We love running a game at native resolution and the simplicity and purity of that – but we feel like this will become less and less common, especially if we begin to see 8K games next generation. Temporal reconstruction techniques offer cheap performance returns allowing for higher quality visual systems with very hard to perceive visual quality loss – especially at higher resolutions. Put another way: pixel counting was a very useful thing a few years ago with games like Gears of War 4, as it would give you a clear indication of image clarity. With games that use reconstruction now I think pixel counts are an interesting thing to make note of but isn’t necessarily that indicative of your final output clarity unless you are seeing some extremely lower-than-native resolutions.
One thing we didn’t want to compromise on for Gears 5 was running the output at native resolution. We had all the controls to allow us to scale down from native output resolution if we wanted to for performance reasons but decided against it to maintain the highest possible quality frame 100 per cent of the time. Additionally, we always render the UI and a large number of processing effects at native resolution after the temporal upscale to ensure we have crisp UI and high quality per pixel effects like motion blur, bloom and tone-mapping.
Digital Foundry: What technical aspects of the visuals, or experiences in development from Gears 5, you are particularly proud of?
The Coalition: We are proud of all the systems and workflow improvements listed above, but overall we are most proud of how we were able to push visual quality noticeably ahead of Gears of War 4 while simultaneously pushing the game to 60fps on Xbox One X and 60fps for all MP modes on Xbox One. It was an immense collaborative effort between engineers, technical artists, and the art team to constantly keep 60fps at the forefront of everything we did while not compromising quality, and in-fact improving quality. We had to adopt many new workflows and techniques – especially around keeping the CPU running at 60fps.
Our CPU optimisation efforts carried over to our dedicated game servers, allowing us to double our server simulation rate from 30Hz to 60Hz. This results in end-to-end lower-latency networking for our multiplayer PvP (player versus player) game modes compared to Gears of War 4.
Targeting 60fps for campaign on Xbox One X turned out to be critical in allowing us to hit our split-screen performance targets. We had to be able to run our entire campaign on Xbox One with three-player split screen at 30fps, which is a non-trivial problem when you consider the burden on the draw thread to render 3x the amount of assets, not to mention the GPU overhead. Had we not targeted 60fps I’m not quite sure how we would have pulled off three-player split screen.
We have improved our HDR support in Gears 5 on Xbox One S/X and added support for HDR on Windows 10. Using a large dataset of pairs of HDR/SDR images from Microsoft’s first-party shipped games, we utilized machine learning to train an inverse tone mapper for colour space conversion (Rec.2020) and optimal tuning of HDR output in Gears 5. Of course, we also have in-game calibration to let players tune the output for their particular monitor’s abilities. You can tune against reference images as well as the real-time game.
With Gears 5, all of our cinematics (except for the Gears of War 4 recap and title sequence) are rendered natively in real-time. In the past these would be locked as 1080p videos. In Gears 5, the cinematics will scale to your system and run in 4K on Xbox One X, they will adapt to your PC video options quality settings providing continuity with the gameplay visuals. We feel that the overall visual quality of the real-time cinematics eclipses the off-line rendered cinematic videos we had on Gears of War 4. Our overall push for higher performance along with significant investments in character performance, visual effects and lighting resulted in tools our exceptionally talented cinematics team, animators, VFX, audio and lighting artists were able to use to push further and deliver something really special.
Ultimately seeing the positive reaction to the Technical Test visuals was a great validation of our approach that doubling frame rate doesn’t have to impact visual quality if you plan towards that goal and have some great innovation to go with it – especially with the power of Xbox One X.