Nvidia Ada Gpu Science
Nvidia Ada Gpu Science
Nvidia Ada Gpu Science
V1.0
Table of Contents
Introduction ........................................................................................................................................................... 3
The Era of Neural Graphics ......................................................................................................................................... 3
The Evolution of DLSS .................................................................................................................................................. 4
DLSS 3 Multiplies Performance by Up to 4X ....................................................................................................... 5
DLSS 3 Can Double CPU Bound Performance .................................................................................................... 8
DLSS 3 and Responsiveness — NVIDIA Reflex.................................................................................................... 9
DLSS 3 and the Three Pillars of Performance .................................................................................................... 9
DLSS 3 and NVIDIA Streamline ...............................................................................................................................10
NVIDIA DLSS 3: Available this October ................................................................................................................11
NVIDIA Encoder and Broadcast ...................................................................................................................... 13
AV1 Encoder ...................................................................................................................................................................13
Broadcast SDK Features ...........................................................................................................................................15
Conclusion ............................................................................................................................................................ 16
List of Figures
Figure 1. Ray Tracing Operations Per Pixel Over Time .................................................................................. 4
Figure 2. DLSS Super Resolution Pipeline .......................................................................................................... 5
Figure 3. DLSS 3 With Frame Generation Pipeline ......................................................................................... 6
Figure 4. DLSS Software and Hardware Components .................................................................................. 6
Figure 5. Engine Motion Vectors and Optical Flow ........................................................................................ 7
Figure 6. AI Pixels Provide Most of the Image ................................................................................................. 8
Figure 7. DLSS Can Double CPU Bound Performance .................................................................................. 8
Figure 8. System Latency Pipeline ........................................................................................................................ 9
Figure 9. The Three Pillars of Performance .....................................................................................................10
Figure 10. NVIDIA Streamline and DLSS .............................................................................................................11
Figure 11. DLSS 3 is Optical Multi Frame Generation, Super Resolution, and NVIDIA Reflex .......11
Figure 12. DLSS 3 Momentum ................................................................................................................................12
Figure 13. AV1 Encoder Performance ..................................................................................................................14
Figure 14. AV1 Image Quality ..................................................................................................................................15
Figure 15. Virtual Background, Face Expression Estimation, and Eye Contact...................................16
Introduction
Technology is more than mere speeds and feeds. What you do with technology, and the novel
applications and innovations it leads to, is what we call science. Science lets us invent things the
world has never seen, and it has led to the concept of neural graphics. In this whitepaper, the
science of GPUs is illustrated in DLSS—how innovative hardware acceleration techniques have
enabled the use of artificial intelligence for game performance.
In late 1999, NVIDIA introduced GeForce 256, the world’s first GPU, which included powerful
hardware-based transform and lighting engines. A few years later, the programmable shading era
was born with the GeForce 3 GPU introducing programmable pixel and vertex shaders.
Programmable shading GPUs revolutionized 3D and made possible the beautiful graphics we see
in games today. Nearly 20 years after our invention of the GPU, we launched NVIDIA RTX—a new
architecture with dedicated processing cores that enabled real-time ray tracing and accelerated
artificial intelligence algorithms and applications. RTX—the fusion of programmable shading, ray
tracing, and AI—started the next era of graphics: neural graphics.
Neural graphics has quickly become integral to how artists create 3D worlds, and how the world is
animated and rendered. One of the most impactful neural graphics inventions is Deep Learning
Super Sampling (DLSS)—an AI that has learned to enhance the resolution of motion graphics and
improve performance of ray-traced games and creator applications.
DLSS analyzes the current and past frame, along with game motion vectors that describe geometry movement, and its
deep learning algorithm provides a new supersampled image. This higher resolution output is based on a lower
resolution input, and dramatically improves game performance.
DLSS 3 combines previous DLSS techniques, such as motion vectors and Super Resolution, with new Frame Generation,
leveraging the Ada architecture’s Optical Flow Accelerator.
and analyze images. Some of the hardest challenges in frame generation require hardware, such
as the Optical Flow Accelerator, to overcome. The Optical Flow Accelerator is able to capture
pixel-level information such as particles, reflections, shadows, and lighting, which are not included
in game engine motion vector calculations. Post processing effects and ray tracing effects are
difficult to reconstruct accurately because they lack the same type of motion vectors associated
with geometry in a game. The standard game engine motion vectors—used in temporal anti-
aliasing algorithms to blend pixels from the prior frame with the current frame—may account for
the positioning of moving geometry, but they do not help determine the position of many types
of post processing effects, and this causes artifacts like ghosting, stutter, and blurriness.
For example, the following game communicates to DLSS the motion vectors for geometry, such
as the street, but has no motion vectors to pass along for the ray-traced shadow that is also
moving while cast against the street. Without motion vectors for ray tracing effects, the shadow
would be reconstructed inaccurately, causing stuttering or ghosting as the motion vectors of the
street drag the shadow in the wrong direction.
DLSS 3 reconstructs 7/8th of the total displayed pixels using AI. Frame 1 uses DLSS Super Resolution to reconstruct a
higher resolution frame, and frame 2 then uses DLSS Frame Generation to entirely generate a new frame before
resuming DLSS Super Resolution in frame 3, and so on.
DLSS 3 reconstructs 7/8th of the total displayed pixels using AI. Frame 1 uses DLSS Super Resolution to reconstruct a
higher resolution frame, and frame 2 then uses DLSS Frame Generation to entirely generate a new frame before
resuming DLSS Super Resolution in frame 3, and so on.
influence or modulate each other: a fast game with poor image quality is not a great experience,
and a game with punishing latency will feel sluggish despite high FPS performance.
NVIDIA Image Scaling, NVIDIA Real-Time Denoisers (NRD), NVIDIA Reflex, and Optical Multi Frame
Generation and DLSS technologies.
Streamline offers a single integration with a plug-and-play framework, including DLSS 2 and 3.
Figure 11. DLSS 3 is Optical Multi Frame Generation, Super Resolution, and
NVIDIA Reflex
AV1 Encoder
The dedicated hardware encoders on our GPUs receive a major update with the 8th Gen NVIDIA
Encoder (also known as NVENC), with support for AV1. This new codec developed by AOM—a
consortium of the leading video companies in the world, including NVIDIA—massively increases
encoding efficiency, allowing users to dramatically improve their video streaming quality. With
AV1 support, our new encoder is 40% more efficient at encoding than H.264. This improvement
makes streams look much better, as if using 40% more bandwidth. It also unlocks new
experiences for end users who face bandwidth limitations. For example, Twitch and Discord limits
streams to 8 Mbps, which was not enough to stream 1440p resolution with H.264. Using AV1,
streams look amazing at 8 Mbps and 1440p resolution. Incidentally, we have worked with Discord
to enable AV1 end-to-end streaming; Discord plans on releasing AV1 streaming to their users
later this year.
NVENC AV1 encoder provides massive encoding speedups at all major resolutions compared to the previous H.264
encoder.
in the next build of DaVinci Resolve, Jianying—China's top video editing app—and Voukoder, a
popular encoder plugin for Adobe Premiere Pro.
Figure 15. Virtual Background, Face Expression Estimation, and Eye Contact
Face Expression Estimation is a new feature in our Augmented Reality toolkit. The feature tracks
face expressions to animate face meshes in digital avatars, providing better transfer of emotions
to the avatar. This feature will help developers create more realistic avatars. It will be available in
VTube Studio, one of the top VTubing apps, in October.
Eye Contact is a feature that tracks the position of the subject’s eyes and morphs their position
to directly view the camera, no matter where the eyes might be looking. The effect looks
extremely natural because Eye Contact maintains your natural movements, like blinking. Eye
Contact helps users chatting over the internet maintain direct eye contact even though the
webcam is not positioned in the exact center of the screen. It is also immensely useful for
removing eye movements while reading from screens or teleprompters.
All these features are available for our partners, free of charge. They are shipping in more than 10
applications. And, the top device manufacturers are using these effects to enhance their devices,
including Logitech, Corsair, and Elgato.
Conclusion
The science of NVIDIA technology is introducing the next era of rapid performance improvements
with help from DLSS 3: using the boundless opportunities of artificial intelligence, paired with
dedicated hardware acceleration, to advance performance. The future will continue to see cutting
edge technological advancements married to powerful neural graphics concepts, and we will
continue investing to invent the hardware and software that powers the neural graphics
revolution.
Notice - The information provided in this specification is believed to be accurate and reliable as of the date provided.
However, NVIDIA Corporation (“NVIDIA”) does not give any representations or warranties, expressed or implied, as to
the accuracy or completeness of such information. NVIDIA shall have no liability for the consequences or use of such
information or for any infringement of patents or other rights of third parties that may result from its use. This
publication supersedes and replaces all other specifications for the product that may have been previously supplied.
NVIDIA reserves the right to make corrections, modifications, enhancements, improvements, and other changes to this
specification, at any time and/or to discontinue any product or service without notice. Customer should obtain the
latest relevant specification before placing orders and should verify that such information is current and complete.
NVIDIA products are sold subject to the NVIDIA standard terms and conditions of sale supplied at the time of order
acknowledgement, unless otherwise agreed in an individual sales agreement signed by authorized representatives of
NVIDIA and customer. NVIDIA hereby expressly objects to applying any customer general terms and conditions with
regard to the purchase of the NVIDIA product referenced in this specification.
NVIDIA products are not designed, authorized or warranted to be suitable for use in medical, military, aircraft, space or
life support equipment, nor in applications where failure or malfunction of the NVIDIA product can reasonably be
expected to result in personal injury, death or property or environmental damage. NVIDIA accepts no liability for
inclusion and/or use of NVIDIA products in such equipment or applications and therefore such inclusion and/or use is at
customer’s own risk.
NVIDIA makes no representation or warranty that products based on these specifications will be suitable for any
specified use without further testing or modification. Testing of all parameters of each product is not necessarily
performed by NVIDIA. It is customer’s sole responsibility to ensure the product is suitable and fit for the application
planned by customer and to do the necessary testing for the application in order to avoid a default of the application or
the product. Weaknesses in customer’s product designs may affect the quality and reliability of the NVIDIA product
and may result in additional or different conditions and/or requirements beyond those contained in this specification.
NVIDIA does not accept any liability related to any default, damage, costs or problem which may be based on or
attributable to: (i) the use of the NVIDIA product in any manner that is contrary to this specification, or (ii) customer
product designs.
No license, either expressed or implied, is granted under any NVIDIA patent right, copyright, or other NVIDIA intellectual
property right under this specification. Information published by NVIDIA regarding third-party products or services
does not constitute a license from NVIDIA to use such products or services or a warranty or endorsement thereof. Use
of such information may require a license from a third party under the patents or other intellectual property rights of
the third party, or a license from NVIDIA under the patents or other intellectual property rights of NVIDIA. Reproduction
of information in this specification is permissible only if reproduction is approved by NVIDIA in writing, is reproduced
without alteration, and is accompanied by all associated conditions, limitations, and notices.
ALL NVIDIA DESIGN SPECIFICATIONS, REFERENCE BOARDS, FILES, DRAWINGS, DIAGNOSTICS, LISTS, AND OTHER
DOCUMENTS (TOGETHER AND SEPARATELY, “MATERIALS”) ARE BEING PROVIDED “AS IS.” NVIDIA MAKES NO
WARRANTIES, EXPRESSED, IMPLIED, STATUTORY, OR OTHERWISE WITH RESPECT TO THE MATERIALS, AND
EXPRESSLY DISCLAIMS ALL IMPLIED WARRANTIES OF NONINFRINGEMENT, MERCHANTABILITY, AND FITNESS FOR A
PARTICULAR PURPOSE. Notwithstanding any damages that customer might incur for any reason whatsoever, NVIDIA’s
aggregate and cumulative liability towards customer for the products described herein shall be limited in accordance
with the NVIDIA terms and conditions of sale for the product.
Trademarks - NVIDIA, the NVIDIA logo, GeForce, and GeForce RTX are trademarks and/or registered trademarks of
NVIDIA Corporation in the U.S. and other countries. Other company and product names may be trademarks of the
respective companies with which they are associated.