- hace 1 semana
Categoría
🎮️
VideojuegosTranscripción
00:00NVIDIA Theosis 5 introduces 3D-guided neuro-rendering,
00:04an AI model designed to bring real-time graphics
00:07closer to Hollywood-grade photorealism.
00:09Producing that level of realism in real-time
00:12is one of the hardest challenges in graphics.
00:14Offline rendering can spend minutes, if not hours,
00:17to refine a single frame,
00:19but games have to render each frame in just a few milliseconds.
00:22So game developers often have to scale back
00:25their final vision of the game
00:26to hit their real-time performance target.
00:28While game developers have been fighting that battle,
00:32generative models learn photorealism,
00:34not by simulating the physics of life,
00:36but by watching the real world.
00:38But those models are big and slow and probabilistic,
00:42so they're hard to control like traditional rendering.
00:45These models take abstract input like text prompt,
00:48which leaves plenty of room for them to improvise.
00:51When given the same prompt 10 times,
00:53a model will output 10 different results.
00:56So DLSS5 takes the best of both worlds.
00:59It's not conditioned on prompts.
01:01It uses the render frame as the foundation,
01:04which was precisely designed by its artist
01:07and built by the game engine.
01:09And generation becomes the final appearance stage,
01:12infusing the scenes with photorealistic lighting and materials.
01:16This enables developers to push beyond the traditional limit
01:20of real-time compute and VRAM
01:22to realize their ultimate artistic vision.
01:25Making generation work inside a real-time renderer
01:28meant solving three fundamental challenges.
01:31The first one is preserving artistic intent.
01:35Generative models alter the images by design,
01:38so we train DLSS5 to preserve the artistic intent
01:41by keeping the geometry, textures,
01:44and light-shadow relationships
01:45exactly as the artist created them.
01:48This guarantees character identity,
01:51composition, and scene semantics are all locked.
01:54It also means that neural rendering model is deterministic.
01:58The same input will always produce the same output.
02:02The second challenge is consistent frame-by-frame generation.
02:06Regular video models have to generate frames in chunks
02:10to make them temporarily coherent.
02:12But in a game, the next frame depends on the player input.
02:15By leveraging motion vectors from the game engines,
02:18DLSS5 runs one frame in, one frame out,
02:20while ensuring the output is stable and consistent
02:23with no shimmer, swimming, or drifting.
02:27The third challenge is speed.
02:29Generative foundation models can create images of anything,
02:32but it takes seconds per frame.
02:34So we designed a fast, specialized generative model
02:37that does only one thing.
02:39Make real-time rendering look more real.
02:43And make no mistake,
02:44it's not a conventional post-process filter.
02:47The model still carries what those big generative models
02:50learned about the real world.
02:52It understands things like object types,
02:54human features, and light sources,
02:56and what cues can make these elements more photo-real.
03:00For example, mirror rendering knows only to add scatter or glow
03:04on hair and fur, but not metal crates.
03:07Making an image more realistic is an artistic decision.
03:11There's no single correct answer.
03:14Which look fits the game,
03:15how strong the effect should be,
03:17and where it should apply,
03:19those are creative calls to make,
03:21and they belong to the game's artist.
03:23So we built artistic control directly into the Elsus 5.
03:27The frame defines what must remain,
03:30the artist directs what may change.
03:34Hi everyone, I'm Gav.
03:35I lead a group of tech artists and art directors here at NVIDIA
03:38who have spent our careers working in game studios.
03:41Our job is to bring an artist's eye
03:43to the heart of NVIDIA technology.
03:45We work directly with NVIDIA researchers
03:47to give developers and artists
03:48the technologies and tools to bring their vision to life.
03:51To demonstrate the benefits of DLSS 5,
03:53we will show scenes from two NVIDIA tech demos
03:56and the controls developers can use
03:58to art direct the final result.
04:00Starting with the Raman chef,
04:01his geometry and facial structure are unchanged
04:03when the LSS 5 is enabled.
04:05What changes are the visual cues and the light response.
04:08Around the head and fabric,
04:09ambient occlusion is more defined
04:11and contact shadows are tighter.
04:13The ears gain more natural subsurface scattering
04:16as light passes through them
04:17and the eyes pick up clearer reflections.
04:20Those additions do not replace the chef
04:22with unrelated interpretation.
04:24They make the existing character,
04:25materials, and lighting feel more complete.
04:28Let's take a look at how DLSS 5 interacts
04:30with the clear render scene from Zora,
04:32which has no materials or textures.
04:34You can see the neural rendering
04:36is not altering any shapes.
04:38Instead, the lighting, materials,
04:40and self-shadowing become more complete.
04:42The model preserves and enhances
04:44the foundation the game provides.
04:46Follow the shape of her mouth
04:48and the proportions around her eyes.
04:49They remain intact.
04:51With neural rendering,
04:53the lips and skin gain more convincing
04:54self-shadowing and reflections.
04:56They air against more natural strand definition,
04:58volume, and light response.
05:00The result remains grounded in the render frame
05:02and the information provided by the game.
05:05The better the foundation,
05:06the more useful data the model has
05:07to generate a realistic output.
05:09More detailed assets,
05:10such as materials and lighting,
05:12contribute to the foundation.
05:13Here is a different section
05:15of the same Zora demo,
05:16rasterized, without ray tracing.
05:18And now, let's take a look at this scene
05:20with DLSS 5 on.
05:21The 3D-guided neural rendering enhancements
05:23are clear.
05:24However, the result is still not as good
05:26as it could be,
05:27because the original frame lacks
05:28accurate lighting information.
05:30But with ray tracing enabled,
05:32we can see that the DLSS output
05:34is far more detailed and realistic
05:35because the input has a higher quality source.
05:38And when the base render leverages path tracing
05:40for even more accurate information,
05:42the neural rendering system
05:44has even more to work with.
05:45Now that we have seen the visual results,
05:47let's take a look how developers can control it.
05:49There are multiple levels of control.
05:51Some come from the model itself,
05:53while others come from the game engine.
05:55Together, they let developers art-direct
05:57how and where neural rendering is applied.
06:00The SDK offers several models
06:02that produce different outputs
06:03because they are trained
06:04with different parameters.
06:05A developer can choose the model
06:07that works best for their content.
06:08And that choice does not have to apply
06:10to the entire game.
06:11An interior can use one model
06:13and outdoors another.
06:15Developers are able to mix and match.
06:17Now that we have selected a model,
06:19Structure Intensity modulates
06:21high-frequency details in the frame,
06:22such as ambient occlusion,
06:24contact shadows, reflections,
06:26and subsurface scattering.
06:28Tone Intensity regulates low-frequency details,
06:30such as broader lighting and color response,
06:33which are more expressive.
06:34Together, they let artists decide
06:36how much enhancement is applied to the output.
06:38But what if enhancing the entire scene
06:40is not the intended direction?
06:41This is where masking comes in.
06:43On the model side,
06:44the LSS-5 understands the content
06:46and object-level semantics of the scene.
06:48It recognizes the base character
06:50and can automatically generate a mask.
06:52Look at how moving the slider
06:54is only affecting the character intensity.
06:56A developer can then enhance
06:58only that character
06:59or hold the character back
07:01while boosting the environment around them.
07:03And now, we're applying Uplift
07:05only to the character
07:06while leaving the environment unchanged.
07:09Here is the same model,
07:11a 25, 50, and 95% Structure Intensity.
07:1525% stays closer to the render frame,
07:18while 95% moves closer
07:20to the model's maximum uplift.
07:21The developer chooses the strength
07:23that fits their artistic direction.
07:25And here is the same Structure Intensity
07:27across different models,
07:29generating distinct outputs.
07:31Let's take a look at this scene
07:33where global controls
07:34enhance the entire image.
07:35Turning the LSS-5 on,
07:37and we have stronger occlusion,
07:38which gives the table
07:39and the background
07:39the appropriate depth and darkness.
07:41The glassware looks more realistic.
07:43The metal cups pick up clearer reflections.
07:45And the grapes
07:46have a more convincing surface response.
07:48These improvements
07:49are all happening across the scene,
07:51all at once.
07:52On the engine side,
07:53developers can implement masks
07:54for the exact objects
07:55or group they want to redirect.
07:57There is no limit
07:58to how many groups can be implemented,
08:00and masking can be as granular
08:01as developers want.
08:02In this scene,
08:03you can apply neural rendering
08:04only to the dev masks objects,
08:06like the picture,
08:08the glass,
08:09or the grapes.
08:10In this other setup,
08:11the cutting board
08:12and the props
08:12at the top of the counter
08:13can be separate group masks.
08:15Each mask can have its own structure
08:17and tone adjusted,
08:18while the rest of the scene
08:19stays unchanged.
08:22Imagine a developer
08:23wants to enhance
08:24subsurface scattering
08:26only for plants,
08:27foliage,
08:28or food.
08:29They could apply it selectively
08:30across the entire game
08:32without affecting anything else.
08:33The key point
08:34is that developers
08:35have full control
08:36where neural rendering is used
08:37and how it is applied,
08:38and how strong the effect is,
08:40allowing them to mix
08:41and match as needed.
08:42Characters,
08:43environments,
08:44gameplay,
08:45and cutscenes
08:45can each be treated differently.
08:47Last but not least,
08:48let's take a look
08:49how DLSS can enhance
08:50an environment.
08:51For this outdoor foliage scene,
08:52we have selected the model
08:53best suited to the scenario.
08:55Now,
08:56we can use the intensity slider
08:57to bring the environment to life.
08:59Even without DLSS 5,
09:00the ray-traced scene
09:01already looks impressive,
09:02but it's still missing the details
09:04that make it feel truly realistic.
09:06As we increase
09:06the structure intensity,
09:07you can see improved shadowing
09:09and contact shadows,
09:10along with more convincing
09:11material transmission.
09:12This is especially noticeable
09:14in the leaves.
09:15Subsurface scattering
09:16allows light to pass
09:18through them naturally,
09:19giving the foliage
09:19greater depth and realism.
09:21Achieving this level
09:22of subsurface scattering
09:23in real time
09:24will require shooting
09:25far too many arrays
09:26to maintain performance,
09:27but neural rendering
09:28makes this possible.
09:29DLSS 5 is not one look
09:31applied everywhere.
09:32It's a flexible toolset
09:33developers can direct,
09:35so the artist's ultimate
09:36creative vision
09:36can reach the screen.
09:38These are the tools
09:39currently available
09:40to developers today
09:41as they integrate
09:42DLSS neural rendering
09:43into their games.
09:44And we're adding
09:45more controls and models
09:46in close collaboration
09:47with our partners.
09:48Their input directly shapes
09:49the features we add
09:50and how we refine the toolset.
09:52DLSS 5 3D-guided
09:54neural rendering
09:54is a breakthrough
09:55in both AI and graphics,
09:58a new level of realism
09:59for games.
10:00DLSS 5 extends the pipeline.
10:03It doesn't replace it.
10:04And that's why we continue
10:05to work on other RTX
10:07and DLSS technologies.
10:09Super resolution,
10:10frame generation,
10:11and ray reconstruction.
10:12And we want to be clear,
10:14we're not claiming
10:14photorealism is solved.
10:16This is the very beginning
10:18of real-time AI generation.
10:20We initially previewed DLSS 5
10:22on two RTX 1590 GPUs
10:25and have continuously improved
10:26both image quality
10:27and performance
10:28to run on a single GPU
10:29with more optimizations coming.
10:32DLSS 3D-guided neural rendering
10:34will be supported
10:35on all GeForce RTX 50 series GPUs.
10:38By combining traditional rendering
10:41with a specialized
10:42generative model,
10:43DLSS 5 has made
10:44a tremendous leap
10:45in image fidelity.
10:47And the model is getting faster
10:49and more intelligent every day.
10:51We're just getting started
10:52and can't wait
10:53for you to try it yourself.
Comentarios