Rendering in UE4（Epic Game TA-Homam Bahnassi講座個人筆記）

04-21

Presented at the Gnomon School of VFX in January 2018, part two of the class offers an in-depth look at the rendering pipeline in Unreal Engine, its terminology and best practices for rendering scenes in real-time. This course also presents guidelines and profiling techniques that improve the debugging process for both CPU and GPU performance.

分7個部分介紹UE4中的渲染管線。

Index

1.Intro
2.Before Rendering
3.Geometry Rendering
4.Rasterizing and Gbuffer
5.Dynamic Lighting/Shadows
6.Static Lighting/Shadows
7.Post Processing

1.INTRO

Everything needs to be as efficient as possible
Adjust piplelines to engine and hardware restrictions
Try to offload parts to pre-calculations
Use the engines pool of techniques to achieve quality at suitable cost
CPU and GPU handle different parts of teh rendering calculations
They are interdependent and can bottleneck each other
Know how the load is distributed between the 2
不僅僅用來渲染高質量的靜態圖片，也用來渲染有交互的動態場景。
Quality Features Performance 三者間的權衡
調節引擎的pipelines和硬體限制
進行預計算

Shadring techniques

Real time rendering techniques are differnt fromm offline rendering
Expensive ray-tracing features are approximated or pre-calculated
Depends on projection(rasterization)
Shading/lighting are mainly done either through defferred or Forward shading UE4 supports both

Deferred Shading

1.Composition based using the GBuffer

2.Shading happens in deferred passes

3.Good at rendering dynamic lighting

4.More flexible when it comes to disabling feature,less flexible when it comes to surface attributes

延遲渲染：通過GBuffer，渲染動態光照更有優勢。當涉及到禁用特性時，更靈活，在涉及表面屬性時不那麼靈活。

============================================

2.BEFORE RENDERING

CPU-Game Thread

Calculate all logic and transforms

1.Animations
2.Position of models and objects
3.Physics
4.AI
5.Spawn and destroy,Hide and Unhide

Anything that relates to the posistion of objects to change

CPU階段計算所有的邏輯和轉換，動畫，坐標，物理屬性，創建和銷毀

============================================

CPU-Draw Thread

Before we can use the transforms to rendering the image we need to know what to include in the rendering

Ignoring this question might make rendering expensive on GPU

Occlusion process-Builds up a list of all visible models/objects

Happens per object-Not per triangle

Stage process-in order of execution

1.Distance Culling
2.Frustum Culling
3.Precomputed Visibility
4.Occlusion Culling

幾種剔除：距離剔除，視錐剔除，預計算，遮擋剔除。

剔除具體到物體，而不是三角面

============================================

Occlusion Performance Implications

UE4 has a list of models to render

1.Set up manual culling(i.e.distance culling,pre-coputed vis)
2.Even things like particles occlude
3.Many small objects cause more stress on CPU for culling
4.Large models will rarely occlude and thus increase GPU
5.Know your world and balance objects size vs count

性能分析，1.設置距離剔除，預計算來提高性能

2.小物體太多影響性能，大物體基本上不影響遮擋

3.找到平衡，場景中物體的大小的數量。

============================================

3.GEOMETRY RENDERING

GPU-Prepass/Early z pass

The GPU now has a list of models and transforms but if we just render this info out we could possibly cause a lot of redundant pixel rendering

Similar to excluding objects,we need to exclude pixels

We need to figure out which pixels are occlluded

To do this, we generate a depth pass and use it to determine if the given pixel is in front and visible

z pass 來處理像素的渲染，被遮擋的不渲染。

============================================

Drawcalls

GPU renders drawcall by drawcall not triangle by traingle

A drawcall is group of tris sharing the same properties

Drawcalls are prepared by the CPU(Draw) thread

Distilling rendering info for objects into a GPU state ready for submission

GPU 渲染物體通過drawcall 而不是三角形，CPU階段提交drawcall到GPU state

============================================

UE4 with current gen high-end PCs

2000-3000 is reasonable

More than 5000 is getting high

4.RASTERIZING AND GBUFFER

Rasterzing

GPU ready to render pixels

Determine which pixels should be shaded called rasterizing

Done drawcall by drawcall then tri by tri

Pixel Shaders are responsible for calculating the pixel color

Input is generally interpolated vertex data, texture samplers

Rasterizing inefficiency

When rasterizing dense meshes at distance, they converge to only few pixels

A waste of vertex processing

A 100k tris object seen from so far away that it would be 1 pixel big,will only show 1 pixel of its closest triangle!

光柵化：ps處理vs階段傳來的頂點信息，距離特別遠的mesh，可能占的像素特別小，會浪費許多vs階段的性能。

============================================

Overshading

Due to hardware design, it always uses a 2x2 pixel quad for processing

If a traingle is very small or very thin then it means it might process 4 pixels while only 1 pixel is actually filled

由於硬體的原因，每次處理2x2 4個像素

============================================

Rasterization and Overshading Performance Implications

Triangles are more expensive to render in great density
When seen at a distance the density increases
Thus reducing triangle count at a distance(lodding/culling) is critical
Very thin triangles are inefficient because they pass through many 2x2 pixel quads yet only fill a fraction of them
The more complex the pixel shader is the more expensive

性能分析：密度大的三角面，性能要求高。距離遠密度會變大，儘可能降低三角面個數，thin tri資源消耗大。

============================================

Results are written out to:

Multiple Gbuffers in case of deferred shading

Shaded buffer in case of forward shading

光柵化後的數據用在延遲光照的Gbuffer中。

GBuffer PPerformance Implications

The GBuffer takes up a lot of memory and bandwidth and thus has a limit on how many different GBuffer images you can render out

Gbuffers memory is resolutions dependent

性能分析：GBuffer佔用大量內存帶寬，能渲染出的GBuffer數量有限。

5.LIGHT AND SHADOWS

Two approaches for lighting and shadows

Dynamic
static

Lighting(Deferred Shading)

Is calclated and applied using pixel shaders

Dynamic point lights are rendered as spheres

The spheres act like a mask

Anything within the sphere is to receive a pixel shader operation to blend in the dynamic light

動態點光源渲染成球體，相當於一個蒙版遮罩，遮罩內的像素，在ps裡面做混合

============================================

Light calculation requires position

Depth buffer used to get pixels pos in 3D

Use normal buffer to appley shading.Direct between Normal and light

計算光照，深度depth buffer和Normal buffer共同作用，計算光照。

============================================

Shadows

Common technique for rendering shadows is Shadow Maps

Check for each pixel if it is visible to the given light or no

Requires rendering depth for light Pov

在light view空間下，渲染shadow map。

Process Pros/Cons

Pros

Is rendered in real time using the GBuffer
Lights can be changed,moved,or add
Does not need any special model preparation

Cons

Especially shadows are performance heavy

利弊分析：

利：利用GBuffer實時渲染可以動態調整燈光

弊：性能代價

============================================

Quality Pros/Cons

1. Shadows are heavy on performance, so usually render quality is reduced to compensate

2. Doea not do radiosity/global illumination for majority of content

3. Dynamic soft shadows are very hard to do well, dyn shadows ofter looks sharp or blocky

質量利弊：

性能代價大，降低質量提高性能；無法渲染自發光和全局光照；動態軟陰影效果差。

============================================

Dynamic Lighting Performance Implications

Small dyn light is relatively cheap in a deferred renderer
The cost is down to the pixel shader operations, so the more pixels the slower it is
the radius must be as small as possible
Prevent excessive and regular overlap

動態光照性能分析：

延遲渲染動態光源小，性能佔用較小。

成本受ps影響，處理像素越多，越慢。

半徑盡量小。避免過度疊加。

============================================

Dynamic Shadows Performance Implication

Turn off shadow casting if not needed

The tri count of geometry affect shadows perf
Fade or toggle off shadows when far away

動態陰影性能分析：

關閉不必要的陰影，三角面數量影響陰影效果，距離遠的時候簡化陰影。

============================================

6.STATIC LIGHTING AND SHADOWS

Dynamic lights and shadows expensive

Thus part of it is offloaded to pre-calculations/pre-rendering

This is referred as static lights and shadows

Lighting data stored mainly in lightmaps

動態光照昂貴，使用lightmap。

Lightmaps

A lightmap is a texture with the lighting and shadows baked into it

An object usually requires UV lightmap coordinates for this to work

This texture is then multiplied on top of the basecolor

將光照信息烘焙到原有的紋理信息上。

============================================

Lightmass

Stand alone application that handles light rendering,baking to lightmaps and integerating into materials

Raytracer supporting Gl

Supports distributed rendering over a network

Bake quality is determined by Light Build Quality as well as settings in the Lightmass section of each level

Better to have a lightmass importance Volume around part of the scene

光照烘焙：單獨的模塊處理光照渲染。支持全局光照，烘焙區域和質量可調節。

============================================

Process Pros/Cons

Super fast for performance in real-time, but increases memory
Takes a long time to pre-calculate the lighting
Each time something is changed,it must be re-rendered again
Models require lightmap UVs,this additional prep step that takes time

利弊分析：

速度更快，但內存增加；須要花時間預處理；場景改變重新烘焙；模型須要光照uv

============================================

Quality Pros/Cons

1. Handles Radiosity and Global Illumination

2. Renders realistic shadows including soft shadows

3. Quality is dependent on lightmap resolution and UV layout

4. May have seams in the lighting due to the UV layout

質量利弊：

可以處理輻射度和全局光照；

可以渲染逼真的陰影；

質量受lightmap解析度和uv布局影響；

uv布局影響可能出現縫隙；

============================================

Static Lighting Performance Implications

1. Static Lighting always renders at the same speed

2. Lightmap resolution affects memory and filesize,not framerate

3. Bake time are increased by:

Lightmap resolutions
Number of models/light
Higher quality settings
Lights with a large attenuation radius or source radius

靜態光照性能分析：

光照貼圖影響內存和文件大小。貼圖解析度增大，燈光和模型增加，質量提高，光源半徑增大都會導致烘焙時間增多。

============================================

7.POST PROCESSING

Visual effects applied at the very end of the rendering process

Uses the GBuffers to calculate its effects

Once more relies heavily on Pixel Shaders

後處理：

使用Gbuffer計算效果。

Example:

light Bloom
Depth of Field/Blurring
Some types of lensflares
Light Shafts
Vignette
Tonemapping/Color correction
Exposure
Motion Blur

光暈效果，

景深/模糊，

光澤貼圖/顏色校正，

曝光，

運動模糊。

Post Processing Performance Implications

Affected directly by final resolution

Affected by shader complexity

Parameter(e.g.DoF blur radius)

後處理性能分析：

受解析度影響；受shader複雜度影響；參數影響，如模糊半徑。

參考視頻：

Gnomon Masterclass Part II: Rendering in UE4 | Event Coverage | Unreal Engine

視頻鏈接?

www.youtube.com