### Control Keys

move to next slide (also Enter or Spacebar).
move to previous slide.
d  enable/disable drawing on slides
p  toggles between print and presentation view
CTRL  +  zoom in
CTRL  -  zoom out
CTRL  0  reset zoom

Slides can also be advanced by clicking on the left or right border of the slide.

### Notation

Type Font Examples
Variables (scalars) italics $a, b, x, y$
Functions upright $\mathrm{f}, \mathrm{g}(x), \mathrm{max}(x)$
Vectors bold, elements row-wise $\mathbf{a}, \mathbf{b}= \begin{pmatrix}x\\y\end{pmatrix} = (x, y)^\top,$ $\mathbf{B}=(x, y, z)^\top$
Matrices Typewriter $\mathtt{A}, \mathtt{B}= \begin{bmatrix}a & b\\c & d\end{bmatrix}$
Sets calligraphic $\mathcal{A}, B=\{a, b\}, b \in \mathcal{B}$
Number systems, Coordinate spaces double-struck $\mathbb{N}, \mathbb{Z}, \mathbb{R}^2, \mathbb{R}^3$

### List of Math Symbols

Symbol Meaning
$\Omega$ Solid angle
$\theta$ Polar angle in the spherical coordinate system
$\phi$ Azimuth angle in the spherical coordinate system
$\Phi$ Radiant flux
$I$ Radiant intensity
$E$ Irradiance
$L$ Radiance
$\mathrm{f}_r$ BRDF (Bidirectional Reflection Distribution Function)
$\mathrm{f}_d$ Diffuse part of the BRDF
$\mathrm{f}_s$ Specular part of the BRDF

### List of Math Symbols

Symbol Meaning
$\mathbf{n}$ Surface normal
$\mathbf{v}$ Unit vector in view direction
$\mathbf{l}$ Unit vector in light direction
$\eta$ Refractive index
$F$ Fresnel reflectance
$\mathbf{h}$ Halfway vector between light and view direction
$(\dots)_+$ Ramp function
$\langle \mathbf{a}\cdot \mathbf{b}\rangle$ Scalar product

### Image-based Lighting

• With image-based lighting, the emitted radiance $L_i$ of the environment is specified by a spherical environment image
• Even without taking occlusion into account, according to the rendering equation, for a surface point all incident radiances $L_i$ must be multiplied by the BRDF and summed up. However, this would be too time-consuming for real-time computation.
• If a parameterized BRDF is given, e.g. Phong BRDF, the integrals can be pre-computed and saved in a parameterized way (Pre-Filtered Environment Map)
• For example, with the Phong BRDF, the diffuse part could be parameterized by the normal direction and the specular part can be parameterized by the reflection direction and stored in a spherical environment image
• Often the results for different Phong glossiness exponents $n_s$ are stored in the mipmap levels of a texture

### Environment Lighting (Modified Phong BRDF)

• Starting from the rendering equation, the modified Phong BRDF is inserted (both equations are known from Part 10, Chapter 1):
\begin{align}L_o(\mathbf{v}) &= L_e(\mathbf{v}) + \int\limits_\Omega \mathrm{f}_r(\mathbf{v}, \mathbf{l})\, \, L_i(\mathbf{l}) \cos(\theta) \, d\omega\\ &= L_e(\mathbf{v}) + \int\limits_\Omega \left(\rho_d \frac{1}{\pi} + \rho_s \frac{n_s+2}{2 \pi} \,\cos(\alpha)^{n_s} \right)\, \, L_i(\mathbf{l}) \cos(\theta) \, d\omega \end{align}
• First, let us consider the diffuse part:
\begin{align}L_{o,d}(\mathbf{v}) &= \int\limits_\Omega \rho_d \frac{1}{\pi} \, \,L_i(\mathbf{l}) \cos(\theta) \, d\omega = \rho_d \frac{1}{\pi} \int\limits_\Omega L_i(\mathbf{l}) \cos(\theta) \, d\omega\\ &= \rho_d \frac{1}{\pi} \int\limits_{0}^{2\pi}\, \int\limits_{0}^{\pi/2} L_i(\mathbf{l}) \cos(\theta) \sin(\theta) \, d\theta \, d\phi \end{align}

### Modified Phong BRDF (Diffuse Part)

• The approximation of the integral with the Riemann sum gives:
\begin{align}L_{o,d}(\mathbf{v}) &= \rho_d \frac{1}{\pi} \int\limits_{0}^{2\pi}\, \int\limits_{0}^{\pi/2} L_i(\mathbf{l}) \cos(\theta) \sin(\theta) \, d\theta \, d\phi\\ &\approx \rho_d \frac{1}{\pi} \sum\limits_{1}^{K}\, \sum\limits_{1}^{J} L_i(\mathbf{l}) \cos(\theta) \sin(\theta) \underbrace{\Delta\theta}_{\frac{\pi/2}{J}} \, \underbrace{\Delta\phi}_{\frac{2\pi}{K}}\\ &= \rho_d \frac{\pi}{K\,J} \sum\limits_{1}^{K}\, \sum\limits_{1}^{J} L_i(\mathbf{l}) \cos(\theta) \sin(\theta) \end{align}

### Importance Sampling

• The Riemann sum is usually not a very efficient solution because the function is sampled uniformly
• Using the theory of importance sampling, we can also approximate any integral by a sum:
$\int\limits_a^b \mathrm{f}(x) \,dx \approx \frac{1}{N} \sum\limits_{n=1}^{N} \frac{\mathrm{f}(x_n)}{\mathrm{p}(x_n)}$
where $p(x)$ is some arbitrary probability density function (PDF) that must fulfill the condition:
$\int\limits_a^b \mathrm{p}(x) \, dx = 1$
• In theory, the best PDF (with the smallest variance) would be
$\mathrm{p}(x) = \frac{\mathrm{f}(x)}{ \int_a^b \mathrm{f}(x)}$
which means the PDF should follow the shape of the function (i.e., the sampling density should be higher if the function values are higher).

### Modified Phong BRDF (Diffuse Part)

• Thus, the approximation of the integral for the diffuse part of the Phong BRDF can be solved with:
\begin{align}L_{o,d}(\mathbf{v}) &= \rho_d \frac{1}{\pi} \int\limits_{0}^{2\pi}\, \int\limits_{0}^{\pi/2} \underbrace{L_i(\mathbf{l}) \cos(\theta) \sin(\theta)}_{\mathrm{f}(\theta_n, \phi_n)} \, d\theta \, d\phi\\ &\approx \rho_d \frac{1}{\pi} \frac{1}{N} \sum_{n=1}^{N} \frac{\mathrm{f}(\theta_n, \phi_n)}{\mathrm{p}(\theta_n, \phi_n)}\\ &= \rho_d \frac{1}{\pi} \frac{1}{N} \sum_{n=1}^{N} \frac{L_i(\mathbf{l}) \cos(\theta_n) \sin(\theta_n)}{\mathrm{p}(\theta_n, \phi_n)} \end{align}

### Split-Sum-Approximation: (2) BRDF Integration Map

// adapted from "Real Shading in Unreal Engine 4", Brian Karis, Epic Games
vec2 integrateBRDF(float roughness, float NoV) {
vec3 V;
V.x = sqrt(1.0 - NoV * NoV); // sin
V.y = 0.0;
V.z = NoV; // cos
vec2 result = vec2(0.0);
uint sampleCount = uint(samples);
for(uint n = 1u; n <= sampleCount; n++) {
vec2 p = hammersley(n, sampleCount);
float a = roughness * roughness;
float theta = acos(sqrt((1.0 - p.y) / (1.0 + (a * a - 1.0) * p.y)));
float phi = 2.0 * PI * p.x;
// sampled h direction in normal space
vec3 H = vec3(sin(theta) * cos(phi), sin(theta) * sin(phi), cos(theta));
vec3 L = 2.0 * dot(V, H) * H - V;

// because N = vec3(0.0, 0.0, 1.0) follows
float NoL = clamp(L.z, 0.0, 1.0);
float NoH = clamp(H.z, 0.0, 1.0);
float VoH = clamp(dot(V, H), 0.0, 1.0);
if(NoL > 0.0) {
float G = G_Smith(NoV, NoL, roughness);
float G_Vis = G * VoH / (NoH * NoV);
float Fc = pow(1.0 - VoH, 5.0);
result.x += (1.0 - Fc) * G_Vis;
result.y += Fc * G_Vis;
}
}
result = result / float(sampleCount);
return result;
}

### Example: Split-Sum-Approximation

#version 300 es
precision highp float;
precision highp int;
out vec4 outColor;

#define PI 3.1415926535897932384626433832795

in vec2 tc; // texture coordinate of pixel (interpolated)
in vec3 wfn; // fragment normal of pixel (interpolated)
in vec3 vertPos; // fragment vertex position (interpolated)
uniform sampler2D envmapImage;
uniform sampler2D prefilteredEnvmap;
uniform sampler2D brdfIntegrationMap;
uniform sampler2D diffuseMap;
uniform sampler2D baseColorTexture;
uniform sampler2D roughnessTexture; // roughness texture
uniform sampler2D metallicTexture; // metallic texture
uniform sampler2D emissionTexture; // emission texture"
uniform float reflectance; // Fresnel reflectance
uniform bool showBackground;
uniform vec3 cameraPos; // camera position in global coordinate system
uniform int mipCount; // number of usable mipmap levels
uniform int gsnMeshGroup;

vec2 directionToSphericalEnvmap(vec3 dir) {
float s = 1.0 - mod(1.0 / (2.0*PI) * atan(dir.y, dir.x), 1.0);
float t = 1.0 / (PI) * acos(-dir.z);
return vec2(s, t);
}

// adapted from "Real Shading in Unreal Engine 4", Brian Karis, Epic Games
vec3 specularIBL(vec3 F0 , float roughness, vec3 N, vec3 V) {
float NoV = clamp(dot(N, V), 0.0, 1.0);
vec3 R = reflect(-V, N);
vec2 uv = directionToSphericalEnvmap(R);
vec3 prefilteredColor = textureLod(prefilteredEnvmap, uv,
roughness*float(mipCount)).rgb;
vec4 brdfIntegration = texture(brdfIntegrationMap, vec2(NoV, roughness));
return prefilteredColor * ( F0 * brdfIntegration.x + brdfIntegration.y );
}

vec3 diffuseIBL(vec3 normal) {
vec2 uv = directionToSphericalEnvmap(normal);
return texture(diffuseMap, uv).rgb;
}

vec3 fresnelSchlick(float cosTheta, vec3 F0) {
return F0 + (1.0 - F0) * pow(1.0 - cosTheta, 5.0);
}

void main() {
vec3 normal = normalize(wfn);
vec3 viewDir = normalize(cameraPos - vertPos);

if(gsnMeshGroup == 0) {
if(showBackground) {
// color of envmap sphere
outColor.rgb = texture(envmapImage, vec2(1.0-tc.x, tc.y)).rgb;
outColor.a = 1.0;
} else {
}
} else {

vec3 baseColor = pow(texture(baseColorTexture, tc).rgb, vec3(2.2));
vec3 emission = pow(texture(emissionTexture, tc).rgb, vec3(2.2));;
float roughness = texture(roughnessTexture, tc).r;
float metallic = texture(metallicTexture, tc).r;

// F0 for dielectics in range [0.0, 0.16]
// default FO is (0.16 * 0.5^2) = 0.04
vec3 f0 = vec3(0.16 * (reflectance * reflectance));
// in case of metals, baseColor contains F0
f0 = mix(f0, baseColor, metallic);

// compute diffuse and specular factors
vec3 F = fresnelSchlick(max(dot(normal, viewDir), 0.0), f0);
vec3 kS = F;
vec3 kD = 1.0 - kS;
kD *= 1.0 - metallic;

vec3 specular = specularIBL(f0, roughness, normal, viewDir);
vec3 diffuse = diffuseIBL(normal);

vec3 color = emission + kD * baseColor * diffuse + specular;
outColor.rgb = pow(color, vec3(1.0/2.2));
outColor.a = 1.0;
}
}

### Are there any questions?

Please notify me by e-mail if you have questions, suggestions for improvement, or found typos: Contact