- Photometric Stereo
- Lambert Law
- Problem formulation
- Algorithms
- Generated images
- Results
- Perspectives
---
## Photometric Stereo
- Estimate the surface normals of an object
- Shiny spheres $\rightarrow$ direction of the lighting
Wikipedia
---
## Lambert law
$I(q) = \rho(Q) \times \vec{n}(Q) \cdot \vec{s}(Q)$
- $\rho(Q)$ is the albedo
- $\vec{n}(Q)$ is the normal vector
- $\vec{s}(Q) = \phi \times \vec{s_0}(Q)$ is the lighting direction
$N$ lightings, $P$ pixels \
$\rightarrow I = M \times S \times D_{\phi}$
- $I \in \mathbb{R}^{P \times N} \rightarrow$ gray scale levels $\rightarrow$ known from image pixels
- $M \in \mathbb{R}^{P \times 3} \rightarrow$ the albedo and the normals $\rightarrow$ **unknown**
- $S \in \mathbb{R}^{3 \times N} \rightarrow$ direction of lightings $\rightarrow$ known from shiny spheres
- $D_{phi} = diag(\phi_1,...,\phi_{N}) \in \mathbb{R}^{ N \times N} \rightarrow$ intensities of lightings $\rightarrow$ **to be determined**
---
## Algorithm 1
Intensities : $[\phi_1,...,\phi_{N}]$
New values : $\phi_j \plusmn \delta, \ j \in [1,..,N]$
Estimation of the matrix $M$
Mean-squared error : $\underset{\phi_i}{\min} || I - M S D_{\phi} ||_2^2$
Update the value of $\phi_j$
Repeat previous steps
---
## Algorithm 2
Algorithm 1 $\rightarrow$ too long
$$I = M S D_{\phi} \iff M = I(S D_{\phi})^\dagger = I (S D_{\phi})^T [(S D_{\phi})(S D_{\phi})^T]^{-1}$$
Lambert law :
$$
\begin{align*}
I &= I (S D_{\phi})^T [(S D_{\phi})(S D_{\phi})^T]^{-1} S D_{\phi} \\
&= I D_{\phi} S^T S^{-T} D_{\phi}^{-2} S^{-1} S D_{\phi}
\end{align*}
$$
New residual :
$$\underset{\phi_i}{\min} || I - I D_{\phi} S^T S^{-T} D_{\phi}^{-2} S^{-1} S D_{\phi} ||_2^2$$
---
## Generated images
---
## Results (1/2)
---
## Results (2/2)
---
## Real images
---
## Results
---
## Perspectives
3D reconstruction
---
# Lighting direction estimation
- Estimation of lighting vector
- Neural Network
- Real data
- Generated data
- Results
- Perspectives
---
## Estimation of lighting vector
```mermaid
flowchart LR
id1[Bounding box of the sphere]
id2[Deduce the normals]
id3[Resolution of I = s * n]
id1 --> id2
id2 --> id3
```
---
## Neural Network
---
## ResNet-50
---
## Real data : creation of mask
---
## Generated data with blender
Simulated matte spheres
Generated data with different lightings
---
## Results
---
## Perspectives
- Create more data to prevent overfitting
- Diversify the types of data lighting (more than 8 directions)
- Transform the model into something more general: \
$\rightarrow$ from {image of sphere, vector lighting} to {image of objects, vector lighting}
LaserFocusWorld
---
## Problem formulation
Intensities : $[\phi_1,...,\phi_{N}]$
New values : $\phi_j \plusmn \delta, \ j \in [1,..,N]$
Estimation of the matrix $M$
Mean-squared error : $\underset{\phi_i}{\min} || I - M S D_{\phi} ||_2^2$
Update the value of $\phi_j$
Repeat previous steps
---
# Lighting direction estimation
---
## Real data : creation of mask
---
## Generated data with blender
---
## Perspectives