projet-long/slides.md

theme	class	coverAuthor	coverBackgroundUrl	coverBackgroundSource	coverBackgroundSourceUrl	coverDate	themeConfig	title
academic	text-white	Laurent Fainsin, Pierre-Eliot Jourdan, Raphaëlle Monville-Letu, Jade Neav	https://plus.unsplash.com/premium_photo-1673553304257-018c85e606f8?ixlib=rb-4.0.3&ixid=MnwxMjA3fDB8MHxwaG90by1wYWdlfHx8fGVufDB8fHx8	unplash	https://unsplash.com/photos/g4I556WCJT0	2023-03-09	paginationX paginationY paginationPagesDisabled r t 1	Projet Long

End of study project

Sphere detection and multimedia applications

---

Contents

• Types of spheres
• Automatic sphere detection
• Lighting intensity estimation
• Lightning direction estimation

Architecture

Cinema

3D Reconstruction

pexels

---

class: text-white custombg

Types of spheres

---

class: text-white custombg2

Chrome sphere

CaveAcademy

---

Acquisition techniques

Louis du Mont

---

Realistic lighting

---

class: text-white custombg3

Shiny sphere

CaveAcademy

---

class: text-white custombg4

Matte sphere

CaveAcademy

---

Automatic detection of spheres

---

Model

End-to-End Object Detection with Transformers, arXiv:2005.12872

---

Datasets

---

Results

---

Estimation of the lighting intensity in an image

---

Photometric Stereo

• Estimate the surface normals of an object
• Shiny spheres \rightarrow direction of the lighting

---

Lambert law

I(q) = \rho(Q) \times \vec{n}(Q) \cdot \vec{s}(Q)

\rho(Q) is the albedo

\vec{n}(Q) is the normal vector

\vec{s}(Q) = \phi \times \vec{s_0}(Q), \vec{s_0}(Q) being the direction of the lighting vector

---

Problem formulation

N lightings, P pixels

I = M \times S \times D_{\phi}

I \in \mathbb{R}^{P \times N} \rightarrow gray scale levels

M \in \mathbb{R}^{P \times 3} \rightarrow the albedo and the normals (unknown)

S \in \mathbb{R}^{3 \times N} \rightarrow direction of lightings

D_{phi} = diag(\phi_1,...,\phi_{N}) \in \mathbb{R}^{ N \times N} \rightarrow intensities of lightings (to be determined)

---

Algorithm 1

Intensities : [\phi_1,...,\phi_{N}]

New values : \phi_j + \delta and \phi_j + \delta, j \in [1,..,N]

Mean-squared error : \underset{\phi_i}{\min} || I - M S D_{\phi} ||_2^2

Update the value of \phi_j

Repeat previous steps

---

Algorithm 2

Algorithm 1 \rightarrow too long

I = M S D_{\phi} \iff M = I(S D_{\phi})^\dagger = I (S D_{\phi})^T [(S D_{\phi})(S D_{\phi})^T]^{-1}

Lambert law :

$$ \begin{align*} I &= I (S D_{\phi})^T [(S D_{\phi})(S D_{\phi})^T]^{-1} S D_{\phi} \ &= I D_{\phi} S^T S^{-T} D_{\phi}^{-2} S^{-1} S D_{\phi} \end{align*}

New residual :

\underset{\phi_i}{\min} || I - I D_{\phi} S^T S^{-T} D_{\phi}^{-2} S^{-1} S D_{\phi} ||_2^2

---

Generated images

---

Results (1/2)

---

Results (2/2)

---

Real images

TODO LOLO : mettre les images comme dans les slides avec carre rouge

---

Results

---

Automatic estimation of lighting vector

• Creation of data
• Estimation of light vector with matte balls
• Training of neural networks

---

Creation of mask

---

Generated data with Blender

---

Estimation of lighting vector for training

---

Verification estimation of lighting vector

---

Which type of neural network ?

---

Results

---

Conclusion

---

Perspectives