ENSEEIHT/projet-probleme-inverse-3D
1
0
Fork 0
Code Issues Pull requests Packages Projects Releases Wiki Activity

the end?

Browse source 
Co-authored-by: Laureηt <laurent@fainsin.bzh>
Co-authored-by: pejour <pejour@users.noreply.github.com>
This commit is contained in:
gdamms 2023-02-01 13:54:26 +01:00
parent 4384319a03
commit 8b7865be22
2 changed files with 101 additions and 75 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

134
docs/slides.md
View file

@ -29,6 +29,7 @@ title: Bureau d'étude de PI3D
- Hyposthèses
- L'algorithme
- Résultats
- Conclusion
</div>
@ -48,6 +49,7 @@ Et pour ce faire on avait besoin de faire les étapes suivantes:
1. Initialisation du volume
2. Mise à jour du volume
- Résultats
- Conclusion
-->
@ -85,6 +87,7 @@ Volume = Ensemble des points de l'espace tel que, l'image de ces points par une
<figure>
<img src="https://upload.wikimedia.org/wikipedia/commons/thumb/a/a7/MarchingCubes.svg/350px-MarchingCubes.svg.png" class="h-50">
<figcaption class="text-center">Marching cubes</figcaption>
<!-- mettre lien ici -->
</figure>
</div>
@ -107,6 +110,10 @@ En ce qui concerne le rendu de la surface 3D, on utilise l'ago de marching cubes
<!--
MVS -> reconstruction 3D à partir des positions des caméras et de leur image.
On obtient un nuage de point dense
Avec la méthode classique, il peut y avoir des décalages entre les nuages de points
Mais avec level set, on peut s'affranchir de ce problème
-->
---
@ -144,16 +151,19 @@ changement de topologie du level set
<img src="https://upload.wikimedia.org/wikipedia/commons/b/bc/Voxels.svg" class="m-auto mr-50 -mt-13 h-100">
<style>
span.katex {
position: absolute;
bottom: 1%;
}
</style>
<div class="absolute top-25">
$\mathbb{R}^3 \to \mathbb{V}$
</div>
### Binarisation du levelset
$\mathbb{R}^3 \to \mathbb{V} \newline \implies \mathcal{V} = \{ \textbf{v} = (x, y, z) \in \mathbb{V}, u(\textbf{v}) > 0 \}, \quad u \colon \mathbb{V} \to \{0, 1\}$
<div class="absolute bottom-3">
$\mathcal{V} = \{ \textbf{v} = (x, y, z) \in \mathbb{V}, u(\textbf{v}) > 0 \}, \quad u \colon \mathbb{V} \to \{0, 1\}$
</div>
<a href="https://en.wikipedia.org/wiki/Voxel" class="absolute bottom-0 font-extralight mb-1 mr-2 right-0 text-xs">Wikipedia</a>
@ -207,8 +217,8 @@ Plus on a de caméra, meilleure sera la définition de l'enveloppe.
### Shape from Silhouette 3D
<div class="flex items-center">
<img src="/figs/example3D.gif" class="m-auto h-110">
<iframe frameborder="0" allowfullscreen mozallowfullscreen="true" webkitallowfullscreen="true" allow="autoplay; fullscreen; xr-spatial-tracking" xr-spatial-tracking execution-while-out-of-viewport execution-while-not-rendered web-share width="100%" height="100%" src="https://sketchfab.com/models/4da17838dd5f497faa147db5febd21d9/embed"></iframe>
<img src="/figs/example3D.gif" class="m-auto h-100">
<iframe frameborder="0" allowfullscreen mozallowfullscreen="true" webkitallowfullscreen="true" allow="autoplay; fullscreen; xr-spatial-tracking" xr-spatial-tracking execution-while-out-of-viewport execution-while-not-rendered web-share width="100%" height="100%" src="https://sketchfab.com/models/e693a18fb2a044518313a42fe43dce86/embed?autostart=1" class="h-100"></iframe>
</div>
<!--
@ -231,75 +241,83 @@ nuage de voxel. si nuage de points -> conversion en mesh possible grace aux marc
---
### Shape from Silhouette 2D
<img src="https://cdn.discordapp.com/attachments/953586522572066826/1069974891023056926/image.png" class="m-auto -mt-2 h-110">
<img src="https://cdn.discordapp.com/attachments/953586522572066826/1069974281175449630/image.png" class="m-auto -mt-1 h-11">
---
## Mise à jour du volume
<img src="/figs/update.png" class="m-auto h-120">
<!--
- Sélection des voxels sur la bordure du "marbre"
- Vérification de la visibilité du voxel par toutes les caméras
- Récupération des couleurs visibles par les caméras
- Si !consensus et air, !update -> air
- Si consensus et air, update -> solide
- Si !consensus et solide, update -> air
- Si consensus et solide, !update -> solide
-->
---
## Itération du Level Set
<div class="flex">
<img src="https://cdn.discordapp.com/attachments/953586522572066826/1069973956787966003/Screenshot_from_2023-01-31_14-33-47.png" class="h-110">
<img src="https://cdn.discordapp.com/attachments/953586522572066826/1069975888382410784/image.png" class="h-110">
</div>
---
## Résultats
### L'environnement
<!-- TODO -->
<img src="https://cdn.discordapp.com/attachments/953586522572066826/1070291885823889408/peanut.png"/>
<img src="https://cdn.discordapp.com/attachments/953586522572066826/1070291885823889408/peanut.png" class="m-auto h-110"/>
---
## Résultats
### Les données
<!-- TODO: mettre les cams a guache et les mask et imagesa droite les unes sur les autres -->
<img src="https://cdn.discordapp.com/attachments/953586522572066826/1070293528288165930/peanut_cams.png"/>
<img src="https://cdn.discordapp.com/attachments/953586522572066826/1070292853282054225/Image0000.png"/>
<img src="https://cdn.discordapp.com/attachments/953586522572066826/1070292853495975988/Image0010.png"/>
<img src="https://cdn.discordapp.com/attachments/953586522572066826/1070292853764407306/Image0020.png"/>
<img src="https://cdn.discordapp.com/attachments/953586522572066826/1070292923322744903/Image0000.png"/>
<img src="https://cdn.discordapp.com/attachments/953586522572066826/1070292923549224980/Image0010.png"/>
<img src="https://cdn.discordapp.com/attachments/953586522572066826/1070292923754741770/Image0020.png"/>
<style>
img.shadowy {
box-shadow: 0 0px 6px rgb(0 0 0 / 30%);
}
</style>
<div class="flex items-center">
<img src="https://cdn.discordapp.com/attachments/953586522572066826/1070293528288165930/peanut_cams.png" class="h-90"/>
<div class="flex-col inline-flex gap-5">
<img src="https://cdn.discordapp.com/attachments/953586522572066826/1070292853282054225/Image0000.png" class="w-100 shadowy"/>
<img src="https://cdn.discordapp.com/attachments/953586522572066826/1070292923322744903/Image0000.png" class="w-100 shadowy"/>
<img src="https://cdn.discordapp.com/attachments/953586522572066826/1070292853495975988/Image0010.png" class="w-100 shadowy"/>
<img src="https://cdn.discordapp.com/attachments/953586522572066826/1070292923549224980/Image0010.png" class="w-100 shadowy"/>
<img src="https://cdn.discordapp.com/attachments/953586522572066826/1070292853764407306/Image0020.png" class="w-100 shadowy"/>
<img src="https://cdn.discordapp.com/attachments/953586522572066826/1070292923754741770/Image0020.png" class="w-100 shadowy"/>
</div>
</div>
---
## Résultats
### L'initialisation (Shape from Silhouette)
<img src="https://cdn.discordapp.com/attachments/953586522572066826/1070287482186383450/init.png" class="m-auto h-120">
<img src="https://cdn.discordapp.com/attachments/953586522572066826/1070287482186383450/init.png" class="m-auto h-110">
---
## Conclusion et axes de développement
## Résultats
### Bordures
<div class="h-100 flex items-center text-2xl">
<img src="https://cdn.discordapp.com/attachments/953586522572066826/1070307308032233532/border.png" class="m-auto h-110">
---
## Résultats
### Visibilité des voxels
<div class="h-100 flex items-center">
<img src="https://cdn.discordapp.com/attachments/953586522572066826/1070312481894973460/ray.png" class="h-105 -ml-15 -mr-15">
<img src="https://cdn.discordapp.com/attachments/953586522572066826/1070312482243104839/selected.png" class="h-105 -ml-15">
</div>
---
## Résultats
### Évolution du level set
<div class="h-100 flex items-center">
<img src="https://cdn.discordapp.com/attachments/953586522572066826/1070319867946872912/evol7bis.gif" class="m-auto h-105">
<img src="https://cdn.discordapp.com/attachments/953586522572066826/1070319868324360252/shape7bis.gif" class="m-auto h-105">
</div>
---
## Conclusion
<div class="h-100 flex flex-col text-2xl justify-center">
### Ce que nous avons fait
- Réalisation du SfS en 2D/3D
- Réalisation du MVS par level sets en 2D avec utilisation du SfS
- Résultat en 3D ?
- Passer en continu au lieu de binaire
- Réalisation du MVS par level sets en 2D avec initialisation par SfS
<br>
### Axes d'amélioration soon™
- Résultat en 3D
- $\{0, 1\} \to [0, 1]$
</div>

42
src/sfs2d.py
View file

@ -137,20 +137,23 @@ for p in range(11):
plt.figure()
plt.axis("equal")
plt.axis("off")
plot_voxels(Z, "#fff", 1.0, True)
# plot_voxels(border, "#00f", 0.5, False)
for i in range(nb_frame):
plot_camera(cam2world_projs[i], str(i), "#f00")
# plt.show()
# plt.savefig(f"border.png", dpi=300, bbox_inches="tight", pad_inches=0, transparent=True)
# exit()
for i, j in track(np.argwhere(border)):
for i, j in track(np.argwhere(border)[30:]):
# if i != 74 or j != 34:
# continue
x, y = X_MIN + (i + 0.5) * VOXEL_SIZE, Y_MIN + (j + 0.5) * VOXEL_SIZE
start = np.array([x, y])
# plt.plot([start[0]], [start[1]], "o", color="#f00")
# plt.fill([x, x + VOXEL_SIZE, x + VOXEL_SIZE, x],
# [y, y, y + VOXEL_SIZE, y + VOXEL_SIZE], "#0f0", alpha=0.7)
# plt.fill([x - VOXEL_SIZE/2, x + VOXEL_SIZE/2, x + VOXEL_SIZE/2, x - VOXEL_SIZE/2],
# [y - VOXEL_SIZE/2, y - VOXEL_SIZE/2, y + VOXEL_SIZE/2, y + VOXEL_SIZE/2], "#00f", alpha=0.7)
values = []
@ -164,7 +167,7 @@ for p in range(11):
d = np.array([x, y + VOXEL_SIZE])
# plt.plot([d[0]], [d[1]], "o", color="#0f0")
for k in range(nb_frame):
for k in range(5,nb_frame):
# plot_camera(cam2world_projs[k], str(k), "#0f0")
@ -192,6 +195,8 @@ for p in range(11):
# color="#f00",
# alpha=0.3,
# )
# plt.savefig(f"ray.png", dpi=300, bbox_inches="tight", pad_inches=0, transparent=True)
# exit()
if len(voxels_intersected) == 0:
visible = True
@ -216,25 +221,27 @@ for p in range(11):
proj = np.round(proj).astype(np.int32)
values.append(frames[k][proj[1, 0]])
# fig = plt.gcf()
# plt.figure()
# plt.imshow(
# np.repeat(frames[k][np.newaxis, :], 100, axis=0), cmap="gray")
# plt.plot(proj[1, 0], 50, "o", color="#f00")
# plt.plot(proj[1, 1], 50, "o", color="#ff0")
# plt.plot(proj[1, 2], 50, "o", color="#0ff")
# plt.plot(proj[1, 3], 50, "o", color="#f0f")
# plt.plot(proj[1, 4], 50, "o", color="#0f0")
# plt.figure(fig)
# fig = plt.gcf()
# plt.figure()
# plt.imshow(
# np.repeat(frames[k][np.newaxis, :], 100, axis=0), cmap="gray")
# plt.plot(proj[1, 0], 50, "o", color="#f00")
# plt.plot(proj[1, 1], 50, "o", color="#ff0")
# plt.plot(proj[1, 2], 50, "o", color="#0ff")
# plt.plot(proj[1, 3], 50, "o", color="#f0f")
# plt.plot(proj[1, 4], 50, "o", color="#0f0")
# plt.figure(fig)
# else:
# plot_camera(cam2world_projs[k], str(k), "#f00")
# print(values)
# plt.show()
# plt.savefig(f"selected.png", dpi=300, bbox_inches="tight", pad_inches=0, transparent=True)
# exit()
occurences = [[x, values.count(x)] for x in set(values)]
occurences = sorted(occurences, key=lambda x: x[1], reverse=True)
if occurences[0][1] >= len(values) * 0.7:
if len(values) > 0 and occurences[0][1] >= len(values) * 0.7:
voxel_[i, j] = 1
color = "#0f0"
else:
@ -259,13 +266,14 @@ for p in range(11):
voxel = voxel_
plt.savefig(f"evol{p:02}.png", dpi=300, bbox_inches="tight", pad_inches=0, transparent=True)
plt.savefig(f"data/peanut/res-0.7/evol{p:02}.png", dpi=300, bbox_inches="tight", pad_inches=0, transparent=True)
plt.figure()
plt.axis("equal")
plt.axis("off")
plot_voxels(Z, "#fff", 1.0, True)
plot_voxels(voxel, "#00f", 0.5, False)
plt.savefig(f"shape{p:02}.png", dpi=300, bbox_inches="tight", pad_inches=0, transparent=True)
plt.savefig(f"data/peanut/res-0.7/shape{p:02}.png", dpi=300, bbox_inches="tight", pad_inches=0, transparent=True)
plt.close("all")
# plt.show()