Mastering 3D Computer Vision & Point Cloud Processing-Mod 12 — Voxel Grid to Mesh Conversion with Code

Introduction:

Welcome to the ✍️“3D Computer Vision & Point Cloud Processing Blog Series”. This series of blogs is your “Hands-on guide to mastering 3D point cloud processing with Python.”

In this post, we’ll delve into how the voxel grid can be converted to Mesh data.

Topics to Be Discussed in this Blog

1. Intro to 3D Data Types
2. Voxel Grid to Mesh Conversion
3. Mesh Original vs Mesh Computed from voxel
4. Summary

Intro to 3D Data Types

• There are several common 3D data types, including point clouds, meshes, and voxel grids.
• Point clouds consist of points in 3D space representing object surfaces or features.
• Meshes are collections of vertices, edges, and faces that define 3D object shapes.
• Voxel grids are three-dimensional grids where each cell represents a value or attribute, often used for volumetric data.

For more details refer my previous blog.

Why Convert Voxel Grid to Mesh Data

1. Converting Voxel Grid to Mesh is essential for several reasons,

1. Surface Representation: Voxel grids are used for volumetric data, but converting to a mesh focuses on extracting and representing the object’s surface. This is more visually intuitive and useful for various applications.
2. Visualization: Meshes are ideal for 3D visualization as they visually represent the object’s surface geometry. They are suitable for rendering and displaying 3D models in computer graphics, simulations, and virtual reality applications.
3. Downstream Processing: Many 3D processing algorithms and software tools work with mesh data. Converting to a mesh format allows you to utilize these tools for tasks like mesh simplification, smoothing, texturing, and further analysis.

2. Applications

1. 3D Scanning and Reconstruction: Converting voxel data to a mesh enables the creation of detailed 3D models from scans, such as CT scans or 3D scans, for objects or environments.
2. Computer Graphics and Gaming: Meshes are essential in computer graphics and gaming for rendering and displaying 3D objects, characters, and environments.
3. 3D Printing: Meshes are widely used in 3D printing as they define the object’s outer surface, crucial for generating the printer’s toolpaths.

3. Pros of Converting to Mesh

1. Visual Appeal: Meshes offer a visually appealing and realistic representation of 3D objects and scenes.
2. Compatibility: Mesh data is compatible with a wide range of 3D software tools, making it suitable for various applications and processing tasks.
3. Downstream Processing: Mesh data allows for further processing and analysis, including simplification, smoothing, and texturing.

4. Cons of Converting to Mesh

1. Data Size: Meshes can be larger in size compared to voxel grids, as they represent detailed surface geometry.
2. Loss of Volumetric Information: Converting to a mesh focuses on the surface, potentially leading to a loss of volumetric information.
3. Complexity: Mesh data can be more complex to work with than voxel grids, especially for algorithms that rely on voxel-level information.

Follow the below steps to run the script.

1. Create a file named “voxel-to-mesh.py” and paste the following code into it.
2. Download “bunny-pcd-to-voxel-001.ply” ply data from https://github.com/MatPixel/dataset-for-3d-pointcloud-processing-3d-deep-learning/blob/main/bunny-pcd-to-voxel-001.ply and place it in the same directory as the Python script.
3. Open the file in viewer such as MeshLab (https://www.meshlab.net/) for understanding the data.
4. Run the script using a Python interpreter such as visual studio code https://code.visualstudio.com/.
5. The script will load the point cloud, estimate normals if needed, convert it to a mesh, and save the resulting mesh as “bunny-voxel-001-to-mesh.ply” in the same directory.

CODE:

# Import libraries
import numpy as np
import open3d as o3d

# Load a point cloud
ply_file_path_read = "bunny-pcd-to-voxel-001.ply"
voxel_grid = o3d.io.read_voxel_grid(ply_file_path_read)
voxelsize = voxel_grid.voxel_size
print(f"Voxel size of the Voxel Grid = {voxelsize}")
voxelorigin = voxel_grid.origin
print(f"Voxel Grid Origin = {voxelorigin}")

# Computer voxel centers from Voxel grid as points in the PCD
voxels = voxel_grid.get_voxels()

"""
# Approach1: get points using 'get_voxel_center_coordinate'
points_from_voxel = []
for voxel in voxels:
    voxel_center = voxel_grid.get_voxel_center_coordinate(voxel.grid_index)
    points_from_voxel.append(voxel_center)
"""
# Approach2: get points manually 
#voxelindex = 0; voxel_center_as_point_in_pcd = voxelorigin + voxels[voxelindex].grid_index*voxelsize
#print(voxel_center_as_point_in_pcd)
points_from_voxel = np.asarray([(voxelorigin + voxel.grid_index*voxelsize) for voxel in voxels])
print(f"Number of points computed from Voxel Grid = {len(points_from_voxel)}")
points_from_voxel = np.asarray([(voxelorigin + voxel.grid_index*voxelsize) for voxel in voxels])

# Create Point cloud and Mesh
point_cloud = o3d.geometry.PointCloud()
point_cloud.points = o3d.utility.Vector3dVector(points_from_voxel)
point_cloud.estimate_normals()
o3d.geometry.PointCloud.orient_normals_consistent_tangent_plane(point_cloud, 10)

# Estimate radius for rolling ball
avg_dist = np.mean(point_cloud.compute_nearest_neighbor_distance())
radius = 1.5 * avg_dist
#mesh, _ = o3d.geometry.TriangleMesh.create_from_point_cloud_poisson(point_cloud, depth=9)
mesh = o3d.geometry.TriangleMesh.create_from_point_cloud_ball_pivoting(point_cloud,o3d.utility.DoubleVector([radius, radius * 2]))
print(f"Number of Vertices in the mesh = {len(np.asarray(mesh.vertices))}")
print(f"Number of Triangles(faces) in the mesh = {len(np.asarray(mesh.triangles))}")

# Visualize and save the Mesh generated from Voxel Grid and PCD
o3d.visualization.draw_geometries([mesh], width=1200, height=800)
o3d.io.write_triangle_mesh("bunny-voxel001-to-mesh.ply", mesh, write_ascii=True)

OUTPUT:

Voxel size of the Voxel Grid = 0.001
Voxel Grid Origin = [-0.0951899  0.0324874 -0.0623736]
Number of points computed from Voxel Grid = 34583
Number of Vertices in the mesh = 34583
Number of Triangles(faces) in the mesh = 65450

In the results,

1. Voxel Size: The voxel size of 0.001 indicates the resolution at which the voxel grid represents the 3D space. A smaller voxel size means higher detail but also increased computational requirements.
2. Voxel Grid Origin: The origin of the voxel grid, [-0.0951899, 0.0324874, -0.0623736], defines its position in 3D space. This information is vital for aligning the mesh with the original data.
3. Number of Points: The voxel grid contains 34583 points, representing the discrete locations where the object exists in 3D space.
4. Mesh Vertices: The resulting mesh has 34583 vertices, which are the endpoints of the mesh edges. These vertices define the object’s shape in the mesh.
5. Mesh Triangles: The mesh comprises 65450 triangles, which are formed by connecting the vertices. These triangles make up the mesh surface.

Mesh Original vs Mesh Computed from voxel Grid

The table below shows the difference between the Original Mesh and the Mesh generated from the voxel grid.

1. Element Vertex and Element Face: The original mesh has 35947 vertices and 71600 faces, while the mesh computed from the voxel grid has 34583 vertices and 65450 faces. This difference is because the conversion process from a voxel grid to a mesh can result in a different number of vertices and faces due to the interpolation and simplification techniques used.
2. Vertex Coordinates: The vertex coordinates (-0.0378297, 0.12794, 0.00447467) in the original mesh and (-0.0331899, 0.173487, -0.0033736) in the mesh computed from the voxel grid are different. This variation occurs because the conversion process involves estimating the vertex positions based on the voxel grid data. Factors such as voxel size, interpolation methods, and mesh generation algorithms can lead to these differences.

Summary

In this blog, we discussed converting a voxel grid to a mesh. The conversion process involves estimating vertex positions and creating a mesh representation of the object’s surface. The resulting can lead to a loss of volumetric information and increased complexity compared to the original voxel grid.

✨Happy exploring! Happy learning!✨

📝Next Blog Preview:

In the upcoming post, 🚀“Mastering 3D Computer Vision & Point Cloud Processing- Mod 13 — Commonly used 3D File formats with Code”.

Topics to Be Discussed in the Next Blog

1. Discussion on Commonly used File formats
2. ASCII vs Bin on Point Cloud
3. ASCII vs Bin on XYZ Data
4. ASCII vs Bin on PCD Data
5. ASCII vs Bin on PLY Data

Topics to Be Discussed in the Upcoming Blogs — Immediate Focus

• Discussion on Commonly used File formats
• Most used Dataset
• Most Point Cloud Preprocessing Techniques