Mastering 3D Computer Vision & Point Cloud Processing-Mod 10-Mesh to Point Cloud and Mesh to Voxel Grid Conversion with Code

Introduction:

Welcome to the ✍️“3D Computer Vision & Point Cloud Processing Blog Series”. This series of blogs is your 🚀 hands-on guide to mastering 3D point cloud processing with Python.

In this post, we’ll delve into how the Mesh data can be converted to Point Cloud and Voxel Grid.

Topics to Be Discussed in this Blog

1. Intro to 3D Data Types
2. Mesh to Point Cloud Conversion
3. Mesh to Voxel Grid Conversion
4. Summary

Intro to 3D Data Types

• There are several common 3D data types, including point clouds, meshes, and voxel grids.
• Point clouds consist of points in 3D space representing object surfaces or features.
• Meshes are collections of vertices, edges, and faces that define 3D object shapes.
• Voxel grids are three-dimensional grids where each cell represents a value or attribute, often used for volumetric data.

For more details refer my previous blog.

Why Convert Mesh to Point Cloud

1. Converting Mesh to Point cloud is essential for several reasons,

1. Simplification: Meshes are complex, composed of vertices, edges, and faces. Converting them to point clouds simplifies the data to just 3D points.
2. Uniform Representation: Point clouds provide a uniform representation of 3D space, regardless of the original mesh’s complexity, making them useful for different data sources or sensors.
3. Downsampling: Converting to a point cloud allows for efficient downsampling by reducing the number of points, which can speed up processing.
4. Accessibility: Some algorithms and libraries may require point cloud input, making conversion necessary for compatibility.

2. Applications

1. Computer Vision: Point clouds are often used in computer vision tasks like object recognition, registration, and reconstruction.
2. Data Compression: Point cloud representations can be more memory-efficient and provide better compression compared to detailed meshes.

3. Pros of Converting to Point Cloud

1. Uniform Representation: Point clouds offer a uniform representation of 3D space and simplify data.
2. Compatibility: Point cloud formats are widely supported, ensuring compatibility with various software and algorithms.

4. Cons of Converting to Point Cloud

Converting to point clouds can result in a loss of detail and increased noise from the original mesh, and they lack explicit topological information, which can make certain operations challenging.

5. Let’s code

Follow the below steps to run the script.

1. Create a file named “mesh-to-pcd.py” and paste the following code into it.
2. Download “bunny-mesh.ply” ply data from https://github.com/MatPixel/dataset-for-3d-pointcloud-processing-3d-deep-learning/blob/main/bunny-mesh.ply and place it in the same directory as the Python script.
3. Open the file in viewer such as MeshLab (https://www.meshlab.net/) for understanding the data.
4. Run the script using a Python interpreter such as visual studio code https://code.visualstudio.com/.
5. The script will load the point cloud, estimate normals if needed, convert it to a mesh, and save the resulting mesh as “bunny-mesh-to-pcd.ply” in the same directory.

CODE:

# Import libraries
import numpy as np
import open3d as o3d

# Load a mesh
ply_file_path_read = "bunny-mesh.ply"
mesh = o3d.io.read_triangle_mesh(ply_file_path_read)
mesh.compute_vertex_normals()
#mesh.scale(1 / np.max(mesh.get_max_bound() - mesh.get_min_bound()),center=mesh.get_center()) # Fit to unit cube
print(f"Number of Vertices = {len(np.asarray(mesh.vertices))}")
print(f"Number of Triangales(faces) = {len(np.asarray(mesh.triangles))}")

# Convert mesh to pcd
points = mesh.vertices
point_cloud = o3d.geometry.PointCloud(points)

# Visualize and save the point cloud
o3d.visualization.draw_geometries([point_cloud], width=1200, height=800)
o3d.io.write_point_cloud("bunny-mesh-to-pcd.ply", point_cloud, write_ascii=True)

OUTPUT:

Number of Vertices = 35947
Number of Triangles(faces) = 71600

From the above code, the commented line (as shown below) is meant to make the mesh fit into a unit cube by scaling around the center. This step is often used in real-time projects where multiple meshes are involved for processing.

#mesh.scale(1 / np.max(mesh.get_max_bound() - mesh.get_min_bound()),center=mesh.get_center()) # Fit to unit cube

The plots below show the results of the mesh to point cloud conversion. The top plot is the original Mesh, and the bottom plot is the converted pcd.

Original Bunny Mesh data

Converted Bunny Point Cloud Data

In the above code, we have used “compute_vertex_normals” is not explained here, and we will discuss it in detail in upcoming blogs, keeping the learning flow in mind. For now, we are focusing on “how to convert” rather than “how it works.” Thank you for your understanding.

Why Convert Mesh to voxel Grid

1. Converting Mesh to voxel grid is essential for several reasons,

1. Discretization: Voxel grids represent 3D space as a grid of cubical cells (voxels), simplifying complex geometry into a structured grid.
2. Quantization: Voxel grids provide a quantized representation, with each voxel storing attributes like occupancy, color, or scalar values.
3. Uniform Representation: They offer a uniform representation of 3D space, simplifying computational tasks with data from different sources.

2. Applications

1. 3D Printing: Used to convert detailed models into printable representations, offering precise control over printed object properties.
2. Medical Imaging: Employed in MRI and CT scans to represent volumetric data for analysis.
3. Computer Graphics: Utilized for real-time rendering, dynamic scene representation, and other graphics tasks.

3. Pros of Converting to voxel grid

1. Structured Representation: Simplifies computational operations with a structured and quantized representation.
2. Compression: More memory-efficient and better compression than detailed meshes, especially for volumetric data.

4. Cons of Converting to voxel grid

1. Loss of Detail: May lose fine-grained geometric detail from the original mesh.
2. Increased Memory Usage: Can consume more memory, especially at high resolutions.
3. Complexity: Requires specialized algorithms and data structures compared to working with meshes.
4. Grid Artifacts: Quantization can introduce artifacts affecting visual quality or accuracy.

5. Let’s code

Follow the below steps to run the script.

1. Create a file named “mesh-to-voxel.py” and paste the following code into it.
2. Download “bunny-mesh.ply” ply data from https://github.com/MatPixel/dataset-for-3d-pointcloud-processing-3d-deep-learning/blob/main/bunny-mesh.ply and place it in the same directory as the Python script.
3. Open the file in viewer such as MeshLab (https://www.meshlab.net/) for understanding the data.
4. Run the script using a Python interpreter such as visual studio code https://code.visualstudio.com/.
5. The script will load the point cloud, estimate normals if needed, convert it to a mesh, and save the resulting mesh as “bunny-mesh-to-pcd.ply” in the same directory.

CODE:

# Import libraries
import numpy as np
import open3d as o3d

# Load a mesh
ply_file_path_read = "bunny-mesh.ply"
mesh = o3d.io.read_triangle_mesh(ply_file_path_read)
mesh.compute_vertex_normals()
mesh.scale(1 / np.max(mesh.get_max_bound() - mesh.get_min_bound()),center=mesh.get_center()) # Fit to unit cube
print(f"Number of Vertices in the mesh = {len(np.asarray(mesh.vertices))}")
print(f"Number of Triangles(faces) in the mesh = {len(np.asarray(mesh.triangles))}")
o3d.io.write_triangle_mesh("bunny-mesh-unitcube.ply", mesh, write_ascii=True)

# Compute points distances
pcd = o3d.geometry.PointCloud()
pcd.points = o3d.utility.Vector3dVector(mesh.vertices)
distances = pcd.compute_nearest_neighbor_distance()
avg_dist = np.mean(distances)
print(f"Average neighbour distance = {np.mean(distances):.6}")

# Guess different voxel sizes based on points' distances (KDTree approach also can be used)
voxelsize = np.round(avg_dist, 5)
voxelsize_half = voxelsize/2
voxelsize_double = voxelsize*2
print(f"Different voxel size considered based on points' distances: {voxelsize}, {voxelsize_double}, {voxelsize_half}")

# Convert mesh to pcd
voxel_grid = o3d.geometry.VoxelGrid.create_from_triangle_mesh(mesh, voxel_size=voxelsize)
print(f"Voxel center in X,Y,Z = {voxel_grid.get_center()}")
print(f"Number of voxels = {len(np.asarray(voxel_grid.get_voxels()))}")
print(f"Sample Voxels: \n {voxel_grid.get_voxels()[:3]}")
o3d.visualization.draw_geometries([voxel_grid], width=1200, height=800)
o3d.io.write_voxel_grid("bunny-mesh-to-voxel-001-unitcube.ply", voxel_grid, write_ascii=True)

voxel_grid = o3d.geometry.VoxelGrid.create_from_triangle_mesh(mesh, voxel_size=voxelsize_half)
print(f"Voxel center in X,Y,Z = {voxel_grid.get_center()}")
print(f"Number of voxels = {len(np.asarray(voxel_grid.get_voxels()))}")
print(f"Sample Voxels: \n {voxel_grid.get_voxels()[:3]}")
o3d.visualization.draw_geometries([voxel_grid], width=1200, height=800)
o3d.io.write_voxel_grid("bunny-mesh-to-voxel-002-unitcube.ply", voxel_grid, write_ascii=True)

voxel_grid = o3d.geometry.VoxelGrid.create_from_triangle_mesh(mesh, voxel_size=voxelsize_double)
print(f"Voxel center in X,Y,Z = {voxel_grid.get_center()}")
print(f"Number of voxels = {len(np.asarray(voxel_grid.get_voxels()))}")
print(f"Sample Voxels: \n {voxel_grid.get_voxels()[:3]}")
o3d.visualization.draw_geometries([voxel_grid], width=1200, height=800)
o3d.io.write_voxel_grid("bunny-mesh-to-voxel-003-unitcube.ply", voxel_grid, write_ascii=True)

OUTPUT:

Number of Vertices in the mesh = 35947
Number of Triangles(faces) in the mesh = 71600
Average neighbour distance = 0.00644491
Different voxel size considered based on points' distances: 0.00644, 0.01288, 0.00322


Voxel center in X,Y,Z = [-0.02873008  0.09729862  0.00515428]
Number of voxels = 71422
Sample Voxels:
 [Voxel with grid_index: (14, 44, 57), color: (0, 0, 0), 
  Voxel with grid_index: (53, 95, 75), color: (0, 0, 0), 
  Voxel with grid_index: (5, 101, 95), color: (0, 0, 0)]


Voxel center in X,Y,Z = [-0.02652538  0.09834554  0.00514782]
Number of voxels = 234502
Sample Voxels:
 [Voxel with grid_index: (84, 231, 161), color: (0, 0, 0), 
  Voxel with grid_index: (108, 190, 132), color: (0, 0, 0), 
  Voxel with grid_index: (125, 147, 82), color: (0, 0, 0)]


Voxel center in X,Y,Z = [-0.02753553  0.09317389  0.00369362]
Number of voxels = 19359
Sample Voxels:
 [Voxel with grid_index: (8, 62, 20), color: (0, 0, 0), 
  Voxel with grid_index: (28, 47, 33), color: (0, 0, 0), 
  Voxel with grid_index: (24, 46, 43), color: (0, 0, 0)]

In results,

a. Mesh Information:

• The mesh has 35,947 vertices and 71,600 triangles (faces), describing its shape and structure.
• The average distance between neighboring points in the mesh is approximately 0.00644 units, indicating the mesh’s level of detail.

b. Voxel Grid Information:

• Voxel Center in X,Y,Z coordinates for three different voxel sizes:

[-0.02873008, 0.09729862, 0.00515428]

[-0.02652538, 0.09834554, 0.00514782]

[-0.02753553, 0.09317389, 0.00369362]

• Number of Voxels for each voxel size:

71422 voxels for size 0.00644

234502 voxels for size 0.01288

19359 voxels for size 0.00322

The below plots show the results of mesh to voxel grid conversion. Plots from left right and top to bottom: voxel grid with voxel size = 0.00644, 0.01288, 0.00322 respectively and the combined one.

Left to Right: voxel grid with voxel size = 0.00644, 0.01288 respectively.

Left to Right: voxel grid with voxel size = 0.00322 and the combined one.

Summary

In this blog, we discussed converting mesh data into point clouds and voxel grids. Mesh to point cloud conversion simplifies complex geometry into a structured grid of 3D points, useful for tasks like computer vision. Meanwhile, mesh to voxel grid conversion offers a quantized representation, beneficial for applications like 3D printing and medical imaging. We also provided code examples for both conversions, highlighting their practical significance in various fields.

✨Happy exploring! Happy learning!✨

📝Next Blog Preview:

In the upcoming post, 🚀“Mastering 3D Computer Vision & Point Cloud Processing- Mod 11— Voxel Grid to Point Cloud Conversion with Code”.

Topics to Be Discussed in the Next Blog

1. Voxel Grid to Point Cloud Conversion
2. PCD vs PCD from voxel

Topics to Be Discussed in the Upcoming Blogs — Immediate Focus

• Discussion on Commonly used File formats
• Most used Dataset
• Most Point Cloud Preprocessing Techniques