"Don't Mesh with Me: Generating Constructive Solid Geometry Instead of Meshes by Fine-Tuning a Code-Generation LLM"

LLM-powered CAD system generates precise 3D parts from text descriptions using code instead of meshes

This paper introduces a novel approach to generate precise 3D mechanical parts using code-generation LLMs instead of imprecise mesh-based methods. The solution converts CAD files into Python scripts representing Constructive Solid Geometry (CSG), enabling accurate and modifiable 3D geometry generation through natural language.

-----

https://arxiv.org/abs/2411.15279v1

🔧 Original Problem:

→ Current AI methods for 3D geometry generation use mesh-based approaches, which lack precision and adaptability for mechanical engineering

→ Existing solutions can't accurately represent smooth curves and surfaces, making them unsuitable for CAD applications

-----

🛠️ Solution in this Paper:

→ The paper creates a pipeline converting Boundary Representation (BREP) geometry into CSG-based Python scripts

→ It uses GPT-4 to generate natural language descriptions of 3D parts

→ The system fine-tunes a code-generation LLM to complete geometries based on positional input and text descriptions

→ The model learns to generate surface-based CSG instead of approximate meshes

-----

💡 Key Insights:

→ CSG representation allows precise control and modification of 3D parts

→ Text annotations quality significantly impacts model performance

→ Model accuracy decreases with increasing geometric complexity

-----

📊 Results:

→ 96.5% correct syntax and logic in generated geometry

→ 82% accuracy in following text instructions for simple geometries

→ Performance drops to 19% accuracy for complex structures with 7+ cells