Under the sea XR
Building a Codex-Centered Production Pipeline
Overview
Undersea XR is an underwater Unity prototype and testbed for a Codex-centered production pipeline.
The first working version took shape in 20 days, with VR constraints shaping the work from the start through LOD strategy, GPU-friendly rendering, asset prep, and in-engine validation. The goal was to keep research, generation, procedural workflows, validation, and delivery connected inside one production ecosystem.
For me, this project was also an exploration of a new way of working. I believe this kind of connected, evolving pipeline is closer to how production will operate in the future than relying on scattered AI tools and temporary wrappers.
Approach
Instead of using AI as scattered prompts, I built around a central orchestration layer in Codex. It handled planning, routing, research, project memory, tool execution, and artifact tracking, while reusable skills, specialized workers, staged workflows, and MCP bridges connected Unity, Blender, and Houdini.
The system was designed to stay close to real tools, keep outputs inspectable, and turn repeated work into reusable workflows.
AI-Driven 3D Production Pipeline
A 3D production pipeline built around Codex, connecting reusable skills, workflows, specialized workers, artifact lineage, and tools such as Unity, Blender, and Houdini.
- Codex as the main workspace
Codex serves as the central orchestration space where project context, workflows, workers, tools, and production decisions remain connected throughout development. - Skills as reusable task patterns
Skills turn repeated pipeline tasks, like research, asset prep, generation handoffs, and validation, into reusable procedures. - Workflows as staged execution
Workflows were created into automated and repetitive stages: skills, tools, outputs, approval points, and validation. - Workers in specialized production roles
Workers handle specific pipeline jobs, like project intake, visual analysis, image generation, video prompting, or execution, using the right skills and workflows for each task. - Tool orchestration across services
Multiple AI services, APIs, repositories, search systems, and generation tools can be coordinated within the same production flow, allowing tasks to move between systems without breaking context. - MCP bridges production tools
MCP bridges allow the pipeline to operate directly across Unity, Blender, and Houdini, keeping production tasks connected from asset generation through integration. - Artifact dispatch and lineage
Generated assets and project outputs remain connected to the references, workflows, and deliverables around them, creating a traceable history of how work moves through the pipeline.
Visual Production Graph and Artifact Lineage
As workflows and artifacts grew, chat alone was insufficient to track the project complexities clearly.
To solve that, I built two connected views: an Artifact Lineage Board and a Production Graph. The lineage board shows how references become production outputs, while the graph shows the operational system behind them: skills, workers, tools, providers, workflow stages, and dependencies.
Together, they make the pipeline visible, traceable, and easier to manage.
Production Use Cases
Undersea XR used the pipeline for four kinds of work: research & knowledge capture, asset generation & optimization, procedural environment production in Houdini, and Unity-side validation for XR readiness.
For environment generation:
- Houdini was integrated directly into the workflow through MCP-driven automation. Procedural trench scenes could be generated from presets, processed into terrain chunks, exported with LODs and colliders, and prepared as Unity-ready outputs without leaving the pipeline.
- Blender was also used as an MCP-driven processing step for asset preparation jobs such as LOD generation, texture atlasing, cleanup, and export preparation. This helped keep generated assets moving toward Unity-ready deliverables without turning each optimization pass into a separate manual task.
- Unity was the destination for the pipeline, where generated assets, procedural environments, runtime systems, and XR interactions came together as a working experience. Codex supported implementation and optimization work across ECS-based flocking, GPU instancing, custom shader deformation, scene tooling, and performance validation.
Runtime Optimization
To support dense underwater ecosystems within XR performance constraints, wildlife rendering uses ECS-based flocking, GPU instancing, and vertex shader deformation rather than traditional skinned mesh animation. This approach reduces CPU overhead, scales efficiently to larger populations, and keeps runtime performance compatible with VR requirements.
Key Takeaways
- Agnostic: The pipeline reduced reliance on scattered wrappers and outside tools by keeping research, generation, validation, and delivery inside one connected system.
- Efficiency: AI accelerated execution through automation, reusable workflows, and specialized workers. This shifted my role from executing individual production tasks toward providing design direction, validation, orchestration, and decision-making.
- Continuous Learning: AI capabilities evolve rapidly. Maintaining an effective pipeline requires ongoing research, testing, refinement, and adaptation as new models, tools, and workflows emerge.
- Guardrails: As projects grow in complexity, strong guardrails become increasingly important. They help keep workflows reliable, repeatable, and easier to trust. Defining, testing, and refining those guardrails has become one of the key lessons from the project and remains an area I want to continue exploring and improving.
- Collaboration Potential: This pipeline was developed for a single operator. An important next step is exploring how similar systems can scale to multiple contributors while preserving structure, visibility, and workflow continuity.
- Human Direction Still Matters: AI can accelerate production, but it does not replace judgment, taste, creative direction, or technical decision-making. The ability to guide, evaluate, and refine outputs remains a critical part of the process.
All of this reinforced my belief that the future of creative and technical work will increasingly combine the speed and capabilities of AI with the judgment, direction, and expertise of a skilled conductor.
