From 3D Data to World Models: Research Challenges in Physical AI
Recent progress in generative 3D and physical AI has accelerated development across both research and industry, but much of the public discussion still abstracts away the underlying technical challenges.
This conversation is designed for researchers and technical practitioners working close to the stack, with a focus on the data and systems questions shaping real-world progress:
how 3D data is captured, structured, enriched, and evaluated;
what world models require from training data;
where current data pipelines remain brittle;
and which representation and infrastructure choices most meaningfully affect downstream performance.
The discussion will examine generative 3D, physical AI, data quality, multimodal representations, and the translation from raw assets to model-ready inputs.
Hosted at the ALLSIDES NYC office, this is a technically grounded exchange for researchers, engineers, and applied AI teams, with time to continue the conversation informally afterward.
Rey Pocius, M.S. is a Machine Learning Researcher at Protege, where he leads research for the spatial and physical intelligence vertical. His work defines what data quality means for the foundation models behind embodied AI and world modeling, and studies how it shapes their performance. His research background spans robotics, explainable AI, reinforcement learning, diffusion and transformer models, and multimodal learning.(datalab.withprotege.ai)
Franz Tschimben is CEO and co-founder of ALLSIDES, where he works at the intersection of computer vision, 3D scanning, and digital twin infrastructure for AI and commerce applications. (ALLSIDES.tech)
Francis Williams is the Creator and Technical Lead of the high-performance 3D deep learning libraries fVDB and fVDB-Reality-Capture at NVIDIA Spatial Intelligence Lab. His research operates at the intersection of computer vision, machine learning, and computer graphics. He specializes in developing 3D shape representations which enable deep learning on real-world geometric data. (https://fwilliams.info/)