CVPR 2023 Tutorial: Hands-on Egocentric research with Project Aria, from Meta

Monday 19 June 2023, afternoon session


Richard Newcombe1    Xiaqing Pan1   Vasileios Balntas1   
Edward Miller1    Pierre Moulon1    Prince Gupta1    Rawal Khirodkar2
1Meta Reality Labs Research
2Carnegie Mellon University

Abstract

Project Aria is a research device from Meta, which is worn like a regular pair of glasses, and enables researchers to study the future of always-on egocentric perception.

In this tutorial, we will introduce two exciting new datasets from Project Aria: Aria Digital Twin, a real-world dataset with hyper-accurate digital counterpart; and Aria Synthetic Environments, a procedurally-generated synthetic Aria dataset for large-scale ML research. Each dataset will be presented with corresponding challenges, which we believe will be powerful catalysts for research.

In addition to introducing new datasets and research challenges, we will also provide a hands-on demonstration of newly open-sourced tools for working with Project Aria, and demonstrate how the Project Aria ecosystem can be used to accelerate open research into egocentric perception tasks such as visual and non-visual localization and mapping, static and dynamic object detection and spatialization, human pose and eye-gaze estimation, and building geometry estimation.

Learn more about associated challenges and datasets at ProjectAria.com.



Agenda

SECTION ONE: An Introduction to Project Aria

13:30 Always on machine perception, by Richard Newcombe
13:45 Introduction to Project Aria, by Prince Gupta

SECTION TWO: Aria Research Kit

14:00 Overview of Aria Research Kit & Partner Program, by Sach Lakhavani
14:12 Machine Perception Services, by Jakob Engel
14:24 Open Source SW ecosystem and roadmap overview, by Carl Ren
14:36 Hands-on demonstration via Jupyter notebook & CLI, by Vijay Baiyya

14:50 BREAK (25 minutes)

SECTION THREE: New Open Datasets, Models and Challenges

15:15 Overview of Open Science Initiatives, by Edward Miller
15:25 New Dataset & Challenge: Aria Digital Twin, Xiaqing Pan and Carl Ren
15:45 New Dataset & Challenge: Aria Synthetic Environments, by Vasileios Balntas
16:05 Partner Highlights: EgoHumans Dataset from CMU for inferring human position, by Rawal Khirodkar

SECTION FOUR: Closing remarks

16:25 Closing Remarks & Joint Q&A


Questions?

Visit the Project Aria website or email projectaria@meta.com to get more information on the project.

Academic and industrial research institutions interested in participating in Project Aria can submit their proposals here.