BMW x Meta AR Research
Stabilizing AR/VR content inside moving vehicles with IMU and GPS sensor fusion.
- Year
- 2024
- Role
- Research Design Engineer
- Tags
- ResearchSensorsSpatial OSAR/VR
Bringing AR and VR into the car, where the world won't hold still.
As part of BMW's AR Research team, I work with Meta Reality Labs on AR and VR in the vehicle. Two questions drive it: can virtual content hold still while the car is moving, and once it can, what is AR actually good for in here?
Bringing AR and VR into the car, where the world won't hold still.
As part of BMW's AR Research team, I work with Meta Reality Labs on AR and VR in the vehicle. Two questions drive it: can virtual content hold still while the car is moving, and once it can, what is AR actually good for in here?
First, stability. Headset tracking alone can't pin content to the world inside a moving car, because the car's motion throws off the IMU. We feed real-time IMU and GPS from the vehicle's own sensors into the tracking pipeline, so objects stay anchored inside and outside the car as it accelerates, brakes, and turns.
Second, use cases. Once content holds still, the question is what it's for: productivity that turns a commute into work time, entertainment for passengers, navigation drawn onto the road, and the apps already running through the BMW ecosystem.
It comes together as mixed-reality demos on Meta's devices, built around one idea: the car and the headset agreeing on where they are. When the vehicle's sensors and the headset share a frame of reference, the content stops drifting and starts to feel like part of the car.
