Robot-Controlled Camera
A KUKA robot arm driving the camera in an Unreal Engine scene.
- Year
- 2021
- Role
- Interaction Designer, Prototyper
- Tags
- Robotic ArmVirtual ProductionHCI
Move the robot, move the camera.
Robot-Controlled Camera uses the IIWA Lady, a collaborative robot built to work alongside humans, to explore spatial perception in virtual space and produce animation effects that are hard to key by hand. The virtual scenes run in Unreal Engine, and TACO IIWA software handles real-time communication between the robot and the engine.
Move the robot, move the camera.
Robot-Controlled Camera uses the IIWA Lady, a collaborative robot built to work alongside humans, to explore spatial perception in virtual space and produce animation effects that are hard to key by hand. The virtual scenes run in Unreal Engine, and TACO IIWA software handles real-time communication between the robot and the engine.
We developed a custom interface between the ROSOCoop robots and Unreal Engine. Camera data streams over UDP from the robot into Rhino Grasshopper and on to the Unreal scene.
An Unreal blueprint receives the UDP packets and decodes them into camera state. The arm controls position and rotation, and also focal length and speed, so one physical movement composes the whole shot.
The system was built during a summer workshop at Feng Chia University, working side by side with robotic engineers.
Credits
- Context
- Unreal Engine Robot-Controlled Camera Workshop, Feng Chia University, Summer 2021
- Advisers
- Alexey Marfin, Yu-Ting Shen, Shih-Yuan Wang
- Tools
- Unreal Engine, C++, KUKA IIWA