Robot teleoperation is often challenging due to lack of depth information, which can lead to collisions with obstacles. Free viewpoint images can address this issue by providing operators with a better understanding of the robot’s surroundings from arbitrary points.
Abstract
In robot teleoperation, a lack of depth information often results in collision of the robot with obstacles in its path or surroundings. To address this issue, free viewpoint images can greatly benefit the operators in terms of collision avoidance as the operators are able to view the robot’s surrounding from the images at arbitrary points, giving them a better depth information. In this project, we installed multiple fisheye cameras and an LRF on a robot, and free viewpoint images are generated under the assumption that walls are perpendicular to the floor. Experimental results show that the proposed method can generate free viewpoint images using cameras and an LRF. Finally, the proposed method is primarily implemented using OpenGL Shading Language to utilize a graphics processing unit computation to achieve a real-time processing of the multiple high-resolution images.
Source code
Calibration of multiple fisheye cameras, LRF, and robot model
Viewer
Citation
Our papers about free viewpoint image generation.
- R. Komatsu, H. Fujii, Y. Tamura, A. Yamashita, and H. Asama, “Free Viewpoint Image Generation in Man-made Environments Using Body Mounted Cameras and a Laser Rangefinder”, Proceedings of the 17th SICE System Integration Division Annual Conference (SI2016), pp.2536-2540, Sapporo, 2016. (In Japanese) (SI2016 Presentation Award)
- R. Komatsu, H. Fujii, Y. Tamura, A. Yamashita, and H. Asama, “Free Viewpoint Image Generation System Using Fisheye Cameras and a Laser Rangefinder for Indoor Robot Teleoperation”, ROBOMECH Journal, vol. 7, 15, pp.1-10, 2020. [pdf]
Contact
If you have any questions, please feel free to create issues here.