Decreasing labor force in the field of construction is one of the biggest social problems. To tackle this problem, we are collaboratively working with various companies on several projects to develop core technologies, with the aim to make the current construction more efficient and automate certain processes.
Action Recognition of Construction Machine
To enhance the efficiency of the current construction, it is crucial to monitor the progress of tasks to identify the bottleneck. We focus on tracking the progress of excavators as they are one of the most commonly used construction machines.
We install a camera at the construction site and apply computer vision algorithms to recognise the excavator’s actions and monitor its progress. One of the problems is that, unlike human action recognition, there is a limited amount of training data for excavator action recognition. Therefore, we utilize computer simulations to prepare the motion data for the training. This is still on-going project.
Visual Stereo Slam in Dynamic Construction Environments
There is a necessity to estimate the poses of construction machines for autonomous construction. In certain scenarios, such as near mountains and valleys, the GNSS signal is unstable. We are working on Visual Simultaneous Localization and Mapping (Visual SLAM) to localize construction machines.
Typically, Visual SLAM operates under the assumption that the environment is static. However, there are usually other construction machines moving around the site, which breaks this assumption and leads to poor localization accuracy. To address this problem, we employed object detection and semantic segmentation to determine the pixels that are unlikely to be static and reject them from the visual SLAM algorithm. We also utilized a hierarchical approach for efficient computation. Please refer to the paper here.
Innovative Constructions
Our laboratory is working on team organization and teleoperation of multiple construction machines as a part of the CAFE project. The CAFE project aims to develop AI robots by 2050 that adapt to diverse environments and act alongside humans. Focusing on natural disaster response and moonbase development, the project utilizes translatable technology between these fields. Please check CAFE project website for detail.
I’m mainly working on the teleoperation of multiple robots by one operator. Ideally, multiple robots should be able to perform the task autonomously without the help of an operator. However, the disaster sites are highly uncertain so in some situations the robots cannot perform the given tasks by themselves. It is also difficult for an operator to remotely control each robot individually. The main idea is to rely on the autonomy of the robots. The operator only provides detailed instructions when the robots cannot handle the situation properly. This is an ongoing project.