Tip-over prediction for hydraulic excavator during traveling using DNN with point cloud bird’s-eye view

Abstract

This study reports on a new system for predicting a risk of tipping aimed at improving the safety of remotely operated hydraulic excavators. In disaster recovery sites, remotely operated hydraulic excavators are sometimes deployed, but operating them involves risks, including the danger of tipping over. To prevent the risk of tipping, a method of predicting the danger of tipping in advance is effective. Existing prediction methods based on physical models do not consider external disturbances or vehicle movement and the time taken for prediction has been a challenge. In this study, we propose a new system that uses a DNN (Deep Neural Network) to quickly predict the maximum inclination angle of a hydraulic excavator up to one second ahead. This system does not require the analysis of complex physical models and directly learns and predicts the relationship between sensor data and future body inclination angles. Furthermore, by converting 3D point clouds into bird’s-eye views and inputting them into a CNN (Convolutional Neural Network), we aim for rapid prediction. Verification by simulation showed a prediction error of 0.056 radians and a prediction time of about 2.79 milliseconds, demonstrating sufficient performance. However, these results are based on simulations, and validation with actual excavators is a future challenge. This study is expected to improve the safety of hydraulic excavators operated remotely. In the future, the goal is to achieve more efficient operations while ensuring the safety of both operators and equipment.