The Automated Extraction of Parking Lot Pavement Distress from UAV SfM Point Clouds
2024 Volume 5 Issue 1 Pages 104-110
Details
2024 Volume 5 Issue 1 Pages 104-110
The road surface maintenance of parking areas is important to maintaining public infrastructure and extending its service life. Concave surface rutting detection is the first step in maintenance procedures and has become a tough task because of the increasing number of parking areas. UAVs were employed to collect data from the parking lot’s surface, providing a faster and more cost-effective solution rather than traditional detection methods. Furthermore, the algorithm of Structure from Motion (SfM) was utilized to reconstruct the 3D point cloud of the target area. RANSAC and DBSCAN algorithms were used to extract the road distress, which was further analyzed. The results illustrated that the proposed method can accurately detect and classify parking damage, achieving an accuracy rate of 90%.
The inspection of road surface pavements, such as highways and parking areas, is crucial for ensuring public safety and maintaining structural longevity. It is important for stakeholders to allocate resources toward the upkeep of these infrastructures and to strategize sustainable inspection methodologies. In particular, the distress of parking lot pavements exacerbates user discomfort and deteriorates vehicle functionality, driving comfort, and traffic safety. Timely inspection and maintenance of such distress are important in bolstering societal infrastructure.
In Japan, the proliferation of plaza-type parking areas aligns with the surge in vehicle ownership. With the increase in the number of cars, the number of parking spaces has also been increasing year by year. The number of parking spaces reached approximately 5.21 million until 2017. Besides, the large parking with an area exceeding 500 m2 was built, which necessitated substantial expenditures and efforts for maintenance checks 1). The Parking Handbook 20222), published by four parking associations in Japan, underscores an ongoing initiative to enhance the quantity and quality of parking spaces. Current endeavors aim to serve not only the patrons of these facilities but also the broader community by implementing accident prevention measures, augmenting user convenience through a user-centric approach, leveraging IoT technologies, embracing novel mobility solutions, considering environmental and scenic impacts, and planning for disasters.
Conventionally, parking lot distress has been subject to regular expert inspections. Japan Ministry of Finance stipulates the expected lifespan of various pavements, with a tenure of 15 years for concrete, block, brick, and stone pavements and 10 years for asphalt surfaces. Meanwhile, the service life of pavements is affected by the volume and load of vehicles. Rutting is a common pavement distress in parking areas caused by vehicle loading on the pavement. It is depressing and the surface subsides as a concave surface in the parking area, which padding water compromises the survivability of the pavement.
To reduce maintenance costs, it is necessary to perform regular inspections and assessments of parking lot pavements. Nevertheless, these manual inspections are laborious, expensive, labor-intensive, and potentially perilous, yielding results that are inherently qualitative and reliant on the inspectors’ expertise. Even experienced inspectors can struggle to produce accurate assessments 3). Consequently, the advent of pavement inspection mechanisms employing artificial intelligence and robotics is gaining increased significance 4).
Rutting can be detected by close look but it is difficult to detect by any image processing technic such as Convolution Neural Network of object detections et. c YOLO from image. Thus there is no literature-introduced method that can detect this type of damage effectively using UAV and AI. This study mainly references studies for road inspection as they share the objective of identifying pavement distress on bitumen surfaces. Recent advancements in technologies have prompted researchers to explore innovative methods for road surface assessment. Studies have generally focused on vibration-based 5), 2D image-based 6) 7) 8), and 3D point cloud- based methods 9) 10). Vibration-based techniques, utilizing vehicle-mounted accelerometers, offer a cost-effective and straightforward approach but provide limited coverage and are unsuitable for low- speed assessments in parking areas. 2D image-based methods employing AI algorithms effectively detect surface cracks and some deep potholes but lack depth information crucial for evaluating ruts and small distress. Conversely, 3D points cloud-based methods, like MMS and laser scanners, while offering accurate surface modeling, face challenges related to data acquisition time, equipment costs, and their applicability to parking areas.
UAV-based photogrammetry and Structure from Motion (SfM) technologies present an alternative for deriving 3D road surface models. For instance, Inzerillo et al. 11) utilized SfM to inspect road pavements, comparing UAV-SfM and ground camera-based SfM (N-SfM) to laser scanning results, and found UAV-SfM to be less accurate but accurate enough for identifying areas requiring detailed analysis.
Biçici et al. 12) proposed an automatic detection method for road surface inspection using UAV- derived point clouds, employing an algorithm to exclude non-relevant elements. However, the applicability of such data to parking lot inspections requires further investigation.
Wu et al.’s 13) use of a DJI Phantom(r) 4 Pro for parking pavement inspection and subsequent 3D mapping to manually identify defects highlights the potential effectiveness of 3D mapping in such contexts, with prospects for further research into automatic distress inspections.
Thus, this study is research to integrate various methodologies, such as UAV-SfM, to construct detailed 3D models of parking lots and develop an algorithm capable of identifying and detecting concave surface rutting automatically.
In the proposed framework of this study, as shown in Fig.1, the UAV operates autonomously, capturing aerial images of the parking lot and generating point cloud data via the SfM method. Next, the parking lot surface point cloud will be extracted. The RANSAC algorithm is utilized to differentiate between planar points and outliers within the point cloud. Subsequently, the DBSCAN algorithm clusters outlier points. Clusters with a point count below a predetermined threshold are identified as distress. Conversely, dense clusters are deemed planar, prompting a reapplication of the RANSAC algorithm. The process iterates until the point count in all clusters falls below the threshold, indicating completion. The subsequent phase involves calculating the depth and ratio of concavity for each identified pavement distress.
The UAV device used in this study was a DJI Mavic 2 Pro, for data gathering in parking lot inspections. The selected parking lot area, covering 1,200 m2 and located in Japan, has an irregular shape. The UAV and its flight route planning are shown in Fig. 2. Utilizing the DroneDeploy software, the flight area is demarcated, and parameters such as altitude and camera angle are selected, automatically generating a flight path as shown in Fig.2 (B). This enables the UAV to autonomously execute flights and capture images using DroneDeploy’s mobile software.
To optimize the point cloud’s accuracy in capturing surface information, the UAV’s camera is downwards during flights while also rotating around the area’s periphery, as shown in Fig. 3 (A) and (B). The photographic locations are recorded as shown in Fig. 3 (C).
The flight parameters, provided in Table 1, include maintaining a 30m altitude with an 80% overlap rate for both frontal and side images to enhance 3D modeling accuracy. The camera’s gimbal angle is set to -90 degrees relative to the flight direction, ensuring perpendicular image capture.
The reconstruction of the 3D point cloud is initiated by applying the Structure from Motion (SfM) algorithm to process photographic data. This algorithm offers a contemporary alternative to classical measurement methods and can reconstruct three-dimensional structures from images captured at different viewpoints with overlapping fields.
Metashape is an advanced image processing software primarily used for generating 3D cloud points from images. Recently, it has been widely applied in the fields of civil engineering 14) 15). This study used Metashape to process the 3D reconstruction of the parking lot data. The software commences by aligning camera positions to establish an initial tie point cloud. This is succeeded by generating a dense point cloud, which further refines the spatial detail. The examples of this process are shown in Fig. 4 (A) and (B).
To balance computational speed and accuracy during the point cloud construction, Metashape was configured to high accuracy mode for both tie point and dense point cloud generation. In this case, the parameters for 3D reconstruction are detailed in Table 2.
The point cloud generated includes not only the parking lot but also adjacent buildings, vegetation, and transient entities like people and vehicles due to the parking lot remaining operational during UAV flighting. Errors in the 3D point cloud reconstruction can occur when moving objects pass by during the taking of images. It’s crucial to isolate an accurate and complete target point cloud of the parking area. This involves computing the point cloud’s confidence level based on the number of images contributing to each point and classifying the point cloud into distinct categories using machine learning algorithms, which assess attributes such as the normal vectors of points. Metashape provides both the confidence values and classification method. As shown in Fig.5 (A), points with high confidence are highlighted in blue, with a value of 100. The classification result, indicating the road surface class in black, is shown in Fig.4 (B). This class includes the parking lot’s surface. Then, the target area will be manually filtered based on high-confidence and road surface class data, as shown in Fig.4 (C).
Due to design considerations like drainage and entrance heights, the parking lot surface isn’t uniformly flat. As shown in Fig.6, a red-to-blue gradient depicts elevation changes, with darker reds indicating higher areas. The parking lot has an approximate elevation difference of 0.28m, the highest point converges to a single point, creating multiple planes, and the entire parking lot elevation takes on a chasing cone shape. Hence, segmenting the surface accurately is crucial. Conventional damage extraction algorithms, which rely on normal directions, are difficult for such a varied surface.
This study introduces the use of the RANSAC algorithm 16) to fit planes and distinguish between planar and uneven regions. It is coupled with the DBSCAN clustering algorithm 17) to group closely situated point clouds. We start by conceptualizing the problem in two dimensions. As shown in Fig.7 (A), we generate a line representing a depression with specific x and y-axis projections. A small residual threshold set in the RANSAC algorithm effectively discriminates between linear and concave features. Increasing to two planes, as shown in Fig.7 (B), the RANSAC algorithm identifies one plane but not the other, prompting the use of DBSCAN to differentiate the clustering of other planes and other features. Again, the RANSAC algorithm is performed for the larger cluster. This process is generalized to three planes in Fig.7 (C), where the method proves capable of discerning all features.
In the study case, RANSAC first identifies larger planes and separates them from other planes and uneven areas. Then, DBSCAN clustering is used to distinguish individual features. As shown in Fig. 8, once the size of the clusters falls below a threshold, the iterations of RANSAC are stopped.
After separating surfaces and uneven areas on the parking lot, we analyze each feature individually, estimating depth and differentiating between concave and convex geometries. Depth is measured using an oriented bounding box for each feature, as shown in Fig.9. It is the smallest enclosing box for each feature, where the box’s vertical span represents the feature’s height.
To differentiate concave from convex elements, the centroid for each feature is calculated. Using the point cloud’s normal vectors, we compute the angle between each point’s vector from the centroid and its normal. This angle helps determine the surface’s concavity. Points within a concave region have a negative dot product of these vectors, as shown in Fig.10 (A), whereas those on a convex surface exhibit a positive dot product, as seen in Fig.10 (B). The concavity ratio is then determined by the proportion of points with a negative dot product within the feature.
For this case study, the parking lot spanned approximately 1,200 square meters. The computational processing was conducted on a MacBook Pro 2021, equipped with an Apple M1 Pro chip. The total time taken for the inspection process was 19 minutes and 28 seconds. Detailed information on the runtime is provided in Table 4, indicating that while the point cloud generation was the most time- consuming step, The inspection of parking lots can be accomplished relatively quickly using this method.
Since this study did not reach the survey of the true value, a photographic example taken after it rained is accepted as the true value and compared with the results of the inspection using 3D point cloud data. It can be seen that the area where water is accumulating is in distress. As shown in Figure 11, the upper figure is the true value and the lower figure shows the result. The red lines indicate that the detection was successful, while the blue lines indicate that it was not. It was found that the pavement was successful in detecting ruttings, but not in detecting pavement distress such as cracks.
In the end, 40 distresses were detected. The detection results are simply classified into three categories according to the concavity ratio. If the concavity ratio is less than 0.33, the distress is classified as convex. If the concavity ratio is between 0.33 and 0.66, it is classified as non-flat. If the concavity ratio is greater than 0.66, it is classified as concave. The examples are shown in Table 3. Sample A is a concave instance, sample B is a minor non-flat instance and sample C is a convex instance. The point cloud color shifts according to height. Red color indicates higher height, and blue has lower height. The detailed depths and concavity ratios were calculated, showing the fact that the parking lot surface condition is not perfect.
The distresses were visually labeled from the depth map of the point. The resulting confusion matrix is shown in Table 5. Finally, the overall accuracy was 90%.
This research proposed a method using UAVs to capture parking lot surface data and SfM techniques be used to construct a 3D point cloud, from which the parking lot’s surface, along with its concave and convex features, are extracted via the RANSAC and DBSCAN algorithms. The method also encompasses analyzing individual distress, assessing depths, and differentiating features by applying the minimum bounding box and analyzing point normals and centroids.
Future improvements identified in this study include:
1. Further study of the effect of parameter settings on extraction accuracy and consider the effect of point cloud reconstruction and the extraction algorithms on this accuracy.
2. Analyzing the algorithm’s applicability across various parking lots with differing areas, shapes, and feature depths.
3. Enhancing the precision of the RANSAC algorithm and reducing its sensitivity to threshold settings.
