2023 年 4 巻 1 号 p. 9-15
Geological surveys conducted at the initial and design stages of a project are imprecise due to the limitation in both the time and the budgets provided to conduct them, and often require further verification. With the advent of artificial intelligence, several attempts have been made to train machine learning models to classify, assess, and predict in situ rock types and properties. The objective of this study was to use readily obtainable data to train machine learning algorithms in classifying rock types from the local geology of a construction site. The output would then be automatically displayed in a 3D digital twin model and made available to all stakeholders of a project, thereby maximizing information utility. The machine learning models were trained to identify several rock types including slate, greenstone, chert, and limestone from the trial site. The models used were a gradient boosted decision tree, a normalization-free net, and a custom neural net, trained using drilling parameter, photographic, and hyperspectral imaging data of drill cuttings, respectively. The best performance was obtained by the model trained using hyperspectral imaging data, likely due to the unique spectral signatures produced by minerals in the near-infrared frequency range.
Geological surveys are an essential part of any construction project to map the geological terrain on which the construction will take place. However, geological surveys conducted at the initial and design stages of a project are limited in their precision, partly due to the limitation in both the time and the budgets provided to conduct them. Creating geological strata maps often requires extensive interpolation between distant borehole locations, which in many cases results in maps which do not reflect the true geology in great detail.
As such, it is important to verify and improve the geological information throughout the initial stages of construction, including excavation and drilling. The design plans and construction methods can then be reviewed in light of new and more detailed geotechnical data. Due to the scope of most civil engineering projects, the resources required to reconfirm and reassess are substantial, both in terms of the number and variety of geotechnical tests required, and in terms of human resources and expertise needed to reliably conduct such reviews.
With the popularization of artificial intelligence, several attempts have been made to train and deploy models to recognize, assess, and predict in situ rock types and properties. Chatterjee & Patel (2016) used probabilistic neural networks trained with photographic data to classify different types of limestone, keeping the misclassification error within 5-6%. Shu et al. (2017) used unsupervised feature learning and self-taught learning techniques with photographic data to identify igneous, sedimentary, and metamorphic rock types. He et al. (2019) trained neural networks to determine the in situ rock strength using drilling parameter data.
This article presents a case study in the development and trial implementation of a system which uses machine learning algorithms to classify rock types during rock drilling operations using drilling parameter data, photographic data, and hyperspectral imaging data, and encodes the model outputs into a 3D digital twin model.
(1) Site description
The developed system was deployed at a tunnel rehabilitation and slope stabilization project currently being undertaken in Nara prefecture, Japan. Sliding instability of the slope over one of the tunnel entrances has caused substantial cracking of the tunnel lining. Arched steel beam bracing was placed at major shear cracking locations to stabilize the tunnel as a primary countermeasure. The aim of the project is to install rock anchors over the face surrounding the tunnel entrance to prevent further sliding. A total of 800 anchors will be drilled into place over a 6,765 m2 area above and surrounding the tunnel entrance using the rotary percussion method (Figure 1). The anchors extend beyond the deepest slip surface, with lengths varying between 11 m near the tunnel entrance, to 83 m near the top of the slope.
(2) Site geology
The site geology consists predominantly of slate, greenstone, chert, and limestone (Figure 2). The rocks were identified by color, grain, texture, hardness, and chemical composition (Table 1). Color, grain, and texture were analyzed using a magnifying glass with a magnification factor of 10. Hardness was estimated by scratch test using a cutter. The presence of carbonate was identified using the rock acid test and was mainly used to confirm the presence of limestone.
(1) Data sources & acquisition method
The objective was to train artificial intelligence models to classify rock types using input data which could be obtained rapidly, practically, and cost effectively, and did not interfere with the regular functioning of the construction activities. Three data sources were chosen for this purpose: drilling parameter data measured during the operation of the drill, photographic data of drill cuttings sampled from the drilling fluid, and hyperspectral reflectometry data those same samples. The hyperspectral data is expected to produce better results in training the artificial intelligence model to classify rock types than the photographic or drilling parameter data. Hyperspectral imaging is commonly used in remote sensing surveys as different kinds of minerals produce distinguishable spectral signatures (Kruse, 2012). However, unlike hyperspectral images, drilling parameters could be measured in real time and with a higher spatial frequency without requiring additional labor. And, photographic data could readily be gathered without specialized equipment.
a) Drilling parameter data
Drilling parameters measured included the feed force, torque, water pressure and flow rate, blow number and energy, as well as drilling speed. These parameters are measured every 2 cm of drilling using a drilling sounding system (DSS) mechanically attached to the rotary percussion drill. 5785 drilling data points were recorded by the DSS and used for training. Previous attempts at using artificial intelligence algorithms with drilling data as input have been limited to predicting measures of soil/rock strength values such as SPT N and unconfined compression strength (He et al., 2019), rather than distinguishing the rock type itself.
b) Photographic data
Training data was obtained during trial drilling of two boreholes, with a total drill length of 169 m. Of the 169 meters drilled, 149 drill cutting samples were taken. The samples were placed in a glass petri dish 40 mm in diameter. Each petri dish is placed adjacent to a grey colored square reference used for light correction purposes and illuminated using a 500 lumen light. A photograph is taken squarely over each sample using an Olympus TG-6 at 40 cm distance. The image dimensions are 4000 by 3000 (width by height), and the resolution is 314 pixels/inch.
c) Hyperspectral imaging data
Although it is common to use hyperspectral data for rock identification, only the limited near-infrared frequency range was used for this study. Figure 3 shows a photograph of the arranged hyperspectral imaging set up. The drill cuttings sample, described in the previous section, is placed in a white rectangular ceramic container, with the container acting as a white balance reference. Two bright lights are placed at either side of the container, evenly illuminating the sample (5200 lumens each). High brightness environment was necessary as low light environments increase the noise in the spectral data. The hyperspectral imaging camera (Specim IQ), capable of capturing 204 contiguous spectral bands within a 400 nm to 1000 nm range with 7 nm resolution is placed between the two lamps squarely over the sample and at a 40 cm distance. The hyperspectral photograph is taken, and the reflectance per wavelength of the sample is collected. The image dimensions are 512 by 512, with a resolution of 100 pixels/inch.
(2) Data pre-processing
a) Drilling parameter data
The drilling parameter data was used as collected to train the model. A label is attributed to every row of data (collected every 20 cm). The data was split 80% for training, 12% for validation, and 8% for testing. A stratified k-fold split was used as the dataset was unbalanced.
b) Photographic data
For the photographic data, each photograph is labeled with one rock type. Each photograph is divided into 48 tiles. The data from the gray card section is separated by channel, averaged, and divided by the reference value 119. These values are then used to adjust the RGB channels of the rest of the sample data. Each section is resized to 128 by 128 pixels, horizontally and vertically flipped, rotated, and the brightness is randomly adjusted. Finally, the data is normalized using the ImageNet mean = [0.485, 0.456, 0.406] and std = [0.229, 0.224, 0.225], and rearranged into an 8*3*128*128 tensor (Figure 4). The data was split into 80% for training and 20% for validation using a stratified k-fold split (5 folds).
c) Hyperspectral imaging data
For hyperspectral data, each spectral image is labeled with a rock category. The white background spectral reflectance data is averaged over each bandwidth. To adjust the sample reflectance data, the reflectance value at each band of each pixel is divided by the averaged value of the white background. The data is then batched into 32*204 tensors (spectral data of 32 pixels) as shown in Figure 5.
(3) Machine learning algorithms
a) Gradient boosted decision tree model
For estimating rock types from DSS and hyperspectral imaging data, the gradient boosted decision tree (GBDM) architecture was implemented using the Light Gradient Boosting Machine (LightGBM) framework developed by Microsoft in 2016 (Ke et al., 2017). Compared to other machine learning methods, LightGBM has the following characteristics: (1) short model training time, (2) high memory efficiency, (3) high inference accuracy, and (4) the ability to train with large datasets (Ke et al., 2017). The final output was obtained by weighting the output values of ten constructed models using logistic regression. The hyperparameters used for the training of the model were set to default except for those described in the Table 2.
b) Normalization-free net model
For estimating rock types from photographic data, Normalization-Free Net (NFNet) architecture was used. NFNet is an image recognition model published by DeepMind in 2021 (Brock et al., 2021). By removing batch normalization commonly present in image recognition using ResNet-like architectures, training time is reduced by a factor of 8.7 with no reduction in accuracy (Brock et al., 2021). The hyperparameters used to train this model are summarized in Table 3.
c) Custom neural network
For hyperspectral data, a neural network with two hidden layers was used, in a funnel configuration (Figure 5). Each layer is composed of a densely connected neural layer with ReLU activation and dropout regularization. The first layer contains 256 nodes, and the second layer contains 128 nodes. The hyperparameters used are summarized in Table 4.
(4) Post-processing and evaluation results
For the photographic data, the model output categorization for each image section was compared to the labeled rock type. Similarly, for the DSS data, the model output for each row of data and compared to the label. As for the hyperspectral data, the model was used to perform inference on each pixel, and the number of pixels per rock type was aggregated. The most represented rock type (with the highest percentage of pixels) was compared to the labeled rock type.
The performance measures for each model are shown in Table 5. Overall, a better performance was obtained using hyperspectral imaging data compared to DSS data and photographic data. In the previous section, it was hypothesized that the hyperspectral method would produce better results due to the spectral signatures of minerals in the near-infrared light range. The larger size of the hyperspectral imaging dataset may have also contributed to the increased performance of the model.
The system developed for this project has two major components (Figure 6). The first component is the desktop software which houses the machine learning models trained to classify rock types using the inputs described previously. The second component is a web-application that was developed to display the source data used for the classification and the classification results in 3D digital twin model and in graphical form. The two components are connected via OneDrive cloud storage where source data are collected, organized, and uploaded. OneDrive was chosen as it is user friendly, accessible via the MS Graph API, and familiar to most workers.
(1) Rock classification desktop software
The software takes as input DSS data, photographic, or hyperspectral image data and outputs a classification matrix containing the likelihood of classification for each rock type (Figure 7). The software then generates a file containing the tabulated output of the classification for each meter of drilling and automatically uploads it to the shared OneDrive cloud storage.
(2) 3D digital twin model web application
The web-application architecture was chosen as the application could easily be accessed from any browser and does not require any installation. The main feature of the web-application is the display of a 3D digital twin model of the project site with its major components, which includes the digital elevation model of the rock face obtained by drone surveying, the tunnel area, the slip surfaces identified by geological survey, and the rock anchors. As drilling progresses and geological data becomes available, the application automatically downloads the data from the OneDrive storage and encodes the data into the 3D model as a colored cylinder (Figure 8). The classification output data is split into distinct groups depending on the training data type and added to the model in separately displayed layers. This functionality was built using the customizable Autodesk Platform Solutions viewer plug-in and accompanying APIs.
In this article, an outline of an artificial intelligence based geological evaluation system and an automated digital twin modeling system was presented. The system as a whole was successfully deployed to the trial site where it will continue to be used for the remainder of the project. The model trained using hyperstrectral imaging data performed best, followed by DSS data model, and lastly the photographic data model.
