Instrumentation, Control and System Engineering
Evaluation of Deep Network-based Methods for Crack Detection of Iron Ore Green Pellet
2022 Volume 62 Issue 8 Pages 1694-1704
Details
2022 Volume 62 Issue 8 Pages 1694-1704
Crack detection for iron ore green pellet is an essential step in the measuring process of drop strength, which is one of the important quality metrics of green pellet. However, current method for crack detection of green pellet is manual inspection, which is rather laborious, tedious and subjective. Although various deep network-based methods are proposed to automatically detect cracks in tunnel, pavement and wall, little effort has been made on pellet crack detection. Therefore, it is still unknown whether the current deep network-based methods can solve the crack pellet detection problem. In the present work, we perform comparison study to evaluate the performance of six state-of-the-art deep networks, using our green pellet dataset with various crack types and complex background. Comprehensive comparatives are conducted to evaluate the performance and computing efficiency of six deep networks on pellet crack detection. Moreover, task-driving comparison is performed to show what to extent the six deep networks affect the measuring accuracy of drop strength. Our experimental analyses demonstrate that CrackSegNet achieves better crack detection accuracy than other five networks (DeepCrack-Z, DeepCrack-L, U-net, CrackSegNet, GCUnet), and thereby performs better in the task of drop strength measurement. However, computing time needed by CrackSegNet (0.26 seconds per image) is longer than other networks (0.05–0.20 seconds per image) in processing one image with the size of 512×512. In future work, the performance of deep networks needs to be improved in crack detection accuracy as well as computing efficiency to ensure more accurate and fast measurement of pellet quality.
Crack detection for green pellet is an essential step in the measuring process of drop strength (DS), which is one of the important quality metrics of green pellet. DS reflects the hardness of green pellet, and should be strictly controlled within a certain range to ensure the stability of manufacturing process.1,2,3) The measuring steps of drop strength are as follows: operator raises a green pellet to the height of 0.5m, and then drops it to the steel plate. The “raise-drop” step is repeatedly conducted until cracks can be observed in pellet surface by manual inspection (see Fig. 1). Then, operator counts the drop number of green pellet until the crack appears, and sets the drop number as drop strength. This manual inspection method for crack detection is rather laborious, tedious, subjective, and not conducive to the optimization of manufacturing process. For these reasons, it is necessary to adopt an automatic crack detection method for green pellet to facilitate the DS measurement.
(a) Green pellets; (c) Crack detection of green pellets by manual inspection. (Online version in color.)
With the rapid development of image processing technology, image-based method can be adopted to achieve the automatic detection of pellet cracks.4) However, here comes some difficulties when adopting the image processing algorithm for pellet crack detection: (1) Cracks with different types appear in pellet surface after dropping, including long crack, short crack, wide crack, short crack, etc. (2) Several material stains and water stains are inevitably remained on the steel plate after pellet dropping, leading to the complex background in captured images. The material stain is similar with green pellet, whereas the water stain brings strong light reflection, which challenges the robustness of crack detection algorithm; (3) Green pellets usually contain high moisture content, leading to strong light reflection in pellet surface. In addition, green pellets have uneven surface, so that the captured images contain several shadows in pellet surface. The above difficulties bring challenges to the crack detection algorithm.
In recent years, various image-based methods for crack detection are proposed using traditional image processing techniques, such as threshold segment,5) Gabor filter6) and wavelets method.7) Such a method uses the characteristic that the intensity of crack pixels is darker than that of non-crack pixels. Although these methods can effectively detect long crack and wide crack, their capabilities are limited when detecting thin crack and short crack. To overcome the problem, machine learning-based methods are proposed to detect crack.8,9,10,11) These methods extract hand-crafted features from images, and then classify the image pixels into “crack” pixels and “non-crack” pixels using classifiers, such as support vector machine (SVM),9) random forest,10) AdaBoost,11) etc. However, it is difficult to find effective hand-crafted features to apply in images with complex background.12)
With the development of artificial intelligence, deep learning techniques provide new opportunities for crack detection.13) Deep learning techniques adopt an end-to-end network to obtain the detection results without using any hand-crafted features, so that the robustness of the crack detection algorithm can be greatly improved.14) Recently, some classic networks such as Fully Convolutional Network (FCN),15) U-net16) and SegNet17) are introduced in crack detection and achieve satisfactory detection results. However, these classic networks are mainly used for scene parsing or biomedical segmentation, and the objects in scene parsing and biomedical images are quite different from cracks. Therefore, many improved networks are proposed mainly used for crack detection, such as DeepCrack-L,18) DeepCrack-Z,19) and CrackSegNet.20) These improved networks usually use the classic networks as backbone and combine feature maps from different layers of deep architecture, and usually achieve better crack detection performance than classic networks in their application scene.
In summary, deep network-based methods have great successful performance in crack detection and can be considered state-of-the-art. However, current deep network-based methods mainly focus on crack detection in concrete, buildings and pavement, it is still unknown whether deep network-based methods can achieve satisfactory detection results of green pellet, and meet the requirements of online monitoring of drop strength. Therefore, it is motivated us to perform comparison study to solve the problem of pellet crack detection using deep network-based methods, so that we can figure out (1) whether the current deep network-based methods can be directly adopted in pellet crack detection; (2) which networks perform better in accuracy and computing efficiency to meet the requirements of online monitoring of drop strength. In the present work, comparison studies of six state-of-the-art deep networks are performed to detect pellet crack using our dataset with green pellet dataset. The contribution of our work includes:
(1) A dataset with green pellet images is established, which contains cracks with various types and complex background. Our dataset provides a benchmark for evaluating the performance of different deep networks on pellet crack detection.
(2) Comprehensive evaluation of six state-of-the-art deep networks on pellet crack detection are performed, including the comparison of crack detection performance, comparison of crack detection efficiency and comparison of task-driving.
(3) Advantages and limitations of six deep networks on pellet crack detection are analyzed, whereas the improvement of deep networks on crack detection are discussed.
This paper is organized as follows: In section 2, the six state-of-the-art deep networks used for pellet crack detection are introduced. In section 3, dataset with pellet images used for network evaluation is established and described, and testing experiments of six networks on our dataset are conducted. In section 4, comprehensive evaluations of six state-of-the-art deep networks on pellet crack detection are performed; In section 5, we conclude the performance of six deep networks on crack detection and put forward the improvements.
Deep learning technology is a subfield of artificial intelligence, which has proven well performance in semantic segmentation using deep networks.21,22,23) The task of crack detection is treated as a semantic segmentation, in which the user provides crack images and then obtains information about cracks.24) Recently, many crack detection methods using deep network have adopted in different case studies due to the high accuracy and good robustness.
In the experiments, we evaluate and compare the crack detection performance of six deep networks on green pellet, including two classic networks named FCN,15) U-net,16) and four improved networks named DeepCrack-Z,19) DeepCrack-L18) CrackSegNet,20) GCU-net.26) We select these six networks because (1) these network structures are chosen from the most popular architecture of deep networks; (2) these networks have been validated and show good performance on various datasets with concrete images, pavement images and wall images. In Table 1, the basic information of the six networks is given, including network skeleton, main components, applicable scene and testing dataset. In the following texts, we will briefly describe the structure and main components of the six networks.
;%0A%09%09%09newWindow.document.open();%0A%09%09%09newWindow.document.write('<img src=%22./Graphics/62_ISIJINT-2022-108_tbl_1.jpg%22>');%0A%09%09%09newWindow.document.close();%0A%09%09)
|
FCN has an encoder–decoder architecture, as seen in Fig. 2. The encoder contains convolutional layers and max pooling layers to extract the abstracted features from original image, and the decoder contains convolutional layers and upsampling layers to extract detailed features and recover the image resolution. The kernel size of the convolutional layers in encoder–decoder is set to 3×3, because convolutional layer with the 3×3 kernel size has fewer parameters and can extract detailed features from images than that with a larger kernel size. The channels number of first layer in encoder is set to 64 to extract richer information. After each max pooling layer, the channels number of layer is increased to extract high-level features and make up for the loss of information caused by the decline of resolution. To obtain more precise upsampling results, max pooling index in encoder is concatenate to its corresponding upsampling layer. After that, a 1×1 convolutional layer is used to obtain the final crack detection results.
Architecture of FCN. (Online version in color.)
U-net has an encoder–decoder architecture including 18 convolutional layers with 3×3 kernel size, 4 max pooling layers and upsampling layers, as seen in Fig. 3. It adopts skip connection to concatenate convolutional layers in encoder to its corresponding upsampling layers in decoder, so that both detailed and abstracted features can be obtained from images. Compared to abovementioned FCN, U-net has deeper convolutional layers to extract more abstracted features and adopts skip connection to reduce the loss of spatial resolution of the image feature.
Architecture of U-net. (Online version in color.)
GCU-net is improved from U-net, and its architecture is shown in Fig. 4. GCU-net consists of two major components: (1) an U-net structure used as the backbone; (2) a global context (GC) block in decoder used to capture the global context in the high-level crack features. The architecture of GC block is shown in Fig. X3 (b), it is seen that GC block contains a context modeling module and two transform modules. Modeling module is used to learn query-independent global context, and two transform modules used to capture channel-wise dependencies and fuse features. The kernel size of convolutional layers in GC block is set to 1×1, so that the number of parameters in this module can be further reduced. Compare to U-net, GCU-net can effectively model global context information to deal with the various crack type of concrete images.
(a) Architecture of GCU-net. (b) Architecture of GC block. ⊗ represents matrix multiplication, ⊕ represents broadcast element-wise addition, ⊙ represents broadcast element-wise multiplication. (Online version in color.)
DeepCrack-Z is improved from Segnet, and its architecture is shown in Fig. 5. DeepCrack-Z consists of two components: (1) an encoder-decoder; (2) a skip-layer fusion module. The encoder-decoder structure is built on SegNet, including 13 convolutional layers and 5 down-sampling pooling layers in encoder and 13 convolutional layers and 5 down-sampling pooling layers in decoder. The skip-layer fusion module is adopted to fuse image feature at different scale. Specifically, skip-layer concatenates the last convolutional layer at each scale in the encoder and last convolutional layer at the corresponding scale in the decoder, followed by a 1×1 convolutional layer and a convolutional layer to decrease the multi-channel feature maps and recovery the image resolution. Finally, the outcomes generated from five different scales are further concatenated and then operated by a 1×1 convolutional layer to obtain the detection results.
Architecture of DeepCrack-Z. (Online version in color.)
DeepCrack-Z is improved from VGG16, and its architecture is shown in Fig. 6. DeepCrack-Z consists of (1) the convolutional layers, (2) side-output layers, (3) the refinement part. The convolutional layers built on classic VGG16 with 12 convolutional layers. Different from the conventional VGG16, encoder part in DeepCrack-L discard connected layers to increase the computation efficiency; Side-output layers used to extract image feature from last convolutional layer at each scale in encoder, so that abstracted features and detailed features at different scales can be obtained. Five side-outputs are fused together and then filtered by guided filter to preserve crack features and remove the noises in low-level detection results.
Architecture of DeepCrack-L. (Online version in color.)
DeepCrack-Z is improved from VGG16, and its architecture is shown in Fig. 7. CrackSegNet mainly consists of: (1) an encoder built on VGG16; (2) dilated convolution layers; (3) spatial pyramid max pooling layers; (4) skip connections. Four dilated convolution layers with dilation rates of k = 2, 2, 4, and 4 are applied in cascades after encoder used to extract deeper feature and maintain the original spatial resolution. Four spatial pyramid max pooling layers are adopted to obtain image features at different scale. Skip connections used to extract image features at different scale from encoder. Finally, the image features at 7 different scales are extracted from encoder and spatial pyramid max pooling layers, and then fed into 1×1 convolutional layers to obtain the final detection results.
Architecture of CrackSegNet. (Online version in color.)
In this section, a new dataset with green pellet images is established, and the image acquisition procedures as well as the characters of our dataset are described in detail. Then, experiential testing of six state-of-the-art networks on our dataset is conducted, and the experimental setting of six network in training process is described in the following texts.
3.1. Experimental Set-up and Image AcquisitionTo test the performance of different deep networks on pellet crack detection, a suitable testing dataset should be adopted. Although some researchers provide available datasets such as CrackForest,9) DeepCrack,18) Crack500,26) SDNET2018,27) they are not suitable for testing the performance of networks on pellet crack detection. The reason is that current available datasets only contain concrete images, pavement images and wall images, while little effort has been made to establish dataset with green pellet images. In addition, the cracks in current dataset are usually continuous and have long length, while the cracks in green pellet have poor continuity and short length. Therefore, it is necessary to establish our own dataset to test the crack detection performance of six deep networks.
The experimental set-up for dataset establishing is shown in Fig. 8. It consists of an industrial grayscale camera (Baumer VCXG-53M, with the resolution 512×512), a steel plate, a LED used to provide illumination. Green pellets used in the present work were collected from local metallurgical factory. To obtain the crack pellet images and non-crack pellet images, the image acquisition procedure is conducted according to the process of drop strength measurement. The detailed image acquisition procedures are described as follows: (1) Turn on the LED light and set the supply voltage, then select a testing green pellet; (2) Raise the testing green pellet to the height of 0.5 m, then drop it to the steel plate; (3) Capture the green pellet image, then detect whether there are cracks in the pellet surface by manual inspection. If there is no crack on pellet surface, repeat (2)–(3); if there are cracks on pellet surface, repeat (1)–(3) until enough images are captured. The above procedures were repeated at different lighting intensities (LED supply voltages: 16 V–21 V).
Experimental set-up. (Online version in color.)
Example of dataset with green pellet images. (Online version in color.)
By using the experimental set-up and conducting image acquisition procedures, a total of 640 images are captured, which contains 320 images with crack pellet and 320 images with non-crack images. Part of the green pellet images are shown in Fig. 13 for example, it can be seen that our dataset has the following characteristics: (1) green pellet images contain various cracks types, including thin crack, wide crack, thin crack, etc. These cracks have great difference in length, width and shape; (2) the pellet image contains complex background, because raw materials and moisture are remained on steel plate after pellet dropping, as seen in image 2-4 and image 2-5; (3) Pellet images are captured under different illumination. The strong light reflection in steel plate and pellet surface exists in steel plate and pellet surface when images are captured under bright illumination (19 V–20 V), and the pellet cracks are blurred and indistinct when images captured under dark illumination (16 V); (4) Green pellet usually contains high moisture content, leading to strong light reflection in the pellet surface (see image 3-5); (5) Green pellet contains uneven surface, which brings shadows in pellet surface (see image 4-6).
Measuring accuracy of drop strength using the crack detection results by different deep networks. (Online version in color.)
The abovementioned diversities of pellet images bring great challenges to the crack detection by deep networks, which is expected to be able to detect pellet cracks with various types as well as be robust to complex background.
3.3. Network TrainingDatasets including 640 green pellet images are used for the network training, in which 320 images are used for network training, and the other 320 images are used for networks testing. The pellet images have the same size of 512×512, and the reference images (ground-truth) are annotated by manual using a labeling tool.
Six networks including FCN, U-net, GCU-net, DeepCrack-L, DeepCrack-Z, CrackSegNet are adopted to detect pellet crack. In the process of network training, there are several parameters and methods for network iteration need to be settled, including initialization method, optimizer, learning rate, loss function, training epoch. In these experiments, the initialization method, optimizer, learning rate and loss function are set in accordance with literature to guarantee the convergence of networks. The training epoch of the six networks is set to 300 for fairly comparison. The details of experimental setting of six networks in the training process are shown in Table 2.
| Networks | Loss function | Learning rate | Optimizer | Initialize method | Epoch |
|---|---|---|---|---|---|
| FCN | Cross entropy | 0.0001 | Adam | He normal | 300 |
| U-net | Cross entropy | 0.0001 | Adam | He normal | 300 |
| GCU-net | Focal loss | 0.0025 | Adam | He normal | 300 |
| DeepCrack-Z | Cross entropy | 0.00001 | SGD | Msra | 300 |
| DeepCrack-L | Modified cross entropy | 0.0001 | SGD | He normal | 300 |
| CrackSegNet | Focal loss | 0.00001 | Adam | He normal | 300 |
After network training, we sent 320 testing images to the trained networks and then obtained probability maps. Optimal dataset scale method (ODS)19) is used to convert probability maps to binary images, so that the crack detection results containing “crack” pixels (positive pixel) and “non-crack” pixels (negative pixel) can be obtained. In order to obtain the convincing detection results, we tried our best to use the open-source code provided by authors themselves.
In this section, various experiments will be performed to evaluate and compare the performance of six different networks on crack detection, using dataset with green pellet images captured in Section 3. The comparison of crack detection performance and the detection efficiency are conducted, and the task driving evaluations are also performed.
4.1. Comparison of Crack Detection Performance of Different Networks 4.1.1. Evaluation MetricsTo quantitatively evaluate the crack detection performance of six different networks, four commonly used metrics including Acc, Precision, Recall and F1 are adopted. Acc measures the overall accuracy of detection results. Precision measures the ratio between true positive pixels and all positive pixels. Higher Precision value indicates more “cracks” pixels can be detected. Recall measures the ratio between true positive pixels and the sum of true positive pixels and false negative pixels. Higher Recall value denotes less pixel is wrongly detected. F1 is the comprehensive evaluation metric for Precision and Recall. Higher value of these four metrics denotes that network has better crack detection performance. These four metrics are defined as
| (1) |
| (2) |
| (3) |
| (4) |
In Fig. 10, six testing images with different crack types and complex background are used to test the performance of different deep networks. Image #1 is captured under dark illumination (16 V), and its crack is blurred and indistinct; In image #2, a block of material stain is remained on the steel plate, and its pixel intensity is similar with that of green pellet; In image #3, there are powder materials in the background; In image #4, there are multiple cracks in pellets, including short crack, thin crack, wide crack, etc. In image #5, the pellet surface is uneven, so that there are shadows in pellet surface; In image #6, water stain is remained on the steel plate, resulting in strong reflected light in image background. The detection results by different networks are given in Figs. 10(b)–10(g), and their corresponding reference images are shown in Fig. 10(h).
Crack detection results of pellet images using six different networks. (Online version in color.)
As can be seen from Fig. 10, the classic networks (FCN, U-net) perform well when detecting cracks with different crack types including short cracks, thin cracks, and wide cracks. They also have good robustness to image background where there are water stains or material stains are remained on the steel plate. However, FCN is easily affected by strong light reflection in pellet images. As seen in image #6, the detected cracks by FCN are discontinuous due to the strong light reflection in pellet surface. In addition, the detection performance of U-net is limited when the pellet surface is uneven. As seen in image #5, U-net wrongly detected the shadows in uneven surface as crack.
The other three improved networks (GCU-net, DeepCrack-L, DeepCrack-Z) didn’t show apparently improved from classic networks. Compared to U-net, GCU-net have satisfactory performance in detecting various cracks, but it is easily affected by the uneven pellet surface and the stains in image background. One possible reason is that the introducing of global context block in networks is suitable for concrete image without complex background, while not suitable for the pellet images; DeepCrack-L is robust to uneven pellet surface and stains in background, but it is easily affected by strong light reflection in pellet images in that the detected cracks are discontinuous when processing images with strong light reflection (seen image #6). In comparison, DeepCrack-Z has satisfactory performance in detecting wide cracks with good robustness to complex background where the water stain or material stain are remained on the steel plate. However, it is easily affected by strong light reflection in pellet surface, and its detection performance is limited when detecting thin cracks and short cracks. As seen in image #4 and #6, most of the thin cracks and short cracks were not detected, and cracks in pellet surface also were not detected due to the strong light reflection. One possible reason is that DeepCrack-Z is mainly adopted for crack detection from pavement images where the background has no stronger light reflection and the cracks are continuous and long. Therefore, DeepCrack-Z is not suitable for pellet image with discontinues and short cracks and stronger light reflection.
On the contrary, CrackSegNet outperforms other five networks. It can effectively detect cracks with different length, width and shape, and has good robustness to the background where the water stain and material stain are remained on plate. Moreover, CrackSegNet is less affected by strong light reflection and uneven surface. The well performance of CrackSegNet may be contributed by the VGG16 skeleton and three components including dilated convolution layers, spatial pyramid max pooling layers and skip connections. The VGG16 skeleton can achieve satisfactory segmentation performance under complex background,28,29) and the three network components are used to extract seven feature maps at different scale from encoder. Therefore, CrackSegNet can not only obtain more accurate detection results, but also have good robustness to complex background.
The metric evaluation results (averaged 320 images) of pellet crack detection by six networks are shown in Table 3. It is seen that CrackSegNet has the best comprehensive performance in Acc (0.998), Precision (0.775) and F1 (0.738) due to the good robustness to various crack type and complex background, indicating that most of pellet cracks can be correctly detected. Other four networks (U-net, DeepCrack-L, FCN, GCU-net) achieve satisfactory metrics value and their F1 value ranging from 0.696 to 0.722. On the contrary, DeepCrack-Z underperforms other five networks due to its poor performance in detecting thin cracks and short cracks. It has relative low metric value in Precision (0.568), Recall (0.603) and F1 (0.585), indicating that most cracks were not detected.
| Networks | Performance metrics | |||
|---|---|---|---|---|
| Acc | Precision | Recall | F1 | |
| FCN | 0.997 | 0.723 | 0.699 | 0.711 |
| U-net | 0.998 | 0.716 | 0.728 | 0.722 |
| GCU-net | 0.997 | 0.733 | 0.733 | 0.696 |
| DeepCrack-Z | 0.998 | 0.568 | 0.603 | 0.585 |
| DeepCrack-L | 0.998 | 0.751 | 0.689 | 0.719 |
| CrackSegNet | 0.998 | 0.775 | 0.705 | 0.739 |
According to the above analysis, CrackSegNet seems more advantageous than the other five networks (FCN, U-net, DeepCrack-Z, DeepCrack-L and GCUnet) in crack detection of green pellet. Since the fast speed of computing time is needed to meet the requirements of online monitoring for drop strength in metallurgy factory.30) It is therefore motivated us to compare the computing time of the six different networks. In the present work, all the crack detection methods using different networks are implemented on the platform Spyder with python 3.6, and computed on a PC with an Intel core i7 and 8 GB of memory.
The average computing time and the platform needed by different networks for 320 pellet images (with the resolution of 512×512) are presented in Table 4. It can be seen that DeepCrack-L costs much less computing time for each image (about 0.05 seconds) than other five networks because DeepCrack-L discards the fully connected layers to improve the network efficiency. Other four networks (DeepCrack-Z, FCN, U-net, GCU-net) cost comparable computing time ranging from 0.12 seconds to 0.20 seconds. Although CrackSegNet outperforms other networks in crack detection, its computing time is relative long (about 0.26 seconds). This means that the performance of crack detection and computing time should be made in a comprehensive manner.
| Networks | FCN | U-net | DeepCrack-L | CrackSegNet | GCU-net | DeepCrack-Z |
|---|---|---|---|---|---|---|
| Platform | Keras | Keras | TensorFlow | Keras | Keras | TensorFlow |
| Computing time | 0.18 s | 0.20 s | 0.05 s | 0.26 s | 0.22 s | 0.12 s |
Crack detection of green pellet is one of the steps in measuring process of drop strength. Therefore, it is necessary to perform task-driven comparison to indirectly evaluate the performance of the six networks, and show what to extent the crack detection results by different networks can affect the measuring accuracy of drop strength. In this section. the procedures of drop strength measurement using deep network-based method are described. Then, the measuring results of drop strength by different networks are demonstrated and discussed.
The procedures of the drop strength measurement using deep network-based method are shown in Fig. 11. Detailed descriptions are illustrated as follows: (1) Select a qualified green pellet; (2) Drop the pellet from 0.5 m height to steel plate, and record the drop number as i (i>0); (3) Capture pellet image using experimental set-up in section 2; (4) Crack detection for green pellet using deep networks, then output the detection results; (5) if any “crack” pixel exists in detection results, the green pellet is detected as crack pellet, and its drop strength is measured as i−1. Otherwise, the green pellet is detected as non-cracking pellet, then, repeat step (2)–(5) until pellet crack can be detected, or the pellet is completely broken.
Procedure of the measurement of drop strength using crack detection results by deep networks. (Online version in color.)
In Fig. 12, two testing pellets with different crack types are used to show how deep networks affect the measuring results of drop strength. For pellet #1, there is a long crack exists in the third drop image, so that the reference value of drop strength (DS) is 2. For pellet #2, there is a short crack appears in the fourth drop image, so that the reference value of DS is 3. When processing images of pellet #1, six networks well detect the long crack in third drop image, and thereby the DS can be correctly measured. However, when processing images of pellet #2, five networks (GCU-net, U-net, FCN, DeepCrack-Z, DeepCrack-L) are affected by the short crack, uneven pellet surface and the strong light reflection, resulting in the false measurement of DS. Specifically, GCU-net falsely detects the “noncrack” pixels as “crack” pixels in uneven surface from the third drop image, so that the measured drop strength value is less than reference value. Other four networks (U-net, FCN, DeepCrack-Z, DeepCrack-L) cannot detect any cracks from the fourth drop image, leading the measured drop strength value larger than reference value. On the contrary, CrackSegNet well detect the short cracks in the fourth image and less affected by the uneven pellet surface as well as the strong light reflection, so that DS can be correctly measured.
Measuring results of drop strength using the crack detection results by different deep networks. The detected cracks are marked by red rectangle. (Online version in color.)
To evaluate the measuring accuracy of drop strength using crack detection results by different deep networks, we measured the reference values of drop strength of 20 green pellets, and compared them with the measured drop strength using different deep network-based methods. The measuring accuracy of drop strength ACCdrop is calculated by
| (5) |
According to the above comprehensive analysis, CrackSegNet behaves better than other five networks on crack detection of green pellet images with various cracks and complex background. In task of drop strength measurement, although the computing time of CrackSegNet is longer than other networks, it achieves highest measuring accuracy of drop strength. In this case, we recommend CrackSegNet when adopting deep network-based method to measure drop strength of green pellet.
Crack detection for iron ore green pellet is an essential step in the measurement of the important quality metric of green pellet called drop strength. The current manual inspection for crack detection has the disadvantages of laborious, tedious and subjective. Although various image-based methods using deep networks have been proposed to automatically detect cracks in bridge, concrete and building, little effort has been made on pellet crack detection. It is still unknown whether current deep networks can be directly adopted in the pellet crack detection. To answer the question, we conduct a set of experiments to establish green pellet dataset, and perform comparable experiments on dataset to test the performance of six state-of-the-art networks (FCN, U-net, DeepCrack-L, DeepCrack-Z, CrackSegNet, GCUnet) using comprehensive evaluations. It can be concluded from the evaluation results that:
(1) In the comparison of crack detection performance of six networks, CrackSegNet can effectively detect pellet cracks with various crack types and has good robustness to complex background. Thus, it achieves highest value in Acc, Precision and F1. Other four networks (FCN, U-net, GCU-net, DeepCrack-L) also have satisfactory performance in detecting cracks with various crack types, but their performance is limited when processing images with strong light reflection or uneven pellet surface. In comparison, DeepCrack-Z underperforms other five networks, its detection ability is limited when detecting thin and short crack.
(2) In the comparison of crack detection efficiency of six networks, DeepCrack-L costs the shortest average computing time (0.05 seconds), followed by DeepCrack-Z (about 0.12 seconds) and FCN (about 0.18 second). Although the CrackSegNet has the excellent performance in crack detection, it costs the longest computing time (about 0.26 seconds).
(3) In task-driven comparison, CrackSegNet has the highest measuring accuracy at 95% due to its good robustness to various cracks and complex background. In comparison, the other five networks have relative low measuring accuracy of drop strength ranging from 60% to 90%. The task-driven comparison is basically consistent with the above comparison of crack detection performance of six networks.
In conclusion, although CrackSegNet costs long computing time than other networks, it outperforms other five networks on crack detection of green pellet and achieves higher measuring accuracy of drop strength. Therefore, we recommend adopting CrackSegNet to solve the crack detection problem of green pellet.
5.2. Future WorkAlthough CrackSegNet outperforms other five networks in crack detection, it costs the longest computing time when processing pellet images. Since crack detection for green pellet is one of the processing procedures in drop strength measurement, the computing time should be improved to meet the requirement of online monitoring for drop strength in metallurgy factory. Therefore, in the future work, the performance of crack detection network on green pellets should be improved in computing efficiency and the accuracy of detection results at the same time to facilitate the drop strength measurement.
Financial support from National Natural Science Foundation of China (No. 61973108 and U1913202) are greatly appreciated. We also thank Qin Zou, Yahui Liu, Yupeng Ren, Yichao Dong, Liu Zhengqin for providing source codes of their crack detection networks in Internet.
