ISIJ International
Online ISSN : 1347-5460
Print ISSN : 0915-1559
ISSN-L : 0915-1559
Instrumentation, Control and System Engineering
A Visual PCI Blockage Detection in Blast Furnace Raceway
Yutao Wang Pu HuangGang Yang
Author information
JOURNAL OPEN ACCESS FULL-TEXT HTML

2020 Volume 60 Issue 3 Pages 519-527

Details
Abstract

The pulverized coal injection (PCI) blockage detection is critical to the stable operation of blast furnace. In recent years, tuyere cameras have been widely applied, which provides a channel to detect the PCI blockage. However, the visual impression of images strongly varies between different raceways, it requires detection method should be robust and convenient to fine-tune for different blast furnace images. This paper presents an intelligent image-based method to detect the PCI blockage. An adaptive image preprocessing technique combining de-noising algorithm and image enhancement algorithm is applied to remove image noise and improve image quality, laying the foundation for subsequent work. The fitting ellipse based on Hough transform is used to locate the tuyere region, which can separate the tuyere region from the background. The adaptive threshold segmentation algorithm combining Otsu and Bernsen is used to obtain binarized image. However, it is difficult to obtain the pulverized coal cloud only by binarization due to the similarity between pulverized coal cloud and lance in gray-level. The multi-scale fully convolutional network (FCN) based on deep learning is investigated to detect the lance region, and pulverized coal cloud can be extracted by removing lance in binarized image. The flow rate of PCI can be characterized by the extracted area information to some extent, which can be used to detect PCI blockage. Extensive videos captured from real production lines are used to evaluate the detection method. The experiment results show that the method can accurately detect the PCI blockage.

1. Introduction

Blast furnaces are the most important and commonly employed facilities for hot metal production due to their superiority in productivity and heat utilization.1) The tuyere raceway is the hearth of the blast furnace where generates heat energy and reducing agent through PCI technology. In fact, the campaign life and hot metal quality of blast furnace is directly influenced by the working condition of PCI.2) During PCI blockage the pulverized coal injected to the tuyere can be discontinuous and uniformity, which causing local inhomogeneous permeability of the burden. To prevent the above situation happening, it is crucial to obtain a more reliable PCI blockage detection.

Simple and common detection to PCI blockage includes temperature difference method3) and optical method.4) The temperature difference method is based on the fact that the pulverized coal temperature is higher than the ambient temperature and the compressed air temperature after dry cooling. Since some factors (wind direction, sunlight and season) exert different effects on lance at different position, it is difficult to generalize the temperature difference judgment rules. The optical method detects the working condition of PCI by installing a photodetector, which requires good light transmission performance in the detection area. However, most coal pipelines are opaque and their internal environment are complex, resulting in the application restriction of the method.

Compared with the above methods, visualization technology can intuitively reflect the raceway condition, which leads an increasing number of scholars to research. Some researchers5,6,7,8,9) studied the temperature distribution of the raceway. Puttinger et al.10) make a discussion of various aspects and difficulties to extract information from tuyere camera images. Qin11) presented an intelligent image-based system for automatic detecting the working condition of PCI. However, it does not solve the problem that separation of lance and pulverized coal cloud region caused by similar gray-level. A static background template is used to represent the lance region. According to the fact that position of lance is changed because the lance needs to be overhauled during damping-down, the static background is obviously not applicable. Zhang et al.12) introduced a dynamic background template by Hough transform for extract the pulverized coal cloud and particles in image. Nevertheless, the tuyere camera has a slight vibration, resulting in the position of lance changes between the adjacent frames. The method is only roughly obtain the lance region. Once the obtained lance region exists deviation, the results will be inaccurate. In fact, the visual impression of images strongly varies between different raceways as shown in Fig. 1, this makes intelligent image-based detection a challenge task. First of all, image data inevitably suffers from various kinds of noises interference in transmission, which causes the degradation of image quality. Some raceway images appear unclear and blurred when the filters of cameras are damaged. The specific performance is that the central region and edge region are unbalanced in gray information. In addition, raceway images also appear strong halo phenomenon generated by the diffuse reflection of furnace wall and pulverized coal particles. The presence of halo lead to uneven changes of gray-level and loss of detail information in image. To solve these problems, an adaptive image preprocessing technique combining image enhancement algorithm and de-noising algorithm is proposed to remove image noise and improve image quality. Second, the ellipse Hough transform is used to locate the tuyere region, which can separate the pulverized coal cloud and inner wall. Finally, the separation of lance and pulverized coal cloud region is a crucial problem. For one thing, the pulverized coal cloud region is close to the lance in gray-level, which means the threshold segmentation method is difficult to extract target region. For another, the position of lance is not fixed, which request the method can accurately and quickly detect changes in position. Convolutional neural network (CNN) has performed superb performance in image segmentation.13) On the basis of traditional CNN, FCN14) replace all full-connected layers with deconvolution layers to retain spatial information of image, which ensures the result more accurately. Therefore, an improved FCN called multi-scale FCN is investigated to detect the lance region. The method proposed in this paper not only can obtain the lance region more accurately, but also has robustness and good adaption compared to other methods. The pulverized coal cloud can be extracted by removing the lance. According to the extracted area information, the variation of PCI flow can be reflected, which can be used to detect the PCI blockage. Extensive videos captured from a 2500 m3 blast furnace are conducted to evaluate the detection method. The experiment results show that the method can accurately detect the PCI blockage.

Fig. 1.

Raceway images captured from different tuyeres.

2. System Description

The schematic diagram of the detection system for single tuyere is shown in Fig. 2. The system is comprised by 1/3″ SONY CCD camera with filter, protective tube, hood and image processing unit, etc. High-resolution camera in front of tuyere collects images signal from visible spectral band (380–780 nm) and transmitted them into the image processing unit as 8-bit RGB digital images. The frame rate of the obtained video is 15 frames/s and the size of raceway image is 192 × 240 pixels. The intelligent image-based detection method is then applied to real-timely detect the working condition of PCI.

Fig. 2.

The schematic diagram of detection system for single tuyere. (Online version in color.)

As shown in Fig. 3(a), a normal raceway image is mainly composed of pulverized coal cloud, raceway, inner wall and lance. Figure 3(b) shows the PCI blockage image. The main purpose of the work is to extract the pulverized coal cloud and judge whether it exists. This requires eliminating the inner wall, raceway and lance region in image. In fact, raceway images appear divergence according to the above analysis, which makes detection method should be robust and convenient to fine-tune for different blast furnace images.

Fig. 3.

The raceway image form real production lines. (a) Normal raceway operation; (b) Abnormal raceway operation.

3. Raceway Image Segmentation and Feature Extraction

3.1. Image Preprocessing

The main purpose of image preprocessing is eliminating the noise and improving image quality. Raceway images captured from production lines are three-channel RGB format, while system puts forward high requirements to detection speed. The RGB images must be converted into the grayscale image, which can reduce the usage of CPU and GPU to improve real-time performance. Then the images filtering is employed. Through observation of image and analysis of histogram, images mainly exist Gaussian noise. In order to remove Gaussian noise, image is divided into smooth region and detail region by canny edge detection. The smooth region is applied by Wiener filtering,15) and pixels in detail region is regained according to the gray-level information of its 3 × 3 neighborhood. In fact, traditional 3 × 3 median filtering is not effective when a small number of abnormal pixels appear in the neighborhood.16) Hence, the group A is constructed by formula 1 to remove the abnormal pixel. The median value of A is regard as the gray-level of filtered pixel in detail region. Experiments show that the algorithm can eliminate Gaussian noise as well as preserve edge.   

A={ x|μ- 2 δxμ+ 2 δ } (1)
Where x is gray-level of edge pixel, μ and δ are the mean and variance in 3 × 3 neighborhood of the edge pixel.

In addition, there are horizontal stripe noises in some images, which influence the detection for pulverized coal cloud. The presence of stripe noise makes image contain a lot of false detail region. It is well known that image spectrogram17) acquired by Fourier transform can demonstrate detail information, which can be used to judge whether the stripe noises exists. Figure 4 shows raceway image, corresponding spectrogram and cumulative distribution function image of spectrogram in vertical and horizontal directions. The value at the midpoint in vertical cumulative distribution function image with stripe noises is obviously higher than the value without stripe noises from Fig. 4(c). Thence, stripe noises can be detected according to the numerical relationship between the value at the midpoint and threshold T (T can be set in the range of 1 × 104 to 1.5 × 104). For stripe noise filtering, the algorithm combing morphological and frequency domain filtering is investigated. Morphological open-close filter18) is firstly to remove the noise in spatial domain. Then the image is transformed frequency domain by Fourier transform and the second-order Butterworth filter is applied. As shown in point P from Fig. 4(d), the cut-off frequency of Butterworth filter is chosen as the frequency corresponding to the secondary peak of cumulative distribution function image in horizontal direction. Finally, the filtered image is transformed back to spatial domain for subsequent processing.

Fig. 4.

Frequency domain analysis of raceway images (a) Raceway image; (b) Spectrogram; (c) Cumulative distribution function in vertical direction; (d) Cumulative distribution function in horizontal direction. (Online version in color.)

In fact, the tuyere camera has been in a harsh environment with high temperature and dust during the blast furnace operation, which brings about the filter lens in front of the camera always damaged. Raceway images present blurry and unclear appearance if the damaged filter is not replaced in time. Figure 5 shows the different raceway images and corresponding histograms. It can be found that the peak position is approximately 50 to 80 in histogram of unclear image through massive experiments, while the peak position of clear image is lower than 20.

Fig. 5.

Raceway image histogram analysis (a) Blurred raceway image; (b) Clear raceway image. (Online version in color.)

The grayscale stretching algorithm19) is mainly used to expand the range of gray-level and enhance contrast for images with excessively concentrated gray-level distribution. For the purpose of improving unclear raceway images quality, the adaptive stretching transform is applied, which can be expressed by Eq. (2).   

g(x,y)={ 0f(x,y)< W L f(x,y)- W L W T ×255 W L f(x,y) W T 255f(x,y)> W T (2)
Where f(x,y) is the gray-level of the pixel on the original image and g(x,y) is the gray-level value of the pixel on the converted image. The top of stretched window WT is selected as the peak of the histogram and the bottom WL is the second peak, as shown in Fig. 5(a).

Although the quality of image is significantly improved by grayscale stretching, images also have obvious halos phenomenon. The presence of halo phenomenon makes images uneven increase in gray-level and loss of the detail information. Power transformation is a nonlinear transformation that transforming high gray-level to low and can be applied to eliminate the halo quickly. In the experiment, the parameters in power transformation are determined by brightness of the tuyere raceway. The performances of image preprocessing are shown in Fig. 6. It can be observed that the adaptive preprocessing algorithm not only ameliorate image quality and remove the noise but also establish good foundation for subsequent processing.

Fig. 6.

Raceway image preprocessing. (a) Original image; (b) De-noised image; (c) Enhanced image.

3.2. Accurate Positioning of Tuyere Region Based on One-dimensional Ellipse Hough Transform

The pulverized coal cloud is connected to inner wall when the flow rate of PCI is relative large. After the binarization method applied, pulverized coal cloud region usually merges with the background and cannot accurately extract the pulverized coal cloud region. It is a reasonable idea that constructing a closed area for tuyere to separate the pulverized coal cloud from the background. Tuyere in image appears an elliptical region created by inclination of camera shooting angle, even though the tuyere is actually a circular shape. Therefore, the ellipse fitting (the circle is a special ellipse) will be more appropriate than circle fitting. In order to construct closed tuyere region, the ellipse Hough transform is employed. The basic idea of Hough transform is to transform the image coordinate space into the parameter space and to search the peaks in parameter space that is corresponding optimal ellipse parameters. Compared with the least squares fitting of ellipses, the ellipse Hough transform method can reliably extract the feature information under the condition of noise, curve discontinuity and incomplete edges. However, the parameter space needs five dimensions in traditional ellipse Hough transform, where it increases the complexity of the space and affects the efficiency of the algorithm.20)

In order to solve the problem, an improved Hough transform21) is proposed. The algorithm finds maximum cumulative midpoint of the connection line by interconnecting the edge points, and the midpoint is taken as the center of ellipse. Moreover, the remaining parameters of the ellipse can be reduced to one dimension parameter for Hough transform according to geometric properties of the ellipse. The algorithm can improve the detection speed due to the reduction of the dimension in the ellipse Hough transform. In experiment, we optimized the transform algorithm by restricting the range of the ellipse parameters to further improve the detection speed. The detected ellipses preform good results as shown in Fig. 7.

Fig. 7.

Results of tuyere region positioning.

3.3. Segmentation of the Target Region

After the position of tuyere has been acquired, the next step is to extract the pulverized coal cloud region. The biggest obstacle in this procedure is the division of the pulverized coal cloud and lance region. For one thing, the lance region is similar to the pulverized coal cloud in gray-level information, traditional binarization segmentation cannot perform well. Several methods are applied and results are shown in Fig. 8. Although the method cannot directly segment the target region, it can be used to distinguish the raceway. The maximum between-cluster variance (Otsu) algorithm22) applies a global threshold to divide the image into two parts. The algorithm has a good overall detection effect, while is not suitable for region with uneven illumination. Hence we used two local segmentation algorithms, Bernsen23) and Niblack24) algorithm. In fact, the local threshold segmentation is realized by setting threshold for each pixel according to its local information, which does not take the integrity of image into account and is susceptible to interference from individual noise. Compared with the Niblack algorithm, Bernsen algorithm has the advantage of small amount of calculation and fast segmentation speed. The local threshold of pixel in Bernsen is calculated by Eq. (3).   

T(x,y)=0.5×( max -rmr -rnr    f(x+m,y+n)+ min -rmr -rnr    f(x+m,y+n)) (3)
Where max(x,y) represents the maximum gray-level of the pixel in the (2r+1) × (2r+1) neighborhood, min(x,y) represents the corresponding minimum gray-level. T(x,y) is the threshold of each pixel.
Fig. 8.

Comparisons among threshold segmentation methods. (a) Processed image; (b) Otsu; (c) Niblack; (d) Bernsen; (e) Ours.

Massive experiments demonstrate the Otsu can highlight integrity of image, but ignore details of the image. The Bernsen algorithm is exactly opposite. An improved binarization algorithm combining advantages of the both is investigated. Figure 9 is the chart of the proposed binarization algorithm. The Otsu is used to binarize the pixels with higher and lower gray-level, which is determined by parameter a, while local threshold map T2(x,y) calculated by Bernsen algorithm is to binarize the rest pixels. Moreover, the local threshold map is applied by median filtering to avoid threshold mutation. As shown in Fig. 8(e), this method can accurately reflect the local nonuniform distribution of pulverized coal cloud.

Fig. 9.

The chart of the binarization algorithm.

For another, the position of lance region is not fixed through observing different raceway images. Moreover, the lance needs to be overhauled when blast furnace is in damping-down, resulting in the position of lance may change during reinstallation. The above features make it difficult to divide the pulverized coal cloud and lance region. Recently, CNN has shown superb performance for tackling with the computer vision tasks.25) In image segmentation, it has been demonstrated that CNN is more efficient than other approaches, especially when there are enough training samples. Compared with the traditional CNN, FCN replace all full-connected layers with deconvolution layers, which retains spatial information of feature map. The spatial invariant property ensure the flexibility in processing various images. However, due to the FCN only deconvolutes the deep features to predict target region, the boundary of detection result is not smooth. In practice, the features obtained from the shallow convolutional layer have local properties including geometric features such as boundary information and image contours, while deep convolutional layers can obtain high-level features with semantic information. According to the above analysis, the multi-scale FCN with the ability of pixel-level prediction is proposed to detect the lance region. Our architecture is shown in Fig. 10, the network is concatenating the shallow and deep feature maps, which can learn the semantic information and local properties.

Fig. 10.

Multi-scale FCN structure.

More specifically, VGG16 network is used as the feature extraction network. It is consisted of 13 convolutional layers and all convolutional layers are divided into five phases attaching a maximum 2 × 2 pooling layer. A Batch Normalization (BN)26) and an activation function layer ReLu are connected after each convolutional layer, where BN can accelerate network convergence. In addition, the network introduces the space pyramid pooling layer27) to aggregate context information of different areas, thereby improving the ability to obtain global information. The pyramid pooling layer is shown in Fig. 11. In this paper, the convolution feature layer conv5_1 is used as the input of the pyramid layer. Different sub-region features are extracted from the input map using three pooling scales 1 × 1, 2 × 2 and 4 × 4. The sizes of different sub-region features are not same. After pooling layer, 1 × 1 convolution operation is applied to reduce the dimension of the corresponding context and maintain the weight of the global feature. The sub-region feature maps of different scales are concatenating, and upsampling layer is then constructed to ensure concatenated map to have same scale as conv5_1 by bilinear interpolation. Finally, upsampled feature maps are concatenated with the feature map conv5_1 as output of the pyramid pooling layer.

Fig. 11.

Pyramid pooling structure.

The Boundary Refinement Network (BRN)28) is put forward to further combine feature information of each layer. The cascaded feature maps (The fusion layer of pool3, pool4 and output of VGG16) are the input of BRN which is a CNN consisting of six convolutional layers. In the structure of the BRN, the convolutional layers are connected with ReLu function, and BRN applies the dilated convolution with dilation ratio of 4 instead of the pooling layer to maintain the same resolution between the input and output feature maps. The output of BRN is the final detection result of lance region.

The target region is obtained by removing the lance from the binarized raceway image. However, due to the inaccuracy of segmentation threshold and the location error of the tuyere region based on Hough transform, there are some small interference regions. The open-close operation of mathematical morphology is applied to eliminate these regions. The results of pulverized coal cloud extraction are shown in Fig. 12. Thence, the flow rate of PCI can be detected according to the obtained region.

Fig. 12.

Pulverized coal cloud extraction result. (a) Preprocessed image; (b) Binarized image; (c) Lance area detection; (d) Extraction of pulverized coal cloud.

4. Results and Analysis

4.1. Image De-noising Analysis

In this section, the de-noising algorithm proposed in this paper is verified by experiments on raceway image data, which is compared with 3 × 3 median filtering, 3 × 3 mean filtering, Wiener filtering and Low-pass filter. The experiment compares the de-noising algorithm by clear real raceway images captured from 10 different tuyeres to ensure the test reliable. The peak signal-to-noise ratio (PSNR) is used as the evaluation index and the calculation expression can be formulated as follows:   

PSNR=10lg 255 2 1 MN i=1 M j=1 N [ I 0 (i,j)- I 1 (i,j)] 2 (4)
Where I0(i,j) is the original image and I1(i,j) is de-noised image. In experiment, the high quality raceway image is the original image I0(i,j), add Gaussian noise with standard deviation of 15 and stripe noise to original image as noisy image. Table 1 shows the comparison of image de-noising results. It can be observed that the performance of proposed algorithm is comparatively better than other algorithms. The reason is that the proposed algorithm not only removes noise well but also preserves the edge details to some extent in filtering process, which demonstrates the algorithm is feasibility and efficiency.

Table 1. Performance in different de-noising algorithms.
Trial No.No. 1No. 2No. 3No. 4No. 5No. 6No. 7No. 8No. 9No. 10
mean filter24.0223.9624.2224.0324.2724.1124.2124.3223.9824.24
median filter24.2424.1224.3424.2324.4024.2624.4524.5024.1324.35
Low-pass filter24.4424.3024.4024.3724.5124.3924.5424.6224.2224.47
Wiener filter24.1524.0924.2324.1524.2924.1324.3524.3824.0724.28
Ours26.1126.0926.2926.1626.3126.1726.3526.4026.0226.25

4.2. Segmentation of Lance Region Analysis

The multi-scale FCN has been implemented with Python 3.5 in Keras framework and a GTX-1080 GPU is used for training. More specifically, our network is trained by end-to-end pattern, the momentum parameter is set 0.9 and learning rate is set 1 × 10−6. In addition, we use the cross-entropy function as the loss function and a mini-batch of 10 images.

The experimental dataset is comprised of 10000 raceway images, involving both blockage images and normal operation images. Among them, 8000 images are randomly chosen as the training set, and others are selected as the testing set. The data fed into the network contains corresponding ground truths in addition to the original raceway images, where ground truths are labeled manually. In practice, lance regions in images are regarded as targets of the multi-scale FCN, and other region as background. All pixel values in ground truths are marked 1 and 0 respectively. In this experiment, the multi-scale FCN is compared with the FCN-8s, FCN-16s and FCN-32s to verify the comprehensive performance in dealing with lance segmentation task.

In order to evaluate the accuracy of the proposed model, PA (pixel accuracy) and MAE (mean absolute error) are used as metrics. In particular, PA is defined as   

pixel_accuracy= i p ii i c i (5)
and MAE is defined as   
MAE= 1 W×H x=1 W y=1 H | S ¯ (x,y)- G ¯ (x,y) | (6)
Where ci is the number of pixels in class i and pij is the number of pixels in class i predicted to be in class j. S and G represent the prediction map and the ground truth that normalized to [0, 1]. As comparisons, results of detection accuracy are shown in Table 2. It can be observed that the proposed model can accurately detect the lance region in image.

Table 2. Detection results for different network architectures.
MethodPAMAE
FCN-32s98.96%0.119
FCN-16s99.14%0.092
FCN-8s99.30%0.071
Multi-scale FCN99.39%0.057

4.3. PCI Blockage Detection

The region of pulverized coal cloud is obtained by image processing technology. The area of the extracted region is defined as characteristic value and can characterize the flow rate of PCI to some extent, which can be applied to detect PCI blockage.

Videos captured from 30 different raceways are used to evaluate the detection method. The variations of PCI characteristic value in normal injection and blockage are shown in Fig. 13. Each characteristic value is the average of consecutive reading in 15 images. When PCI is in steady injection, the values are basically stable and the degree of deviation is small. Figure 13(b) shows the change of values when the PCI is in blockage condition. It reveals that the characteristic values are initially at stable state. Subsequently, the values drop sharply and produce some fluctuations. Until the lance is completely blocked, no pulverized coal is injected and the characteristic values remain near zero. In accordance with the above detection result, it makes a judgment that the PCI is in blockage condition when there are five consecutive values below the set threshold. Fifty video data with five minutes (twenty of them exist PCI blockage) are used to verify the algorithm proposed in this paper and all judgments are correct. In addition, the method executed by GTX-1080 GPU can meet the requirements of real-time performance.

Fig. 13.

The characteristic curve of coal injection. (a) Normal injection (b) PCI blockage.

5. Conclusion

In this paper, an intelligent method based on image processing is presented for the study on PCI blockage detection. Considering the PCI blockage detection by image processing is not easy because the appearance of raceway changes significantly over time. We analyzed extensive raceway images captured from a 2500 m3 blast furnace to set the parameters in the method to be adaptive. The adaptive preprocessing technique combining image enhancement algorithm and de-noising algorithm is firstly applied to lay the good foundation for subsequent processing. Secondly, the fitting ellipse based on Hough transform locates the tuyere region, which can separate the pulverized coal cloud and inner wall when the flow rate of PCI is relative large. Moreover, an improved binarization algorithm combining Otsu and Bernsen algorithm is used to obtain binarized image. Finally, the multi-scale FCN trained by enough samples is to detect the lance region, and the pulverized coal cloud can be extracted through removing the lance in binarized image. The extracted pulverized coal is able to reflect the variation of flow rate of PCI, which can accurately detect the PCI blockage. In summary, the method proposed is feasible and the detection results will become significant reference in practical control of blast furnace raceway.

References
 
© 2020 by The Iron and Steel Institute of Japan
feedback
Top