2019 Volume 59 Issue 10 Pages 1820-1827
In optical surface inspection of steel products under hot conditions, overdetection caused by signals from harmless surface such as scale texture pattern is usually a problem. Therefore, the authors proposed a new inspection technique called the “twin-illumination and subtraction technique,” which is able to remove only harmless signals, based on the finding that most harmful defects on these products have concave shapes, whereas most harmless features that might be overdetected have flat or convex shapes. In this technique, an image in which the concave and convex parts are emphasized could be acquired from the difference between two images of the same position illuminated from two opposite directions. As an application of this technique, after laboratory tests to confirm its effectiveness, we conducted inline test for hot pipes which have poor surface. Although harmless signals from flat surfaces could be removed as expected, harmless signals from convex surfaces such as peeled scale and micro-surfaces which cause specular reflection conditions remained in addition to harmful signals from concave defects. To improve detectability, we introduced a discrimination method based on the characteristic bright/dark patterns of concave defects, a detection and decision tree judgment function using features from images. Based on the above, we constructed a prototype system which can obtain two images with a slight time difference by using strobes and dual CCD cameras. As a result, we have confirmed that the system has sufficient performance for prevention of massive rejects and is an effective technique for detection with poor surface.
Surface inspection of steel products is very important in terms of quality assurance. Recently, automation of inline surface inspections, especially of sheet products, has increased as a result of many kinds of development.1,2,3) For example, in continuous galvanizing lines and continuous annealing lines, automatic detection of defects has been realized by polarization optics, and surface quality has been assured by shipping products with the detected defect parts marked.
On the other hand, in surface inspections in the field of mill scale steel products like steel pipes, visual inspection by the operator is still the mainstream, and automatic inspection has not reached the stage of practical application at present. Compared with surface inspection of sheet products, the largest problem for practical use is the difficulty of discriminating between harmful defects and harmless overdetected parts caused by poor surface such as scale surface pattern.
In particular, there have been only a few reports on methods of surface inspection of steel pipes in the hot condition, and an eddy current flaw detection method4) and several optical methods have been reported. However, the eddy current flaw detection method requires a small target distance, distance between target and sensor, and durability is a problem in surface inspection in the hot condition, and for these reasons, this technique has not been put into practical use.
On the other hand, none of the reports on the optical methods are practical examples. First, the method of detecting defects by applying a light-section method to the circumferential direction of steel pipes to measure their shape has been proposed.5) However, it is difficult to apply this method to steel pipe rolling lines because changes in the outer diameter of the pipe and fluctuations of the pass line make alignment of the optical system difficult.
In addition, a method6) has been proposed in which light sources and cameras are installed in the circumferential direction of hot rolled steel, reflection images of the illuminated light are acquired using wavelength filters, and surface defects are detected by frequency analysis. However, this is merely defect detection by threshold processing of target images in which the entire surface is covered by scales, and only severe defects can be detected.
Furthermore, a thermal radiation measuring method,7) which detects defects from the difference in luminance between the harmful defect part and harmless part, has also been tried, but as with the abovementioned method, the many scale texture patterns that occur on the surface of a hot steel pipe cause overdetection, making it difficult to apply this method. As a result, this method also has not been put to practical use.
In contrast to the above-mentioned methods, the authors devised the “twin illumination and subtraction method,” which is a completely new optical inspection technology, to solve such problems as overdetection of scale texture patterns, etc.8) In this paper, we explain the principle of this method and report the verification of its validity by a laboratory experiment. Furthermore, a prototype for surface inspection of steel pipes has been installed in a production line and is extremely effective in detecting defects in the hot condition. The results that can be obtained in actual operation of this prototype system are also reported.
Overdetection is an extremely important problem due to the poor surface condition in hot surface inspections. From observation of harmful defects and harmless scale texture patterns, which are an overdetective factor, in the hot condition, the authors focused on the fact that the harmful defects are concave, whereas many harmless scale texture patterns are flat, and proposed a new technology called the “twin illmination and subtraction method,” which discriminates between defects and scale texture patterns. A schematic diagram of this method is shown in Fig. 1.
Twin-illumination and subtraction technique. (Online version in color.)
First, the surface of the object steel pipe is illuminated from the right and left by the light source. As the shape of the harmless scale texture pattern is flat, there is no difference in the shading of the two captured images due to the direction of illumination. On the other hand, because the shape of the harmful defect is concave, the appearance of the shading differs depending on the light source direction. Therefore, when the difference between the two images is calculated by subtraction, it is expected that scale texture patterns without a difference in shading will disappear, and only the signal in the defect part will remain.
An evaluation test was conducted to confirm the effectiveness of the principle described above. The test piece was obtained by cutting out part of an actual pipe with defects which has been generated in a production line. The test piece had a poor surface from scale texture patterns as well as painted parts. The illumination angle was set to 45°, and the test piece was in the cold condition, room temperature the obtained images and the subtraction result are shown in Fig. 2. As shown in the two reflected images in the upper part of the figure, when the test piece is illuminated from either the left or the right side, it is difficult to distinguish the defect portion from the images. The subtraction image obtained from the two images is shown in the lower part of the figure. Signals with a flat surface property were canceled by subtraction processing, and it was possible to detect only the concave and convex parts, especially shadow parts. Thus, this experiment proved that the proposed technique is effective as an overdetection measure.
Result of laboratory experiment in cold condition. (Online version in color.)
In this paper, we apply the proposed technique to a production line and attempt to realize full-circumference, full-length outer surface inspection of steel pipes with poor surface in hot condition. The purpose of the inspection is to detect continuously generated surface defects at an early stage and provide feedback from the inspection results to the line operator in order to prevent massive rejects.
Harmful concave defects which are generated in the product manufacturing process reduce the first-pass yield rate because the conditioning process is necessary to remove the defects, and may also deteriorate product yield, since the product must be rejected if it is not possible to maintain the guaranteed thickness tolerance. Moreover, these surface defects are often generated consecutively unless the cause of occurrence such as a protuberance on the roll is eliminated. Therefore, early detection of defects in the hot condition is very important, because delayed detection of defects will lead to massive rejects.
Based on this background, the purpose of this inspection is to assist the operator by providing information such as images and the positions of defects at an early stage. For this purpose, we set the target values of the detection rate and overdetection rate to the extent that operation by this guidance is established. One example of conveying speed is about 2.0 m/s, and the outer diameter of the target pipe is about 300 mm.
The target defect is defined as an open and concave defect, as shown in Fig. 3(a). One example of defect size is a depth of about 0.5 mm and length of about 10 mm. We decided to exclude the closed defects of the type shown in Fig. 3(b) from the target defects because most are caused by material-related factors attributable to continuous casting process, making it difficult to take action in case of massive rejects.
Profile of defects (Cross-sectional view). (a) Open defect. (b) Closed defect.
The optical system shown in Fig. 4 was studied for application of this method to steel pipes at the production line. First, in the pipe conveyance system, there are mainly two conveyance methods, the horizontal path, in which the pipe is conveyed while rolling in the circumferential direction, and the longitudinal path, in which it is conveyed in the longitudinal direction by rolls. In order to inspect the entire outer surface of a steel pipe with a small number of cameras and light sources, we adopted the approach of installing the inspection system in the longitudinal path and illuminating the pipe surface from the longitudinal direction. In this case, the system has no sensitivity to the inclined surface in the direction perpendicular to the illumination direction, that is, the inclined surface in the circumferential direction. However, since most actual target defects have a slope in the longitudinal direction, there is no problem with illumination only in the longitudinal direction. The illumination angle of the light source was set to 45° in order to ensure long target distance between light source and hot pipe in consideration of removal of radiant heat effect from hot pipe.
Configuration for pipe inspection. (Online version in color.)
Defect detection performance was evaluated under these optical system conditions by using an artificial defect processed on the surface of a steel pipe test piece. The defect shape is shown in Fig. 5(a), and the position of the defect tested is shown in Fig. 5(b). Therefore, artificial defects inclined in the illumination direction compared to the non-defective part were processed on the steel pipe test piece by changing the inclination angle θ because the intensity of the reflected light depends on the inclination of the surface in the longitudinal direction. The defect positions on the steel pipe were changed as the angle φ in the circumferential direction from the front surface of the test piece. Since the surface property of the defect-processed portion has high secularity, and thus differs from the non-defective part, the surface property of the defective part was made to approximate the non-defective part by performing heat treatment to form scale.
Detectability of defects by changing depth and position. (a) Shape of defect. (b) Position of defect. (c) Subtraction images of defects. (Online version in color.)
As a result, the obtained subtraction images are shown in Fig. 5(c). The defect signal becomes stronger as the inclination of the defect increases and as the position of the defect becomes closer to the front of the optical axis of the camera. Furthermore, it was possible to detect concave defects under the condition that a defect with an inclination angle θ of 10° or more had an SN ratio of 3.0 or more, even at the position where φ differed by 60° from the optical axis of the camera.
For application of the proposed technique to a production line, it is necessary to capture two images illuminated from one side and the other side, respectively, without pixel-by-pixel misalignment. However, because of the distortion due to lens aberration and variation in the amount of movement between the two images, image registration in pixel units is practically difficult to carry out if two images are acquired by simply switching the light source and performing position alignment afterward by image processing such as parallel movement. In order to solve this problem, we proposed a new imaging method.
In this method, imaging by two optical systems is performed with virtually no delay by first taking one image at the timing of illumination by one light source such as a strobe light source with a very short illumination time, and then continuously taking an image at the timing of illumination by a second strobe light source immediately after the first illumination and imaging are completed. An example of a specific timing chart is shown in Fig. 6.
Timing chart of time-separation method. (Online version in color.)
There are two methods for taking two images with a slight time difference. One method uses a high-speed camera, and the other uses two imaging devices with a prism or beam splitter in order to obtain two images on the same optical axis. In the method described here, we use dual CCD camera, in which two CCD sensors are precisely aligned and assembled into one housing in advance. the imaging of each CCD sensor is controlled separately, for easy alignment of the optical system.
When using this imaging method, for example, assuming that the illumination time of the strobe light source and the exposure time of camera imaging are both 100 μsec, the camera resolution is 1.0 mm/pixel, and the target conveying speed is 1.0 m/sec, and it is possible to take two images with only a 0.1 pixel gap and acquire two illumination images from two directions in a substantially aligned state. Therefore, the twin illumination and subtraction technique can be applied to a moving target in a production line.
We set up a prototype system which inspects the entire surface of the steel pipes in the production line in order to evaluate the detectability in the hot condition. The optical arrangement and the system configuration are shown in Fig. 7. In order to inspect the entire circumference of the outer surface of the steel pipe, the circumferential direction was divided into three areas, and each region was illuminated from upstream and downstream to obtain the subtraction image. In addition, full length inspection was also realized by sequentially applying twin illumination and subtraction processing of three sets each time the target steel pipes are conveyed the same distance as the inspection range. Each strobe light source is controlled by a trigger signal which is designed so that the respective light illuminations do not interfere with each other, and two images are taken by each CCD sensor in accordance with each light illumination. In the prototype, the above imaging method was realized by positioning and alignment using dual CCD cameras. Using strobe light sources also has the effect of eliminating motion blur due to flapping and vibration during conveyance.
Configuration of prototype system. (Online version in color.)
Since this inspection is conducted in the hot condition, at about 800°C, we use infrared cut-off filter in order to cut off the thermal radiation.
The images obtained by the optical system are processed by using the image processing algorithm for defect detection described in the next chapter by PCs, which are connected one by one to the dual CCD cameras for imaging in the three directions.
An example of images obtained by the prototype is shown in Fig. 8. The optical condition differs depending on the positions due to the differences in the incident angle and reflection angle, so that different light quantity unevenness occurs under left and right illumination. For this reason, it is anticipated that the light quantity unevenness cannot be removed simply by subtraction processing. Therefore, we developed an image processing algorithm that can eliminate the above-mentioned light quantity unevenness and detect defects with high accuracy. The processing flow is shown in Fig. 9.
Raw images from prototype system. (a) Illuminated from left side. (b) Illuminated from right side. (Online version in color.)
Flow of image processing algorithm.
First, the boundary between the steel pipe and the background is calculated by edge detection, and the inspection area is determined. Thereafter, shading correction for removing light quantity unevenness is performed on each of the two images. Generally, since unevenness of light intensity has a low frequency component with respect to harmful signals from defects, a high-pass filter is frequently adopted. Examples of images showing the processing results and a subtraction image are shown in Fig. 10. It can be confirmed that light quantity unevenness is removed in the corrected images. As can be seen from Fig. 10, in the subtraction image, the signals due to scale texture pattern are clearly removed.
Result of shading correction and subtraction. (a) Raw images of target pipe. (b) Shading corrected images of target pipe. (c) Subtraction image of target pipe.
Furthermore, harmful signals from defects are extracted from the subtraction image by threshold processing. An example of images before and after subtraction processing obtained by the prototype system is shown in Fig. 11. Therefore, the signals of the scale texture pattern which cause overdetection are canceled by subtraction processing of the image of the noisy surface, and it is possible to detect only harmful signals from concave and convex defects by threshold value processing.
Effects of subtraction processing. (a) Image illuminated from left side. (b) Image illuminated from left side. (c) Subtraction image. (Online version in color.)
The purpose of subtraction processing is to cancel the signals from flat surfaces and to detect only the signals from concave defects. However, while proceeding with the inline test, there were examples in which the signals from harmless peeled scale with a convex shape could not be removed by subtraction processing because their appearances differed when illuminated from two directions.
Figure 12 shows three kinds of subtraction images, together with and schematic illustrations of their features. Figure 12(a) shows a harmful concave defect, Fig. 12(b) shows a harmless convex shape such as peeled scales, and Fig. 12(c) shows peeled micro scale in which one slope has a specular reflection condition. The results of observation revealed that the images of the defects with concave shapes displayed a dark area on the illuminated side, while the images of convex shapes such as those in peeled scale displayed a bright area on the illuminated side, so that the patterns of bright and dark were reversed depending on the shape. Among these images, in the case where minuscule peeled scales reflect the illumination from one side in a specular condition, the reflection signal becomes very strong in comparison with the background. These signals are not canceled in the subtraction image and become a small and strong light or dark signal.
Subtraction images obtained from prototype. (a) Harmful defect. (b) Peeled scale. (c) Peeled micro scales under specular reflection conditions. (Online version in color.)
These signals from harmless surface cause overdetection, therefore, we devised an original logic to detect only defects with concave shapes by detecting the bright and dark pattern by using threshold processing of each type of pattern and calculating their positional relationship. Namely, signals from a convex shape such as peeled scale are discriminated by the characteristic that the position of the light and dark areas is opposite to that in the signals from a concave shape, or that they are strong bright or dark signals. In this logic, if the right side illumination image is subtracted from the left side illumination image, harmful defects are detected by performing image processing of the positional relationship pattern in which the left side is dark and the right side is bright. Thus, overdetection can be dramatically reduced compared with the case in which the bright portion or the dark portion is simply detected by a threshold value.
By using the proposed algorithm described in the previous section, it was possible to detect only the concave defects. When this algorithm was applied to images actually obtained by the prototype, almost all the signals caused by convex signals and the microscopic surface were removed, but overdetected signals having a bright and dark pattern, as shown in Fig. 13, were also detected. Most of these overdetection signals occurred when the surface was poor and signals such as peeled scales or the like were accidentally adjacent to each other, and thus could be attributed to a positional relationship between the bright and dark portions similar to that of defect signals. When comparing the defect signal shown in Fig. 13(a) with the overdetection signal shown in Fig. 13(b), each has its own characteristics and can be discriminated visually to some extent. Therefore, by applying machine learning9) of binary decision tree using quantified features commonly used in ordinary surface inspection, such as the area, width, and length calculated from images, it is possible to discriminate between harmful signals from defects and overdetection signals.
Bright and dark patterns in subtraction images. (a) Defects images. (b) Harmless surfaces images. (Online version in color.)
A reliability test was conducted to evaluate the detection performance when using the installed prototype and image processing logic. Figure 14 shows the transition of the condition of overdetection when applying a simple light projection method, subtraction processing, bright and dark pattern recognition, and feature discrimination, under the condition that detection is possible to the extent that continuous harmful defects can be suppressed. Since the steel pipe’s surface in hot condition is very poor, in the existing simple light projection method, over 1000 detection signals were generated per steel pipe, and surface inspection was not established. By contrast, a remarkable reduction to one overdetection signal per 10 steel pipes became possible by applying the proposed method.
Transition of number of over-detections per pipe in each processing step. (Online version in color.)
In order to confirm the detection performance described above in preventing the occurrence of massive rejects, defect information was presented to the operator quickly. When defects are detected, the operator is notified by an on-screen display and an alarm is sounded, and the operator can grasp the situation of defect occurrence. If the proportion of defective steel pipes exceeds a certain level, there is some fear that massive rejects will occur. In that case, the operator tries to minimize the number of defects by stopping the line temporarily to identify and eliminate the cause, such as a convex shape on the roll or problem in the rolling conditions.
As a result of the experimental introduction of this prototype system and guidance to operators, it has been possible to suppress continuous open defects on the external surface over a long period of time. Thus, this study verified the fact that surface inspection by the twin illumination and subtraction technique is effective for prevention of massive rejects.
In this research, we proposed the twin illumination and subtraction technique, which is a completely new optical inspection method, and realized surface inspection in which overdetection signals such as peeled scale and scale texture pattern were removed, which had been difficult with conventional methods. As an example of application to a production line, a prototype system was installed at a steel pipe production line to detect concave defects in the hot condition. We also developed an image processing algorithm for automatically detecting defects from the raw images and obtained good results. Furthermore, it is possible to prevent massive rejects caused by open defects on the external surface by guidance to operators and elimination the cause of the defect. Thus, the results of the prototype test showed that the surface inspection system using the twin illumination and subtraction technique had excellent effectiveness and practicality.
The twin illumination and subtraction technique can also be widely applied to the surface inspection of steel materials other than hot steel pipes, which was the application example described in this paper. As further developments in the future, there is currently a great need for automatic inspection of other types of steels with poor surface such as hot steel bars and slabs, as well as hot steel pipes. There is a high possibility that our proposed framework can also be applied to these objects. Moreover, in order to make effective use of this technique, analysis of physical reflection characteristics in order to establish the optimal optical design method for the new inspection targets is a subject for future work.