2021 Volume 62 Issue 6 Pages 719-725
The classification of the microstructures of Al–Si–Mg casting alloy in different cooling rates was accomplished by using our originally developed methods and machine learning techniques. The mechanical properties of the samples were slightly increased with the increase of cooling rates. The microstructures of the samples were similar because of the approximate cooling rates. High classification rates of about 80% to 90% were obtained using the software with machine learning techniques developed by us. The classification rate change with the number of images in training data was tested and a suitable number of images for training in the machine learning process was found.
Fig. 4 The flow diagram of classification process.
The microstructures of metallic material have a crucial impact on their properties. Therefore, the analysis of microstructure is one of the most important objects in the analysis of metallic materials.1,2) The classification of microstructure is traditionally accomplished manually by specialists, but the subjectivity of human being will give rise to uncertainties. With the development of computer technology in recent years, various intelligent image analysis methods have emerged, providing new methods for microstructure analysis. As a result, there have been many attempts at classification and characterization of the microstructures by using machine learning techniques and related technology,3–12) for the purpose of improving the accuracy and efficiency of microstructure analysis compared to the traditional methods.
Jessica Gola et al.8) used data mining methods to demonstrate a method of determining various steel structures in two-phase steels though the evaluation of their morphological parameters. In the work of Seyed M. Azimi et al.6) they proposed a Deep Learning method to do the microstructural classification in some examples of microstructural composition of low carbon steel, and their system achieved a very high classification accuracy of more than 90%. Dmitry S. Bulgarevich et al.7) reported a novel and efficient pattern recognition method for light microscope images of steel. This method can reliably and automatically segment typical steel microstructures based on machine-learned random forest statistical algorithms. This method helps you process large amounts of image data in a short amount of time for quality control and find new steels with the required properties.
There have been many excellent achievements in the research of identifying and classifying the microstructure of steel. But the related research on aluminum alloy, which is a structural material as important as steel, is not much. The morphology and distribution of second phase particles play an important role in the mechanical properties of casting aluminum alloys.
In the previous work, we have defined a 2-dimensional local number, short as LN2D,13,14) and proposed a quantitative method that uses the relative-frequency distribution of LN2D to evaluate the spatial distribution of particles. This method has already been used to evaluate the spatial distributions of all SiC particles and the delaminated SiC particles in the Al–SiC composites, for the purpose of investigating whether the volume fraction of second-phase particles will affect the distribution of damaged particles, and further caused the influence to the tensile deformation behavior.15,16)
And we also proposed a statistical method to evaluate the mean free path of dislocation motion by the use of image processing, which is called image mean free path (IMFP), and investigated the relation of the measured mean free path to mechanical properties of Al–Si casting alloys.17)
In this work, we attempted to classify the microstructures of aluminum–silicon alloys at different cooling rates by using these originally developed methods and machine learning techniques, hope to help improve the accuracy and efficiency of microstructural analysis for quality control and for the design of new aluminum alloys with desirable properties.
The flow chart of the experimental process using machine learning techniques is illustrated in Fig. 1. The material used in this experiment was Al–7%Si–0.3%Mg (mass%) alloy, casting by the mixture of pure aluminum, Al–24.3%Si (mass%) and Al–10%Mg (mass%) ingots. Three samples solidified in different cooling rates were casting in a copper mold. After the preparation and polishing of samples, many images of samples microstructures will be taken by optical microscopy. Before the later steps, the image binarization must be accomplished first. In the next stage, the features representing the morphology and distribution of second phase particles could be extracted from the binary images. Finally, the implement of classification process can be carried out by using the classifiers of support vector machine (SVM) and random forest, based on the features extracted.
The flow chart of experimental process in this work.
A copper mold that can be heated up was used to make the samples solidify in different cooling rates. Three sample ingots were cast in the copper mold when it was not heated (room temperature), heated up to 80°C and heated up to 140°C, respectively. The sample 1 was solidified in the mold heated up to 140°C, the sample 2 was solidified in the mold heated up to 80°C, and the sample 3 was solidified in the mold when it was not heated. A HIOKI machine of model LR8431 with a K-type thermocouple was used to record the temperature change with time during the casting process, so that the cooling curves, by which cooling rates can be calculated, can be drawn.
2.3 Microstructural characterizationFor the characterization of microstructures of three samples, the samples were cut though the cross-section from the long strip sample ingots. The specimens were ground with 400 to 2000 grit SiC abrasive paper, and then polished carefully by using 3 and 1 µm diamond suspension into a smooth mirror surface, in order to prevent the influences of scratches in later process. Hundreds of microstructure images of each sample will be shot by 400× optical microscopy in the central area of observation surface, preparing for the later treatment. The resolution of each image was 1280 × 960 pixel.
2.4 Image binarizationThe process of image binarization including by brightness adjustment, filter processing, threshold processing, and noise reducing, that is shown in Fig. 2. The purpose of brightness adjustment and filter processing is to adjust the contrast of the images and ensure that each image has the same contrast. Threshold processing could invert the adjusted images into binary images, while noise reducing will eliminate noises and further reduce the impact of dirt or minor scratches.
Process of the image binarization.
In the next step, a broad variety of features describing the morphological structure and the distribution of the second phase particles were measured on the binary images using the software developed by us. The description of features used in this work were shown in Table 1. The gravity center, area, perimeter, and circumscribed rectangle of each second phase particle will be detected by the program, and the image mean free path (IMFP) and 2-dimensional local number (LN2D) will be measured.
Besides, there will be 2 output data types of each feature, histogram data and statistical data. The histogram data is relative-frequency distribution of the data variable, and the statistical data is average value and standard deviation.
2.5.1 Definition of the image mean free pathFigure 3 is a binary image of aluminum alloy, the white part is the aluminum matrix while the black part is second phase particles. Image mean free path (IMFP) could be measured by following procedures.17) (1) A starting point A in matrix and a traveling direction are selected randomly. (2) A length, a and b, between the starting point and encountered second phase is measured. Measurement is excluded when some edge of the image is found. Procedure (1) and (2) were repeated at least one million times, and the image mean free path was calculated by averaging a (IMFP Single) or a + b (IMFP Double). IMFP relates to the mean free path of dislocation motion. It is reported that mechanical properties such as ultimate tensile stress and proof stress increased with decreasing IMFP.
Schematic diagram shows the measurement of image mean free path.
The 2-dimensional local number (LN2D) is defined as the number of the center of gravity (GCs) of the second phase particle in the circle centered on the GC of the specific particle on the tangent plane.13) Determine the radius of the measurement circle so that the number density in the GC circle containing seven particles corresponds to the number density of all particles in two dimensions. The measured radius R2D is expressed as
| \begin{equation} \frac{7}{\pi R_{2D}^{2}} = \lambda_{A}\to \left(\frac{7}{\pi\lambda_{A}}\right)^{1/2}{} = \frac{1{,}493}{\lambda_{A}^{1/2}}, \end{equation} | (1) |
When the GC is randomly distributed and the number density λA is maintained, the same measuring sphere with a radius of R2D will be used for measurement. Since the center of the measurement window (circle) is located on the GC of the two-dimensional particle, each particle has a specific LN2D value, which is considered to be similar to the particle size or shape factor. Then, we think we can evaluate the spatial distribution by observing the frequency distribution of LN2D and using the size distribution to characterize the overall particle characteristics.
The measurement of LN2D is carried out in two steps. First, determine the GC of all second phase particles on the image and determine the total density λA. Calculate the measuring radius R2D according to the formula. (1). Second, define a measurement circle, whose center is located at GC of a noticed particle (LN2D) or at a random position (LN2DR), with a radius of R2D for all GCs, and calculate the GC that exists in each measurement circle.
2.6 ClassificationThe process of classification is shown in Fig. 4. The images will be randomly taken out and divided into 2 parts, one part for training, and the other part for test, which means that the images belonging to training data were only used for machine learning, and the images belonging to test data were only used for classification. After the preprocessing, the data of extracted features will be used to train the model and a classifier will be used for the classification. Two kinds of classifiers, SVM and random forest, were used in this work.18–21)
The flow diagram of classification process.
A support-vector model is a learning machine for two-group classification problems. The input vector is non-linearly mapped into a very high dimensional feature space to form a linear decision plane. Due to the special properties of the deterministic surface, the learning machine has a high generalization ability.19)
Random forest works by building a large number of decision trees during training and outputting the class as a class pattern for each tree. The essence of this method is to build multiple trees in a randomly selected subspace of the feature space. Trees of different subspaces can summarize their classifications in a complementary way and monotonically improve the classification of their combinations.20,21)
Our system is suitable for two-class problems, which is to compare and classify two different structures at one time. Both the training data and the test will contain microstructural images of two different samples (have the same amount), and in the training data, the different images will be marked by label 0 or label 1, depending on whether it belongs to the same sample. After the machine learning process, the test data of mixed images of two samples will be marked label 0 or 1 by the classifier model being trained. If one image marked as label 0 in the result belongs to the same sample as the image with the same label in the training data, it is regarded as a correct recognition, otherwise, it is an error. The classification rate will be the percentage of the total correct recognition number in all images of test data.
2.7 Mechanical propertiesThe tensile tests were carried out by a Shimadzu tensile testing machine with a load of 2500 kgf at a stretching speed of 0.5 mm/min. Vickers hardness tests were performed on the central area of cross-section of the casting sample ingots using a load of 3 kgf holding for 10 s.
The measured cooling curves of three samples were shown in the Fig. 5. The cooling rate can be defined by dT/dt computed from the approximately straight-line portion during the later stages of primary dendrite growth. The calculated cooling rates of three samples were 5.7 K/s, 8.0 K/s and 9.5 K/s, respectively. The sample 1 gained the lowest cooling rate, while the sample 3 obtained the highest cooling rate. But the gaps between the samples were not large.
Measured cooling curves of three samples.
The microstructures of samples taken by optical microscopy was shown in Fig. 6, which were classic microstructures of Al–Si–Mg alloys. Microstructure analysis with dendrite arm spacing (DAS) is traditionally used in the Al–Si alloys. Rui CHEN et al.22) found that secondary dendritic arm spacing (SDAS) of Al–Si–Mg alloy is very sensitive to cooling rate, and the size of SDAS decreases with the increase of cooling rate. Besides, the decrement of the size of second phase particles and DAS of Al–Fe–Si alloy with increasing cooling rate was reported by B. Dutta et al.23) But in our work, it could be noticed that because of the approximate cooling rates, the microstructures of three samples were similar, which resulted in the classification of the microstructures was difficult by traditional methods.
Images of microstructures of sample 1 (a), sample 2 (b), and sample 3 (c).
We did the principal component analysis (PCA), trying to find out the difference of microstructures between the samples. The result was shown in Fig. 7, from where we can find that the most effective components were LN2DRVar (Variance of LN2DR), Long SideSTD (standard deviation of Long Side), and Narrow SideSTD (standard deviation of Narrow Side), respectively. The average of Long Side, Narrow Side (Long SideAv and Narrow SideAv) and variance of LN2DR (LN2DRVar), standard deviation of Long Side and Narrow Side (Long SideSTD and Narrow SideSTD), for three samples were tested, and the results were shown in Table 2.
The principal component analysis (PCA) of the samples, using 40 images for each sample.
The difference of values of Long SideAv and Narrow SideAv are almost negligible, demonstrating the similarity of the particle size between the samples. And it can be noticed that the difference of LN2DRVar, Long SideSTD and Narrow SideSTD are much bigger, and have the increasing trends with the increase of cooling rates. As we know that the cooling rate will affect the nucleation and growth of eutectic phase particles. A higher cooling rate will accelerate the nucleation of aluminum and the eutectic phase, thereby making the eutectic phase particles in the eutectic region finer and dense. A shorter solidification time may result in a decrease in the uniformity of the microstructure. Although a slower cooling rate will result in a coarser crystal, a longer solidification time will increase the uniformity of the eutectic region. These may explain that the average rate of Long Side and Narrow Side of the samples had very small differences, but standard deviation of which were obvious and increased with the cooling rates.
The mechanical properties of samples were also proved the slight difference between three samples, which was shown in Fig. 8. Both the ultimate tensile strength and the average hardness of the samples were increased with the increase of cooling rates. The reason why the change in mechanical properties was so slight should be the approximate cooling rates of samples.
Mechanical properties of three samples.
Such a small difference is difficult to detect through traditional manual methods, and through computer image recognition and machine learning methods, it can be classified just using microscopic pictures. The number of images used for the training data was 80 (40 images of each sample), and the number used for the test data was 20 (10 images of each sample). And 4 groups of test data were set, while the result was the average rate of 4 tests to reduce the interference of special factors.
We tested 2 sets of data for contrast, sample 1 versus 2 and sample 1 versus 3, respectively. The difference between sample 1 and 3 was bigger than which between sample 1 and 2. The result of classification was shown in Table 3. It can be noticed that the classification rate reached a high level of 80∼90%. When using the histogram data, the classification rate of Random Forest was higher, while using the statistical data, the accuracy of SVM was higher.
Besides, the classification rate of sample 1 versus 3 was always higher than sample 1 versus 2. The bigger difference of the cooling rates will cause the bigger difference of microstructures. As what we mentioned above, the standard deviation of some features was increased with the cooling rates. Therefore, appearance of the higher classification rates should be predictable.
And in this work, we want to figure out whether the number of micrographs for training will affect the result. Increasing the number of sample images can make the feature values more consistent with the features of the entire sample, but it will also increase processing time and reduce efficiency. Therefore, we hope to find the number of images most suitable for training data. And for this reason, we did the classification rate change with number of images in training data, the number was increased from 20 to 140.
The results of classification rate change with number of images using in training data were shown in Fig. 9. Figure 9(a) and (b) were the results of using histogram data. The classification rate of Random Forest was increased when the number of training increased to 60, and then dropped slightly. Figure 9(c) and (d) were the results of using statistical data. The classification rate was also increased when the number of training data raised to 60 and 80, furthermore, the classification rate of SVM had an overall increment of about 5∼10%, got the highest rate. Based on these, it can be inferred that 60 or 80 should be a suitable number for training. When using the statistical data and SVM as the classifier, the highest accuracy in classification could be obtained in this work.
The classification rates change with number of images used for training data.
The Al–7%Si–0.3%Mg alloys were solidified in a copper mold with three different cooling rates. The microstructures of the samples were similar because of the approximate cooling rates. Originally developed methods were used to detect the difference between the microstructures of the samples. The mechanical properties of three samples were slightly increased with the increase of cooling rates.
The classification of the microstructures was accomplished by using machine learning technique, and high classification rates of about 80% to 90% were obtained. The classification rate was highest when using the statistical data and using SVM as classifier. Besides, the classification rate was higher when the difference of cooling rates of samples was bigger. A suitable number of images for training in the machine learning process, in order to get the highest classification rate in shortest time, was discovered.
