2020 Volume 2 Issue 3 Pages 72-79
With the development of information technology, digital pathology, including image analysis, and automatic diagnosis of pathological tissue, has developed remarkably. It has become possible to recognize and quantify histopathological features using artificial intelligence (AI). We have attempted to analyze and quantify various histopathological findings using image processing software. In this report, we introduce the latest results of recognition and quantification of various pathological findings in the liver, kidney, and lung using image processing software, including Image-Pro Plus, Tissue Studio, and HALO. HALO is an image analysis platform specialized for the study of pathological tissues, which enables tissue segmentation by using AI. Using HALO, histopathological changes, which are difficult or impossible to analyze with conventional image analysis, could be easily, and accurately analyzed. Quantification of pathological findings by image analysis can contribute to improve objectivity, precision, and persuasiveness of pathological evaluation. Quantification of morphological changes of histopathological findings using digital pathology including AI in robust pathology is a new and innovative evaluation technique. In the future, a versatile and useful tool is expected to enable faster and more efficient pathological evaluation in both non-clinical and clinical fields.
Pathological examination is powerful in investigating the toxicity and efficacy of compounds, or disease condition by directly observing morphological changes in various tissues in both clinical and nonclinical settings. Evaluation by experienced pathologists and peer review by multiple individuals increases objectivity. However, the evaluations are qualitative or semiquantitative and introducing a bias between individuals and between facilities is unavoidable; moreover, human evaluation is time- and labor-intensive. Many semiquantitative scoring systems are used as an index of morphological changes of pathological tissues [1]. Although these scoring systems are useful and sufficient for pathological evaluation, quantifying morphological changes of pathological tissues has the following advantages. Morphological changes of pathological tissues can be quantified objectively, accurately, and precisely. If these quantitative results are visualized by image enhancement or expressed graphically, it persuades researchers who are not familiar with pathology. Moreover, the reliability is improved by statistically analyzing the quantitative values. If quantitative analysis of pathological evaluations can be automated, it can improve efficiency, save labor, and speed-up evaluation. A quantification tool that can detect morphological changes that are difficult for a pathologist to judge can revolutionize pathological evaluation.
Quantification of histopathological changes by morphometry have been performed for a long time. Recently, with the development of information technology, digital pathology has also developed remarkably. An increasing number of researchers have been attempting to analyze pathological tissue and diagnose the pathology using machine learning or artificial intelligence (AI) [2,3,4,5,6,7,8]. Notably, in the clinical field, some cancer tissues can be recognized by automatic diagnosis software with high accuracy [9,10,11]. In contrast, in the nonclinical field, specific areas of interest are stained using special staining or immunostaining, and the area or number may be simply quantified in many cases. However, it is diffucult to analyze and quantify complex and highly variable tissue morphology using hematoxylin and eosin (HE) stained specimens.
With the widespread use of digital pathology, image processing software that can quantify morphological changes of pathological tissues has enabled analysis. We have analyzed and quantified various histopathological findings using a conventional image processing software such as Image-Pro Plus (IPP) and Tissue Studio to convert quantitative results into indicators of drug efficacy or toxicity [7, 12]. However, these quantitative analyses require high expertise in the setting of conditions and construction of algorithms for image processing software in addition to pathological knowledge. Therefore, it is not easy for everyone to analyze tissue image, and this practice tends to be of low versatility. Recently, we have analyzed and recognized tissue morphological changes using HALO which is an image analysis platform specialized for the study of pathological tissues [8]. By learning the morphological features of pathological tissues using AI, including random forest or neural network, it is possible to recognize complex features of pathological tissues that could not be recognized and analyzed by conventional software. In this review, we introduce some examples of morphological feature recognition and quantification of histopathological findings using image processing software, and discuss their potential and future prospects.
HALO has an image analysis function based on parameter settings, including color and form, of pathological tissues. By adding a functional module specialized for analysis of various morphologies of pathological tissues, these quantitative analyses are possible [8, 13,14,15,16,17]. Furthermore, by learning and recognizing the morphological structures and features of pathological tissues, these regions (tissue class) and structures can be easily separated and quantified. Recent studies have used HALO to easily separate tissue classes and quantify various morphological features in pathological tissue [18,19,20]. Table 1 shows the summary of quantitative analysis of pathological findings that we have attempted using various image processing software. Examples of image analysis, mainly using HALO in the main organs of rodent or humans, are described below.
| Organ | Findings | Measurement parameter | Image-pro premier | Tissue studio | HALO | HALO module |
|---|---|---|---|---|---|---|
| Liver | Degeneration/necrosis hepatocytes | Area of degeneration/necrosis | No | - | Yes | RF Classifier |
| Hepatocellular hypertrophy | Size and number of hepatocyte (simulated) | No | Yes | Yes | (RF Classifier) + Cytonuclear | |
| Hepatocellular vacuolation (lipid) | Size of each vacuole | Yes | No | Yes | Vacuole | |
| Hepatocellular vacuolation (phospholipid) | Size of each vacuole | Yes | No | Yes | Vacuole | |
| Bile duct proliferation | Area of bile duct | No | - | Yes | RF Classifier | |
| Fibrosis (with azan stain) | Area of fibrosis | Yes | Yes | Yes | RF Classifier or Area quantification | |
| Kidney | Basophilic tubules (regeneration) | Area of basoplilic tubule | No | - | Yes | RF Classifier, NN Classifier |
| Hyaline casts | Area of cast | - | - | Yes | RF Classifier, NN Classifier | |
| Degeneration/necrosis tubules | Area of degeneration/necrosis | No | - | Yes | NN Classifier | |
| Glomeruli | Area and number of glomerulus | No | No | Yes | NN Classifier | |
| Fibrosis of glomeluri (with sirius red stain) | Area and numer of fibrosis in each glomerulus | No | No | Yes | NN Classifier | |
| Glomeruli, mesangial cell proliferation | Area and number of mesangial cell proliferative glomerulus | No | - | Yes | NN Classifier | |
| Glomerulosclerosis | Area and number of glomerulosclerosis | No | - | Yes | NN Classifier | |
| Vacuolation of renal tubular epithelium | Area of vacuole | No | No | - | - | |
| Lung | Dilation of alveolar space | HALO: Area of alveolar space IPP: Mean linear intercept (MLI) | Yes | - | Yes | (RF Classifier) + Muscle fiber |
| Fibrosis (with azan stain) | Area of fibrosis | No | - | Yes | RF Classifier | |
| Spleen | Atrophy of marginal zone | Area of white pulp, red pulp, and marginal zone | No | No | Yes | RF Classifier |
| Decrease of extramedullary hematopoiesis | Area of erythroblast (simulated) | No | No | Yes | RF Classifier + Cytonuclear | |
| Thymus | Cortical atrophy | Area of cortex and medulla | No | No | Yes | RF Classifier |
| Adipocyte | Adipocyte hypertrophy | Size and number of adipocyte | Yes | No | Yes | (RF Classifier) + Muscle fiber |
| Pancreas | Atrophy (Hypertrophy) of islets of Langerhans | Area of islets of Langerhans (α, β cell) | No | Yes | Yes | Islet |
| Skeletal muscle | Degeneration/necrosis (with IgG-immunostain) | IgG-positive area of myocyte | No | - | Yes | RF Classifier |
| Parotid gland | Atrophy of acinar cells | Size and number of acinar cell | No | - | Yes | RF Classifier + Vacuole |
| Sublingual gland | Atrophy of acinar cells | Size and number of acinar cell | No | - | Yes | RF Classifier + Vacuole |
| Adrenal grand | Adrenocortical hypertrophy | Size and number of adrenocortical cells (simulated) | No | Yes | Yes | (RF Classifier) + Cytonuclear |
| Intestine | Increase of mucin | Area of mucin-derived vacuole | Yes | - | Yes | RF Classifier or Area quantification |
| Intestinal villi | Length of intestinal villi | Yes | - | Yes | - | |
| Skin | Thickening of (epi) dermis | Thickness of (epi) dermis | Yes | - | Yes | - |
Yes: quntificable, No: not quantificable, -: not examine, RF: random forest, NN: neural network, IPP: Image-Pro Plus.
Pathological changes may occur in the liver as it metabolizes foreign substances absorbed by the body. Although vacuoles derived from lipids or phospholipids can be analyzed by IPP [7, 12], the size of vacuoles is easily quantified using HALO (Fig. 1A–C) [8]. The area of fibrosis (stained with azan stain, Sirius red stain, or Masson’s trichrome stain) was quantifiable using both IPP and HALO. By quantifying changes in the size of lipid droplets and the area of fibrosis, conventional pathological evaluation of the liver in the nonalcoholic steatohepatitis (NASH) model using nonalcoholic fatty liver disease (NAFLD) activity score (NAS) [21, 22] was more precise and objective [12]. In this image analysis, some sinusoidal spaces were recognized as lipid droplets, which is a drawback of the evaluation system. Therefore, automated image analysis should be accompanied by visual examination. Hepatocyte hypertrophy was analyzed using Tissue Studio or HALO [7, 8], which includes the use of an algorithm that simulates the cytoplasm by recognizing hepatocyte nuclei. However, the nucleus does not appear in extremely enlarged hepatocytes, which cannot be simulated and evaluated. Furthermore, hepatocyte degeneration or necrosis such as eosinophilic cytoplasm and atrophied nuclei (Fig. 1D–F) and bile duct area (Fig. 1G–I), which were difficult or impossible to analyze with conventional image processing software, were recognized and the areas were quantified using HALO [8].
Quantification of pathological changes in hematoxylin and eosin (HE) stained liver using HALO. (A–C) Analysis of the hepatocellular vacuole derived from lipid in the liver of mouse fed high fat diet. Bar=50 µm. (A) Original image. (B) Analyzed image. The lipid-derived vacuoles were recognized by the criteria of size and roundness and are represented in red using the vacuole module of HALO. (C) Quantitative results for vacuole area (% of the whole area of the liver section). Grade of fatty change was evaluated by pathologists: very slight, ±; slight, +; moderate, ++. (D–F) Quantification of hepatocellular degeneration or necrosis area in the liver of mouse administered compound A. Bar=1 mm. (D) Original image. (E) Segmented image of degeneration or necrosis (green), red blood cells (red), and other regions (pink) using the tissue classifier module. (F) Quantitative results for degeneration or necrosis area (% of the whole area of the liver section). Grade of degeneration or necrosis was evaluated by pathologists: no change, −; slight, +; moderate, ++. (G–I) Quantification of the bile duct area in the liver of a patient with cirrhosis. Bar=100 µm. (G) Original image. (H) Segmented image of the bile duct (green), infiltrating cells (light blue), and other regions (pink) using the tissue classifier module. (I) Quantitative results for bile duct area (% of the whole area of the liver section). Grade of bile duct proliferation was evaluated by pathologists: no change, −; slight, +; moderate, ++.
The structure of renal pathology is complex; therefore, in many cases, semiquantitative evaluation methods, including scoring the degree of tubular damage [23] and glomerular lesions, are performed and morphometric changes of pathological tissue have never been quantified. Using random forest classifier of HALO, hyaline casts, basophilic tubules (Fig. 2A–C), and degenerated or necrotic tubules (Fig. 2D–F) were recognized, and the areas were quantified [8]. In addition, using the neural network classifier (deep learning) of HALO, morphometric changes in the glomeruli, including mesangial cell proliferation or sclerosis, were recognized and the area of lesion or number of abnormal glomeruli were quantified (Fig. 2G–I). Furthermore, we investigated whether various histopathological findings in animal renal disease model could be recognized and analyzed. If histopathological changes in complex renal tissues can be recognized automatically, easily, and accurately using image processing software, it can revolutionize pathological examination.
Quantification of pathological changes in hematoxylin and eosin (HE) stained kidney using HALO. (A–C) Quantification of the area of basophilic tubules in the kidney of mouse fed high fat and cholesterol diet. Bar=200 µm. (A) Original image. (B) Segmented image of basophilic tubules (blue), hyaline casts (orange), red blood cells (red), and other regions (pink) using the tissue classifier module. (C) Quantitative results for segmented areas (% of the whole area of the kidney section). Grade of tubular basophilic change evaluated by pathologists: no change, −; slight, +. (D–F) Quantification of the area of tubular degeneration or necrosis in the kidney of mouse administered aristolochic acid. Bar=200 µm. (D) Original image. (E) Segmented image of tubular degeneration or necrosis (green), hyaline casts (orange), red blood cells (red), and other regions (pink) using the neural network classifier module. (F) Quantitative results for segmented areas (% of the whole area of the kidney section). Grade of tubular degeneration or necrosis evaluated by pathologists: no change, −; slight, +; moderate, ++; severe, +++. (G–I) Quantification of pathological changes of glomeruli in the kidney of mouse administered anti-glomerular basement membrane antibody. Bar=100 µm (G) Original image. (H) Segmented image of normal glomeruli (yellow), mesangial proliferative glomeruli (green), glomerulosclerosis (blue), basophilic tubules (light blue), hyaline casts (orange), red blood cells (red), and other regions (pink) using the neural network classifier module. (I) Quantitative results for segmented glomeruli (% number of glomeruli in the whole area of the kidney section). Grade of glomeruli of mesangial proliferation and sclerosis evaluated by pathologists: no change, −; slight, +.
The size of alveolar spaces that have changed by tobacco exposure is evaluated using an indirect indicator such as the mean linear interception (MLI) [24], and we have quantified it easily and efficiently using morphological processing and macro tool equipped in IPP (Fig. 3B) [7]. The size of alveolar spaces was directly quantified using HALO (Fig. 3A, 3C, 3D). For the evaluation of pulmonary fibrosis, semiquantitative methods such as the Ashcroft scale are used frequently (Fig. 3F) [25, 26]. Recently, a method based on evaluating pulmonary tissue density has been reported [27, 28]. Pulmonary fibrosis is stained blue or red with azan or Sirius red, respectively; however, tissues other than fibrosis are also stained. Therefore, it is impossible to distinguish and quantify using color settings of the conventional image analysis method. Using HALO, we have quantified the fibrosis area by recognizing morphological features of fibrosis, normal alveolar walls, and foamy macrophages, which are stained in the same color as the fibrosis (Fig. 3E, 3G, 3H).
Quantification of pathological changes in the lung using HALO. (A–D) Quantification of alveolar space in the lung of mouse exposured tabacco. The specimens were stained with hematoxylin and eosin (HE). Bar=200 µm. (A) Original image. (C) Analyzed image. Each alveolar space was recognized by the criterion of size and is represented in red using the muscle fiber module of HALO. Bar=200 µm. (D) Quantitative results for alveolar space area. The average size of each alveolar space was measured in the whole area of the lung section. Grade of dilation of alveolar space was evaluated by pathologists: no change, −; slight, +. (B) Image analysis of mean linear interception (MLI) using Image-Pro Plus (IPP). The lung tissue image was superimposed on the grid lines (horizontal lines). The number of intersections (green points) of the alveolar wall and the grid line was measured automatically using the macro tool. MLI was calculated from the ratio of the total length of the grid lines to the total number of intersections. (E–H) Quantification of fibrosis area in the lung of mouse administered bleomycin. The specimens were stained with azan. Bar=200 µm. (E) Original image. (G) Segmented image of fibrosis (light blue), foamy macrophage (green), alveolar wall (yellow), and red blood cells (orange) using the tissue classifier module. (H) Quantitative results for segmented areas (% of the area of the lung section). Grade of fibrosis was evaluated by pathologists: no change, −; slight, +. (F) Image analysis of the Ashcroft scale using IPP. The degree of fibrosis in multiple microscopic fields was graded from 0 to 8.
Furthermore, if the type of infiltrating inflammatory cells, such as macrophages and mononuclear cells, in the pulmonary lesion can be identified and analyzed in detail, it can be applied to pulmonary pathological evaluation including pneumonia due to chronic obstructive pulmonary disease.
Pathologically, the spleen has red pulp, white pulp, and marginal zone, which is the border region between the red pulp and white pulp. The region of red pulp, white pulp, and marginal zone in the spleen often change as a result of immune stimulation. The morphological features of the red pulp, white pulp, and marginal zone were learned and separated using HALO, and the degree of findings such as atrophy of the white pulp and the marginal zone were quantified (Fig. 4A–C) [8].
Quantification of the red pulp, white pulp, and marginal zone areas in hematoxylin and eosin (HE) stained spleen of mouse administered compound B (B, C) using HALO. Bar=100 µm. (A) Original image. (B) Segmented image of the red pulp (pink), white pulp (blue), and marginal zone (light blue) using the tissue classifier module. (C) Quantitative results for segmented areas. Grade of atrophy of the marginal zone evaluated by pathologists: no change, −; slight, +; moderate, ++.
The size of adipocytes can be quantified using Adobe Photoshop [29] or IPP (Fig. 5B) [7], but it is necessary to connect the contours of the adipocytes that are not connected at several places either manually or by morphological processing, which are labor- and time-intensive. Using HALO, the size of the adipocyte was analyzed and quantified easily (Fig. 5A, 5C, 5D) [8].
Quantification of adipocyte size in hematoxylin and eosin (HE) stained epidydimal adipose tissue of mouse fed high fat diet using HALO. Bar=100 µm. (A) Original image. (C) Analyzed image. Each adipocyte was recognized by the criterion of size and is represented in yellow. (D) Quantitative results for adipocyte size. The average size of adipocytes was measured in the entire section. Grade of hypertrophy of adipocyte was evaluated by pathologists: no change, −; slight, +. (B) Image analysis using Image-Pro Plus (IPP). Several steps of morphological processing were performed on the adipocyte image, and the divided regions were defined as the adipocyte.
As described in Table 1, various morphological changes of pathological tissues can be quantified.
All animal experiments were in accordance with the institutional guidelines and were approved in advance by the Committee of Animal Experiments in the Research Division of Mitsubishi Tanabe Pharma Corporation.
With the development of digital pathology in recent years, we have examined whether digital image analysis of various pathological tissues is possible using several image processing software. By using HALO (random forest classifier and neural network classifier), various morphological features of pathological tissues that could not be recognized using conventional image processing software could be recognized, thereby making it possible to recognize and quantitatively analyze significant morphological changes. It was difficult for the random forest classifier to recognize detailed tissue morphology such as glomeruli. However, it was possible to recognize morphological changes in and around the glomerulus using the neural network algorithm with deep learning algorithm. In addition, complex pathological changes including basophilic changes and degeneration or necrosis of renal tubules was learned and recognized using HALO.
Inflammatory cells, which often appear in various pathological conditions, can be quantified using IPP or HALO if they have a simple size and color like lymphocytes. However, it was difficult to extract when they formed clusters, and could not be distinguished from other types of inflammatory infiltrating cells, such as macrophages or neutrophils. It seems that organizational structure as small as nucleus is difficult to recognize and we expect development of software to enable these functions in the future.
It is possible to recognize and quantify various organizational structures and morphological changes. However, it is not a versatile method and it is necessary for a pathologist to confirm the results of image analysis. To establish suitable image analysis methods, many verifications, and validations are required. In addition, it is necessary to seek the existence and utility value as a tool for final diagnosis and evaluation. On the other hand, it is possible that advanced morphological feature recognition using AI enables detection of morphological changes that are difficult for humans to distinguish. Thus, subtle drug effects and changes in toxicity, which are difficult to analyze via conventional pathological evaluation using microscopy, may be detected.
Histopathological changes are easy to analyze when fibrosis or infiltrating cells can be specifically stained using immunohistochemistry. However, it will be more useful and versatile if the morphological changes of pathological tissue in HE stain, which is the most common in pathological specimens, can be recognized and quantified.
If various histopathological structures and morphological changes were analyzed and quantified by recognizing the features of pathological tissues automatically and easily, it will not only assist evaluation by conventional microscopy, but also significantly revolutionize conventional pathological evaluation. In the future, we expect that many users will verify and utilize image processing software, including HALO, to make quantitative evaluation of pathological tissue a versatile and useful method.
All authors are employees of Mitsubishi Tanabe Pharma Corporation. The authors declare that they have no conflicts of interest.
The authors thank Tetsuhiro Kakimoto, Masaharu Tanaka, Atsushi Fukunari, Hiroyuki Utsumi, Yuri Fujimoto, and Yoko Takada for supporting the present study and helpful discussions.
