Ancestry estimation using image analysis of orbital shapes from Thai and Japanese skulls

Abstract

The orbital shape is one of the most ambiguous features for morphological study in ancestry estimation. Recent research has used digital image analysis to obtain more objective and better results. The aim of this study was to create a method and formula to determine ancestry using image analysis of orbital shapes from skulls of Thai and Japanese individuals. This pilot study applied two-dimensional digital image processing techniques to analyze 440 skulls comprising 220 modern Thai and 220 modern Japanese samples of known identities. The image analysis involved four steps: pre-processing, segmentation, feature extraction, and classification. Five shape measurements of both the left and right orbital rims were selected and a formula was derived using multivariate discriminant analysis. Another set of 68 Thai and Japanese skulls was used as a blind test set for accuracy. The formula had a predicted and cross-validated accuracy of 80.7% and a tested accuracy of 86.8%. This methodology potentially increases the utility of orbital shapes for ancestry estimation, especially between these two subgroups of Asians.

Introduction

Forensic ancestral research from all over the world has generally focused on three major population groups: European, African, and Asian. Very few researchers (e.g. Tallman, 2016) have studied human variation within the same Asian group. Moreover, only Native American and East Asian (i.e. Chinese and Japanese) are typically chosen to be representatives of the Asian group in these studies (Lahr, 1996; Gill, 1998; Hefner, 2009; Xing et al., 2013).

Forensic anthropology in Thailand has been widely known for around 20 years. It began with the recovery of murder victims’ skulls from famous cases in Thailand during 1998–99. It gained prominence with the biological identification of the victims of the tsunami in Southern Thailand in 2004. However, forensic and bioanthropological research examining data from Thai samples have mostly focused on sex, age, and stature estimations, and have only very rarely assessed ancestry (Uthayanung, 1951; Sangvichien, 1971). At present, the number of visitors in Thailand from other Asian countries, including Japan, has increased. There are Japanese villages located in some provinces in Thailand, e.g. Chiang Mai (Northern region) and Chon Buri (Central region). The number of elderly Japanese people moving from Japan to stay long term in these villages has been increasing every year. In addition, there are remains of Japanese soldiers from World War II abandoned in some provinces, still awaiting recovery and identification. As the present study focuses on ‘Asian populations,’ two sample groups of Mongoloid skulls were examined: modern Thai (representing ‘Southeast Asian’) and Japanese (representing ‘East Asian’). It is interesting to explore whether population affinities of these two Asian groups would reflect any differences. The present study focused on the outer rim of the orbital aperture. This is a distinguishing feature of the facial bones and is usually found in good condition in forensic casework.

Orbital shapes or forms or openings have been used as one of the categories of craniofacial trait variations for many decades (Krogman, 1962, 1973; Olivier, 1969; Brues, 1977; Gill, 1986; Rhine, 1990). They are included in the ancestry estimation section of many textbooks and laboratory manuals (Pickering and Bachman, 1997; Gill, 1998; Blumenfeld, 2000; Byers, 2011). Although regarded as one of the most ambiguous traits for morphological study for ancestry estimation, the characteristics of the orbital have nevertheless played a role in reflecting population affinity and assessing ancestry (Masters, 2008). Several studies of orbital shape have evolved further by applying digital image analysis methods to discriminate the shape of the orbital rim among different population groups (Gore et al., 2011; Urbanová, 2011; Xing et al., 2013; Rubin and DeLeon, 2017). This study aims to develop the use of the orbital shapes for anthropological examination and forensic casework in Thailand.

For ancestry estimation in Thailand, the cranial morphology method has normally been used for real casework. The lack of digitizing equipment and high-tech software for biological and forensic anthropology research has been problematic, and this is also likely to be the case in many other countries. This is why the present study chose a practical method which would be beneficial to the user undertaking real casework. Digital image processing techniques provide more refined numeric results of orbital measurements compared with using basic measuring instruments. A previous study (Xing et al., 2013) identified different orbital shapes among the three major population groups. It stated that the Asian orbital shape is clearly taller and has a rounder contour compared to that of Europeans and Africans. From this it might be assumed that Thai and Japanese orbital shapes tend to be not much different from each other. However, applying digital image processing techniques might lead to the possibility of detecting differences between these two groups.

Digital image processing is a useful method that can be applied to skull morphological feature measurements. It uses computer software to acquire and analyze numerical data from a two-dimensional (2D) photograph (Auephanwiriyakul, 2012). The data output from this process yields more details and better results because the software has been developed to work in a way similar to the human eye. Nowadays, digital image processing is increasingly applied in areas of scientific study, especially biomedical research (Auephanwiriyakul et al., 2011). Beyond the software’s ability to function like the human eye, it can generate, transfer, and compute the shape of orbits as numerical data (Gonzalez and Woods, 2002). This enables the study of the orbital shape to be less subjective and more consistent than measuring by hand.

The objective of this study was to create a method and derive a formula to determine ancestry using image analysis of the orbital shapes of skulls of Thai and Japanese individuals. The results of this study could contribute to ancestry estimation in both physical anthropological study and forensic casework in Thailand.

Materials and Methods

This study assessed 2D photographs of 440 complete skulls for the training set, and 68 complete skulls for the test set (Table 1). The training set comprised 220 modern Thai and 220 modern Japanese skulls of known identities to derive a formula. The test set comprised 34 modern Thai and 34 modern Japanese skulls of known identities to test the accuracy of the formula. The photographs of Thai skulls were from the human bone collection of the Forensic Osteology Research Center (FORC), Faculty of Medicine, Chiang Mai University. These were collected from cadavers donated to the Chiang Mai Medical School between 2003 and 2017. For the Japanese skulls, the photographs were taken from samples curated at Kyoto University Museum, Kyoto, Japan. There were collected from hospitals in Kyoto between 1901 and 1921. All samples used for this study were documented, recording sex, age, and ancestry. Any skulls with deformities from pathology, trauma, cranial surgery, tooth loss, or damage to the orbital region were excluded.

Table 1 Age and sex composition of the skull samples

Ancestry group	Age (years, average and range)	Male	Female	Total
Training set				440
Thai	60.2 (26–89)	110	110	220
Japanese	37.5 (18–66)	115	105	220
Test set				68
Thai	67.1 (46–88)	17	17	34
Japanese	33.3 (17–55)	17	17	34

Four steps of image analysis (Gonzalez and Woods, 2002) were applied to the training set in order to derive the formula in this study.

1. Pre-processing

The first step of this method was to photograph the objects and prepare these 2D images to be suitable for segmentation. The photographs of the skull samples were taken with an Android smartphone camera in automatic mode with a focal length of 27 mm and an f-stop of 2.2. Every skull sample was photographed in anterior view with a centimeter scale and a black fabric background. Skulls were placed in the standard position, with the Frankfort horizontal plane parallel to the floor.

The camera was mounted on a tripod 30 cm away from the midpoint of the interorbital distance at the level of the superior orbital rim of the skull samples, perpendicular to the floor. The camera’s viewfinder was set at about the same height as the orbital region. The distance between the anterior part of the skull and the camera was the same for every skull. After the photograph was taken, noise and unwanted objects were removed so that segmentation would give optimum results. The photograph of a skull with a centimeter scale was then calibrated using MS Paint to determine the number of pixels per centimeter. The image was then imported into the image analysis software to begin the process of segmentation and feature extraction.

2. Segmentation

This step separates the region of interest (ROI), both the left and right orbital rims, from the other parts of the skull. The program, which was developed from MATLAB, works as a semi-automated application as both left and right orbital rims were digitized manually by the user. The program then automatically proceeds to segmentation and feature extraction.

To manually select the sample region for left and right orbits, the researcher digitized points at the most outer or anterior margin including the lacrimal bone and supraorbital notch, if present, to ensure that the complete rim of each orbit was selected. This was shown by blue dots and lines along both left and right orbital rims in the image software (Figure 1a). This step automatically generated the numerical data of the major and minor axes in the MS Excel spreadsheet.

Figure 1

Definition of the measurements used in this study. (a) The orbital rim was manually digitized (blue spots) using a semi-automated program. The solid and dashed line show the major and minor axes, respectively. (b) Three points determine the angle degree of the orbit: (1) a point on the curved line of the superolateral corner of the orbital rim, which locates the end of the superior rim before curving downward to form the lateral rim of orbit; (2) a point on the most superior margin of the supraorbital rim; and (3) a point at the lateral side of that orbit wherever the horizontal line passes through the second one.

When using the Angle Degree Measurement function, the program automatically provided the movable horizontal line on the screen for selecting the points. To measure the angle of the orbit, three points were selected. The first point was on the curved line of the superolateral side of the orbital rim, located at the end of the superior rim before curving downward to be the lateral rim. The second point was on the most superior margin of the supraorbital rim, excluding the supraorbital notch, if present. The third point was any point on the lateral side of that orbit where the horizontal line in the image analysis software passes through the second point previously selected (Figure 1b). This step automatically calculates the angle of the orbit, in degrees, which appears in the MS Excel spreadsheet.

3. Feature Extraction

In this process, the program automatically extracts features of the orbital shapes, and generates the numerical data of parameters for the orbital shape analysis. The parameters used in the formula are shown in Table 2.

Table 2 Parameters used in this study and their definition

Parameter	Definition
Major axis length (mm) (Ma.Ax) (R)	The longest axis of the orbit, passing through the centroid of the ROI in any direction, from one side of the orbital rim to the other (Gonzalez and Woods, 2004)
Minor axis length (mm) (Mi.Ax) (R/L)	The shorter axis of the orbit, passing through the perpendicular to the major axis at the centroid, from one side of the orbital rim to the other (Gonzalez and Woods, 2004)
Angle degree (°) (Angle.De) (R/L)	The downward angle of the superolateral orbital rim relative to the horizontal line (see Figure 1)

ROI, region of interest.

4. Classification

The final step was to develop the formula for classification. The numerical data of all parameters automatically computed and provided by the image analysis software were imported to generate the formula. Using the Discriminant Function Analysis feature of Statistical Package for the Social Sciences (SPSS) software version 23, five shape measurements were selected. The percentage accuracy was used in the cross-validation. The significance threshold was set at 5%. Numerical data from the extracted features were produced, and this formula was used to classify the orbital shapes of Thai and Japanese populations.

The group of 68 randomly selected skull photographs was assessed using the same four steps. This tested the accuracy of the formula derived from the experiment using the training set. The numerical data of the five parameters generated by the program were calculated using the formula for ancestry estimation. The percentage accuracy of the formula for predicting Thai and Japanese was calculated by using the outcomes of this test set.

For the intra- and inter-observer errors, this study used the ‘technical error of measurement’ (TEM). The 68 skull photographs from the training set were randomly selected. To test for intra-observer error, the observer produced image analysis data for the selected skulls, then repeated this process a second time within a month. Three observers produced the image analysis data to determine the inter-observer error. The first observer was the first author. The second observer was a physical anthropologist who had more than 5 years of experience studying human skulls. The third observer was a person who had experience working only with image analysis software, but had no hands-on experience with real human skulls.

Results

Intra-observer error

The high values of R (Table 3), presented in the minor axis lengths of right (Mi.Ax.R) and left orbit (Mi.Ax.L) and also the major axis length of the right orbit (Ma.Ax.R), suggested that the lengths of the major and minor axes were more reliable and repeatable than the angle degree (Angle. De). However, the means of the first and second measurements of all parameters showed a very small gap. This could mean that the high values of TEM and rTEM (relative technical error of measurement) of the angle degree in this study were acceptable, but should be used with caution due to their low coefficient of reliability.

Table 3 Evaluation of intra-observer error

Parameter	N	Mean (first trial)	Mean (second trial)	TEM	R	rTEM
Ma.Ax.R (mm)	68	40.11	40.08	0.51	0.97	1.26
Mi.Ax.R (mm)	68	34.04	33.95	0.29	0.99	0.86
Angle.De.R (°)	68	19.55	19.86	2.33	0.63	11.83
Mi.Ax.L (mm)	68	34.72	34.76	0.28	0.99	0.80
Angle.De.L (°)	68	17.28	17.66	1.98	0.65	11.36

TEM, technical error of measurement, calculated as $\text{TEM}=\sqrt{\left(\sum {\text{D}}^2\right)/2\text{N}}$ (Weinberg et al., 2005; De Menezes et al., 2009).

R, coefficient of reliability, calculated as $R=1-\left(\frac{{\text{TEM}}^2}{{\text{SD}}^2}\right)$ (Weinberg et al., 2005).

rTEM, relative technical error of measurement, calculated as $\text{rTEM}=\frac{\text{TEM}}{\text{VAV}}\times 100$ (VAV = variable average value) (Weinberg et al., 2005; Perini et al., 2005).

Inter-observer error

The reliability and repeatability of the minor axis group between the first and second observers were acceptable, considering the high values of R (Table 4). In contrast, the R values of the angle degree were quite low, reflecting the difficulty of repeatability of this measurement between the different users.

Table 4 Evaluation of inter-observer errors

Mean					1st vs 2nd			1st vs 3rd
Parameter	N	First observer1	Second observer2	Third observer3	TEM	R	rTEM	TEM	R	rTEM
Ma.Ax.R (mm)	68	40.11	40.89	43.71	1.03	0.89	2.53	4.76	0.19	11.36
Mi.Ax.R (mm)	68	34.04	34.05	35.92	0.42	0.97	1.23	3.60	0.27	10.29
Angle.De.R (°)	68	19.55	20.69	22.41	3.66	0.43	18.17	4.43	0.32	21.10
Mi.Ax.L (mm)	68	34.72	34.27	36.64	0.48	0.97	1.38	4.08	0.28	11.42
Angle.De.L (°)	68	17.28	17.62	19.52	2.76	0.41	15.82	3.56	0.40	19.33

TEM, R, rTEM, see Table 3.

1  First author.

2  An observer experienced in working with human skulls.

3  An observer inexperienced in working with human skulls.

Comparing the values between Table 3 and Table 4, the results were more reliable and repeatable when the observer had experience working both with real human skulls and image analysis software. A lack of experience in either one of these could produce a higher degree of error. However, of the two skill-sets, lacking expertise of real human skulls could affect the reliability and repeatability more than having no experience working with the image analysis software. All values presented in Table 4 (first observer vs. third observer) reflect the lack of expertise of the third observer in studying or measuring real human skulls.

Statistical results of the training set

The negative values of the mean differences of all measurements suggested that the major and minor axes of the orbits of Japanese skulls were longer and have a greater degree of sloping on the superolateral orbital rim than the orbits of Thai samples (Table 5). The t-values of the minor axis length of the left orbit (Mi.Ax.L) (t = 12.251) represented the largest different feature between Thai and Japanese skulls. Although the plots (Figure 2a, b) show an overlap of data between them, the median line of the box in each group could be identified separately.

Table 5 Descriptive statistics of measurements and test of differences between Thai and Japanese skulls from the training set

Parameter	Thai				Japanese				Mean difference	Level of significance
	N	Mean	SD	Range	N	Mean	SD	Range
Ma.Ax.R (mm)	220	38.70	2.41	32.6–46.4	220	41.46	2.82	35.6–52.4	−2.76	P < 0.001, t = −11.216
Mi.Ax.R (mm)	220	33.44	1.87	28.9–39.1	220	35.15	2.25	29.4–43.9	−1.71	P < 0.001, t = −8.698
Angle.De.R (°)	220	17.45	3.66	8.4–32.0	220	21.14	3.72	10.4–30.2	−3.69	P < 0.001, t = −10.485
Mi.Ax.L (mm)	220	33.71	1.87	29.1–38.5	220	36.22	2.39	30.2–43.1	−2.51	P < 0.001, t = −12.251
Angle.De.L (°)	220	16.56	3.36	6.8–27.5	220	18.84	3.25	10.2–32.6	−2.28	P < 0.001, t = −7.263

Figure 2

Boxplots demonstrating the measurements used. (a) Linear lengths (mm). (b) Angles (degrees).

Paired t-test was performed to test the difference of variables between the left and right sides of the orbit. The results showed highly significant differences (P < 0.001) between both sides of all measurements. Using the direct discriminant function analyses, all variables were individually examined to evaluate how well each one distinguishes between Thai and Japanese. For generating the formula of ancestry estimation, the stepwise method excluded the major axis of the left orbit as it was not a powerful component of the formula.

The results of the discriminant function analysis derived from the training set of skull samples are presented in Table 6. The coefficients of all variables used to create the formula of ancestry estimation between Thai and Japanese skulls, are as follows:   

$$x=-13.937+0.149·\text{Ma}.\text{Ax}.\text{R}-0.389·\text{Mi}.\text{Ax}.\text{R}+0.118·\text{Angle}.\text{De}.\text{R}+0.506·\text{Mi}.\text{Ax}.\text{L}+0.076·\text{Angle}.\text{De}.\text{L}$$

Table 6 Coefficients of the discriminant formula (a) and percentage accuracy (b) derived from the training set

(a)
Parameter	DF coefficients
Ma.Ax.R (mm)	0.149
Mi.Ax.R (mm)	−0.389
Angle.De.R (°)	0.118
Mi.Ax.L (mm)	0.506
Angle.De.L (°)	0.076

(b)
Correct classification (%)			% Cross-validated
Overall	Thai	Japanese	Overall	Thai	Japanese
81.4	85.0	77.7	80.7	84.1	77.3

The cut point of this formula was −0.4075. If the discriminant score (x) was under the cut point, the skull tended to be Thai. On the other hand, if x was over the cut point, the skull tended to be Japanese. The formula had a predicted and cross-validated accuracy of 80.7%.

Percentage of accuracy of the test set

The percentage accuracy calculated overall and separated by group (Table 7) indicated that the discriminant equation tended to yield results favoring Japanese ancestry estimation rather than that of Thai. However, the calculation of the false positive ratio indicated that the error of ancestry estimation being Thai (3/31 = 9.7%) was lower than that of the Japanese (6/37 = 16.2%).

Table 7 Confusion matrix of the discriminant analysis on the test set.

predicted	Thai	Japanese
actual
Thai (n = 34)	28 (82.4%)	6 (17.6%)
Japanese (n = 34)	3 (8.8%)	31 (91.2%)

Discussion

The statistical results of the training set indicated that there were significant differences between Thai and Japanese orbital shapes. The multivariate discriminant function analysis revealed that the shape measurement of the orbits (reflected by the major and minor axis lengths and the angle degree of sloping of the superolateral aspect) played a part in determining the difference in ancestry estimation between Thai and Japanese individuals. The mean differences demonstrated that a Japanese orbit has a wider orbital rim and is more sloping at the superolateral corner of the orbit than that of a Thai orbit. The angle degree of sloping of the orbital rim was consistent with the results of previous research (Gore et al., 2011; Rubin and DeLeon, 2017), which suggested that the inclination or curvature of the orbits appeared to play a significant role in orbital shape variation. For the experiment using the formula on a tested sample set of 68 individuals, data suggested that the predicted result is higher in Japanese skulls than in Thai skulls. This reflects a greater variation in the orbital shape of Thai skulls compared with that of Japanese skulls.

The orbital shape is one of the morphological traits of a skull which has been used for ancestry estimation of individuals. It has also been discussed as one of the most confusing traits for assessing ancestry. Even though the differences of the orbital shape among the three major population groups, European, African, and Asian, has been affirmed by numerous studies for decades (Krogman, 1973; Rhine, 1990; Pickering and Bachman, 1997; Gill, 1998; Blumenfeld, 2000; Hefner, 2009; Byers, 2011; DiGangi and Hefner, 2013), difficulties when using these for estimating ancestry in forensic cases can still occur. When assessing the morphological differentiation of the orbits among these three major groups with the naked eye, it is possible to differentiate ancestry. However, expertise is required to distinguish the rectangular shape of the orbit of African skulls from the more angular shape of the orbit of European skulls, and the round shape of the orbit of Asian skulls (Hefner, 2009; DiGangi and Hefner, 2013). More problematically, it is almost impossible to distinguish the orbital shapes of Thai and Japanese skulls (both Asian groups) using only the naked eye.

Transferring the 3D perspective of a real skull to a 2D photograph inevitably results in measurement error. The facial part of the human skull probably curved backwards on both lateral sides, affecting the perspective of the lateral part of the eye orbit in the 2D photograph. Xing et al. (2013) stated that placing the skull in the Frankfort plane and measuring the orbital shape from an anterior aspect was inappropriate as it provided limited information of the orbital contour in the orbital plane. However, this did not appear to greatly affect the results of the present study. Using this placement yielded over 80% classification on both training and test sets, sufficient for discriminating the orbital shape between these two Asian groups. Even though this setting might not reflect the ‘actual’ orbital shape, it was the standard position. This helped to control the consistency of the pre-processing step and reduce technical errors in measurements on repetition.

Although the major and minor axes do not reveal the entire shape of the orbital rim, they might reflect differences in the Japanese orbit, which is likely to be taller and wider than a Thai orbit. The difference of angle degree of the superolateral rim of the orbit provided clearer pictures of the orbital variation between these two sample groups. Greater sloping of the Japanese orbit suggests that the superolateral corner of the orbital rim is rounder than that of a Thai orbit. This agrees with a previous study using the morphological method (Tallman, 2016) which suggested that the morphology of the orbit of Japanese skulls was circular and rhombic, while the orbit of Thai skulls was rectangular. This is consistent with previous research (Masters, 2008) which states that the orbital shape relates to facial prognathism. With decreasing facial prognathism, orbital shape tends to be more rectangular. Again, this concurs with Tallman (2016), who proposed that Japanese skulls have moderate to large prognathism, while prognathism in Thai skulls is likely to be absent. Apart from facial projection, orbital shape variation was found to co-vary with facial height, supraorbital breadth, and orbital volume (Brown, 1992; Brown and Maeda, 2004). Considering orbital shape variation, Enlow and Hans (1996) mention the mechanisms of facial growth. These interact as a factor of the differences in orbital shape among a population group.

Finally, we note the accuracy and repeatability of our measurements. The analysis of intra-observer error and inter-observer error between the first and the second observers revealed that the major and minor axis lengths have an R value greater than 0.75 (Table 4), which is acceptable by the current standard (Ward and Jamison, 1991; Weinberg et al., 2005). The angle degree measurements of both intra- and inter-observer errors analyses had low values of R, but the corresponding TEM was less than 3°, which is also acceptable (De Menezes et al., 2009). However, the inter-observer error of the right orbital angle between the first and second observers exceeded 3°. This suggests that this measurement should be used with caution and that researchers and users would benefit from training to use the image analysis software more effectively. The TEM, R, and rTEM of the inter-observer error analysis between the first and the third observers clearly points the importance of training and experience working with real skulls before using this method (Table 4).

The most reliable and repeatable measurement was presented in the minor axis length of both left and right orbits, followed by the major axis length of the right orbit. To obtain these data, the program selected the ROI by creating a line from dot to dot through the complete rim of the orbit. The less space between the dots inserted by the researchers, the more precise and accurate the numerical data is when considering repeatability (not less than 40 dots is recommendable). However, the medial border of the orbit is generally difficult to identify in a photograph because of the absence of sharp edge. Therefore, it is essential that researchers have both expertise and relevant experience working with real skulls. For the angle degree of the orbit, as shown in the relatively large measurement error of the angle degree, pointing at the superolateral corner on the curved line of the orbital rim might be inconsistent if the user is unfamiliar with variation in orbital morphology. Future development of the image analysis software should focus on this parameter, as it produced a high percentage of classification inaccuracy compared to other measurements.

Conclusions

This study, which used image analysis of orbital shapes, provides a method and a formula for ancestry estimation that distinguished Thai and Japanese skulls. This could be used as a forensic application for those of either Thai and Japanese ancestry. It could, for example, be useful for identifying the skulls of Japanese soldiers from World War II found in Thailand. This semi-automated system, using image analysis software, could provide reliable and repeatable measurement data with satisfactory and acceptable precision. Five parameters of the left and right orbits as a component of one formula were able to produce 80.7% accuracy (total sample = 440), with 86.8% accuracy of the formula test experiment (n = 68). It was found that using a mobile phone camera to capture images was advantageous in that it was easy to use in the field and was cost effective. This approach could be combined with other ancestry assessments to confirm the results predicted by other methods.

Acknowledgments

This study was fully granted by the Royal Golden Jubilee PhD scholarship (the Thailand Research Fund, under the Office of the Prime Minister, the Royal Thai Government) (grant number PHD/0168/2557). This research work was partially supported by Chiang Mai University. The authors would like to thank Professor Masato Nakatsukasa and Kyoto University Museum for providing access to modern Japanese skull collections, Associate Professor Karnda Mekjaidee, MD, Head, Department of Forensic Medicine, Chiang Mai University for useful research advice, and two anonymous reviewers for their helpful comments on the manuscript.

References

•  Auephanwiriyakul  S. (2012) Digital Image Processing. Computer Engineering Chiang Mai University, Chiang Mai.
•  Auephanwiriyakul  S.,  Sumonphan  E.,  Theera-Umpon  N., and  Tayapiwatana  C. (2011) Automatic nevirapine concentration interpretation system using support vector regression. Computer Methods and Programs in Biomedicine, 101: 271–281.
•  Blumenfeld  J. (2000) Racial identification in the skull and teeth. Totem: The University of Western Ontario Journal of Anthropology, 8: 20–30.
•  Brown  P. (1992) Recent human evolution in East Asia and Australia. Philosophical Transactions of the Royal Society London, Series B, 337: 235–242.
•  Brown  P. and  Maeda  T. (2004) Post-Pleistocene diachronic change in East Asian facial skeletons: the size, shape and volume of the orbits. Anthropological Science, 112: 29–40.
•  Brues  A.M. (1977) People and Races. Macmillan, New York.
•  Byers  S.N. (2011) Introduction to Forensic Anthropology, 4th edn. Prentice Hall, Boston.
•  De Menezes  M.,  Rosati  R.,  Allievi  C., and  Sforza  C. (2009) A photographic system for the three-dimensional study of facial morphology. The Angle Orthodontist, 79: 1070–1077.
•  DiGangi  E.A. and  Hefner  J.T. (2013) Ancestry estimation. In:  DiGangi  E.A. and  Moore  M.K. (eds.), Research Methods in Human Skeletal Biology. Elsevier, Oxford, pp. 117–149.
•  Enlow  D.H. and  Hans  M.G. (1996) Essentials of Facial Growth. W.B. Saunders, Philadelphia.
•  Gill  G.W. (1986) Craniofacial criteria in forensic race identification. In:  Reichs  K.J (ed.), Forensic Osteology. Charles C. Thomas, Springfield, pp. 143–159.
•  Gill  G.W. (1998) Craniofacial criteria in the skeletal attribution of race. In:  Reichs  K.J. (ed.), Forensic Osteology: Advances in the Identification of Human Remains. Charles C. Thomas, Springfield, pp. 293–315.
•  Gonzalez  R.C. and  Woods  R.E. (2002) Digital Image Processing, 2nd edn. Prentice Hall, Upper Saddle River.
•  Gonzalez  R.C.,  Woods  R.E., and  Eddins  SL. (2004) Digital Image Processing Using MATLAB. Prentice Hall, Upper Saddle River.
•  Gore  T.,  Nawrocki  S.,  Langdon  J., and  Bouzar  N. (2011) The use of elliptical Fourier analysis on orbit shape in human skeletal remains. In:  Lestrel  P.E. (ed.), Biological Shape Analysis: Proceedings of the 1st International Symposium, Tsukuba, Japan. World Scientific, Singapore, pp. 242–265.
•  Hefner  J.T. (2009) Cranial nonmetric variation and estimating ancestry. Journal of Forensic Sciences, 54: 985–995.
•  Krogman  W.M. (1962) The Human Skeleton in Forensic Medicine. Charles C. Thomas, Springfield.
•  Krogman  W.M. (1973) The Human Skeleton in Forensic Medicine, 2nd edn. Charles C. Thomas, Springfield.
•  Lahr  M.M. (1996) Multiregional evolution as the source of modern human cranial diversity. In:  Lahr  M.M. (ed.), The Evolution of Modern Human Diversity: A Study of Cranial Variation. Cambridge University Press, Cambridge, pp. 64–115.
•  Masters  M.P. (2008) Modern variation and evolutionary change in the hominin eye orbit. PhD dissertation, Ohio State University.
•  Olivier  G. (1969) Practical Anthropology. Charles C. Thomas, Springfield.
•  Perini  T.A.,  De Oliveira  G.L.,  Ornellas  J.S., and  De Oliveiria  F.P. (2005) Technical error of measurement in anthropometry. Revista Brasileira de Medicina do Esporte, 11: 86–90.
•  Pickering  R.B. and  Bachman  D.C. (1997) The Use of Forensic Anthropology. CRC Press, Boca Raton.
•  Rhine  S. (1990) Non-metric skull racing. In:  Gill  G.W. and  Rhine  S. (eds.), Skeletal Attribution of Race: Methods for Forensic Anthropology. Maxwell Museum of Anthropology, Albuquerque, pp. 9–20.
•  Rubin  K.M. and  DeLeon  V.B. (2017) Ancestral variation in orbital rim shape: a three-dimensional pilot study. Journal of Forensic Sciences, 62: 1575–1581.
•  Sangvichien  S. (1971) Physical anthropology of the skull of Thail. PhD thesis, Faculty of Medicine Siriraj Hospital, Mahidol University (in Thai).
•  Tallman  S.D. (2016) The evaluation and refinement of nonmetric sex and ancestry assessment methods in modern Japanese and Thai individuals. PhD dissertation, University of Tennessee.
•  Urbanová  P. (2011) Variation of the orbital rim by using elliptic Fourier analysis. In:  Lestrel  P.E. (ed.), Biological Shape Analysis: Proceedings of the 1st International Symposium, 2009 June, Tsukuba, Japan. World Scientific, Singapore, pp. 221–241.
•  Uthayanung  T. (1951) Cranial module, volume, size, and indices in Thai and Chinese skulls. PhD thesis, Faculty of Medicine Siriraj Hospital, Mahidol University (in Thai).
•  Ward  R.E. and  Jamison  P.L. (1991) Measurement precision and reliability in craniofacial anthropometry: implications and suggestions for clinical applications. Journal of Craniofacial Genetics and Developmental Biology, 11: 156–164.
•  Weinberg  S.M.,  Scott  N.M.,  Neiswanger  K., and  Marazita  M.L. (2005) Intraobserver error associated with measurements of the hand. American Journal of Human Biology, 17: 368–371.
•  Xing  S.,  Gibbon  V.,  Clarke  R., and  Liu  W. (2013) Geometric morphometric analyses of orbit shape in Asian, African, and European human populations. Anthropological Science, 121: 1–11.