Automated identification of Lauraceae by scale-invariant feature transform

Abstract

An image dataset of the cross-sectional optical micrographs of the Lauraceae species including 39 species in 11 genera, capturing at least one full annual ring, was investigated by scale-invariant feature transform (SIFT), a computer vision-based feature extraction algorithm. We found an image of 900 × 900-pixel size at a pixel resolution of ca. 3 µm, corresponding to the actual size of 2.65 × 2.65 mm², as the minimum requirement for the image dataset in terms of the accuracy of the recognition at both the genus and species levels. Among the several classifiers investigated, the linear discriminant analysis (LDA) presented the best performance reaching a maximum of 89.4% in the genus with a species identification of approximately 96.3%. Cluster analysis of all the SIFT descriptors for each image yielded practical information regarding the descriptors; they recognize selectively the cell lumina, cell corners, vessels, and axial and ray parenchyma cells. Therefore, the difference between the genus or species levels was determined per the variation in the quantities of these computer-based properties. Another clustering approach, the hierarchal dendrogram, was applied to visualize the numerical distances between the genus and species. Interestingly, even Machilus and Phoebe, which are difficult to distinguish by conventional visual inspection, are quite distantly classified at the genus level. In contrast, some species in Cinnamomum, Machilus and Litsea were categorized into different subgroups rather than the original genus. Microscopic wood identification is found to be possible at the genus level; however, the numerical dataset of the morphological features has various overlapping clusters, causing the genus-level identification of the Lauraceae to be more difficult than species-level identification.

Introduction

The Lauraceae family is distributed worldwide mainly in warm temperate and tropical regions. It is also found in East Asia, and is regarded as an important wood species. For example, in ancient times, Phoebe was used as a coffin material in China, and Cinnamomum camphora was used as a material for building ships and sculptures in Japan. Thus, the Lauraceae are a culturally and historically important family in East Asia. Although the taxonomy of the Lauraceae is not entirely understood, they are known to consist of approximately 45 genera and over 2800 species. The Lauraceae are known to be relatively difficult to distinguish because of their vast diversity and wide variety within the family [1]. Therefore, the classification of the Lauraceae is complex and not easy. However, an unambiguous classification is essential for understanding their cross-cultural utilization.

Identifying wood species accurately is very important in ethnobotanical research on wood usage. Among the several methods proposed until now, the microscopic method is the most accurate and widely used owing to the tremendous efforts by the wood anatomists and the corresponding identification keys proposed by the IAWA (the international association of wood anatomists) community. The method allows us to mostly identify it up to the genus level.

More recently, identification methods are gradually expanding to new techniques such as DNA-based identification [2,3,4,5,6] and spectroscopy-based chemometrics [7,8,9,10,11,12]. These methods are used primarily for specific purposes. DNA-based identification is a method that uses the inherent DNA sequence of each species, and has the advantage of being able to identify up to the species level. However, owing to the lack of a database, it is difficult to use it for generalized identification. Chemometrics, combining the spectroscopic data and multivariate analysis, is representative of the facile and nondestructive analysis. However, this application is restricted to limited or selected species identification, and its accuracy is negatively affected by aging that is critical when this method is applied to old wood used in cultural properties [9].

Recently, wood identification using computer vision-based methods to extract features from the wood images has been receiving an increasing interest. Several methods such as the gray level co-occurrence matrix (GLCM) [13, 14], local binary pattern (LBP) [15, 16], and higher order local autocorrelation (HLAC) [17] have been tested to extract features from wood images. Computer vision is a part of the diverse field of artificial intelligence, and is one of the technologies with the most potential. Currently, computer vision algorithms that can be applied to various problems are being developed by computer scientists. The selection and implementation of these algorithms will vary according to the problem that is considered. For instance, it is known that Haar-like features and histogram of gradient (HOG) exhibit good performance for face recognition and human detection and its motions, respectively [18, 19]. Scale-invariant feature transform (SIFT) is one of the useful algorithms to detect the local features from an image. It was first introduced by D.G. Lowe in 1999 [20], and was discussed in an extended paper in 2004 [21]. The features detected by SIFT are invariant to scale and rotation, and are robust to the changes in the illumination and affine. The usefulness of SIFT has already been demonstrated in various studies [22,23,24], and advanced algorithms such as the speeded-up robust features (SURF) [25], principal components analysis-based local descriptor (PCA–SIFT) [26], and gradient location and orientation histogram (GLOH) [24] are being developed starting from SIFT, for inspiring the various descriptors proposed later.

In this study, a wood classification system based on the SIFT algorithm was designed to investigate the Lauraceae family, i.e., the structural and anatomical diversity in the genera and species is systematically and statistically quantified by computer vision. The relationship between the observed computer-based features and anatomical characteristics of wood was comparatively discussed, and the applicability of computer-based features for automated classification was presented. In addition, a hierarchical clustering was performed to analyze the correlations as well as the distances from the genera and species. This paper describes a new approach for the quantification of wood structure and its application to the anatomy that, we believe, can be applied to numerous other fields in wood science.

Experimental procedure

Image dataset

The species in the Lauraceae family that are used for wood classification are summarized in Table 1. The wood samples of 11 genera including 39 species of the Lauraceae were received from the RISH Xylarium, Kyoto University. The cross-section of the samples were cut using a sliding microtome, and stained with safranin. The images were captured at a low magnification with Olympus™ 2 × (0.08NA) PlanApo objective lens, using a BX51 optical microscope equipped with DP73 CCD (charge-coupled device) camera (Olympus, Japan). Most of the acquired images have an area of at least one annual ring. The original size of an image was 4800 × 3600 pixels, and the resolution was 0.74 µm/pixel. The Lauraceae dataset consisted of 1557 images, and each species had more than 8 images.

Table 1 List of the species in the image dataset

Image pretreatment

For feature extraction, all the images in the dataset were converted from RGB color to 8-bit grayscale. An initial dataset of square 3600 × 3600-pixel images of 0.74 µm/pixel resolution was cropped from the original micrographs, corresponding to an actual area of 2.65 × 2.65 mm². Then, a series of datasets with decreasing resolutions, 1.47, 2.94, 5.88, and 11.76 µm/pixel were prepared by bilateral filtering. These images of 3600 × 3600 pixels to 1800 × 1800, 900 × 900, 450 × 450, and 225 × 225 pixels in the same area were tested for determining the necessary size and resolution information for the classification.

Classification

The flowchart of the experimental procedure including the image pretreatment is shown in Fig. 1. The image features were calculated from a series of downsized image datasets, followed by classification using multiple classifiers.

Fig. 1

Experimental flowchart of image processing, machine learning, and classification

SIFT algorithm

SIFT is selected as the algorithm to extract image features that are known to be invariant to the image scale and rotation. A feature detected by SIFT is called a keypoint. The algorithm generates a difference of Gaussian (DOG) from the image pyramid, and it detects the local extremes as the keypoint candidates in the DOG. Subsequently, the algorithm determines the keypoints that are normally inappropriate as features, by eliminating the candidates present on the edge or having a low-contrast.

In the calculation, we adopted the default parameters proposed by Lowe [21], namely, the number of layers in each octave (nOctaveLayers = 3), value of the Gaussian filter applied to the image of each layer (σ = 1.6), contrast threshold (ct = 0.06), and edge threshold (et = 10).

Classifiers

Four different classifiers, namely, k-nearest neighbor (k-NN), logistic regression (Logistic_R), support vector machine (SVM) with a linear kernel, and linear discriminant analysis (LDA) were used for the classification, and the performances of the classifiers were compared. The test set and training set were randomly selected in a ratio of 2:9, and the calculation was repeated 20 times with different sets of test and training data to obtain the average and standard deviation of the accuracy of recognition.

k-means clustering

To visualize the objects that SIFT selects as keypoints, k-means clustering was performed on all the detected features from one image of Cinnamomum camphora, and eight representative centroid patterns of the SIFT descriptors were calculated. The keypoints derived from each pattern were re-projected on the corresponding image, and their distribution was scrutinized in comparison with the anatomical features.

Hierarchical clustering

An unsupervised hierarchical clustering was performed to draw a dendrogram based on the numerical distances between the genus and species. The simple Euclidean distances calculated from the original SIFT descriptors did not cause a reasonable clustering; therefore, the dimensionality of the descriptors was reduced by LDA. The clustering was performed by the agglomerative Ward’s minimum variance method [27] using LDA components up to ten as a dataset. The above setting was selected by considering the cophenetic correlation coefficient [28] that was proposed as a measure of how closely a dendrogram preserved the pairwise distances between the original unclusterized data points.

Programming

All the programs for the image pretreatments and classification were written in Python 3.5.2 [29] running on MacOSX. The SIFT algorithm and image pretreatments were implemented by the libraries OpenCV [30], NumPy [31], and SciPy [32]. The k-means clustering and hierarchical clustering were implemented by the Scikit-learn [33] and SciPy libraries, respectively.

Results and discussion

SIFT descriptor and classification

An example of the keypoints detected by the SIFT operation is presented in Fig. 2a. This algorithm determines various local gradient-based keypoints that are expressed by a 128-bins histogram representing the orientation and magnitude of the gradient around them, as shown in Fig. 2b. Numerous keypoints are found from an image. Generally, the descriptor obtained by SIFT and used for further analyses is a single 128-dimensional vector that is in fact the sum of the histograms from all the keypoints in each image (Fig. 2c). Briefly, all the images have their own single SIFT descriptor, and they have only 128-bins to represent themselves.

Fig. 2

Keypoints and descriptors in the SIFT algorithm. a Keypoints detected in a cross-section of Cinnamomum camphora. The image is partially cropped to a size of 225 × 225 pixels from the original size, and a total of 220 keypoints are detected; b descriptors of all detected keypoints in the image in a; c The sum of the SIFT descriptors represented in the image in a as well as in the histograms in b

A summary of the accuracies in the classification of the Lauraceae dataset at various image resolutions is presented in Table 2. Among the four classifiers tested, LDA and SVM yield better results that are depicted in Fig. 3. Clearly, in both the cases, a good performance is obtained from the image dataset having a better resolution, with LDA exhibiting a slightly better performance than SVM. The best accuracy is 89.4% by LDA and 88.0% by SVM in the genus classification, and it is 96.3% by LDA and 95.4% by SVM in the species classification mostly, when the 900 × 900 image dataset was tested. This implies that the information obtained from the 2.65 × 2.65 mm² area with a resolution of 2.94 µm possesses the critical information required for identification.

Table 2 Classification accuracies of the Lauraceae image dataset at the genus and species levels by the SIFT algorithm with the four different classifiers

Fig. 3

Variation in the classification accuracy and number of keypoints by reducing the image size at the genus and species levels

More importantly, it is worth to note that the accuracy of the prediction is better at the species level than the genus level.

Structure of SIFT

To determine the features that can be extracted from the 2.65 × 2.65 mm² area with a resolution of 2.94 µm in the cross-section of C. camphora through computer vision, the SIFT descriptors were examined via cluster analysis. As mentioned earlier, each image provided numerous SIFT descriptors and image-size dependency of the keypoints was plotted as a dashed line in Fig. 3. Obviously, a damping of as much as 5 to 3 digits difference in the number of keypoints occurred in between 900 × 900 and 450 × 450-pixel images.

All the SIFT descriptors from the image of 900 × 900-pixel size were subjected to unsupervised clustering into eight virtual classes, and the keypoints within each class were re-plotted on the initial image. Four representative classes are shown in Fig. 4; the remaining four classes were either similar to one of these presented four classes or were a mix of multiple classes. Expectedly, the keypoints in a given class are found on exactly similar anatomical features, and in this case, they are located at the cell corners, cell lumina, vessel elements, and axial parenchyma cells. The centroid histograms for each class are presented in Fig. 5, showing clear differences in the histogram patterns. In fact, the number of keypoints that belong to these patterns of histograms differs with the image, species, and genus, as a result of which SIFT allows us to perform the classification.

Fig. 4

SIFT keypoints clustered by k-means clustering in the cross-section of Cinnamomum camphora. The images show an enlarged part, and the keypoints are detected in an image of 900 × 900 pixels. The keypoints in each image are mainly located at the cell corners (a), cell lumina (b), vessel and axial parenchyma cells (c), and cell wall of the vessel and ray parenchyma cells (d)

Fig. 5

SIFT descriptors of each class indicating different anatomical features as categorized by k-means clustering

Resolution dependent keypoints distribution

The 900 × 900-pixel images with 2.65 × 2.65 mm² area and a resolution of 2.94 µm in the cross-section, appear to be the most suitable setting for performing further analysis in terms of the classification accuracy and memory and computational time load. To confirm, the composition of the shift descriptors of the C. camphora images was re-investigated with respect to the change in the image resolution. As shown in Fig. 6, the composition of the features is almost equivalent for 3600 × 3600 and 1800 × 1800-pixel images. In these images, the keypoints detected at the cell corners are approximately 50% of the total, of which approximately 40% are detected in the cell lumina. In the case of the 900 × 900-pixel image, the compositional ratio at the cell corner and cell lumina decreases to 47.9% and 36.8% respectively, which is slightly less than for the larger images. This may be owing to the fact that the feature extraction capability of SIFT is influenced by the image size, and thus, the ratios of the composition of the same anatomical features are somewhat different. However, in comparison with the images smaller than 450 × 450 pixels, the 900 × 900-pixel image set is found to possess reasonable information that can be provided from the images larger than itself.

Fig. 6

Variation in the compositional ratio of the anatomical features due to the reduction in the image size. V, vessel; AP, axial parenchyma cell; R, ray parenchyma cell; * Standard deviation

Overall, the image size, i.e., the resolution, is a very important factor in the classification, and the 900 × 900-pixel images with 2.65 × 2.65 mm² area and a resolution of 2.94 µm in the cross-section are essential to expect a satisfactory performance. Furthermore, it is interesting to note that computer vision suggested that the numbers or localization of the cell corners are species-specific, and this could never be provided from a human vision.

Unsupervised hierarchical clustering

Figure 7 shows the dendrograms at the genus and species levels with cophenetic correlation coefficients of 0.60 and 0.65, respectively. As described in the experimental section, LDA reduced the datasets of the SIFT descriptors for the 900 × 900-pixel images. The cophenetic coefficient is the parameter, used in biostatistics, that in the dendrogram closely represents the actual numerical distances between any paired data. Thus, the variation in the data in a given branch is considered to be larger than in the genus compared with in species clustering. This is consistent with the result of the classification accuracy, as described earlier. In genus level clustering shown in Fig. 7a, eleven genera are blanched into 4 groups. Machilus belonged to the same group with Cinnamomum and both the species are in very close distance to each other, although in the actual phylogenetic relationship [5, 6], Machilus is close to Phoebe. In practice, with microscopic inspection, some species in Machilus are known to be difficult to be distinguished from those of Phoebe, even by skilled wood anatomists. As shown in Fig. 7a, in contrast computer vision, Pheobe is sub-grouped differently from the majority of species in Cinnamomum, Machilus and Litsea.

Fig. 7

Dendrograms calculated by the Euclidean distance with Ward’s clustering method. Dendrogram at the a genus level and b species level. In the figure, * is the cluster size; **is the dominant species in the cluster (number of images of the species in the cluster/total number of images of the species in the dataset), and *** is the minor species in the cluster

In the dendrogram at the species level (Fig. 7b), various species are sub-grouped independently from their original genus. Specifically, the species in Cinnamomum, Machilus and Litsea have been randomly mixed to form new subgroups. The accuracy of species prediction by LDA being as high as 96%, the dendrograms suggest the presence of morphological similarity, although it is invisible unless computer vision is applied.

Numerical and real phylogenetic trees

With the classification by SIFT depending only on the cross-sectional morphological features of wood, it is not surprising that both at the genus and species levels, the relative numerical distances of each genus and species are slightly different from the reported phylogenetic relationships [5, 6]. According to Liu et al. [5], who studied the phylogenic relationships of 409 individuals representing 133 species of 12 genera in the Lauraceae using DNA barcoding, 44 individuals from the ten species were misclassified by the form taxonomists. This supports the fact that the morphological features of the Lauraceae are indeed complex to be classified unambiguously. As shown in Table 2 and Fig. 3, the fact that the numerical classification by SIFT is more accurate at the species level than at the genus level, implies that the structural features of the species in a given genus are similar to those in another genus, making the genus level classification difficult. This accounts for what the anatomists say about the Lauraceae.

Conclusion

Feature extraction by SIFT followed by LDA as a classifier allowed us to classify accurately 11 genera and 39 species in the Lauraceae. The extracted features are found to be related to the distributions of the vessels, parenchyma cells, fibers, and ray cells, and thus, providing a new perspective on species-specific morphology. Though such an approach is still relatively new in the wood anatomy, the potential of the technique seems promising, as the computer vision allows us to observe cell corners as a feature which so far has not been considered as species-specific. Ultimately, deep learning by a convoluted neural network may achieve complete automated species identification in reality. Finally, the quantification of two-dimensional image data by computer vision would also allow us to predict the physical and mechanical features of a given wood, if we design the features to reflect the porosity, anisotropy, and ratios of cell types for instance. These lines of investigation are in progress and will be reported elsewhere.