We propose a new method to extract the informations of microscopic structure from the extended X-ray absorption fine structure by the application of sparse modeling based on a simplified single-scattering approximation of photo-electron waves. This method can extract the sparse radial distribution function of the atoms located nearby the target atom and can estimate the Debye-Waller factor without any assumption of the micro-structures. Therefore, this method is expected to exhibit considerable ability in the crystallographic researches on new materials and such cooperative researches of data-driven science and crystallographic measurements are strongly expected to extend the frontiers in various research fields.

In this study, we propose a sparse phase retrieval algorithm （SpPRA）, which involves an iterative Fourier transform and sparse modeling （LASSO）, and demonstrated that it can retrieve the phase from diffraction data including noise and missing information. Considering the sparseness of the target sample as regularization enables preventing over-fitting the noise and estimating the diffraction data of the missing region.

Recently, the atomic arrangement analysis of dopants in crystal using atomic resolution holography, such as photoelectron holography, X-ray fluorescence holography and neutron holography become possible. The equation for converting an atomic arrangement to an atomic resolution hologram is given. In order to obtain an atomic arrangement from a measured hologram, it is necessary to perform inverse conversion, which is an underdetermined system. L1 regression used for machine learning is effective to solve it. Taking photoelectron holography as an example, the principle of the atomic resolution holography, algorithm to a tree-dimensional atom image, and the analysis of atomic arrangement of the phosphorus dopant in diamond by photoelectron holography are shown.

A data-driven approach to X-ray response non-uniformity in photon-counting detectors, which is referred to as ReLiEf （Response to Light Effector）, has been developed to realize synchrotron total-scattering measurements for materials with both crystalline and non-crystalline domains. A total-scattering measurement system corrected by ReLiEf, which is called OHGI （Overlapped High-Grade Intelligencer）, can give wide-Q-range （0.1～31 Å^－1） and small-Q-step （10^－3 Å^－1） data with an accuracy of 0.2％ by a single measurement.

Along with the advancement of measurement technologies, a large amount of experimental crystallographic data can be obtained easily, so that objective and rapid analysis suitable for batch process is strongly required. This article introduces recent research on objective and rapid analysis of crystal structure using X-rays or electron beams. For one-dimensional X-ray diffraction pattern, non-negative linear regression （NNLS） using diffraction patterns provided by public database has achieved more objective and faster analysis than conventional manual works. With regard to two-dimensional electron and X-ray diffraction, good objective and rapid analysis was realized for several types of materials by using artificial intelligence （AI） based on convolutional neural network. On the other hand, the sparsity inherent in the diffraction pattern was found to cause a lack of information, and universal applicability of material property prediction using AI that depends on training data is still open to question. However, the fact that AI itself has the potential ability to solve complex images, a breakthrough technology for universal two-dimensional objective and rapid analysis might be found in progressing research.

The structure of disordered materials is still not well understood, due to the lack of structural information from diffraction data. In this article, attempts are made to reveal the information on the topology of crystalline and disordered materials by utilizing persistent homology analyses. The persistence diagram of silica （SiO₂） glass indicates that the shape of rings in the glass is similar not only to those in the crystalline phase with comparable density （α-cristobalite）, but also to rings present in crystalline phases with higher density （α-quartz and coesite）. This behavior is a result of disorder, because the shape of rings in silica glass is buckled, which can be observed in high density crystalline phases. Furthermore, we have succeeded in revealing the differences, in terms of persistent homology, between tetrahedral networks and tetrahedral molecular liquids, and the difference between liquid and amorphous states. A combination of diffraction data and persistent homology analyses is a useful tool for allowing us to uncover structural features hidden in the pairwise correlations.

Prediction of crystal structure from the chemical composition has been a long-standing challenge in natural science. Although various numerical methods have been developed over the last decades, it remains still tricky to numerically predict crystal structures comprising more than several tens of atoms in the supercell. We propose a new “data assimilation” approach for crystal structure prediction from numerical simulations with support of X-ray diffraction experimental data. We show that even if the experimental data is insufficient for conventional structure analysis, it can still support and substantially accelerate structure simulation.

Applying long wavelength X-ray （ca. longer than 2 Å） in macromolecular crystallography has been limited due to the lack of dedicated measurement environment. Photon Factory BL-1A, as an MX beamline optimized for diffraction experiment using the wavelength from 2.7 to 3.3 Å, offers the opportunity to routinely perform data collection for Native SAD phasing or light atoms identification with much sophisticated manner. Sample optimization is another key for successful experiment at the wavelength where severe absorption is inevitable. We describe how the difficulties in use of long wavelength X-ray were overcome at the beamline and the auxiliary facility, as well as the current results of Native SAD phasing.