Scientific machine learning (SciML) is a research field aimed at solving various scientific problems through the mutual complementation and synergy of data science and physical laws/mathematical models. In this talk, I introduce examples of the application of SciML in earthquake studies. Among the representative methods of SciML are Physics-Informed Neural Networks (PINNs) and operator learning. PINN is a method that has attracted attention due to its widespread use across many engineering fields. For instance, in seismic travel time calculations, the introduction of PINNs has enabled simultaneous solutions and rapid inference for various source conditions without labeled data. It also allows solving inverse problems such as travel time tomography using methods different from conventional approaches. On the other hand, operator learning includes two types: DeepONet and Fourier Neural Operator (FNO). These learn mappings from functions to functions. With the capability of operator learning, it becomes possible to instantly predict seismic motions for various velocity structure. Although these architectures are still somewhat complex and incomplete, remarkable future developments are anticipated.

The act of reading is crucial for learning, and by analyzing this activity, we can gain insights into both the quantity and quality of reading and learning. This presentation will introduce the German Research Center for Artificial Intelligence (DFKI) and highlight its learning-related projects. For example, the “Reading-Life Log” project records the reading activities of users wearing sensors, enabling mutual analysis between the individual and the book. Further learning support tools include a word meter to measure the amount of language read, actuators that encourage increased reading volume, and learning support based on confidence estimation. These are integrated into the Intelligent Textbook, which estimates cognitive load and identifies areas of interest or difficulty in understanding. In the HyperMind project, supplementary materials are provided to help users overcome areas they struggle with. However, since the effectiveness of learning support varies among individuals, a personalized prescription is necessary.

The category of Vision and Language includes multimodal understanding, which outputs recognition results from both visual and textual inputs, Image2Text, which generates text from visual input, and Text2Image, which generates visuals from text. Currently, research in this field is accelerating. One example from the authors’ research is the development of an AI robot that collaborates with humans to create and transcend knowledge. This requires building a scientific foundational model that learns from scientific literature, conducts experiments autonomously, and becomes smarter through discussions with researchers. Other research examples include studies on automating experimental procedures into manuals, AI-driven discovery of scientific laws and principles from data, and research on discovering new materials. In the discovery of new materials, two approaches are being explored, one of which involves creating generative AI that uses highly accurate decoders for generating crystal structures.

At NICT, research and development of large language models (LLMs) and the systems that utilize them are underway. Pre-training data that were used by LLMs developed overseas contain only a small proportion of Japanese data, and the widespread adoption of these LLMs increses risks such as the loss of Japan’s cultural uniqueness. Therefore, enhancing the development capabilities of domestic LLMs that utilize a large amount of Japanese data becomes crucial. For training Japanese LLMs, high-quality Japanese data is necessary. Over the past 15years, NICT has accumulated a large quantity of Japanese web data, and this is being utilized to construct high-quality training data. The countermeasures against hallucinations, fake news, and other proposed risks are being developed. We are currently developing “WISDOM-LLM”, a platform that combines a diverse range of AIs, which can fact-check generated outputs from LLMs as well as generate well-founded counterarguments. To protect Japanese society from “stray generative AIs” that are likely to increase in the future, a world of “democratic AIs” is being considered, where multiple “justice-oriented AIs” engage in discussion with each other, and subsequently, their output is reviewed by humans to make the final decision.

Ports and Harbours Bureau of Ministry of Land, Infrastructure, Transport and Tourism is promoting the automation of construction sites based on “i-Construction 2.0.” At port construction sites, ICT construction methods and 3D data are being introduced to improve productivity and working conditions. There are al- ready achievements such as the use of dredging vessel construction history data for managing the quality of floor excavation work. BIM/CIM is also applied as a general rule, and is being used in construction projects of a certain scale or larger. A cloud system for sharing 3D data is also under development. Fur- thermore, a data platform that centralizes information from the logistics, management, and infrastructure sectors has started operation. In the infrastructure sector, information from planning to maintenance of port facilities is being digitized, enabling centralized access via GIS. Additionally, efforts to decarbonize port construction projects are underway, including the development of CO2 emission calculation guidelines and the assessment of CO2 emissions by structural type for port construction projects nationwide, in order to explore efficient reduction measures.

In this study, we examined a method to automatically calculate the structural information of bridges using point cloud data measured with the Matterport Pro3. We calculated the dimensions of railings, decks, and main girders and evaluated their accuracy. The point clouds were pre-annotated, and this information was utilized during processing. The length of the railings was estimated by projecting the point cloud onto a 2D image from a top-down view and using image processing techniques. The RMSE was 0.289 m, suggesting a possible error of about 5cm. The length and width of the deck were compared using principal component analysis and ellipse fitting, with RMSEs of 0.10 m and 0.50 m, respectively. Additionally, we attempted to detect the intersections of the main girders, but estimating the position of intersections was difficult in areas with missing data. Future work should aim for more accurate estimation by combining the method with automatic point cloud classification using deep learning.

The advent of sophisticated photographic technology and artificial intelligence (AI) for the identification of cracks and other defects from image data has led to the partial implementation of bridge inspections combining digital images and AI as an alternative to close visual inspection for reinforced concrete bridge piers and slabs. The present paper reports the results of the use of digital images and AI in two bridge inspections: an inspection for cracks on reinforced concrete slabs to improve the efficiency of bridge span inspections where inspection time is limited, and an inspection for corrosion on steel members to improve the efficiency of pedestrian bridge inspections where traffic control is required. The paper also reports on the benefits and points to be noted in the use of digital images and AI obtained from these efforts.

The civil engineering industry faces low productivity issues; hence, it is adopting Building/Construction Information Modeling, Management (BIM/CIM) to improve productivity in this domain. BIM/CIM utilizes 3D data models to streamline whole construction projects, design construction, operation and maintenance (O&M), and demolition processes, leading to enhanced efficiency and quality of construction projects. However, creating these 3D models by hand is time-consuming and labor-intensive. This issue is particularly true for infrastructures that engineers have been designing based on 2D data for a long time. This study proposes an interactive method for generating 3D data models using generative artificial intelligence (AI) to address this issue. Using the proposed method, engineers provide instructions in natural language and generate 3D data models automatically. The proposed method reduces the burden of engineers operating 3D modeling software or inputting complex parameters, resulting in labor savings and time reduction. This study focused on Industry Foundation Classes (IFC) 4.3 and conducted experiments to generate data models of simple shapes (rectangular cuboid, cylinder, and sphere). The proposed method utilizes one-shot prompting to generate 3D models collectively because IFC is a particular professional file format. This study evaluated the accuracy of the proposed method based on the ratio of the number of collectively generated models to the total number of generated models. The results show an accuracy of 64% for rectangular cuboids, 31% for cylinders, and 44% for spheres. The proposed method is currently limited to low accuracy and generating simple shapes. However, future development aims to improve accuracy using fine-tuning and AI agents and support complex shapes used in the civil engineering industry, such as beams, columns, and footing, as well as attributes. The proposed generative AI-based 3D data model generation method is expected to accelerate BIM/CIM adoption in the civil engineering industry, significantly contributing to productivity improvement.

To improve efficiency by utilizing digital data in the civil engineering and construction fields, we verified the application and effectiveness of 3D models generated from point cloud data to reinforcement inspection and repair inspection, by verifying their accuracy and comparing them with conventional methods. The accuracy of the reinforcement inspection was within a relative error of 0.3ϕ for the rebar spacing and within a relative error of 0.6ϕ for the cover thickness (where ϕ is the rebar diameter). The accuracy of the repair inspection was within 6% of the volume and within 7 mm of the depth and the cover thickness. Compared with the conventional reinforcement inspection, the work time was reduced by about 80%. Compared with conventional repair inspection, the work time was reduced by about 85%.

In general, it is said that in responding to road flooding, if it is possible to predict flooding 30 minutes in advance, the necessary response can be taken. For this purpose, it is necessary to improve the accuracy of rainfall information 30 minutes ahead. In this study, a model for predicting road flooding was constructed and verified based on past cases of road flooding using the Multi-Parameter Phased Array Weather Radar (MP-PAWR), and based on the results, a demonstration experiment was conducted to predict road flooding by applying online observation data compression and restoration technology. Based on the results of these experiments, the company aims to contribute to the facilitation of countermeasures through early action prior to road flooding, based on the improvements identified in the verification experiments.

This paper proposes an automatic findings generation technique based on a Multimodal Large Language Model to enhance the efficiency of bridge maintenance management. The generative AI learns the relationship between distress images and their findings from a few examples and generates findings for desired distress images. To effectively leverage this capability, the proposed method employs clustering to identify the most informative inspection information for each findings category in learning task, and selects past inspection records based on this information. This approach not only improves the accuracy of the generated findings but also reveals the connection between the inspection information identified through clustering and the implicit knowledge in engineers’ findings creation Experiments verify that the proposed method successfully reflects the insights of engineers and achieves high-precision findings generation.

This study conducted a technical survey on the design and implementation of a system for the visualization of river flow conditions using three-dimensional representations. The implemented system reflects changes in real time by superimposing the water surface, generated based on river information managed in a graph database, on a three-dimensional model generated based on open data representing geographic features. Finally, the study summarized the points that can be improved to enhance the performance and presented the items to be considered when constructing a flow visualization system.

In recent years, initiatives for utilizing digital twins in road traffic management have been progressing. To realize a digital twin, it is essential to accurately determine the location of roads. In this paper, we propose a system that enables high-precision positioning of road locations using inexpensive and readily available equipment. The system is equipped with a camera directly beneath the antenna, allowing drivers to continuously verify whether the antenna is passing over the targeted lane markings. Additionally, the system calculates the displacement and direction of the antenna’s position relative to the lane markings through image analysis. By presenting these calculation results to the driver, the system prompts them to correct any deviation of the antenna’s position from the lane markings. Finally, we conduct an experiment to measure the location of the outer edge line of the roadway, confirming the effectiveness of the proposed system.

Local governments inspect road surfaces to detect deformations in their early stages. Since current road surface inspection methods are costly and time-consuming, image analysis using machine learning has been studied as a way to reduce cost and time. However, this method has difficulty detecting new types of deformations that are not included in the training data. In this paper, we propose a method to detect unnormal road surfaces using only image patches of normal road surfaces, where normal is defined asa road surface with no deformation. The proposed method is expected to enable efficient inspections by focusing on areas that are detected as unnormal. In the experimental section, the effectiveness of the proposed method is demonstrated using road surface images from Japan and Nigeria.

Monitoring road and transportation conditions in real-time using edge computing is conducted to achieve more advanced road and transportation management. In order to accurately monitor various road and transportation conditions, it is necessary to re-learn discriminators that analyze data sequentially at each distributed edge. In order to efficiently re-learn discriminators, decentralized learning, in which edges directly exchange data with each other, is desirable. In this paper, we propose a method to re-learn a discriminator based on a neural network using the knowledge possessed by other edges. In the proposed method, a part of the weight matrices of other discriminators is transferred to its own discriminator, and the process of adapting the replaced weight matrices to its own discriminator is repeated using the error backpropagation method. By repeatedly replacing the weight matrices and applying the error backpropagation method, stable re-training of discriminators can be expected.

We attempted to use a generative AI that may predict natural disasters. Focusing on river flooding and inland water inundation caused by recent climate change, we used an existing large-scale language model (LLM) to create a system that can output images and texts of flood waveforms, inundation conditions, etc. by querying the generation AI with pictures or text (hereinafter, referred to as prompt input). We verified whether better flood calculations could be made by linking a tank model capable of flood calculations based on physical laws to the LLM and by having LLM refer to the precipitation and runoff volumes related to floods as additional information. We confirmed that an internal parameter of the tank model can be appropriately adjusted based on flood information from prompt input and further confirmed that continuous interaction with the LLM can suggest better estimates of the internal parameter.

This study aims to examine the effectiveness of evacuation behavior analysis by integrating roadside camera images with object detection. YOLO was applied to images captured by roadside cameras installed in Toyama Prefecture to analyze traffic volume before and after the 2024 Noto Peninsula Earthquake. The results revealed a significant increase in traffic volume from coastal areas to inland regions, as well as behaviors suggesting evacuations to higher ground and emergency supply purchases. The spatiotemporal patterns of traffic volume changes across Toyama Prefecture were successfully captured. Image-based analysis using YOLO is simple and provides a more intuitive understanding than numerical data, making it accessible to the general public. This approach has the potential to enhance disaster awareness among local residents.

In this paper, we propose a method for automatically correcting the results of a winter road surface condition estimation model using in-vehicle cameras. Typically, the estimation accuracy of road surface condition models decreases when applied to surfaces other than those used for training. However, collecting data and training the model for all road surfaces we wish to estimate is not feasible due to cost constraints. The proposed method mitigates this accuracy degradation without requiring model retraining. Specifically, when the confidence level of the predicted road surface is low, the method applies the k-nearest neighbors (k-NN) algorithm to the feature vectors of other road surface conditions within the model to relabel the prediction. The road surface conditions targeted for estimation are the six types commonly used in road management: Dry, Slightly wet, Wet, Slushy, Icy and Snowy. The proposed method was validated through experiments using real vehicle data collected on general roads, applying a model trained on winter highway data, and its effectiveness was confirmed.

A Virtual Reality (VR) educational material was developed for beginners in civil engineering. This educational material has three main objectives. First, by using familiar civil engineering structures as learning content, users can become familiar with the names of structures in their environment. Second, through multi-perspective observation enabled by 3D capture technology, learners can examine structures in more detail. Thirdly, by incorporating VR and 3D Gaussian splatting, the material allows learners to experience state-of-the-art technologies. In addition, 15 students of National Institute of Technology, Gunma Collage were asked to use this educational material, and a questionnaire survey was conducted. As a result, more than 86.6% of the respondents answered “agree” or “somewhat agree” to “the usefulness of using 3D models,” “the usefulness of using VR,” and “the usefulness of using the familiar environment as a target”. The results of the questionnaire survey on the level of understanding of civil engineering structures showed that the cumulative response rate of “can answer” and “probably can answer” for each name was over 80%, suggesting that the teaching materials developed this time have a certain learning effect.

With advancements in measurement technology, digital twins leveraging large-scale real-time data have garnered significant attention. In digital twins, measurement and data analysis technologies play a critical role, and GNSS (Global Navigation Satellite System), known for its high-precision positioning capabilities, is one such promising technology. This study attempted real-time 3D displacement monitoring during bridge construction using GNSS. While no significant movement was observed in the direction orthogonal to the bridge axis, displacements in the longitudinal and vertical directions were found to be interlinked. These displacements were inferred to result from both small rotation of the entire bridge, including its foundation, caused by concrete casting, and deformations contributing to vertical and longitudinal displacements. The study demonstrated the potential of continuous monitoring using GNSS for three-dimensional behavior of bridges during construction.

This paper proposes a damage level estimation model that incorporates in-context learning with data augmentation to achieve accurate classification of inspection images in road infrastructures, even with a limited amount of data. Conventional methods face challenges such as the ambiguity between damage levels in inspection images and the requirement for large-scale datasets to construct advanced models. In contrast, the proposed method utilizes a pre-trained Large Multimodal Model (LMM) and introduces incontext learning with inspection images using data augmentation (augmented images) to construct an LMM capable of damage level estimation. Furthermore, by integrating the estimation results of each augmented image, the method generates a final output and enables accurate damage level estimation even for challenging images. Finally, we validate the effectiveness of the proposed method using actual inspection images from road infrastructures by comparing its performance with other methods for damage level estimation.

Location information of workers at construction sites is widely used to improve safety and productivity. Various devices, such as cameras, GNSS, and BLE beacons are used to obtain this information. However, cameras are susceptible to illuminance variations, GNSS is difficult to use indoors and in underground spaces, and BLE beacons require a substantial number of receivers for comprehensive coverage of expansive sites. These limitations present practical challenges.

To address these challenges, this study proposes a novel method for detecting workers using low-cost LiDAR. Unlike other technologies, LiDAR is unaffected by lighting conditions and performs well in indoor, underground, and expansive environments. The proposed approach combines deep learning with motion detection techniques to enhance accuracy and reliability. Verification experiments achieved an F-measure detection accuracy of approximately 0.9, demonstrating the method’s potential for precise workers localization at construction sites.

This study was conducted to investigate the application of an infrastructure management platform that integrates data from 3D models and IoT sensing to remotely monitor damage to bridges, with the goal of quickly assessing damage status during maintenance management and earthquakes, and to large bridges such as expressway bridges. The study was carried out on large bridges such as motorway bridges. As a preliminary preparation, an integration platform was created for small- and medium-sized bridges using Rhinoceros 3D modelling software and Grasshopper visual programming language to test whether the integration of 3D models and local bridge information obtained through IoT sensing could be performed smoothly. The results showed that the girder expansion due to temperature change was not affected by the temperature change. As a result, it was confirmed that the integrated platform worked smoothly on a PC with general specifications, even with photographic resolution information that allowed confirmation of the shear deformation of the laminated rubber bearings following the expansion and contraction of the girders due to temperature changes.

In snowy and cold regions, snowfall leads to changes in road surface conditions, which can become a factor in traffic accidents. If road administrators can anticipate the road surface conditions on the routes under their management, road safety can be improved. This paper proposesa method to predict road surface conditions using Multi-LSTM, leveraging past road surface and weather data. Specifically, the approach first constructs spatiotemporal data sequences from accumulated road surface and weather data. Next, a prediction model based on Multi-LSTM is developed, integrating temporal and spatial sequence predictions to forecast road surface conditions. Finally, the proposed model predicts road surface conditions using the constructed spatiotemporal data as input. At the conclusion of this paper, experiments are conducted using actual observation data from the Soya region of Hokkaido, and the effectivenessof the proposed method is validated through comparisons with six alternative methods.

In Japan, blue lanes are being developed as part of the improvement of bicycle traffic spaces. However, investigations into the effectiveness of blue lane development conducted so far have not established widely applicable methods. Meanwhile, AI-based image analysis technology has been introduced for motor vehicle traffic volume analysis. However, research specifically focused on bicycle traffic volume surveys using image analysis technology within the country is still limited, and while its utility is recognized, research accumulation in this field has not significantly progressed.This study aims to develop a model for measuring bicycles by type using image analysis technology. Next, a visual survey of traffic volumes will be conducted on both blue lanes and sidewalks before and after blue lane implementation in Sapporo City. Using the results, an algorithm will be constructed to clarify the factors influencing bicycle users’ section selection behavior, thereby conducting an analysis of the factors affecting the positioning of bicycles within designated cycling spaces.

In the field of civil engineering, the proportion of maintenance work has been increasing. Maintenance work, which involves construction on existing structures, presents unique challenges, such as limitations on the scope of operations compared to new construction projects. These challenges necessitate heightened consideration of production efficiency and safety. This paper presents the findings of a study on the development of a DX construction model leveraging AI, using spatial information to address real-world issues encountered in accident cases during maintenance work. Additionally, future prospects for the application of this model are discussed.

This study investigates the trends of AI and digital twin technologies in achieving carbon neutrality and explores the future challenges. Since GHG emission sources are diverse, the role of AI and digital twin technologies is significant in areas such as measurement and evaluation of emission factors or activity level, data integration, and optimization. In the infrastructure sector, research and development on emission calculation and optimization based on BIM/CIM will be required. Moreover, civil engineering structures have a substantial impact on the overall emissions of cities. Therefore, it is necessary to consider emission reductions not only from the perspective of individual structures but also from the urban scale. In the future, complex optimization problems will need to be solved, and the introduction of advanced computer science technologies, such as quantum computing and large-scale databases, will be essential.

Road administrators typically conduct road condition surveys and visual inspections to assess the state of road pavements, but these methods are labor-intensive and costly. To address this issue, the authors developed a simple method for road pavement crack diagnosis using video images captured by a vehicle-mounted camera and deep learning techniques. However, the authors discovered that patching, manholes, and vehicles in front of the pavement were often misidentified as cracks. Therefore, in this study, the existing method was enhanced by incorporating a function to detect patching, manholes, and vehicles in advance, and a new approach was developed to diagnose cracks in road pavements by filtering out multiple geographical features. The results from the demonstration tests showed that the calculated cracking rate improved by approximately 5% compared to the existing method, confirming the effectiveness of the proposed method.

Road managers typically conduct visual inspections and road surface condition surveys to assess pavement damage. However, there are challenges related to manpower shortages and cost constraints. To address these issues, pavement rut detection method using in-vehicle camera images and deep learning has been developed. The authors have also proposed a pavement rut detection approach that utilizes video images and image domain segmentation, successfully reducing false positives by considering the characteristic that pavement ruts occur in the longitudinal direction of the road. However, since the reflection of pavement ruts in video images is influenced by vehicle speed, and t assuming a constant speed, there remains a challenge of inadequate response to false detections caused by rough road surfaces and detection omissions.

In this research, correction method for rut detection that account for vehicle speed was developed. Demonstration experiments confirmed that this approach successfully addresses the issues identified in previous research.

Digital twins require an accurate reproduction of physical spaces in digital form. Although it is preferable to capture information from the physical environment in high density, errors or omissions in the data may lead to accidents or malfunctions. The cross-verifying multiple data sources is necessary correcting and supplementing digital twins. This study examines the feasibility of verifying JARTIC data-indicating traffic regulation for large vehicles- by combining geotagged photo data (e.g., from Google Street View) with road coordinate data. Traffic signs in the photographs were detected using YOLOX, and restricted areas for large vehicles were estimated by integration of the corresponding coordinates with the road network information obtained from OpenStreetMap. These estimates suggests that cross-verification is feasible.

In this study, to reveal the influences of print paths on the mass transport resistance of 3D printed mortals, several tests related to mass transfer resistance were conducted using specimens printed by different print paths. The results of this study show that differences due to the print path are observed in the shape and location of void structure, moisture loss, and moisture penetration resistivity. The differences in mass transfer resistance caused by print paths may be due to changes in the location of continuous voids formed at the intersection of the vertical and horizontal interface of the filaments. In addition, the relationships between cumulative void volume and water absorption and between cumulative void volume and water penetration depth were analyzed, and changes in these relationships due to the print paths were observed. Based on the results of this study, a print path that does not arrange continuous voids, formed at the intersection of the vertical and horizontal interface of the filaments, in the same direction as the water penetration may be effective in ensuring mass transfer resistance of 3D printed mortal.

The use of 360° cameras for capturing spherical images of infrastructures has gained attention to enhance maintenance efficiency. However, common viewers are generally limited to displaying images, making it cumbersome to search for and view specific ones. Additionally, they do not accommodate essential maintenance needs, such as recording and verifying structure names, capture dates, damage locations, and damage types. This study developed an inspection information management system that enables seamless verification of capture locations and damage information by linking the camera poses of multiple spherical images using Structure from Motion. This system utilizes images taken with commercially available 360° cameras and was applied to multiple bridges to evaluate its effectiveness. It is designed for both cloud-based and local environments, allowing administrators to adapt its use to their operational requirements.

During maintenance work on steel bridges, new members are added to or replaced with existing members for repair and reinforcement. By measuring three-dimensional point clouds, it is possible to grasp the shape and dimensions of existing members in detail. However, it takes time to create a three-dimensional model from point clouds, and automatic processing is required to improve efficiency. Furthermore, although existing high-strength bolt holes are often used when replacing parts, there is no established program to automatically extract the center position of high-strength bolts. In this study, we examined the algorithm for searching the center position of high-strength bolts from point clouds and the parameters for the types of high-strength bolts, and demonstrated that the search can be performed with the accuracy required for joining steel bridge members.

Under severe financial constraints, the consolidation and removal of aging road bridges are considered effective measures for long-term regeneration of regional infrastructure. In this context, it is important to gain understanding from residents and users from the perspective of human-centered design. Therefore, this study explores the current challenges and future directions for the smooth consolidation and removal of aging bridges, considering the views of residents and others under financial constraints, based on responses to previous surveys regarding infrastructure maintenance and management. Multivariate and ChatGPT analyses suggest that differences in residents’ attributes and living environments influence their perceptions of these challenges and demands. The paper also proposed a reference formula as a guideline for determining the relative position (horizontal distance) of alternative bridges to be secured during consolidation or removal.