In order to apply multimodal LLM to a life insurance company's inquiry response task, we constructed a benchmark using business data to compare and evaluate the actual performance of multiple models. We evaluated three models, Claude 3.5 Sonnet, Gemini 1.5 Pro, and GPT-4o, focusing on document QA and textualization of image content tasks. As a result, Claude 3.5 Sonnet showed the highest accuracy in document QA, and Gemini 1.5 Pro showed the highest accuracy in the image content text conversion task. In addition, we identified the characteristics of charts and tables in in-house documents that were difficult for LLM to recognize. Through these evaluations, we confirmed that benchmarking using business data yields results that are different from those obtained by general-purpose benchmarks that are publicly available.

Sales of physical books have been continuously declining. A reasons is the difficulty of managing them compared to e-books. To make purchasing physical books more convenient, a simple system that requires no specialized knowledge is considered. In the field of library management, various inventory inspection systems have been developed to reduce the burden of book inventory checks. This study focuses on ChatGPT, a type of AI chatbot. Since ChatGPT enables image and text processing with no-code or low-code solutions, the feasibility of extracting book information from images using OCR and converting it into a list is examined. For verification, spine images of different book categories, including paperbacks, general books, and comics, are captured, and the extracted text data is converted into a CSV file using ChatGPT’s OCR capabilities. To determine the minimum readable text size, images are taken at different resolutions, and OCR processing is carried out.

In recent years, the use of chatbots has been rapidly expanding in corporate inquiry handling and mental health care applications. In particular, chatbot personality in user interaction has attracted attention as a factor that significantly influences user satisfaction and engagement. In this study, we propose a method to customize chatbot personalities by combining multiple base characters based on 16 types of classification by MBTI using ChatGPT. Through evaluation experiments, we verified the degree to which the personalities reproduced by the combination of base characters match the pre-defined target personalities.

In today's "stressful society," many people experience daily stress, yet not everyone has someone to turn to for venting. While AI-based counseling services have been proposed and implemented in recent years, most focus on problem-solving, making them less suitable for simply venting frustrations. This study proposes a ChatGPT-based chatbot specialized in listening to users' venting. The chatbot does not attempt to solve problems but instead generates empathetic and encouraging responses, aiming to serve as an easily accessible listener for users. In an evaluation experiment, the proposed chatbot was compared with a counseling chatbot that actively provides solutions and a chatbot that engages in casual conversation. Experimental results demonstrated certain positive effects of the chatbot in terms of conversational smoothness, progression, and mood improvement.

To realize robots and intelligent systems that comprehend human behavior, it is essential to segment and classify target behaviors in an unsupervised manner. However, conventional unsupervised segmentation methods do not account for individual differences, resulting in reduced segmentation accuracy when behaviors vary among individuals. In this paper, to address this issue, we propose the LIC-GP-HSMM (Latent Individuality Conditioned Gaussian Process-Hidden Semi-Markov Model), an extension of the GP-HSMM known for its high accuracy in previous unsupervised segmentation studies. The LIC-GP-HSMM introduces latent variables (latent individuality vectors) that represent behavioral individuality into the GP-HSMM. These latent individuality vectors can be inferred in a manner equivalent to the GPLVM (Gaussian Process Latent Variable Model). In the experiments, using synthetic data that simulates behaviors with individual differences, we demonstrate that the proposed method enables more accurate segmentation than GP-HSMM. Furthermore, we show that the inferred latent individuality vectors effectively represent the individuality of behaviors.

It is known that there is a discrepancy between cognitive functions measured using objective indicators and those assessed through self-evaluation and that individuals with a larger discrepancy are more likely to be involved in traffic accidents. In this study, we estimated the degree of self-awareness change in cognitive function from in-vehicle sensor data obtained via the Controller Area Network (CAN) using three machine learning models: Logistic Regression (LR), Support Vector Machine (SVM), and Random Forest (RF). As a result, for the "Change in Ability to Assess Traffic Conditions" item from the Self-Awareness Questionnaire on Elderly Driving Characteristics, the values on major roads were 0.507 for LR, 0.646 for SVM, and 0.726 for RF, while at intersections they were 0.767 for LR, 0.808 for SVM, and 0.258 for RF, demonstrating that this characteristic can be estimated with relatively high accuracy.

In this paper, we used a skill model that had learned physical movement to investigate how the differences in skill between novice and expert users would manifest themselves in the results of a virtual reality ball-throwing simulation. The results of the experiment showed that the stochastic time series prediction of the ball-throwing motion using the skill model based on GPDM drew a trajectory related to the hand position information that reflected the user's skill, and that the variance of the trajectory was reduced in the expert case. In addition, we confirmed that the variance of the landing point of the ball also becomes smaller when using a skill model constructed from learning data at the time of proficiency in the ball throwing simulation, just as the variance of the ball throwing motion also becomes smaller in actual ball throwing. The accurate prediction of the success rate of hitting the target in ball throwing using a skill model and ball throwing simulation is a future issue.

In this study, we developed a character conversation simulator by controlling the degree of overlap and investigated the relationship between turn-taking behaviors and conversational atmosphere types. The experimental results show that the major conversational atmosphere types can be determined by changing only the degree of overlap.

This study aims to develop an online dialogue system that enables simultaneous participation in multiple conversations based on the concept of participation frameworks. Specifically, the framework proposed by Goffman (1981) is represented using a graph structure, allowing participants to mutually express and communicate their attitudes by adjusting this graph. The system also incorporates a sound image localization mechanism, whose effectiveness in facilitating conversation listening under multi-dialogue participation scenarios was experimentally examined. The results did not indicate an improvement in information acquisition capabilities due to sound image localization. However, subjective evaluations demonstrated its effectiveness in increasing the clarity of speech perception.

For more natural and smooth interactions between humans and computers, a model that can predict a person's internal state from multiple modalities of information and generate appropriate responses accordingly is desirable. However, current multimodal dialogue systems face challenges such as the high cost of dataset collection and noise caused by variations in data due to users' personality traits. In this study, we apply the meta-learning method called Model-Agnostic Meta-Learning (MAML) to the multimodal sentiment estimation task to verify its effectiveness in addressing these challenges. As a result, we demonstrate that MAML achieves higher accuracy in predicting the internal states of dialogue system users compared to existing methods in multimodal sentiment estimation.

Tsundere is a character type defined by a mixture of cold and favorable reactions in their words and actions, or by alternating between these reactions. In the previous studies on Tsundere and their characters in dialogue models, there are not many examples of quantitative analysis of style and intention and application of such analysis to dialogue models. In this study, in order to construct a dialogue model that reproduces the reactions of Tsundere characters, we take several famous Tsundere characters and analyze their reactions. The purpose of this analysis is to clarify the patterns of reactions and how the Tsundere character's reactions change depending on the situation and the relationship with the other party, and to contribute to the realization of the dialogue model. We constructed a data set in which the reactions collected from the analyzed works can be explained by nine parameters and conducted factor analysis and co-occurrence frequency analysis to extract the distinctive traits of Tsundere character's reactions. From the results of the analysis, it was revealed that reactions such as “teasing” and “picking a fight”, which are frequently used by Tsundere characters, occur under certain conditions and in certain relationships.

This study proposes a feedback system to support smooth communication in group discussions (GD). Designed based on student ideas, this system features alert signs that encourage participants to improve their communication behaviors. Inspired by previous research, including "Presentation Sensei", the system addresses key GD challenges, such as excessive speaking speed, hesitation, monotone speech, lack of eye contact, and closed body language. The alert signs are categorized into four types: (1) Speech Improvement (adjusting speaking speed and hesitation), (2) Non-Verbal Communication Support (enhancing gestures and posture), (3) Active Listening Reinforcement (encouraging nodding and smiling), and (4) Participation and Discussion Flow Management (promoting active speech and time control). By utilizing visual feedback, this study presents a novel approach to improving GD quality and fostering participants' communication skills.

Background: Improving interpersonal skills is crucial in employment support for people with mental disorders. AI-driven dialogues are expected as a new support method. Methods: Participants engaged in AI prompt development and practiced cognitive behavioral therapy (CBT) using conversational AI. Surveys and open-ended responses were analyzed. Results: AI was found useful for self-understanding and emotional regulation but lacked empathy and personalized responses. Conclusion: Developing AI for psychological support for users and assessment tools for supporters is needed.

In recent years, interactive tourist information bots for the purpose of improving hospitality have attracted much attention, and competitions are being held continuously to improve the technology. The author has participated in competitions for dialogue systems for the past four years, all of which have made it to the finals. Although the development of large-scale language models (LLMs) has advanced the technology of tourist information bots, issues such as naturalness of dialogue and improving user satisfaction still remain. The author has introduced ice-breaker elements such as humor, praise, and mini-games, and has studied their impact on user impressions. This paper presents an overview of the dialogue system developed and discusses the issues solved and future prospects.

This research group has collected multimodal information during debates between groups of four university students, analyzed the quality of communication, and considered building a support system to improve communication skills by utilizing the analyzed data. Transformer-based AI, which forms the core of analysis, requires a large amount of training data, but it is actually extremely difficult to secure the required amount of data for student debate experiments. Therefore, in this research, we focus on transcription of debate utterances, and propose to create pseudo data of the transcription using generative AI and use it for training the AI model.

``Evaluation'' is essential in showing advantages of the proposed approach/frameworks in our research and making contributions clear. A questionnaire paper is well used for evaluation in a research field on interactions, such as human-human interaction or human-agent interaction. However, because the interaction is a complex interplay of multiple elements, it is difficult to isolate specific elements, and because of the effects of individual differences and sequential effects, evaluation of interaction studies is still difficult, and questionnaire evaluations are not always absolute. Against this background, we, organizers of this OS, have examined various approaches to evaluation in interaction research. In this paper, we introduce several efforts from the viewpoint of evaluation of interaction research. Through the introduction of studies from multiple aspects, we hope to provide a basis for discussion on evaluation in interaction research.

We address space debris classification using light curves. Light curves, time-series data recording the apparent magnitude of debris observed by optical telescopes, are widely used for identifying debris types. In previous studies, 1D-CNNs were the mainstream classifiers for space debris, where they were pre-trained on light curves generated through simulations and fine-tuned using observational light curves. However, CNNs have limitations in capturing long-term temporal dependencies, and simulated models cannot fully replicate real-world observational conditions, leading to discrepancies between simulated and observed light curves. To address these challenges, we propose a Transformer-based space debris classifier and introduce a pre-training method utilizing observational light curves. Experiments show that the F1 score of the Transformer classifier improves by 30% compared to the baseline 1D-CNN. Additionally, pre-training with observational light curves leads to an improvement of 7.8% in F1 score.

To deliver comfortable driving experience free from unpleasant noises and vibrations, we are building an efficient and globally optimized development process by optimizing and visualizing feasible region through CAE. We developed surrogate models using simulation results of powertrain system as training data to reduce CAE simulation time. However, in early stage of development where system structures and specifications frequently change, effective use of surrogate models is limited because the repeated re-collecting of training data costs a significant amount of time. In this study, we suggested a hybrid approach using LSTM to represent non-linear components that cause simulation difficulties in a powertrain multibody dynamics model. As a result, we accurately reproduced CAE simulation results across 16 conditions including different powertrain systems (FF, FR, HEV) and drive scenes. A maximum 90% reduction of simulation time was confirmed, and we applied this approach to optimization of drivetrain vibration performance.

In analyzing human work behaviors at production sites, traditional methods primarily rely on visual observation, which is time-consuming and labor-intensive. To address this issue, we proposed a method that uses a Gaussian Process Hidden Semi-Markov Model (GP-HSMM) to segment workers’ continuous motions into discrete work behavior classes in an unsupervised manner. However, since this method relies solely on motion data, it has a limitation where motions with similar characteristics, even if they represent different behaviors, tend to be segmented into the same class. To overcome this issue, this paper proposes a novel method that incorporates multimodal information—including object data such as tools and components used by workers—in addition to motion data, for unsupervised segmentation. This approach enables more accurate segmentation, distinguishing behaviors involving similar motions based on the objects used. Preliminary evaluations were conducted using a simple work behavior dataset. The results demonstrated that the proposed method, utilizing multimodal information, achieved higher segmentation accuracy compared to the conventional method that relied only on motion data.

In this study, we propose a recommendation list generation method that considers the mutual influence between items displayed simultaneously in a recommendation list. Traditionally, ”bundle recommendation” has been known as a method that aims to optimize the entire recommendation list while considering inter-item influence. However, few studies explicitly address the mutual influence within the list, i.e., the range that users actually see. This study tackles this issue by incorporating users’ behavior of ”comparing simultaneously presented items while making a selection” into the model, thereby improving prediction accuracy. The proposed method utilizes a Transformer- based model to predict item scores while considering the features of other items displayed simultaneously in the list. Furthermore, to address the challenge of the combinatorial explosion in optimizing the recommendation list, we propose an algorithm that employs a greedy method to generate recommendation lists within a feasible computation time. Finally, numerical experiments using open datasets are conducted to validate the effectiveness of the proposed method.

Behavioral cloning, a branch of imitation learning, has been attracting attention as a method for learning control laws from the demonstrations of experts in complex systems that are difficult to control using rule-based methods, such as autonomous driving and robot arm control. On the other hand, behavioral cloning has a significant challenge of vulnerability to covariate shift: performance degradation due to states that are not included in demonstrations. In this study, we propose a method of adding P-control-based calibration to the control input by behavioral cloning. Numerical experiments using a continuous stirred tank reactor (CSTR) model were conducted to confirm the behavioral cloning problem under covariate shift and to verify the effect of the P-control-based calibration.

Using a multi-agent simulation, this paper compared prediction of the economic impact of a basic income (BI) and a negative consumption tax. Simulation models with modifiable inheritance, income, and consumption taxes were employed to analyze the results of the prediction. The parameters of each tax system were simulated under a variety of scenarios. The results showed that consumption taxes were likely to have a pronounced impact on the saving gap. The results also suggested that income taxes were likely to have a direct impact on the aptitude rate. Furthermore, it was cleared that excessive provision of BI leads to disparities in household savings.

Annual securities reports serve as a crucial resource for analyzing corporate operations, with the Risk Factors section encompassing a wide range of information, including the underlying causes and potential consequences of risks, significant events deemed material by the company, and the perspectives of report authors. In this study, we employ BERT and fin-BERT to vectorize and cluster these riskrelated disclosures, thereby elucidating how companies conceptualize and assess risk. Subsequently, we examin the relationship between the resulting clusters and various economic indicators. Specifically, we classify the events referenced within the risk items and assessed their reliability. As a result, we identified distinct groups of companies that recognize specific economic indicators as critical risk factors.

We collected and formatted the minutes of plenary sessions held between April 2019 and March 2023 from 47 prefectural assemblies across Japan to construct the Prefectural Assembly Minutes 2019-term Dataset. This dataset is organized by speaker name, with publicly available election-time information assigned as speaker attributes for assembly members. This paper provides an overview of the dataset by comparing it with the previously constructed 2011- and 2015-term datasets, analyzing assembly member composition, utterance volume, and characteristic words identified using TF-IDF.

The Tokyo Metropolitan Assembly follows a batch questioning and answering format, where multiple questions are asked consecutively and answered collectively. Despite being available online, the minutes are often difficult to navigate when searching for specific answers. To address this issue, the Question Answering-2 task in NTCIR-17 QA Lab-PoliInfo-4 aimed to develop a system that automatically generates summaries of responses to user-input questions. However, the system built using the provided training data contained factual errors, particularly in numerical information such as years and months. This study addresses one cause of these errors—relative date representations—by aligning them with the date of the plenary session, thereby reducing inaccuracies. Furthermore, we manually evaluated the improved summaries and analyzed the types and patterns of factual errors.

This study examines online harassment against Diet members in Japan, with a particular focus on gender-based differences. We collected replies, mentions, and quote posts on Twitter directed at members of the lower and upper houses from the period following the 2022 upper house election to February 2023. Using machine learning models trained on human-labeled data, we classified these posts as abusive or non-abusive. Additionally, we assessed the sentiment (positivity or negativity) of legislators' posts through semi-supervised learning. Our analysis of gender differences in the likelihood of receiving abusive language revealed that posts directed at female lower house members with fewer terms in office were more likely to be abusive than those directed at their male counterparts. Furthermore, while negative posts by legislators were generally more likely to attract abusive responses, this tendency was more pronounced for female lower house members compared to males. However, these patterns were not observed for members of the upper house. These findings provide valuable insights into the issue of women's underrepresentation in Japanese politics and contribute to the broader literature on gender stereotypes.

With the spread of cashless payments, the market for mobile payment apps is becoming increasingly diverse. The growing availability of apps has resulted in increased complexity in the quality factors considered by consumers. However, most of the existing studies on mobile payment apps focus on pre-use expectations rather than on the actual use evaluation of the apps. It is not clear what quality factors which consumers perceive after use are related to the success of the apps. This study exploratorily identifies the quality factors (topics) that attract consumers to choose payment providers using app review data that reflect the actual use evaluation of the app. Specifically, using data from 2019 to 2024 for five major payment apps, we conducted a structural topic model analysis. This allows us to discuss the differences in topics emphasized by each provider and the dynamics accompanying market maturation. The results reveal that the review rating is strongly related to the usability of the apps and the cumbersomeness of the UI/UX. In addition, the topics of quality expectations for point program and UI/UX cumbersomeness increase over time.

Thumbnails on e-commerce search result pages play a crucial role in influencing user click behavior. However, there has been limited research on identifying which images within the same product are most effective at driving clicks and how to optimize their selection. In this study, we conduct experiments on a real-world fashion e-commerce platform to analyze whether different thumbnails lead to variations in click-through rate (CTR) and whether the most effective thumbnails differ based on search queries. In addition, we apply the Multi-Armed Bandit algorithms to the collected data to explore the potential for optimizing thumbnail selection strategies tailored to search queries. Our experiments reveal significant differences in CTR across thumbnails and demonstrate that the effective thumbnails vary depending on the gender associated with the search query. Furthermore, applying Thompson Sampling based on gender-specific CTR distributions improves reward performance compared to other methods.

In recent years, with the spread of the Internet, more and more people are booking hotels online. In particular, online travel agencies have simplified hotel bookings by providing convenient functions, and some services now have a large number of users. For such a large-scale service, it is important not only to acquire new users, but also to maintain relationships with existing users in order to gain long-term profits. However, a one-size-fits-all approach to all users is inefficient. In this study, we tested a method to predict when a user will make his/her next booking based on the user's past booking and cancellation history and demographic information, and to appropriately estimate the probability of such a booking. The results of evaluating multiple models and probability calibration methods using real-world data showed differences in prediction accuracy and probability calibration effects among the methods, but also suggested that further study is needed to improve performance, including recall rates.

In recent years, educational reforms have highlighted the growing importance of English language learning, emphasizing the need to acquire English proficiency required to pass the Eiken Test in Practical English Proficiency (Eiken). In this study, we developed a recommendation system that adjusts question content based on users' learning volume and answer tendencies and applied it to English learning aimed at passing the Eiken test. Using the learning log data and Eiken pass/fail results obtained through this system, we evaluated its effectiveness using T-learner, a causal inference method. The analysis suggested that the use of this system could improve the probability of passing the Eiken test. Future studies need to verify whether similar effects can be observed in different periods and under various learning conditions. The results of the analysis suggested that the use of this system may improve the probability of passing the EIKEN. Further studies are needed to verify the effectiveness of this system in English language learning by using other log data.

In recent years, the advancement of diffusion models for text-to-image generation has enabled users to quickly produce visual content from textual prompts. However, these models may still fail to capture specialized expertise in creative domains such as advertising, where an art director’s ideation process is crucial for effective communication and brand storytelling. This study proposes a method that fine-tunes a large-scale model using data based on an art director’s conceptual thinking, including brand insight, metaphor formulation, and design rationale. We collected detailed textual representations of real advertising projects and performed supervised fine-tuning. An online evaluation with 1,000 participants was then conducted to compare our proposed approach against a baseline model, using eight key metrics related to creative quality. The results showed significant improvements (5%–12%; p<0.001) in overall appeal, visual impact, metaphorical expression, and other dimensions. Our findings suggest that embedding an art director’s thought process within the training data can help generate more compelling and conceptually rich visual outputs, thereby bridging the gap between automated image generation and professional-level art direction. Such an approach highlights the importance of domain-specific knowledge in shaping AI-based image generation workflows.

Divergent thinking in engineering design involves brain regions supporting memory retrieval and imposing cognitive constraints. Use of generative AI for such thinking is becoming more widespread, yet underlying neural mechanisms during collaborative scenarios with AI remain poorly understood. This study investigated performance and neural correlates in participants who completed a creative thinking task individually or with AI assistance when proposing product concepts addressing a social problem. Thirty-six male students took part in this study, and EEG data were recorded to examine neural activity during tasks. Although performance of participants was improved with using GPT-4o, activation of the left inferior frontal gyrus under AI-assisted conditions suggested potential suppression of divergent thinking. These findings deepen our understanding of neural processes in AI-supported collaboration and may inform the development of novel engineering design support methods. Consequently, they highlight the importance of investigating individual differences in AI-mediated creative performance.

The number of IT administrators with expert knowledge tends to decrease worldwide because Public Cloud, which provides easy management of IT systems, is becoming more popular. The purpose of this study is to propose a method to generate scripts that customers without expert knowledge can use to automate managing IT systems. To generate scripts that use IT system’s API, we propose the RAG method using sample codes. Each sample code uses only one API. Generative AI generates a script using multiple APIs by using these sample codes. And we provide Chatbot that asks users to input parameters as easily as to input parameters of Public Cloud services. Our prototype for managing block storage showed that when we use GPT4, we can execute the generated script with little modifications. We get prospect of decreasing the time to create scripts with 75% of less than the time to manually create scripts.

Large language models (LLMs) developed in China are required to "adhere to the core socialist values." Previous studies have constructed sensitive questions to examine this issue. This study aims to further elucidate the details of censorship by first introducing the Basic Requirements for the Security of Generative Artificial Intelligence Services, published in February 2024. Next, we evaluated LLMs using a benchmark questions created by the China Electronics Standardization Institute and Fudan University. The models analyzed included major Chinese open-source LLMs, derivative models fine-tuned for the Japanese market, and Western LLMs.The analysis revealed evidence of censorship in Chinese models and their derivative versions. The findings suggest that users of these LLMs should be aware of the censorship-based fine-tuning applied to Chinese models and conduct thorough checks to ensure their suitability for specific applications.

This study introduces a novel unlearning technique to address the unauthorized reproduction of copyrighted materials by large language models (LLMs). Although unlearning techniques have recently been introduced as an efficient, low-cost solution for addressing copyright infringement, they require access to model parameters and are therefore not applicable to black-box LLMs. In this study, we propose negative in-context learning, an unlearning method that can be applied for black-box LLMs based on in-context learning. In-context learning allows LLMs to learn knowledge given a few examples without access to model parameters. In contrast, negative in-context learning makes LLM unlearn knowledge by providing negative in-context examples made by using contrastive decoding. By learning these negative in-context examples, LLMs can selectively forget specific knowledge without updating model parameters. Experimental results show that the introduction of negative in-context examples leads to a significant decrease in BLEU, Jaccard, and ROUGE-L scores, confirming that our method effectively interferes with the model’s recall of the original information.

Ensuring fairness in large language models (LLM) is one of the challenges in AI governance. The purpose of this paper is to utilize the characteristics of LLM discovered using psychological methods in existing research, and to find the possibility of a new index of fairness. One of the characteristics of LLM is that when instructions imitating a specific gender or race are given to LLM, unexpected differences occur in the percentage of correct answers to specific questions based on the instructions. By using this characteristic, we show the possibility of using it as an index to measure hidden stereotypes inherent in LLM. As another characteristic, higher STICSA scores (“anxious ”) were associated with a higher proportion of LLM biased responses. Based on this relationship, we show that the STICSA score can be used as a bias evaluation index for various inputs. As a conclusion of this paper, we discuss the significance of applying these psychological characteristics of LLM as a fairness evaluation index in AI governance and find its possibility.