Understanding the nature of nonverbal communication (eye contacts, face expressions, body postures, hand gestures, body motions, etc...) is one of the core issue in behavioral neuroscience. In this study, we demonstrated the data-driven dynamical segmentation of the body expressions in free moving small non-human primate, common marmoset. We developed a new marker-less 3D motion tracking system optimized to marmoset. Then, we proposed unsupervised segmentation using a Gaussian process-hidden semi-Markov model (GP-HSMM). As a result, we succeeded to classify three types of marmoset feeding behavior (high position feeding, low position feeding, and low position feeding with hands) only based on body parts positions, face direction, and body angle information. This result suggested that proposing system could represent high versatility to quantify the animal nonverbal body expressions without qualitative teacher labels.

In recent years, with the development of IoT and sensor technology, various data can be acquired. In this case, it is expected to establish analytical methods capable of extracting the characteristics of relevances of each variable of multimodal data. In this study, time series variables with different dimensions on the same time axis are converted to color change images as RGB which is the three primary colors of light, and Convolution Neural Network(CNN) is applied to this. Next, we propose a method to perform more effective feature extraction by converting the image using XYZ, Lab color space reflecting the color visual stimulus with RGB as the base. We compared accuracy with existing classification method and showed the effectiveness of the proposed method. Moreover, by converting time series in various color spaces. It is suggested that higher performance feature extraction can be realized than when processing each variable as independent.

We have been developing IoT sensors for the evaluation of the comfort levels of real estate building properties. In this paper, we additionally evaluated thermal insulation performance and sound performance. In addition, we added a PM2.5 sensor. We are now conducting a large-scale experiments using $60+$ properties to assess the system' usability and reliability.

When searching for a rental property, information such as rent, breadth, time to station and age of a building is used. These indicators are quantitative data and can be compared. However, some conditions such as quiet and sunny are often described in comments in the remarks column are difficult to compare because there is ambiguity in the state recognition of each property. Therefore, indices that can quantitatively evaluate noise and sunrise are required. For indexing, it is necessary to collect data, but because the target is vacant, a data collection device can not use power from the outlet. Therefore, it is necessary to construct an IoT device capable of sensing environmental information without supplying power from the outlet. In this paper, we develop an environmental information sensing device under the constraint that there is no electricity in the vacant rental property and a no Internet environment. As a result of the demonstration experiment based on the cooperation of real estate agents, we showed the possibility that the proposed device can collect data.

This paper proposes a route search system for supporting users to find their living place based on railway information. When we want to find a living place, the access to the place of work or school is usually considered. However, usual railway search systems are designed to search a route from departure station to destination at a specified time. Given the destination station and conditions such as commuting time and the number of connection, the proposed method finds multiple stating stations satisfying the conditions. This paper describes the search algorithm and explains the feedback about a prototype system from the salesperson in a real-estate company.

This study constructs rent prediction models with/without floor plan images in order to validate whether such images contribute the prediction accuracy. In addition, applications of PCA (principal component analysis) and convolutional neural network are considered as a feature extractor from floor plan images. The prediction accuracy is measured using properties of 90,000 rental housings in Tokyo. In the experimental results, the root mean squared error values of the prediction model with floor plan images and PCA tend to be higher than without floor plan images. This suggests that the use of floor plan images contributes to accuracy of rent prediction.

The apartment loan is a loan for rentals such as for condos, apartments. This loan is a very large loan which is the account for a percentage of more than 10 percents of the whole banks' loan. However, a risk model of the apartment loan with the appropriate accuracy has not been provided in Japan mainly due to the lack of data. Thus, in order to develop the risk model, we analyzed the duration in which a vacant room become occupied by a tenant by using the housing information website data, as a first step. As a result, it was found that (i) This duration can be explained by the geometric distribution, and (ii) The mixture geometric regression model considering nonlinear effects can describe the data properties. In addition, coefficients of this model roughly consistent with the empirical common senses.

This presentation describes attempts to innovate user experience (UX) in real estate field by artificial intelligence technologies. (1) We applied deep learning to property photographs to improve UX, both by implementing automated annotation of photographs, and by creating a new UX "search properties simply by holding a smartphone camera over the street corner". (2) For the UX problem of real estate sales transaction that it is difficult to properly pricing, we developed a service that provides reference prices of apartments in Japan on the map, using a reference price calculation algorithm by machine learning.

Recently, in the field of brain science, it has been reported that brain network related to fear sympathetic nervous activity was clarified by fMRI measurement. However, fMRI measurement under daily circumstances is difficult. If it is possible to detect fear with electroencephalograph in daily circumstances, objective fear indicators will be determined and used to control fear in the entertainment field and to reduce fear in medical services and so on. In this paper, we analyzed the characteristics of electroencephalogram data during fear and no fear by using frequency analysis from two experiments using VR horror game and experiments using horror movie with a dry electrode. As a result, there was significant difference in power spectral density between alpha wave fear and no fear.

The purpose of this study is to elucidate a method that uses somatosensory evoked potential as an estimation index of workload. Realtime estimation of the workload in the brain by AI system will enable to increase the concentration of people by keeping a moderate tension. The quality of interaction between the system and people could be largely increased by intervention of interacting when people lost the concentration. We measured and analyzed the activity of the brain during the execution of the N-back task while presenting somatosensory stimuli to 10 participants without inpairment. As a result, amplitude modulation was seen with N9 which is one of somatosensory evoked potential components. In addition, a t test was performed on it, and a significant difference was observed. The Bonferroni method was used to correct the p value.

Human have the ability to consciously aware of the presence or absence of their memory. This kind of the ability called metamemory plays important roles in human cognition. We aim to evolve arti cial neural networks with neuromodulation, that have a metamemory function. Our constructive approach is based on the repetition of evolutionary experiments, analysis of the evolved networks and re nement of the measure, so as to reducing the gap between the functional properties of behavior and subjective reports of phenomenal experience. This paper reports on the current state of the approach based on the evolutionary experiments with a delayed matching-to-sample task.

Enumerating all Maximal Satisfiable Subsets (MSSes) or all Minimal Correction Subsets (MCSes) of an unsatisfiable CNF Boolean formula is a cornerstone task in various AI domains. This paper considers MCSes enumeration with a SAT solver. We aim to develop a procedure which outperforms several MCSes enumerators proposed so far. The paper presents a basic enumeration procedure and compares it with a state-of-the-art enumerator Enum-ELS-RMR-Cache. The experimental results show that the proposed procedure is more efficient than Enum-ELS-RMR-Cache to solve Partial-MaxSAT instances but it is inefficient than Enum-ELS-RMR-Cache to solve plain MaxSAT instances.

Constraint Satisfaction Problem (CSP) is the combinatorial problem of finding a variable assignment which satisfies all given constraints over finite domains. CSP has a wide range of applications in the research domains of Artificial Intelligence and Operations Research. XCSP3 is one of the major constraint languages that can describe CSPs. More than 23,000 instances over 105 series are available in the XCSP3 database. In 2018, the international XCSP3 competition was held and 18 solvers participated. This paper describes the under development CSP solver sCOP and its results on the 2018 XCSP3 competition. sCOP is a SAT-based solver which encodes CSPs into SAT problems and finds a solution using SAT solvers. Currently, sCOP equips the order and log encodings, and uses off-the-shelves backend SAT solvers. We registered sCOP to two competition tracks CSP-Standard-Sequential and CSP-Standard-Parallel of the 2018 XCSP3 competition and won both tracks.

Recently, IC card systems are popular and their log data are used for analyzing human behaviors. In this paper, we extract user behavior patterns using Non-negative Multiple Matrix Factorization (NMMF) and propose an analysis method to analyze patterns and attribute information by decision tree learning using clustering factor matrix. We examine our proposed method using actual entry and exit data and confirm the effect.

Hybrid systems are dynamical systems involving continuous and discrete changes. Various models such as cyber-physical systems and control systems can be discribed as hybrid systems. We are developing HydLa, a modeling language of hybrid system and its symbolic simulator HyLaGI. HyLaGI performs exhaustive search to find time points of discrete changes, but its computation becomes a bottleneck for some programs having a large number of guarded constraints. In this research, we propose a efficient search method using a branch-and-bound algorithm and implement a prototype. The result of simple experiment shows that our approach reduces most of the search cost in the motivating example.

HydLa is a language for modeling hybrid systems - dynamical systems that intermix discrete and continuous behavior. Its adoption of a constraint-based framework benefits the language in various ways, such as allowing a concise representation of systems and performing error-free high precision simulations. In spite of all the advantages, the computations among sets of constraints become a bottleneck in simulation time when handling some large-scale models. The purpose of this research lies in providing a method of improving the computational efficiency and the scalability of the language and its simulator. This is achieved by considering the monotonic aspects in HydLa models to dynamically reduce the size of guarded constraints. Results show that this approach is effective for models that contain multiple objects represented by guard conditions. As for the model evaluated in an experiment in the research, the overall computational time has reduced to approximately half the original length.

We developed an English verb learning system for elementary and junior high school students by acting the verb meaning on their own and showing a character’s acting in the game. The students learn English verbs by listening to the pronunciation of English words, understanding the meaning, pronouncing the verb, acting on their own, and watching the character’s actions that was reflected their own actions. The result shows that the system is effective for learning English verbs by conducting an experiment using elementary school sixth graders as subjects.

In this paper, VR-based tennis training support system using HMD is proposed. In this system, user can practice the swing by shaking the racket at a virtual tennis court constructed in the virtual environment. By using a HMD and a tracking sensor attached on the real racket, the head position and the racket position can be measured. From measured information, interaction between virtual racket and virtual ball is calculated, and this calculation result are show for user as feedback in the virtual environment. Experimental results verified that using HMD is more effective than 2D video display for showing trajectory of a virtual ball.

We propose a novel virtual try-on method based on Generative Adversarial Networks(GAN), which uses 3D surface model of body. In existing GAN-based methods(CAGAN, SwapGAN) sometimes do not work on a human image of rare posture. In our proposed method, by using DensePose to estimate a point corresponding to 3D surface model for each pixel point of 2D image, 3D surface based information is incorporated into our model. Therefore, it is possible to change clothes of people in various postures. Our proposed method uses a coase-to-fine strategy. First, {\it Parts Generation Network} generates parts and they are mapped to 2D image to produce coarse dressing image. After that, {\it Refine Network} refines the coarse dressing image. In our experiment, we show the result of the proposed method and our method has effect on rare postures by comparison with existing methods.

In thinking about human-computer interface, it is important to know how human beings recognize, judge and act. Therefore, Card et al. devised a model of human cognitive processing process called Model Human Processor (MHP). However, this MHP predicts processing time from empirical rules, and is not much considered from the aspect of brain activity. In this study, we focused on readiness potential (RP), which is a characteristic brain wave, and verified MHP from the aspect of brain activity by measuring EEG when performing basic user performance. Experimental results show that MHP can be roughly explained from the aspect of RP, which is a characteristic brain activity.

The session contains two parts: 1) Brief introduction to the ‘Creative Genome by AOI TYO Holdings’ that is a study on the relevance between storytelling & impressions, that I have mentioned in Jsai2018. 2) A latest prototype by ‘Relevant Intelligence Communication Consortium’ (composed of Synergy Marketing Inc, Amana Inc, and AOI TYO Holdings Inc.), focused on studies about psychological suggestion. In this app we integrated the consortium's two databases, ’Societas’ by Synergy Marketing (for Human Values) and ’Creative Genome‘ by AOI TYO Holdings with connecting each data-definition in order to model the relevance between one’s value-type and the bias for one’s preferable story-type. With utilizing this model we so far aim to define original emotional data labels onto all affairs and things in the world. Additionally I would refer to our HI (Human Intelligence) study which is the foundation of this prototype production.

Communication based on understanding of human is very important. However, it is not always important to understand the mechanism. In this paper, we present ‘Mind design’ to extract only emotional value (sensory sensibility value) from enormous information.

The author has surveyed and analyzed the elements and techniques in kabuki from the viewpoint of narrative generation, in particular in the framework of a Geino Information System (GIS) with an Integrated Narrative Generation System (INGS). The objective of this paper is to select two kabuki elements, person and story, and two kabuki techniques, naimaze and tsukushi or zukushi, to consider their combinatorial, or multiple, usage. As a result, the author presents an example that contains the automatically generated and edited descriptions of an actor, the synopses of two works, and a scenario edited using the techniques of naimaze and tsukushi. This example corresponds to a story in a certain point of time in the circular process of GIS with INGS.

The purpose of this research is to propose a mechanism to generate advertisement plots in TVCM automatically. TVCM is a short video. However, TVCM has some common structure, and in that respect, it has a story subject to the theory of story. In this paper, we focused on stories in advertisements and proposed an integrated story generation system and an advertising plot generation method using “Creative Genome.” The former is a narrative generation system that integrates narrative theory and artificial intelligence technologies and the latter is a data set describing the influence on the recipient’s emotion and the production method for the existing CM. The proposed method gives a specific generation procedure and constituent elements to the latter by the former knowledge system. Through the above, our proposed mechanism became possible to generate advertisement plot by fusing “Creative Genome” which is research based on the theory of narrative, and various results of INGS.

The purpose of this study is to use the folk tale’s motif in narrative generation. A motif is a unit including a major act made by a major character composing it and an action corresponding to it in a certain folk tale. We have developed a motif program for these purposes. This program is based on the folk tale recorded in “Nihon Mukashibanashi Taisei”. The motif program has a tree structure data based on the structure of the original folk tale and a mechanism to convert the tree structure data into a narrative tree that uses it in narrative generation. With the above developments it became possible to create a story using the folk tale’s motif.

A comparative analysis of labor requirement among several cropping systems is important when crop rotation systems are designed to reduce production cost. We proposed a comparative analysis of labor requirement among vegetable cultivation systems based on Agriculture Activity Ontology(AAO). AAO was used to establish correspondences among terms to describe work in labor force survey as a core vocabulary. Working time aggregated on each AAO - defined activity enabled to compare the labor requirement among vegetable cultivation systems.

Terminology in the agricultural field are difficult to process data automatically since they are often used in various expressions. It is especially difficult for crops whose names vary depending on the agriculture activity, edible parts or cultivation method in which various names are used. This study suggests semantic analysis using agricultural knowledge graph to solve these problems. This study also confirmed that the use of knowledge graph's semantic structure not only enabled the extraction of agriculture activity and the crop name clearly, but also enabled agriculture-specific analysis.

When evaluating Dashi in Japanese cuisine, the chef uses the word umami and miscellaneous taste. Umami has been used as an important factor. On the other hand, there is no definite definition of miscellaneous taste, and quality evaluation viewed from miscellaneous taste has hardly been performed. In this research, therefore, the objective was to clarify the characteristics of the Konbu-Dashi which the chef is conscious of miscellaneous taste. We established a quality evaluation method of Konbu-Dashi by using Quantitative Descriptive Analysis (QDA) used to express the overall taste perceived by human beings, and made a comprehensive evaluation. As a result of the multi-spectroscopic analysis, the characteristic of Konbu-Dashi that felt miscellaneous taste had "little umami, relatively much minerals and saccharides" was confirmed. In addition, in the QDA, evaluation terms were created by using 32 kinds of Konbu-dashi to capture the characteristics of both delicious and not delicious Konbu-dashi. From the results using the QDA, 1) focusing on fragrance and flavor as an approach to harshness, 2) the necessity to consider the influence of minerals such as potassium, etc. were derived. From now on, it is necessary to develop machine learning and analysis using AI.

In order for cattle farmers to detect calving sign beforehand and assist it to reduce risks of fatal accidents, recent work proposed a pattern recognition approach based on video information from cameras. To reduce false alarms given by the pattern recognizer, crowdsourcing can be used for double-checking the result of the automatic event detection. However, calving sign detection from videos is not a common task for crowd workers, where most of them are not experts of cattle farming; it is therefore not clear about how microtasks can be designed for the workers and thus their answers contribute to better prediction accuracy. In the present study, a calving detection system of beef cattle is designed aiming for real-world deployment. Exposure of the amnion and allanto from the buttocks of cattle is considered as a sign of calving and identified by the crowdworkers in microtasks designed. As a result of simulation evaluation of detecting two birth prognostic events, precision obtained when using only the pattern recognizer were 0.049 and 0.22, whereas in the case of using crowdsourcing it improved to 0.91 and 0.83, respectively. This result demonstrated that verification of the pattern recognition result by crowdworkers successfully reduced detection errors.

“Nudge” advocated in behavioral economics is known as a strategy that guides people to health and well-being with “a little modest way.” Thaler is known as the Nobel Prize in Economics winner in 2017 with this achievement. Now that redesign (reconstruction) of social system is required due to the rapid development of information technology such as AI and IoT, in this research we regard “nudge” as “mental navigation” through implementing as a “Nudge Agent” by using the basic technology of Human-Agent Interaction (HAI) which we have been carrying out. Specifically, by using HAI-based technology, we construct a “Nudge Agent” that can manipulate the cognitive environment (selection architecture) that directs human decision-making and conduct field experiments at actual home and commercial facilities. Then, we will examine whether this agent can “guide” people’s decision to health and happiness to some extent.

With the rising of information technology, we can conveniently harness many others' opinions. However, it remains unclear how we utilize them. Online review sites, such as Amazon.com, display a lot of products. Especially, it is well-known that products could have two types of consumption mode: hedonic or utilitarian. In this paper, we examined how these product types affected people's behavior on others' opinions. Concerning others' opinions, we focused on a rating distribution. In our experiment, participants conducted a binary product choice task. The task composed of single product name and two rating distributions which had similar average of ratings but different variance of ratings (high / low). The results showed that product types affected people's choices on rating distributions. For hedonic products, participants tended to prefer high-variance rating distribution more often than for utilitarian products. We discussed how the results emerged and illustrated how our findings could 'nudge' people.

Based on previous nudging and embodied cognition approaches, we proposed a new type of nudge: manipulating people’s body posture by changing a decision-making environment could lead decisions in a certain direction. In this paper, our purposes were to investigate whether a forward posture really affect decision making, and if yes, what types of decisions could be likely to be influenced. Through two behavioral experiments, we found that participants’ forward posture could enhance their decisions, especially willingness to change their own attitude. Our findings will provide an opportunity to consider the design of the real-world environment and artifacts for leading people to their predictable decisions.

The problem that autonomous agents and humans cooperate to solve one task is one big theme in the field of Human Agent Interaction. We are interested in cooperative agents that improve human plans naturally not just assisting human plans. As a situation where such an agent is useful, there is situation where a person and an agent have only part of information to achieve tasks. We developed a framework of agents that cooperates on the premise that humans and agents implicitly communicate information through actions with each other in this situation. We formulated the target situation as a Human-Agent Team problem and developed an agent planning method for this problem. This method is composed of two parts, the model that human's predict other purpose and the planning algorithms under the models, which are realized by improving existing methods called CIRL and Bayesian Inverse Planning, respectively. We evaluate our method through participant experiments that humans achieve simple tasks with autonomous agents and confirmed that our method has good performance of collaborative work.

In this aging society, to improve convenience of electric wheelchairs is an issue. Especially how to adapt automatic operating systems, which has developed in recent years, to electric wheelchairs is important. In this paper, we propose Mizusaki system, a screen agent that informs drivers gain changes in an electric wheelchair with automatic gain adjustment system. Mizusaki system, based on the Nudge effect, aims to improve operability and safety not with directly teaching how to drive, but with presenting surrounding environment and internal information. Since Mizusaki system uses a screen effect that is easily recognized even in peripheral vision such as vection and color, it can attract the attention of the driver even while driving, and expect that it will not request gazing at unnecessary time. In designing Mizusaki system, in order to investigate the optimum presentation timing, we conducted an evaluation experiment of the system by changing the presentation timing. As a result of the experiment, we found that when presenting at an earlier timing, drivers give better impressions.

Poor trust calibration in human-AI collaboration often degrades the total system performance in terms of safety and efficiency. Existing studies have primarily examined the importance of system transparency in maintaining proper trust calibration, with little emphasis on how to detect over-trust and under-trust nor how to recover from them. With the goal of addressing these research gaps, we propose a novel method of adaptive trust calibration, which consists of a framework for detecting the status of calibration and cognitive cues called ``trust calibration cues''. Our framework and four types of trust calibration cues were evaluated in an online experiment with a drone simulator. The result showed that presenting the simple cues at the time of over-trust could significantly promote the trust calibration.

Animation has been playing an important role in the economy and culture. However, making animation is a hard work and costly. To solve the problem, we propose new solution for animation video generative model, which of baseline is Cartoon-GAN. Cartoon-GAN is a kind of image to image style transfer network, and it has attained to generate cartoon style image from real-world scenes. In deep learning, there are usually two ways to process videos: 3d convolution or 2d convolution added with temporal processing. However, existing method doesn’t achieve enough smoothness in the cartoon-style video making task. For our cartoon-style transfer task in video to video, our new solution is to use each two image frames and optical flow as an input for the generator. In this paper, we generated cartoon videos by adopting optical flow, which is effective to predict object motion.

By evaluating the effect of the video content, the content creator can know how the viewer felt his work. In this evaluation process, traditionally self-report type questionnaire data has been used. However, since this method involves participant bias, experimenter bias, or human diversity, accurate evaluation is difficult. Also, in order to eliminate these as much as possible, it is necessary to obtain an appropriate subject, which is high cost. In order to deal with these problems, this study proposes a method to complement physiological information in addition to questionnaires when evaluating emotional response to video contents. Specifically, it is a combination of subjective self-report questionnaire and heart rate variability. It is the viewer of short television commercials and news programs that are watched via the VR headset platform. Analysis was done using support vector machine and random forest. As a result, effective models and analysis results were obtained.

Automatic music composition is one of the most difficult and attractive challenges in the artificial intelligence (AI) field. In order to tackle this challenge, an approach using interactive evolutionary computation (IEC) is drawing attention because IEC takes human emotions into consideration. We have proposed an automatic music composition system based on IEC with a surrogate model called an evaluation model. In the previous study, the model is constructed with a Variational Recurrent Auto-Encoder (VRAE) to achieve quantitative evaluations. However, it is not easy for a simple VRAE to map tunes' features into a meaningful latent space regardless of their lengths. This paper focuses on the way to map tunes with different length into a good latent space and the application for IEC. The evaluation model employs a hierarchical VRAE called segmented VRAE. The experiments are carried out to show the effectiveness of the proposed method.

Understanding the creation of human by artificial intelligence (AI) is increasing; however, those are still known as one of the most difficult tasks. In this research, we are challenging for the understanding of four-scene comics by AI. To aim at this challenge, we use a novel dataset what is called “Four-scene Comics Story Dataset", which is the first dataset made by researchers and cartoonists to develop AI creation . We focused on illustration touches of comics which is determined by cartoonists. First, we applied autoencoder (AE) models to this dataset to get distributed representations, then applied classifiers to that and predict a touch. The prediction offers an indirect measure of the distributed representations. The effectiveness of the proposed method is confirmed by computer simulations taking data of various pattern of removing parts in koma images of the four-scene comics story dataset structure as an example.

In this paper, we represent a realization method of behavior prediction by using behavior log in the hospital and attribute data of each nurse. In recent years, the busyness and complexity of medical staff’s work has become a social problem. It is important to realize the operational efficiency of medical staff by applying information technology. We are collecting behavior data by attaching RFID tags to medical staff and hospital patients in an actual medical field under the cooperation of Sapporo Dohto Hospital Medical Corporation. Our method realizes a behavior prediction based on behavior order by using these actual behavior data by applying Long-Short Term Memory (LSTM). We realize improving work efficiency such as optimal placement of medical staff by applying our behavior prediction method.