This paper focuses on a multimodal language understanding method for ``Carry and Place'' tasks with domestic service robots. We address the case of ambiguous instructions, that is when the target area is not specified. For instance ``Put away the milk and cereal.'' is a natural instruction where there is ambiguity on the target area, considering daily life environments. Conventionally, this instruction can be disambiguated from a dialogue system, but at the cost of time and cumbersomeness. Instead, we propose a multimodal approach, where the instructions are disambiguated from the robot state and environment context. We develop MultiModal Classifier Generative Adversarial Network (MMC-GAN) to predict the likelihood of the different target areas considering the robot physical limitation and the target clutter. Our approach, MMC-GAN, significantly improves accuracy compared to baseline methods using instructions only or simple deep neural networks.

We propose a tool-use model that robots choose and use tools to carry out tasks. In these days, research on the tool-use by robots have been done aiming at robots that are useful in daily life. However, conventional research has two problems. (1)experimenters need to label tools or environment. (2)it is impossible to perform a series of operation from tool selection to task execution. In this research, we propose a model which can solve the two problems, we let a robot select a tool, hold it and perform the task, and have a series of experiences. Then, train the sensory-motor data that acquired during the experience and task command with deep learning. At last, to evaluate the model, we confirmed the ability of motion generation in the untrained situation.

Maritime meteorological observation is critical for a safe voyage, and general ships are required in Japan to report the observations to parties concerned. Since it is difficult to recognize the meteorological conditions for non-experts, the demand of automatic recognition arises. Many studies have tackled the classification of cloud genera and the regression of cloud cover. However, less attention has been paid for cloud conditions. Thus, we developed a machine learning system for classification of cloud conditions. We first developed a dedicated equipment for photographing whole sky images and collected data samples. Then, we tagged cloud genera and conditions in each cloud layer (high, middle, and low). Using the dataset, we built a deep convolutional neural network to classify the cloud genera and conditions via fine-tuning ResNet50. The network achieved accuracies higher than 0.9 for both cloud genera and conditions.

Layered neural networks (LNNs) have realized high recognition performance for various real datasets, however, it is difficult for human beings to understand their training results. Conventionally, we have proposed network analysis methods for extracting simplified structure of a trained LNN, by detecting communities of units based on the similarity of connection patterns. In this work, we propose a new method for representing the community structure in a LNN, by using connection weights between pairs of communities. By experiment using the dataset of diagram recognition, we show that our new method provides clues for interpreting the roles of each community in a LNN, in terms of which community in input-side adjacent layer is the most important for it in prediction.

With the remarkable development of Intelligent Transportation System in recent years, it is possible to easily collect traffic information and various information of the vehicle. Probe information provides more extensive traffic information in addition to the observation information. In this paper, we consider the traffic flow prediction method on urban road using probe information. Accurate and real-time traffic information is indispensable for the deployment of high-performance intelligent transportation systems. Traffic flow is complicated, but by deep learning that can acquire feature quantities automatically, it is possible to express the characteristics of the traffic flow without the prior knowledge such as the characteristics of the site, and it is expected to improve the prediction accuracy. Therefore, in this research, we consider a traffic flow prediction model using deep learning. Also, we compared it with other traffic flow prediction method.

Recent advancements in computer-assisted learning systems have increased research in the area of knowledge tracing, and it is reported that leveraging neural networks enables efficient estimation. However, such a development of neural network-based knowledge tracing methods suggests the necessity to review the definition of "knowledge", which previously has been designed by human experts and treated as given. In this context, recently a method to automatically learn efficient knowledge representation from student exercise logs has been proposed, and it is becoming important to designing more machine-friendly knowledge representation, which enables efficient performance of knowledge tracing. In this paper, we analyze the properties of knowledge representation learned to maximize the performance of knowledge tracing, and investigate the important factors for machines to efficiently perform knowledge tracing. Experimental results provide useful insights for reviewing the definition of knowledge, which previously has been treated as given, and designing machine-friendly knowledge representation, which could help improve the learning experience of students in more diverse environments.

Explaining the output of Convolutional Neural Networks (CNNs) is a challenging topic. A typical explanation is to identify which pixels are contributing to the output of CNN. In this paper, we propose a new approach for explaining the output of CNNs by finding pixels that are \emph{not} contributing to the output. To highlight non-contirbuting pixels, we propose optimizing a noise level so that additive noise to the input image does not change the CNN output. The experimental results on MNIST show that the proposed method can idntify non-contributing pixels adequately.

It is known that EEG drastically changes depending on external influences and health condition of an experimental participant. BCI based on the focused features of EEG signal, such as frequency band or measurement sites is suitable only for the users with reproducible, major EEG features evoked by a certain cognitive task. To avoid this problem, we developed a BCI using Learning-type-Fuzzy-Template-Matching (L-FTM) method. In addition, we implemented pruning procedure that deletes unsuitable fuzzy rules with high compatibility degree to both of task and non-task status. We confirmed that was BCI system detected the EEG features of a participant during imaging movement task, after learning the EEG features accompanied by motor task.

Quantitative analysis of human brain activity has been actively studied in brain and neuroscience. In the analysis, there has recently been getting many opportunities that deep leaning is applied to deal with brain activity data observed with fMRI. Whereas, it costs expensive to collect such data and this always causes the lack of enough data to train a model in deep learning framework. With this background, we aim to artificially increase the number of such data and apply them to a caption generation task with deep learning for raising the accuracy of the model.

The Whole Brain Architecture (WBA) is considered to be a strong candidate for the computational cognitive architecture of an artificial general intelligence (AGI) computing platform which includes empirical neural circuit information of the entire brain. The WBA is constructed with the aim of developing a biologically plausible general-purpose artificial intelligence with can exert brain-like multiple cognitive functions and behaviors in a computational system. In this study, we created Whole Brain Connectomic Architecture (WBCA), which is based on the datasets of quantified experiment results in mouse brain provided by Allen Institute for Brain Science to construct a unified platform of WBA. Strengths and hierarchies of connections between brain areas were computed to the provided data and confirmed the consistency in well-studied connections with previous studies. We suggest that computational cognitive architecture defined by connectomic data can enhance the development of AGI algorithms.

We discuss how we can approach to embodied knowledge and skills in practical and clinical field of rehabilitation. There is a dilemma in studying them: Within traditional scientific framework, if researchers want to investigate a particular object or phenomenon in real and natural world by means of hypothesis testing experimental way, they should remove other factors apparently unrelating to their hypothesis in order to avoid affecting the result. However, most real and natural phenomena cannot be separated from various factors surrounding them (e.g., environment, situation). In other words, they are embedded in their environment or situation. This dilemma represents some difficulties of approaching to embodied knowledge and skills in practical and clinical field. We pick up related issues and show our approach to study them.

This paper discusses the understanding embedded in the traditional form of vernacular houses and the landscape composed of those houses and other traditional forms. The authors introduce “Photo Diary” - a form of expressing findings in field-works and a process of associating findings with each other. “Photo Diary” is composed of a photograph expressing a finding in a field-work, the description of remarkable facts taken in the photograph, the description of interpretation of the facts, and the observer’s first-person experiences of the facts. Every photo diary is associated with each other to construct a whole structure of the findings. The authors have been exploring the spatial schema underlying the form of vernacular houses. On the way of exploring, we have been accumulating pieces of findings. This paper explains a process of constructing design inquiries for regional revitalization of Izena Village, Okinawa, where traditional Ryukyu houses and landscapes are remaining.

This study examined the feature of the handling for the treatment of the sit-to-stand motion of the patient in the rehabilitation domain and "knacks" that I could suppose from those features. We had four objects; an occupational therapist who has 18 years of experience and a student from a department of occupational therapy as a role of the therapist, and people who assumed a person of virtual single paralysis as a role of a patient. It was suggested the expert made the patient move forward by maintaining a certain distance between the patient and him, and this motion is very important. What supported the characteristic was the coordinative structure of the whole body, which limits the movement of arms and makes lower limbs move more freely. On the other hand, such a physical structure is seen in the characteristic of the martial arts-like physical campaign for ancient Japanese martial arts. Therefore when we compared sit-up exercise of ancient Japanese martial arts and usual one, the distance with a head and each body parts was smaller the first one.

The present study is an attempt to explore the notion of “mobility” through an experiential learning program. In Fall 2017, five groups of college students attempted to design a series of “mobile kits” for place-making activities. That is to envision and identify when, where, with whom, and how our “precious moments” will be realized. From a carry cart to a bicycle, each group tried to modify and redesign a range of selected items (vehicles) based on their own “issues”at hand. By referring to the process and outcome of the project, this paper begins a discussion about the ways in which we can speculate upon human communication behavior though understanding the use of artifacts/objects in our day-to-day activities.

We proposed "the constructed exhibition". This exhibition method was collaboration between "a traditional exhibition" and "a hands-on exhibition based on constructivism" and "Visible storage". As a result, beholders were aware of interesting facts by "the constructed exhibition".

To elucidate the brain function, it is important to understand dynamics of nonlinear neuronal activity patterns. In this study, we attempted to extract features of activity patterns including evoked responses in the cultured rat neuronal network. Thus, we employed the multi-layered artificial neural network (ml-ANN) as the Deep-Learning method with stacked-autoencoder as the pre-training method to classify the network activity. As the result, activity pattern 2 seconds after the electrical stimulation was barely classified, however, discrimination ability of ml-ANN against the activity pattern of later time domain after the stimulation was not enough to classify the patterns of evoked responses, because of the insufficient amount of learning-data, which is difficult to be gathered in large quantities. This indicates a huge number of pre-learning data is absolutely necessary to improve the discrimination accuracy in order to identify patterns by the Deep-Learning method for large phenomena with "fluctuation" such as neural activity.

We have been developing a number problem solver for National Center Test for University Admissions (NCTUA). The solver searches for a series of equivalence-preserving transformation of an input formula that results in a quantifier-free formula, from which the answer can be easily deduced. However, the increase of the search space hampered the capability of the solver. We introduced formula normalization and extended matching rules in the search in order to reduce the search space. Experimental results show that this method drastically reduces the search space and shorten the execution time.

This research focuses on Kaoomji (Asian style emoticons) used in text-based communication. We analyze Kaomoji that are made from particular characters’ sequence (original shapes). In this paper, we construct a neural network with embedding function to estimate the original form of Kaomoji. As a result, we find out that our proposed method can estimate the original form of Kaomoji with 0.746 of accuracy ratio.

This study examines a method of effective utilization as knowledge in organizational stored information of business activities. As an approach, a information is presented by the document summarization using the real bulletin board system data. For a question sentence, the system retrieves information based on categories and feature words, and documents are summarized.

Recursive Neural Tensor Network (RNTN) is a neural network model that recursively computes the synthetic distributed vector representation for phrases of various lengths and syntax types from the distributed vector representation of words along the syntax tree. Distributed vector representation is used as a feature to classify each phrase and it is used to classify sentiment of phrases in sentiment analysis. In this paper, we apply the RNTN to sentiment analysis of Japanese sentences. For this purpose, based on the Stanford Sentiment Treebank corpus for sentiment analysis, we first create a corpus of Japanese sentences with teacher labels only for words and sentences. Then we evaluate the accuracy of sentiment analysis on Japanese sentences when we learn from only teacher labels for words and sentences. We also consider the effect of attaching teacher labels of phrases by heuristic rules.

Document classification is an important technology in modern information society. In recent years, distributed representation (DR) which embeds semantic relationships of words into vectors has attracted attention and the methods applying DR to document classification have been reported. DR can be generated mainly by using a tool called Word2Vec. Word2Vec has the learning structure using a neural network, and we use the weights on the input side as DR. However, Word2Vec learns different characteristic weights on the output side from DR, which is not focused on and not commonly used. In this paper, we propose a document classification method by ensemble learning using DR and the output side weights and suggest the usefulness on the proposed method.

In recent years, the negative aspects of information creation by individuals, such as fake news, flaming and echo champer phenomena, have been paid attention. We propose a hypothesis "social porn" as one of the causes of these negative phenomena. We define "social porn" as "information that a user belonging to a specific community wants to diffuse and share instantly". In this paper, we define the scale of user reaction time as preparation for observation of social porn. We examine the difference of user reaction time distribution for some tweets. As a result, analysis results suggest that the distribution of user response times may differ between tweets randomly extracted and tweets spreaded by specific community.

Due to the advancement of E-commerce in recent years, recommendation for not only mass-produced daily items but also special items that are not mass-produced is an important task. In this research, we present an algorithm for real estate recommendation. There is no identical property in the world, properties already occupied by someone else can not be recommended, and users rent or buy properties only a few times in their lives. Therefore, automatic recommendation of property is one of the most difficult tasks. As the first step of property recommendation, we predict users' preference for properties by combining content-based filtering and multilayer perceptron (MLP). In the MLP, we used not only attribute data of users and properties but also deep features extracted from Floorplan images of properties. As a result, we succeeded in predicting preference with accuracy of 60.7%.

As a current trend, in the consumption of durable consumer goods such as cars, real-estate properties, increasing number of buyers are making their purchasing decisions based on information on the Internet. Despite that, since real-estate contraction process is barely common or frequent, more users (consumers) than not suffer from a lack of adequate knowledge. Consequently, searching for a house on-line could become confusing. Renting a house, specifically, often comes time restriction, and from times requires compromise in search terms. To help, search terms and listing recommendations have to be more properly generated according to each user's needs and preferences. In this analysis, we propose a method of user preference extraction and provide insights on the results.

The apartment loan is a loan for rentals such as for condos, apartments. This loan is a very large loan which is the account for a percentage of more than 10 percents of the whole banks' loan. However, a risk model of the apartment loan with the appropriate accuracy has not been provided in Japan mainly due to the lack of data. Thus, in order to develop the risk model, we preliminarily analyse and compare two types of data set: the servey data which is made by the real estate appraiser and the housing information website data. As a result, it was found that (i)the web data is approximetly corresponding to the survey data which was made by exparts with respect to statitical properties, (ii)The AR value of the simple multivariate regression model developed in this study which explains the transion of rooms from vacancy to occupation takes about 0.4 and (iii)this transition probability of the model is mainly explained by an age of a building.

Image processing enhanced by deep learning is now being actively applied to real-world issues. The real estate industry, which owns lots of property information, also expects to develop new services using deep learning. In this paper, we present the classification methods for property images using bottleneck features generated by convolution networks (CNN), and experimentally show that we can achieve 88% accuracy in Top-1.

Conventional lecture substitution systems with humanoid robots use pre-defined gestures created by hand. Automatically generating these gestures makes it possible to create gestures without requiring expert knowledge and work, which is expected to lead to further progress in research on lecture substitution systems. This paper proposes an automatic gesture generation method which is expected to consider the semantic context of an utterance. Our proposed method is implemented by using a deep neural network with Bi-Directional LSTM units, applying filters for data correction, and axis conversion.

We proposed a Circulate-Learning Robot for solving isolation of elderly people. Circulate-Learning Robot is a teachable robot which is designed to go to various places and be taught by people. It was found from the previous research that it is important to equalize the amount of teaching from the user. In addition, we focused on the importance of empathy in new relationship building. We propose a dialog selection algorithm to equalize the amount of user’s emotional expression for a user of Circulate-Learning Robot. The number of morphemes in the sentence in which the emotion expression word appears was defined as the amount of emotional expression. In the proposed method, users are clustered using this amount of emotional expression. The topic is decided from the co-occurrence probability of an independent word and an emotion expression word. The proposed system switches dialogue strategy between empathy expression and promote to expressing impressions. This paper proposes the algorithm that selects a dialog strategy according to the amount of user's emotional expression.

In this research, we have been developing a robot system that encourages improvement of driving behavior at home. Mixed reality technology is used to implement driving reflection function for driving behavior improvement in this system.An existing mixed reality system uses a special input interface, therefore smooth operation is expected to be difficult. In this paper, we propose two interfaces that are made with reference to swipe behavior and laser pointer, respectively. The input interface that is imitated laser pointer indicated good perfomance in the experiments.

Multi-camera multiple object tracking methodology that uses probability occupancy map (POM) and graph optimization can efficiently track multiple people in spite of significant occlusion, even without appearance model and prerequisite knowledge about the number of people in the target area. However, the computation time of POM increases according to grid size and number of cameras, which limited latency and throughput. In this paper, we report the effect of hierarchically applying two parallelization techniques to enhance the performance of POM generation. As a result, we achieved more than 20 times higher throughput compared to sequential processing, and reduce latency by 66%.

Object tracking is a challenging problem and it has been improving dramatically in recent years. In this paper, we perform parallelized multi-object tracking system. Object tracking problem has 2 difficulties; one is to detect objects collect, and the other is to track collect using the collect object detection. Jerome et al. performed a multi-object tracking system using K-Shortest Paths to avoid these problems efficiently. However, it is difficult to calculate in parallel because of the iterations calculation of shortest paths on the graph while changing the weight of graph. In our method, we divided time intervals to apply KSP method from Probability Occupancy Map(POM), which is also obtained via using KSP method. Performance evaluation shows our algorithm is 5.4 times faster than the original KSP with 87% accuracy.

According to the population growth of the aged people, the needs of action recognition system for elderly care are increasing. Upcoming IoT era, the privacy issues of the vision based human action recognition is still primary concerns. With due consideration of the privacy protection of users, we have been proposing a system using an infrared array sensor with high abstraction and low resolution. This paper reports the developed simulator to reproduce the behavior of the human, the data collection under various situations using infrared sensor and the simulator, and classification results whose accuracy was more than 90% using machine learning.

Animal receives high-dimensional and complex raw sensory information. Deep learning can recognize such complex sensory information. We studied a deep learning model called hierarchical recurrent neural network (HRNN) that develops spatial recognition through visuomotor integration learning. In a simulation experiment, the HRNN developed the cognitive map, which is an objective map-like internal model, through only subjective visuomotor experiences. Furthermore, the HRNN also developed spatial recognition through visuomotor sequences by a human subject. These results imply that deep learning model can be used to study real animals’ cognition.

Static text communications in online spaces lack the rich embodied information we experience in face-to-face communication. In this research, we aim at charging liveliness into digital text communication by developing TypeTrace, a Web software that can record and play back writing processes and visualize the micro fluctuations of time contained within. We show the design and result of a preliminary experiment we conducted, and discuss further research directions.