We are studying music and color to create a new way of enjoying music namely "wearing", and we are exploring its entertainment. Concretely, images of colorful costumes are formed from music in real time, and projected on fog generated around a person by projectors to realize "wearing music". In this paper, we propose a method to color music in three dimensions, a method to generate costume images from music, and a method to wear music images. The core ideas are the three dimensional coloring of music focusing on the spiral structure of the musical scale and synesthesia, the image generation of the costume by the technique of generative art, and the image projection from three directions to the fog which is the media of projection. However, it was difficult to control the fog, and the images projection onto the fog did not work as a result. Therefore, instead of fog, we also conducted experiments such as projecting costume images directly onto a white coat or mannequin body. By this research, even ordinary people without special senses can experience a new way of enjoying music namely "wearing", and we can expect a possibility of creating a new entertainment world where music and fashion are fused.

View full abstract

This paper is describing a method of using Unreal Engine 4 to control music according to the change of images by artist operation. Specifically, a sound playing object is placed in a 3DCG space. Artist is operating a camera to moving a 3DCG space, thereby sound combination is change. For using this method on work production, I examined motifs, expression methods, programs. In a research created one prototype video and four works. And, clarify an influence on this method to production process of work.

View full abstract

In recent years, entertainment is becoming diversifying, and traditional game, animation, movies, and attractions are experienced crossingly. Curriculum of specific field such as game and animations are relatively systematized, but it’s difficult to cover variety of entertainment. In this paper, we describe the framework and practices of entertainment engineering education. We organize competencies of entertainment engineering into basic, design, and produce, and describe examples of curriculum composition for university and graduate schools. Based on the improvements since 2002, we summarize the interaction between curriculum and practice, and possible future plans.

View full abstract

The world health organization (WHO) reported that around sixty percent of people at the age of fifteen years and older in Japan lack of daily exercise. The importance and necessity of both oxygen-free exercises and aerobic ones for staying healthy is obvious, but practicing those exercises constantly, especially those which are too simple and boring, accompanies so much mental load that makes people reluctant from doing so. One direction of research efforts that have been paid for encouraging exercises is investigation of the environmental factors of exercise such as background music. Here we report the experimental results about the effect of illumination color and intensity that surrounds the person who pedals a bike. The subjects were asked to pedal a aero bike under several illumination conditions and to answer a questionnaire for measuring how they felt about the exercise and the pedaling workload. Their heart-beat rate was also measured during the exercise. The comparison of the subjective evaluation and the physical performance among those illumination conditions indicated that red illumination has made the subjects more excited than each of blue and white illuminations, whereas white has made them relaxed than red. Red illumination seemed to have affected the mental aspect without influencing the physical aspect of heart-beat rate.

View full abstract

Japanese animation called "Japanimation" is highly regarded for its artistry. Many people abroad are interested in Japan through animes and start learning Japanese and listening practice. We focused on the possibility of using animes as teaching materials for listening practice. An episode of anime contains not only one conversation but also several partial conversations. Since the words and expressions in each partial conversation are different, the degree of difficulty might also be different. This paper proposes a method to estimate the degree of difficulty of the partial conversation of anime. By providing partial conversations with estimated levels, we support learners in their listening practices. In evaluation experiments, participants were asked to take a pre-test, practice their listening with the degree assigned partial conversations, and take a post-test. The participants were divided into two groups. An experimental group used the degree set partial conversations while a control group used just partial conversations for their listening exercises. The experimental results showed that the increase in correct rate between pre-test and post-test was 26.4% in the experimental group of L2 of JLPT while that was 10.6% in the control group.

View full abstract

Local "Moe" characters, the manga-like characters that are used for appealing local authority, are widely used in Japan. However, it is not clear that how the appearance of Local "Moe" character affects on the impression perceived by consumers. In this research, we conducted a factor analysis to reveal the factors of impression of Local "Moe" characters. We classified the Local "Moe" characters into two groups by the ratio of height of head to height of body, and conducted an ANOVA on the factor scores between these two groups. Also we calculated the correlation coefficient between the factor scores and the scores of perceived trustworthiness, emotional richness, recommendation effect and suitability perceived. As a result, we revealed two factors, familiarity and reality, and found that the real characters were more effective than deformed characters.

View full abstract

This paper reports research on automatic lyrics generation considering listenability from a vocal melody of popular music. In lyrics generation, it is necessary to consider both the meaning and the melody at the same time, which makes this task difficult both in the generation and evaluation steps. We built a system in which the language model automatically generates lyrics just from a vocal melody. We aimed to learn the data and generate lyrics with flexibility for the melody without using human analysis. By considering the notes as sequences, we applied seq2seq and Transformer, each of which is often used in machine translation. We introduce word density as a quantitative measure of evaluation. The lyrics output by each language model was voice-synthesized, and seven participants performed a subjective evaluation on listenability, meaning, and overall quality. We also briefly evaluated the difference in learning achievement depending on the number of data. All the methods showed some linguistic disturbance in generating the lyrics of the test songs. Among them, Transformer generated the most melodious and natural lyrics.

View full abstract

Among utility-based AI in game AI, there are two types of indicators: goal-based indicators, which have evaluation criteria that guarantee the achievement of the final goal, such as reaching the destination, and non-goal-based indicators, which determine actions based only on local evaluations and do not guarantee the achievement of the final goal. When goal-based and non-goal-based indices are added together and both are taken into account, the problem arises that the goal-based property is lost and the guarantee of goal achievement is lost. The purpose of this research is to realize a behavior that achieves the final goal as quickly as possible, even if both goal-based and non-goal-based are used simultaneously. We propose a method that uses a fill function with a Discrete-Laplacian to maintain the law of transition and guarantee the achievement of the final goal. As an example, we assume that the goal base is the arrival at the destination and the non-goal base is the avoidance of the tracking agents, and the AI that avoids the tracking agents and reaches the destination quickly is realized using the fill function. The adjustment coefficients of the fill function were changed dynamically using fuzzy reasoning based on the numerical values of the AI's fitness and stagnation.

View full abstract

In recent years, there have been many cases of utilizing VR technology for education. Especially in history education, it is used as experiential learning. However, construction of VR contents has some problems. One is the difficulty of creating a 3D model. In particular, the creation of a 3D model from a handwritten map without numerical data is a more difficult task. In order to solve the problems, in this paper, we have proposed the method of creating a 3D model from a handwritten map without numerical information. We have developed a VR system that can be experienced by multiple people at the same time using the created 3D model.

View full abstract

We propose a visualization technique for analyzing relationship between sea condition and catch data. In order for people to predict fish catches in the short term and at the local level, we believe that understanding the relationship between sea conditions and fish catches is essential. We develop an information visualization tool that allows users to investigate the relationship between the two. In this research, we design a coarse-to-fine user interface to grasp the overall trend and then see the details through interaction. Specifically, our tool start with overall visibility through time series graphs of fish catches, then identify representative sea conditions that satisfy the survey conditions, and investigate partial characteristics of the sea conditions. In order to realize the user interface, we have developed a new method for collecting sea condition data similar to the target condition, and a method for visualizing partial similarities and differences in sea condition. We implemented the proposed tool and confirmed the effectiveness of our user interface.

View full abstract