Multimedia data and information management is an important task according to the development of media processing technology. Multimedia is a useful resource that people understand complex situations such as the elderly care domain. Appropriate annotation is beneficial in several tasks of information management, such as storing, retrieval, and summarization of data, from a semantic perspective. However, the metadata annotation for multimedia data remains problematic because metadata is obtained as a result of interpretation depending on domain-specific knowledge, and it needs well-controlled and comprehensive vocabulary for annotation. In this study, we proposed a collaborative methodology for developing ontologies and annotation with domain experts. The method includes (1) classification of knowledge types for collaborative construction of annotation data, (2) division of tasks among a team composed of domain experts, ontology engineers, and annotators, and (3) incremental approach to ontology development. We applied the proposed method to 11 videos on elderly care domain for the confirmation of its feasibility. We focused on annotation of actions occurring in these videos, thereby the annotated data is used as a support in evaluating staff skills. The application results show the content in the ontology during annotation increases monotonically. The number of “action concepts” is saturated and reused among the case studies. This demonstrates that the ontology is reusable and could represent various case studies by using a small number of “action concepts”. This study concludes by presenting lessons learnt from the case studies.

With the development of cameras and sensors and the spread of cloud computing, life logs can be easily acquired and stored in general households for the various services that utilize the logs. However, it is difficult to analyze moving images that are acquired by home sensors in real time using machine learning because the data size is too large and the computational complexity is too high. Moreover, collecting and accumulating in the cloud moving images that are captured at home and can be used to identify individuals may invade the privacy of application users. We propose a method of distributed processing over the edge and cloud that addresses the processing latency and the privacy concerns. On the edge (sensor) side, we extract feature vectors of human key points from moving images using OpenPose, which is a pose estimation library. On the cloud side, we recognize actions by machine learning using only the feature vectors. In this study, we compare the action recognition accuracies of multiple machine learning methods. In addition, we measure the analysis processing time at the sensor and the cloud to investigate the feasibility of recognizing actions in real time. Then, we evaluate the proposed system by comparing it with the 3D ResNet model in recognition experiments. The experimental results demonstrate that the action recognition accuracy is the highest when using LSTM and that the introduction of dropout in action recognition using 100 categories alleviates overfitting because the models can learn more generic human actions by increasing the variety of actions. In addition, it is demonstrated that preprocessing using OpenPose on the sensor side can substantially reduce the transfer quantity from the sensor to the cloud.

In this letter we analyze the economic worth of GPU on analytical processing of GPU-accelerated database management systems (DBMSes). To this end, we conducted rigorous experiments with TPC-H across three popular GPU DBMSes. Consequently, we show that co-processing with CPU and GPU in the GPU DBMSes was cost-effective despite exposed concerns.

The Topics over Time (TOT) model allows users to be aware of changes in certain topics over time. The proposed method inputs the divided dataset of security blog posts based on a fixed period using an overlap period to the TOT. The results suggest the extraction of topics that include malware and attack campaign names that are appropriate for the multi-labeling of cyber threat intelligence reports.

In this paper, the effective Long Range (LoRa) based wireless sensor network is designed and implemented to provide the remote data sensing functions for the planned smart agricultural recycling rapid processing factory. The proposed wireless sensor network transmits the sensing data from various sensors, which measure the values of moisture, viscosity, pH, and electrical conductivity of agricultural organic wastes for the production and circulation of organic fertilizers. In the proposed wireless sensor network design, the LoRa transceiver module is used to provide data transmission functions at the sensor node, and the embedded platform by Raspberry Pi module is applied to support the gateway function. To design the cloud data server, the MySQL methodology is applied for the database management system with Apache software. The proposed wireless sensor network for data communication between the sensor node and the gateway supports a simple one-way data transmission scheme and three half-duplex two-way data communication schemes. By experiments, for the one-way data transmission scheme under the condition of sending one packet data every five seconds, the packet data loss rate approaches 0% when 1000 packet data is transmitted. For the proposed two-way data communication schemes, under the condition of sending one packet data every thirty seconds, the average packet data loss rates without and with the data-received confirmation at the gateway side can be 3.7% and 0%, respectively.

With the widespread diffusion of Internet of Things (IoT), the number of applications using wireless sensor devices are increasing, and Quality of Service (QoS) required for these applications is diversifying. Thus, it becomes difficult to satisfy a variety of QoS with a single wireless system, and many kinds of wireless systems are working in the same domains; time, frequency, and place. This paper considers coexistence environments of ZigBee and Wi-Fi networks, which use the same frequency band channels, in the same place. In such coexistence environments,ZigBee devices suffer radio interference from Wi-Fi networks, which results in severe ZigBee packet losses because the transmission power of Wi-Fi is much higher than that of ZigBee. Many existing methods to avoid interference from Wi-Fi networks focus on only one of time, frequency, or space domain. However, such avoidance in one domain is insufficient particularly in near future IoT environments where more ZigBee devices and Wi-Fi stations transfer more amount of data. Therefore, in this paper, we propose joint channel allocation and routing in both frequency and space domains. Finally, we show the effectiveness of the proposed method by computer simulation.

Currently, the IEEE 802.11n wireless local-area network (WLAN) has been extensively deployed world-wide. For the efficient channel assignment to access-points (APs) from the limited number of partially overlapping channels (POCs) at 2.4GHz band, we have studied the throughput drop estimation model for concurrently communicating links using the channel bonding (CB). However, non-CB links should be used in dense WLANs, since the CB links often reduce the transmission capacity due to high interferences from other links. In this paper, we examine the throughput drop estimation model for concurrently communicating links without using the CB in 802.11n WLAN, and its application to the POC assignment to the APs. First, we verify the model accuracy through experiments in two network fields. The results show that the average error is 9.946% and 6.285% for the high and low interference case respectively. Then, we verify the effectiveness of the POC assignment to the APs using the model through simulations and experiments. The results show that the model improves the smallest throughput of a host by 22.195% and the total throughput of all the hosts by 22.196% on average in simulations for three large topologies, and the total throughput by 12.89% on average in experiments for two small topologies.

In this paper, in order to avoid the cascading failure by increasing the number of links in the physical network in D2D-based SNS, we propose an autonomous device placement algorithm. In this method, some relay devices are placed so as to increase the number of links in the physical network. Here, relay devices can be used only for relaying data and those are not SNS users. For example, unmanned aerial vehicles (UAV) with D2D communication capability and base stations with D2D communication capability are used as the relay devices. In the proposed method, at first, an optimization problem for minimizing node resilience which is a performance metric in order to place relay devices. Then, we investigate how relay devices should be placed based on some approximate optimal solutions. From this investigation, we propose an autonomous relay device placement in the physical network. In our proposed algorithm, relay devices can be placed without the complete information on network topology. We evaluate the performance of the proposed method with simulation, and investigate the effectiveness of the proposed method. From numerical examples, we show the effectiveness of our proposed algorithm.

Network function virtualization (NFV) enables network operators to flexibly provide diverse virtualized functions for services such as Internet of things (IoT) and mobile applications. To meet multiple quality of service (QoS) requirements against time-varying network environments, infrastructure providers must dynamically adjust the amount of computational resources, such as CPU, assigned to virtual network functions (VNFs). To provide agile resource control and adaptiveness, predicting the virtual server load via machine learning technologies is an effective approach to the proactive control of network systems. In this paper, we propose an adjustment mechanism for regressors based on forgetting and dynamic ensemble executed in a shorter time than that of our previous work. The framework includes a reducing training data method based on sparse model regression. By making a short list of training data derived from the sparse regression model, the relearning time can be reduced to about 57% without degrading provisioning accuracy.

Application-aware Traffic Engineering (TE) plays a crucial role in ensuring quality of services (QoS) for recently emerging applications such as AR, VR, cloud gaming, and connected vehicles. While a deterministic application-aware TE is required for these mission-critical applications, a negotiation procedure between applications and network operators needs to undergo major simplification to fulfill the scalability of the application based on emerging microservices and container-based architecture. In this paper, we propose a NetworkAPI framework which allows an application to indicate a desired TE behavior inside IP packets by leveraging Segment Routing over IPv6 (SRv6). In the NetworkAPI framework, the TE behavior provided by the network operator is expressed as an SRv6 Segment Identifier (SID) in the form of a 128-bit IPv6 address. Because the IPv6 address of an SRv6 SID is distributed using IP anycast, the application can utilize the unchanged SRv6 SID regardless of the application's location, as if the application controls an API on the transport network. We implement a prototype of the NetworkAPI framework on a Linux kernel. On the prototype implementation, a basic packet forwarding performance is evaluated to demonstrate the feasibility of our framework.

In this paper, we used the data set of domain names Global Top 1M provided by Alexa to analyze the effectiveness of Fallback in ECN. For the same test server, we first negotiate a connection with Not-ECN-Capable, and then negotiate a connection with ECN-Capable, if the sender does not receive the response to ECN-Capable negotiation from the receiver by the end of retransmission timeout, it will enter the Fallback state, and switch to negotiating a connection with Not-ECN-Capable. By extracting the header fields of the TCP/IP packets, we confirmed that in most regions, connectivity will be slightly improved after Fallback is enabled and Fallback has a positive effect on the total time of the whole access process. Meanwhile, we provided the updated information about the characteristics related to ECN with Fallback in different regions by considering the geographical region distribution of all targeted servers.

A global content delivery plays an important role in the current Internet. Information-Centric Networking (ICN) is a future internet architecture which attempts to redesign the Internet with a focus on the content delivery. However, it has the potential performance degradation in the global content delivery. In this paper, we propose an ICN performance enhancing proxy (ICN-PEP) to mitigate this performance degradation. The key idea is to prefetch Data packets and to serve them to the consumer with the shorter round trip time. By utilizing ICN features, it can be developed as an offline and state-less proxy which has an advantage of scalability. We evaluate the performance of ICN-PEP in both simulation and experiment on global testbed and show that ICN-PEP improves the performance of global content delivery.

In this study, we propose a secure data-providing system by using a verifiable attribute-based keyword search (VABKS), which also has the functions of privacy preservation and feedback to providers with IP anonymous server. We give both theoretic and experimental result, which show that our proposed system is a secure system with real-time property. One potential application of the system is to Integrated Broadcast-Broadband (IBB) services, which acquire information related to broadcast programs via broadband networks. One such service is a recommendation service that delivers recommendations matching user preferences (such as to TV programs) determined from the user's viewing history. We have developed a real-time system outsourcing data to the cloud and performing keyword searches on it by dividing the search process into two stages and performing heavy processing on the cloud side.

In Internet applications, when users search for information, the search engines invariably return some invalid webpages that do not contain valid information. These invalid webpages interfere with the users' access to useful information, affect the efficiency of users' information query and occupy Internet resources. Accurate and fast filtering of invalid webpages can purify the Internet environment and provide convenience for netizens. This paper proposes an invalid webpage filtering model (HAIF) based on deep learning and hierarchical attention mechanism. HAIF improves the semantic and sequence information representation of webpage text by concatenating lexical-level embeddings and paragraph-level embeddings. HAIF introduces hierarchical attention mechanism to optimize the extraction of text sequence features and webpage tag features. Among them, the local-level attention layer optimizes the local information in the plain text. By concatenating the input embeddings and the feature matrix after local-level attention calculation, it enriches the representation of information. The tag-level attention layer introduces webpage structural feature information on the attention calculation of different HTML tags, so that HAIF is better applicable to the Internet resource field. In order to evaluate the effectiveness of HAIF in filtering invalid pages, we conducted various experiments. Experimental results demonstrate that, compared with other baseline models, HAIF has improved to various degrees on various evaluation criteria.

Computer networks are facing serious threats from the emergence of sophisticated new DGA bots. These DGA bots have their own dictionary, from which they concatenate words to dynamically generate domain names that are difficult to distinguish from human-generated domain names. In this letter, we propose an approach for identifying the callback communications of DGA bots based on relations among the words that constitute the character string of each domain name. Our evaluation indicates high performance, with a recall of 0.9977 and a precision of 0.9869.

Two-state number-conserving cellular automaton (NCCA) is a cellular automaton of which cell states are 0 or 1, and the total sum of all the states of cells is kept for any time step. It is a kind of particle-based modeling of physical systems. We introduce a new structure of its value-1 patterns, which we call a “bundle pair” and a “bundle quad”. By employing this structure, we show a relation between the neighborhood size n and n - 2 NCCAs.

Metabolic networks represent the relationship between chemical reactions and compounds in cells. In useful metabolite production using microorganisms, it is often required to calculate reaction deletion strategies from the original network to result in growth coupling, which means the target metabolite production and cell growth are simultaneously achieved. Although simple elementary flux mode (EFM)-based methods are useful for listing such reaction deletions strategies, the number of cases to be considered is often proportional to the exponential function of the size of the network. Therefore, it is desirable to develop methods of narrowing down the number of reaction deletion strategy candidates. In this study, the author introduces the idea of L1 norm minimal modes to consider metabolic flows whose L1 norms are minimal to satisfy certain criteria on growth and production, and developed a fast metabolic design listing algorithm based on it (minL1-FMDL), which works in polynomial time. Computational experiments were conducted for (1) a relatively small network to compare the performance of minL1-FMDL with that of the simple EFM-based method and (2) a genome-scale network to verify the scalability of minL1-FMDL. In the computational experiments, it was seen that the average value of the target metabolite production rates of minL1-FMDL was higher than that of the simple EFM-based method, and the computation time of minL1-FMDL was fast enough even for genome-scale networks. The developed software, minL1-FMDL, implemented in MATLAB, is available on https://sunflower.kuicr.kyoto-u.ac.jp/~tamura/software, and can be used for genome-scale metabolic network design for metabolite production.

Recently, researchers paid more attention on designing optical routers, since they are essential building blocks of all photonic interconnection architectures. Thus, improving them could lead to a spontaneous improvement in the overall performance of the network. Optical routers suffer from the dilemma of increased insertion loss and crosstalk, which upraises the power consumed as the network scales. In this paper, we propose a new 7×7 non-blocking optical router based on the Dimension Order Routing (DOR) algorithm. Moreover, we develop a method that can ensure the least number of MicroRing Resonators (MRRs) in an optical router. Therefore, by reducing these optical devices, the optical router proposed can decrease the crosstalk and insertion loss of the network. This optical router is evaluated and compared to Ye's router and the optimized crossbar for 3D Mesh network that uses XYZ routing algorithm. Unlike many other proposed routers, this paper evaluates optical routers not only from router level prospective yet also consider the overall network level condition. The appraisals show that our optical router can reduce the worst-case network insertion loss by almost 8.7%, 46.39%, 39.3%, and 41.4% compared to Ye's router, optimized crossbar, optimized universal OR, and Optimized VOTEX, respectively. Moreover, it decreases the Optical Signal-to-Noise Ratio (OSNR) worst-case by almost 27.92%, 88%, 77%, and 69.6% compared to Ye's router, optimized crossbar, optimized universal OR, and Optimized VOTEX, respectively. It also reduces the power consumption by 3.22%, 23.99%, 19.12%, and 20.18% compared to Ye's router, optimized crossbar, optimized universal OR, and Optimized VOTEX, respectively.

Emerging byte-addressable non-volatile memory devices attract much attention. A non-volatile main memory (NVMM) built on them enables larger memory size and lower power consumption than a traditional DRAM main memory. To fully utilize an NVMM, both software and hardware must be cooperatively optimized. Simultaneously, even focusing on a memory module, its micro architecture is still being developed though real non-volatile memory modules, such as Intel Optane DC persistent memory (DCPMM), have been on the market. Looking at existing NVMM evaluation environments, software simulators can evaluate various micro architectures with their long simulation time. Emulators can evaluate the whole system fast with less flexibility in their configuration than simulators. Thus, an NVMM emulator that can realize flexible and fast system evaluation still has an important role to explore the optimal system. In this paper, we introduce an NVMM emulator for embedded systems and explore a direction of optimization techniques for NVMMs by using it. It is implemented on an SoC-FPGA board employing three NVMM behaviour models: coarse-grain, fine-grain and DCPMM-based. The coarse and fine models enable NVMM performance evaluations based on extensions of traditional DRAM behaviour. The DCPMM-based model emulates the behaviour of a real DCPMM. Whole evaluation environment is also provided including Linux kernel modifications and several runtime functions. We first validate the developed emulator with an existing NVMM emulator, a cycle-accurate NVMM simulator and a real DCPMM. Then, the program behavior differences among three models are evaluated with SPEC CPU programs. As a result, the fine-grain model reveals the program execution time is affected by the frequency of NVMM memory requests rather than the cache hit ratio. Comparing with the fine-grain model and the coarse-grain model under the condition of the former's longer total write latency than the latter's, the former shows lower execution time for four of fourteen programs than the latter because of the bank-level parallelism and the row-buffer access locality exploited by the former model.

During the recent several years, deep learning has achieved excellent results in image recognition, voice processing, and other research areas, which has set off a new upsurge of research and application. Internal defects and external malicious attacks may threaten the safe and reliable operation of a deep learning system and even cause unbearable consequences. The technology of testing deep learning systems is still in its infancy. Traditional software testing technology is not applicable to test deep learning systems. In addition, the characteristics of deep learning such as complex application scenarios, the high dimensionality of input data, and poor interpretability of operation logic bring new challenges to the testing work. This paper focuses on the problem of test case generation and points out that adversarial examples can be used as test cases. Then the paper proposes MTGAN which is a framework to generate test cases for deep learning image classifiers based on Generative Adversarial Network. Finally, this paper evaluates the effectiveness of MTGAN.

With the development of the Internet of Vehicles, License plate detection technology is widely used, e.g., smart city and edge senor monitor. However, traditional license plate detection methods are based on the license plate edge detection, only suitable for limited situation, such as, wealthy light and favorable camera's angle. Fortunately, deep learning networks represented by YOLOv3 can solve the problem, relying on strict condition. Although YOLOv3 make it better to detect large targets, its low performance in detecting small targets and lack of the real-time interactively. Motivated by this, we present a faster and lightweight YOLOv3 model for multi-vehicle or under-illuminated images scenario. Generally, our model can serves as a guideline for optimizing neural network in multi-vehicle scenario.

An eye model expressed by a revolution about the optical axis of the eye is one of the most accurate models for use in a 3D gaze estimation method. The measurement range of the previous gaze estimation method that uses two cameras based on the eye model is limited by the larger of the two angles between the gaze and the optical axes of two cameras. The previous method cannot calculate the gaze when exceeding a certain limit of the rotation angle of the eye. In this paper, we show the characteristics of reflections on the surface of the eye from two light sources, when the eye rotates. Then, we propose a method that extends the rotation angle of the eye for a 3D gaze estimation based on this model. The proposed method uses reflections that were not used in the previous method. We developed an experimental gaze tracking system for a wide projector screen and experimentally validated the proposed method with 20 participants. The result shows that the proposed method can measure the gaze of more number of people with increased accuracy compared with the previous method.

The progress of immersive technology enables researchers and developers to construct work spaces that are freed from real-world constraints. This has motivated us to investigate the role of the human body. In this research, we examine human cognitive behaviors in obtaining an understanding of the width of their virtual body through simple yet meaningful experiments using virtual reality (VR). In the experiments, participants were modeled as an invisible board, and a spherical object was thrown at the participants to provide information for exploring the width of their invisible body. Audio and visual feedback were provided when the object came into contact with the board (body). We first explored how precisely the participants perceived the virtual body width. Next, we examined how the body perception was generated and changed as the trial proceeded when the participants tried to move right or left actively for the avoidance of collision with approaching objects. The results of the experiments indicated that the participants could become successful in avoiding collision within a limited number of trials (14 at most) under the experimental conditions. It was also found that they postponed deciding how much they should move at the beginning and then started taking evasive action earlier as they become aware of the virtual body.

An encoder-decoder (Enc-Dec) model is one of the fundamental architectures in many computer vision applications. One desired property of a trained Enc-Dec model is to feasibly encode (and decode) diverse input patterns. Aiming to obtain such a model, in this paper, we propose a simple method called curiosity-guided fine-tuning (CurioFT), which puts more weight on uncommon input patterns without explicitly knowing their frequency. In an experiment, we evaluated CurioFT in a task of future frame generation with the CUHK Avenue dataset and found that it reduced the mean square error by 7.4% for anomalous scenes, 4.8% for common scenes, and 6.6% in total. Some other experiments with the UCSD dataset further supported the reasonability of the proposed method.

In recent years, checking sleep quality has become essential from a healthcare perspective. In this paper, we propose a respiratory rate (RR) monitoring system that can be used in the bedroom without wearing any sensor devices directly. To develop the system, passive radio-frequency identification (RFID) tags are introduced and attached to a blanket, instead of attaching them to the human body. The received signal strength indicator (RSSI) and phase values of the passive RFID tags are continuously obtained using an RFID reader through antennas located at the bedside. The RSSI and phase values change depending on the respiration of the person wearing the blanket. Thus, we can estimate the RR using these values. After providing an overview of the proposed system, the RR estimation flow is explained in detail. The processing flow includes noise elimination and irregular breathing period estimation methods. The evaluation demonstrates that the proposed system can estimate the RR and respiratory status without considering the user's body posture, body type, gender, or change in the RR.

This work presents a DNN accelerator architecture specifically designed for performing efficient inference on compressed and sparse DNN models. Leveraging the data sparsity, a runtime processing scheme is proposed to deal with the encoded weights and activations directly in the compressed domain without decompressing. Furthermore, a new data flow is proposed to facilitate the reusage of input activations across the fully-connected (FC) layers. The proposed design is implemented and verified using the Xilinx Virtex-7 FPGA. Experimental results show it achieves 1.99×, 1.95× faster and 20.38×, 3.04× more energy efficient than CPU and mGPU platforms, respectively, running AlexNet.

To date, many studies have employed clustering for the classification of unlabeled data. Deep separate clustering applies several deep learning models to conventional clustering algorithms to more clearly separate the distribution of the clusters. In this paper, we employ a convolutional autoencoder to learn the features of input images. Following this, k-means clustering is conducted using the encoded layer features learned by the convolutional autoencoder. A center loss function is then added to aggregate the data points into clusters to increase the intra-cluster homogeneity. Finally, we calculate and increase the inter-cluster separability. We combine all loss functions into a single global objective function. Our new deep clustering method surpasses the performance of existing clustering approaches when compared in experiments under the same conditions.

Cross-component linear model (CCLM) chromaticity prediction is a new technique introduced in Versatile Video Coding (VVC), which utilizes the reconstructed luminance component to predict the chromaticity parts, and can improve the coding performance. However, it increases the coding complexity. In this paper, how to accelerate the chroma intra-prediction process is studied based on texture characteristics. Firstly, two observations have been found through experimental statistics for the process. One is that the choice of the chroma intra-prediction candidate modes is closely related to the texture complexity of the coding unit (CU), and the other is that whether the direct mode (DM) is selected is closely related to the texture similarity between current chromaticity CU and the corresponding luminance CU. Secondly, a fast chroma intra-prediction mode decision algorithm is proposed based on these observations. A modified metric named sum modulus difference (SMD) is introduced to measure the texture complexity of CU and guide the filtering of the irrelevant candidate modes. Meanwhile, the structural similarity index measurement (SSIM) is adopted to help judging the selection of the DM mode. The experimental results show that compared with the reference model VTM8.0, the proposed algorithm can reduce the coding time by 12.92% on average, and increases the BD-rate of Y, U, and V components by only 0.05%, 0.32%, and 0.29% respectively.

Thanks to the excellent learning capability of deep convolutional neural networks (CNNs), CNN-based methods have achieved great success in computer vision and image recognition tasks. However, it has turned out that these methods often have inherent vulnerabilities, which makes us cautious of the potential risks of using them for real-world applications such as autonomous driving. To reveal such vulnerabilities, we propose a method of simultaneously attacking monocular depth estimation and optical flow estimation, both of which are common artificial-intelligence-based tasks that are intensively investigated for autonomous driving scenarios. Our method can generate an adversarial patch that can fool CNN-based monocular depth estimation and optical flow estimation methods simultaneously by simply placing the patch in the input images. To the best of our knowledge, this is the first work to achieve simultaneous patch attacks on two or more CNNs developed for different tasks.