Deep convolutional neural networks (deep CNN) show a large power for robust recognition of visual patterns. The neocognitron, which was first proposed by Fukushima (1979), is a network classified to this category. Its architecture was suggested by neurophysiological findings on the visual systems of mammals. It acquires the ability to recognize visual patterns robustly through learning. Although the neocognitron has a long history, improvements of the network are still continuing. This paper discusses the recent neocognitron, focusing on differences from the conventional deep CNN. Some other functions of the visual system can also be realized by networks extended from the neocognitron, for example, recognition of partly occluded patterns, the mechanism of selective attention, and so on.
Deep neural networks are highly nonlinear hierarchical systems. Statistical neurodynamics studies macroscopic behaviors of randomly connected neural networks. We consider a deep feedforward network where input signals are processed layer by layer. The manifold of input signals is embedded in a higher dimensional manifold of the next layer as a curved submanifold, provided the number of neurons is larger than that of inputs. We show geometrical features of the embedded manifold, proving that the manifold enlarges or shrinks locally isotropically so that it is always embedded conformally. We study the curvature of the embedded manifold. The scalar curvature converges to a constant or diverges to infinity slowly. The distance between two signals also changes, converging eventually to a stable fixed value, provided both the number of neurons in a layer and the number of layers tend to infinity. This causes a problem: When we consider a curve in the input space, it is mapped as a continuous curve of fractal nature, but our theory contradictorily suggests that the curve eventually converges to a discrete set of equally spaced points. In reality, the numbers of neurons and layers are finite and thus, it is expected that the finite size effect causes the discrepancies between our theory and reality. Further studies are necessary to understand their implications on information processing.
Bifurcations of limit cycles in an H-bridge LC resonant inverter are reexamined taking into account a time delay in the switching transition. The analysis is accomplished by means of a model of the inverter that compresses, in only three parameters, all the elements associated to both series and parallel topologies of the inverter, to the parasitic effects, to the state feedback control, and to the switching time delay. Emphasis is made in the deviation of preexisting bifurcations without delay and the new ones arising when the time delay is taken into account. It is shown from the analysis and numerical simulations that the delay can degrade the quality of oscillations and even inhibit them, but it is also demonstrated that to some extent, this drawback can be compensated by an appropriate state feedback.
This paper studies cellular automata governed by time-variant rules. The time-variant rule is constructed by switching two simple rules. Depending on the rule and initial conditions, the cellular automata can exhibit various spatiotemporal patterns. In order to investigate the dynamics, we present two simple feature quantities. The first quantity characterizes the expression ability of various spatiotemporal patterns. The second quantity characterizes the error correction ability of target spatiotemporal patterns. Performing numerical experiments, we have found three typical classes of the cellular automata. First, the expression ability is the highest and error correction ability is strong. Second, the error correction ability is very strong whereas the expression ability is very weak. Third, the expression ability is very high whereas the error correction ability is very weak.
A deep Neural Network (DNN) is used to estimate the sleep onset and awaking time of a subject with physiological data during sleep, in order to control the sleeping environment based on sleep phase. The results of the estimation from 40 minutes before the actual sleeping time show approximately 2.8 minutes mean error. Regarding awaking time, the results of the estimation from 120 minutes before show approximately 9.9 minutes mean error. Furthermore, the results of estimation in case of the range from 60 to 20 minutes before the actual awakening time show approximately 7.5 minutes mean error. The proposed DNN estimation is found to be effective for control of a comfortable awaking environment.
Although research on the inference phase of edge artificial intelligence (AI) has made considerable improvement, the required training phase remains an unsolved problem. Neural network (NN) processing has two phases: inference and training. In the training phase, a NN incurs high calculation cost. The number of bits (bitwidth) in the training phase is several orders of magnitude larger than that in the inference phase. Training algorithms, optimized to software, are not appropriate for training hardware-oriented NNs. Therefore, we propose a new training algorithm for edge AI: backpropagation (BP) using a ternarized gradient. This ternarized backpropagation (TBP) provides a balance between calculation cost and performance. Empirical results demonstrate that in a two-class classification task, TBP works well in practice and compares favorably with 16-bit BP (Fixed-BP).
Complex patterns can be often retrieved in spatially-extended systems formed by coupled nonlinear dynamical units. In particular, Turing patterns have been extensively studied investigating mathematical models related to different contexts, such as chemistry, physics, biology, and also mechanics and electronics. In this paper, we focus on the emergence of Turing patterns in a circuit architecture formed by coupled units in which a memrsitive element is considered. Furthermore, the unit is formed by only two elements, namely a capacitor and a memristor. The analytical conditions for which Turing patterns can be obtained in the proposed architecture are discussed in order to inform the design of the circuit parameters. Moreover, the characterization of the different types of patterns is performed with respect to the strength of the diffusion occurring between the units. Finally, it is worth to note that the proposed architecture can be considered as the simplest electronic circuit able to undergo Turing instability and give rise to pattern formation.
We propose a novel single-electron (SE) circuit with unique information-processing that mimics the behavior of bubble film, i.e., a “bubble-inspired SE circuit.” It is known that the behavior of bubble film can be assumed to solve the shortest Steiner problem. In this study, we focus on the behavior to design a novel SE circuit. For this, there are three important points, we believe, for mimicking the behavior for the circuit: 1) the film gradually shrinking, 2) the film stopping shrinking when reaching points like pillars, and 3) films stopping shrinking when they collide with each other. We here designed and tested our SE circuit. By computer simulation, we confirmed that the designed circuit displayed the three processes correctly as desired. Therefore, the “bubble-inspired SE circuit” we designed has the potential for novel information processing.
This paper presents a method of computing sample trajectories of particle models of electrons on symmetric electron-wave stub-filters. The time-independent Schrödinger equation describing the electron-wave filters is transformed to a system of linear algebraic equations. The Fourier series expansion transforms the eigenvalues of the system to continuous and differentiable eigenfunctions. A electron wave packet is constructed from the obtained smooth eigenfunctions and a nonlinear stochastic ordinary differential equation (NSODE) which governs the behavior of the electron particle model is established by applying the stochastic quantization to the wave packet. Sample trajectories of the particles are computed by integrating the NSODE. Numerical experiments have shown that the statistical behavior of the particles is almost equal to that of the wave packets.
This paper provides a strict system formulation for a class of nonholonomic systems with Lie bracket motions via rough path analysis. The dynamics of the resulting systems are represented by rough differential equations as augmented versions of ordinary differential equations. The rough differential equations are allowed to have rough signals generated by unbounded-variation functions and derived by classifying the functions according to “orders” of the boundedness. This paper clarifies rough differential equations driven by third-order rough signals, and the validity for control design problems is confirmed by considering a fourth-order chained system, which is a typical form of driftless affine nonholonomic systems.
In this paper, we apply reinforcement learning to control of an auto-driven vehicle following after a platoon of human-driven vehicles. The auto-driven connected vehicle receives data from the preceding vehicles via a network. We consider the case where dynamics of the human-driven vehicles are unknown and that there are network delays between the human-driven vehicles and the auto-driven one. To stabilize the auto-driven vehicle, we introduce a linear discrete-time model with the delays of a platoon and propose a design method of an adaptive controller for the auto-driven vehicle using reinforcement learning. By simulation, it is shown that the proposed controller learns an optimal gain.
A power packet dispatching system and power packet routers with the power storages have been developed. The system is expected to reduce the number of wires, to provide energy on demand and to be applied to various system working with battery such as electric vehicles, robots and so on. In the previous studies, when a router receives a power packet, it is temporarily charged to the storage and immediately transferred to another router or a load. In this paper, we propose more flexible routing protocol for power packet dispatching systems. We have designed a novel protocol with several commands to operate the routers, to charge power to a storage of a router, to reproduce power packets using the charged power, and so on. We have implemented the proposed protocol on the power packet routers and shown that our protocol can transmit the power packets flexibly.
Nurmuhammad et al. proposed two Sinc-Nyström methods for initial value problems. These two methods use different variable transformations; one method uses a single-exponential transformation and the other uses a double-exponential transformation. In the previous study, we improved their methods by replacing those variable transformations to achieve better performance. However, no error analysis of the improved methods has been conducted yet. Therefore, in this study, we perform error analyses of the improved methods.
Discovering network structures among social actors is one of the most fundamental issues related to social networks. In this paper, we propose a novel and effective algorithm for building a human-interaction network from the location data of individuals gathered by sensors such as the GPS system. We model the problem using Markov random field. The proposed approach combines statistical machine learning with sparse modeling, i.e., the L1 regularized maximum likelihood approach. We demonstrate the validity of our method through numerical experiments using artificial location data generated from a simulator of quasi-human-transfer.
For complex large scale networks, like social networks, it is usually impossible to observe complete information about their topology structure or link weight directly. A recent proposal, the network resonance method, can estimate the eigenvalues and eigenvectors of the Laplacian matrix for representing network structure, by using the resonance phenomena of oscillation dynamics on networks. However, it is generally not possible to observe all the eigenvalues and eigenvectors. This paper uses compressed sensing to create a new method of reconstructing the original Laplacian matrix from a partial set of its eigenvalues and eigenvectors. Since very few node pairs in social networks have links, we can expect that compressed sensing will be effective. The estimated Laplacian matrix of a social network enables to us to determine its structure and link weights.