Updated on 2024/10/10

写真a

 
MURATA, Noboru
 
Affiliation
Faculty of Science and Engineering, School of Advanced Science and Engineering
Job title
Professor
Degree
Doctor in Engineering ( 東京大学 )

Professional Memberships

  •  
     
     

    The Iron and Steel Institute of Japan

  •  
     
     

    計測自動制御学会

  •  
     
     

    電子情報通信学会

Research Interests

  • Mathematical engineering

 

Papers

  • Hawkes process modeling quantifies complicated firing behaviors of cortical neurons during sleep and wakefulness

    Takeshi Kanda, Toshimitsu Aritake, Kaoru Ohyama, Kaspar E Vogt, Yuichi Makino, Thomas McHugh, Hideitsu Hino, Shotaro Akaho, Noboru Murata

       2023.07

     View Summary

    Abstract

    Despite the importance of sleep to the cerebral cortex, how much sleep changes cortical neuronal firing remains unclear due to complicated firing behaviors. Here we quantified firing of cortical neurons using Hawkes process modeling that can model sequential random events exhibiting temporal clusters. “Intensity” is a parameter of Hawkes process that defines the probability of an event occurring. We defined the appearance of repetitive firing as the firing intensity corresponding to “intensity” in Hawkes process. Firing patterns were quantified by the magnitude of firing intensity, the time constant of firing intensity, and the background firing intensity. The higher the magnitude of firing intensity, the higher the likelihood that the spike will continue. The larger the time constant of firing intensity, the longer the repetitive firing lasts. The higher the background firing intensity, the more likely neurons fire randomly. The magnitude of firing intensity was inversely proportional to the time constant of firing intensity, and non-REM sleep increased the magnitude of firing intensity and decreased the time constant of firing intensity. The background firing intensity was not affected by the sleep/wake state. Our findings suggest that the cortex is organized such that neurons with a higher probability of repetitive firing have shorter repetitive firing periods. In addition, our results suggest that repetitive firing is ordered to become high frequency and short term during non-REM sleep, while unregulated components of firing are independent of the sleep/wake state in the cortex. Hawkes process modeling of firing will reveal novel properties of the brain.

    DOI

  • Geometry of EM and related iterative algorithms

    Hideitsu Hino, Shotaro Akaho, Noboru Murata

    Information Geometry    2022.11  [Refereed]  [International journal]

    DOI

  • Detecting cell assemblies by NMF-based clustering from calcium imaging data

    Mizuo Nagayama, Toshimitsu Aritake, Hideitsu Hino, Takeshi Kanda, Takehiro Miyazaki, Masashi Yanagisawa, Shotaro Akaho, Noboru Murata

    Neural Networks    2022.02

    DOI

    Scopus

    5
    Citation
    (Scopus)
  • Fast and robust multiplane single-molecule localization microscopy using a deep neural network

    Toshimitsu Aritake, Hideitsu Hino, Shigeyuki Namiki, Daisuke Asanuma, Kenzo Hirose, Noboru Murata

    Neurocomputing   451   279 - 289  2021.09

    DOI

    Scopus

  • Fast Multi-Objective Optimization of Magnetic Shield Shape by Combining Auto-Encoder and Level-Set Method

    Kanae Oda, Ryota Kawamata, Shinji Wakao, Noboru Murata

    IEEE Transactions on Magnetics   57 ( 7 )  2021.07

     View Summary

    We proposed a new multi-objective optimization approach combining both gradient calculation in actual high-dimensional design space and that in a low-dimensional latent space that is an appropriately compressed design space with Auto-Encoder. The proposed approach enables us to quickly and precisely search for more global pareto solution group in multi-objective optimization by using the optimization results in the low-dimensional space as the initial value of the gradient calculation in the actual design space.

    DOI

    Scopus

    7
    Citation
    (Scopus)
  • Designing Comprehensive Cyber Threat Analysis Platform: Can We Orchestrate Analysis Engines?

    Takeshi Takahashi 0001, Yuki Umemura, Chansu Han, Tao Ban, Keisuke Furumoto, Ohnori Nakamura, Katsunari Yoshioka, Jun'ichi Takeuchi, Noboru Murata, Yoshiaki Shiraishi

    19th IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events     376 - 379  2021

    DOI

    Scopus

    15
    Citation
    (Scopus)
  • Single-molecule localization by voxel-wise regression using convolutional neural network

    Toshimitsu Aritake, Hideitsu Hino, Shigeyuki Namiki, Daisuke Asanuma, Kenzo Hirose, Noboru Murata

    Results in Optics   1   100019 - 100019  2020.11

    DOI

    Scopus

    2
    Citation
    (Scopus)
  • Automation of Vulnerability Classification from its Description using Machine Learning.

    Masaki Aota, Hideaki Kanehara, Masaki Kubo, Noboru Murata, Bo Sun, Takeshi Takahashi 0001

    IEEE Symposium on Computers and Communications(ISCC)     1 - 7  2020

    DOI

    Scopus

    37
    Citation
    (Scopus)
  • Design Optimization of Magnetic Material Distribution by Using Encoder-Decoder with Additive Mixing for Design Conditions

    Ryota Kawamata, Shinji Wakao, Noboru Murata, Yoshifumi Okamoto

    2019 19th International Symposium on Electromagnetic Fields in Mechatronics, Electrical and Electronic Engineering, ISEF 2019    2019.08

     View Summary

    Recently, the deep learning technology attracts much attention in various industrial fields. In our previous research, we developed an Encoder-Decoder precisely reproducing the optimization process of conventional optimization method, that is, the level-set method which is one of the gradient methods, by means of Convolutional Neural Network (CNN) and Long Short-term Memory (LSTM). The developed method enables us to implement high speed search of solutions, which means the possibility of better and effective optimization starting with various initial shapes. This method can deal with only the initial shape as design parameter for optimization. Thus, it is necessary to re-train the Encoder-Decoder when the design conditions change, e.g., the displacement of permanent magnet. To overcome this drawback, we have developed a novel network structure to incorporate the design conditions into the training data. Finally, to confirm the validity of the proposed method, we evaluate its calculation time and computational accuracy by using a magnetic circuit design model.

    DOI

    Scopus

  • Application of Conditional Variational Auto-Encoder to Magnetic Circuit Design with Magnetic Field Computation

    Ryota Kawamata, Shinji Wakao, Noboru Murata

    COMPUMAG 2019 - 22nd International Conference on the Computation of Electromagnetic Fields    2019.07

     View Summary

    In the design optimization of electric machine, we ordinarily derive the objective physical quantities, e.g., the shape of the investigated model as design variables, by using numerical method such as the finite element method with the analysis conditions. In recent years, the representation learning using Deep Learning much attracts attention because it can acquire the features of data as a distributed representation and reproduce corresponding data. In this paper, utilizing machine learning technology, we propose an application of Conditional Variational Auto-Encoder (CVAE) to reproduce the more adequate shape of magnetic materials, i.e., design variables, corresponding to the intended magnetic energy, i.e., objective function values.

    DOI

    Scopus

    2
    Citation
    (Scopus)
  • Development of Encoder-Decoder Predicting Search Process of Level-set Method in Magnetic Circuit Design

    Ryota Kawamata, Shinji Wakao, Noboru Murata, Yoshifumi Okamoto

    COMPUMAG 2019 - 22nd International Conference on the Computation of Electromagnetic Fields    2019.07

     View Summary

    The finite element analysis (FEA) of magnetic field generally requires a lot of calculation time. Especially, design optimization methods such as the level-set method with FEA result in large computational effort to find better solution. In this paper, we propose a novel method of precisely and quickly reproducing the conventional optimization steps by means of Convolutional Neural Network (CNN) and Long Short-term Memory (LSTM). The developed method enables us to implement high speed search of solution, which means the possibility of effective optimization with various initial conditions for better solution. Finally, we evaluate calculation time and computational accuracy of the proposed method by using a magnetic circuit design model.

    DOI

    Scopus

    2
    Citation
    (Scopus)
  • Information Geometry of Modal Linear Regression

    Keishi Sando, Shotaro Akaho, Noboru Murata, Hideitsu Hino

    Information Geometry    2019.07  [Refereed]

    DOI

    Scopus

    4
    Citation
    (Scopus)
  • 非負値行列因子分解を用いたカルシウムイメージングデータからの睡眠状態解析

    永山瑞生, 有竹俊光, 日野英逸, 上田壮志, 宮崎峻弘, 柳沢正史, 赤穂昭太郎, 村田昇

    情報論的学習理論と機械学習研究会 (IBISML)    2019.06

  • Localizing Current Dipoles from EEG Data Using a Birth-Death Process

    Keita Nakamura, Sho Sonoda, Hideitsu Hino, Masahiro Kawasaki, Shotaro Akaho, Noboru Murata

    Proceedings - 2018 IEEE International Conference on Bioinformatics and Biomedicine, BIBM 2018     2645 - 2651  2019.01

     View Summary

    A common approach to the electroencephalogram (EEG) source localization problem is to estimate the states of current dipoles. However, the dipole estimation problem is difficult because not only is it an inverse problem but also the number of dipoles can change over time. In this paper, we model the relationship between current dipoles and EEG observations using a state-space model where the creation and annihilation of dipoles is represented as a birth-death process. We estimate the dipoles' positions and moments with a Rao-Blackwellized particle filter and estimate whether a new dipole has been created or an existing one annihilated via the Bayesian information criterion. Experiments on both synthetic and real data show that the proposed model and estimation method can effectively estimate the number and positions of the dipoles.

    DOI

    Scopus

    1
    Citation
    (Scopus)
  • Retrieved Image Refinement by Bootstrap Outlier Test

    Hayato Watanabe, Hideitsu Hino, Shotaro Akaho, Noboru Murata

    Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)   11678   505 - 517  2019

     View Summary

    Outlier detection is used to identify data points or a small number of subsets of data that are significantly different from most other data in a given dataset. It is challenging to detect outliers using an objective and quantitative approach. Methods that use the framework of statistical hypothesis testing are widely used by assuming a specific parametric distribution as a data generation model, but there is no guarantee that the distribution of data can be adequately approximated by a parametric distribution in practical problems. In this paper, a simple method is proposed to objectively detect outliers by hypothesis testing without assuming a specific distribution of outlier scores. By using an arbitrary outlier score function, hypothesis testing is used to determine whether each given sample is an outlier. The distribution of the test statistics is needed for the hypothesis test, and is estimated based on the given data using the bootstrap method. The effectiveness of the proposed outlier test was verified by applying it to outlier detection for text-based image retrieval, where it improved the quality of image searches by removing irrelevant images.

    DOI

    Scopus

    1
    Citation
    (Scopus)
  • Learning Scale and Shift-Invariant Dictionary for Sparse Representation

    Aritake, T., Murata, N.

    Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)   11943 LNCS  2019

    DOI

    Scopus

  • On a Convergence Property of a Geometrical Algorithm for Statistical Manifolds

    Shotaro Akaho, Hideitsu Hino, Noboru Murata

    Communications in Computer and Information Science     262 - 272  2019

    DOI

    Scopus

    1
    Citation
    (Scopus)
  • Real-time botnet detection using nonnegative tucker decomposition.

    Hideaki Kanehara, Yuma Murakami, Jumpei Shimamura, Takeshi Takahashi 0001, Daisuke Inoue, Noboru Murata

    Proceedings of the 34th ACM/SIGAPP Symposium on Applied Computing(SAC)     1337 - 1344  2019

    DOI

    Scopus

    8
    Citation
    (Scopus)
  • Transport Analysis of Infinitely Deep Neural Network

    Sho Sonoda, Noboru Murata

    Journal of Machine Learning Research   20 ( 2 ) 1 - 52  2019  [Refereed]

  • Sleep State Analysis using Calcium Imaging Data by Non-negative Matrix Factorization.

    Nagayama, Mizuo, Aritake, Toshimitsu, Hino, Hideitsu, Kanda, Takeshi, Miyazaki, Takehiro, Yanagisawa, Masashi, Akaho, Shotaro, Murata, Noboru

    Artificial Neural Networks and Machine Learning – ICANN 2019: Theoretical Neural Computation.   11727   102 - 113  2019  [Refereed]

    DOI

    Scopus

    2
    Citation
    (Scopus)
  • EEG dipole source localization with information criteria for multiple particle filters.

    Sho Sonoda, Keita Nakamura, Yuki Kaneda, Hideitsu Hino, Shotaro Akaho, Noboru Murata, Eri Miyauchi, Masahiro Kawasaki

    Neural networks : the official journal of the International Neural Network Society   108   68 - 82  2018.12  [Refereed]  [International journal]

     View Summary

    Electroencephalography (EEG) is a non-invasive brain imaging technique that describes neural electrical activation with good temporal resolution. Source localization is required for clinical and functional interpretations of EEG signals, and most commonly is achieved via the dipole model; however, the number of dipoles in the brain should be determined for a reasonably accurate interpretation. In this paper, we propose a dipole source localization (DSL) method that adaptively estimates the dipole number by using a novel information criterion. Since the particle filtering process is nonparametric, it is not clear whether conventional information criteria such as Akaike's information criterion (AIC) and Bayesian information criterion (BIC) can be applied. In the proposed method, multiple particle filters run in parallel, each of which respectively estimates the dipole locations and moments, with the assumption that the dipole number is known and fixed; at every time step, the most predictive particle filter is selected by using an information criterion tailored for particle filters. We tested the proposed information criterion first through experiments on artificial datasets; these experiments supported the hypothesis that the proposed information criterion would outperform both AIC and BIC. We then analyzed real human EEG datasets collected during an auditory short-term memory task using the proposed method. We found that the alpha-band dipoles were localized to the right and left auditory areas during the auditory short-term memory task, which is consistent with previous physiological findings. These analyses suggest the proposed information criterion can work well in both model and real-world situations.

    DOI PubMed

    Scopus

    4
    Citation
    (Scopus)
  • Temporal Interpolation of Gridded Solar Radiation Data for Evaluation of PV Fluctuations

    Daigo Hirooka, Noboru Murata, Yu Fujimoto, Yasuhiro Hayashi

    Energy Procedia   155   259 - 268  2018.11  [Refereed]

    DOI

    Scopus

    5
    Citation
    (Scopus)
  • Estimationofneuralconnectionsfrompartiallyobservedneuralspikes

    Taishi Iwasaki, Hideitsu Hino, Masami Tatsuno, Shotaro Akaho, Noboru Murata

    Neural Networks   108   172 - 191  2018.08  [Refereed]

  • Distributed Energy Management for Comprehensive Utilization of Residential Photovoltaic Outputs

    Yu Fujimoto, Hiroshi Kikusato, Shinya Yoshizawa, Shunsuke Kawano, Akira Yoshida, Shinji Wakao, Noboru Murata, Yoshiharu Amano, Shin-ichi Tanabe, Yasuhiro Hayashi

    IEEE TRANSACTIONS ON SMART GRID   9 ( 2 ) 1216 - 1227  2018.03  [Refereed]

     View Summary

    The introduction of photovoltaic power systems is being significantly promoted. This paper proposes the implementation of a distributed energy management framework linking demand-side management systems and supply-side management system under the given time-of-use pricing program for efficient utilization of photovoltaic power outputs; each system implements a consistent management flow composed of forecasting, operation planning, and control steps. In our framework, demand-side systems distributed in the electric distribution network manage individual energy consumption to reduce the residential operating cost by utilizing the residential photovoltaic power system and controllable energy appliances so as not to inconvenience residents. On the other hand, the supply-side system utilizes photovoltaic power maximally while maintaining the quality of electric power. The effectiveness of the proposed framework is evaluated on the basis of an actual Japanese distribution network simulation model from both the supply-side and demand-side viewpoints.

    DOI

    Scopus

    49
    Citation
    (Scopus)
  • Estimation of neural connections from partially observed neural spikes

    Iwasaki T, Hino H, Tatsuno M, Akaho S, Murata N

    Neural Networks   108   172 - 191  2018  [Refereed]

    DOI

    Scopus

    2
    Citation
    (Scopus)
  • Information geometric perspective of modal linear regression

    Keishi Sando, Shotaro Akaho, Noboru Murata, Hideitsu Hino

    Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)   11303   535 - 545  2018

     View Summary

    Modal linear regression (MLR) is a standard method for modeling the conditional mode of a response variable using a linear combination of explanatory variables. It is effective when dealing with response variables with an asymmetric, multi-modal distribution. Because of the nonparametric nature of MLR, it is difficult to construct a statistical model manifold in the sense of information geometry. In this work, a model manifold is constructed using observations instead of explicit parametric models. We also propose a method for constructing a data manifold based on an empirical distribution. The em algorithm, which is a geometric formulation of the EM algorithm, of MLR is shown to be equivalent to the conventional EM algorithm of MLR.

    DOI

    Scopus

    2
    Citation
    (Scopus)
  • Geometrical formulation of the nonnegative matrix factorization

    Shotaro Akaho, Hideitsu Hino, Neneka Nara, Noboru Murata

    Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)   11303   525 - 534  2018

     View Summary

    Nonnegative matrix factorization (NMF) has many applications as a tool for dimension reduction. In this paper, we reformulate the NMF from an information geometrical viewpoint. We show that a conventional optimization criterion is not geometrically natural, thus we propose to use more natural criterion. By this formulation, we can apply a geometrical algorithm based on the Pythagorean theorem. We also show the algorithm can improve the existing algorithm through numerical experiments.

    DOI

    Scopus

    2
    Citation
    (Scopus)
  • Integral representation of the global minimizer.

    Sho Sonoda, Isao Ishikawa, Masahiro Ikeda, Kei Hagihara, Yoshihiro Sawano, Takuo Matsubara, Noboru Murata

    CoRR   abs/1805.07517  2018  [Refereed]

  • Neural network with unbounded activation functions is universal approximator

    Sho Sonoda, Noboru Murata

    APPLIED AND COMPUTATIONAL HARMONIC ANALYSIS   43 ( 2 ) 233 - 268  2017.09  [Refereed]

     View Summary

    This paper presents an investigation of the approximation property of neural networks with unbounded activation functions, such as the rectified linear unit (ReLU), which is the new de-facto standard of deep learning. The ReLU network can be analyzed by the ridgelet transform with respect to Lizorkin distributions. By showing three reconstruction formulas by using the Fourier slice theorem, the Radon transform, and Parseval's relation, it is shown that a neural network with unbounded activation functions still satisfies the universal approximation property. As an additional consequence, the ridgelet transform, or the backprojection filter in the Radon domain, is what the network learns after backpropagation. Subject to a constructive admissibility condition, the trained network can be obtained by simply discretizing the ridgelet transform, without backpropagation. Numerical examples not only support the consistency of the admissibility condition but also imply that some non-admissible cases result in low-pass filtering. (C) 2015 Elsevier Inc. All rights reserved.

    DOI

    Scopus

    196
    Citation
    (Scopus)
  • Local Intrinsic Dimension Estimation by Generalized Linear Modeling

    Hino, Hideitsu, Fujiki, Jun, Akaho, Shotaro, Murata, Noboru

    Neural computation   29 ( 7 ) 1838 - 1878  2017.07  [Refereed]

     View Summary

    We propose a method for intrinsic dimension estimation. By fitting the power of distance from an inspection point and the number of samples included inside a ball with a radius equal to the distance, to a regression model, we estimate the goodness of fit. Then, by using the maximum likelihood method, we estimate the local intrinsic dimension around the inspection point. The proposed method is shown to be comparable to conventional methods in global intrinsic dimension estimation experiments. Furthermore, we experimentally show that the proposed method outperforms a conventional local dimension estimation method.

    DOI

    Scopus

    5
    Citation
    (Scopus)
  • Local Intrinsic Dimension Estimation by Generalized Linear Modeling

    Hideitsu Hino, Jun Fujiki, Shotaro Akaho, Noboru Murata

    NEURAL COMPUTATION   29 ( 7 ) 1838 - 1878  2017.07  [Refereed]

     View Summary

    We propose a method for intrinsic dimension estimation. By fitting the power of distance from an inspection point and the number of samples included inside a ball with a radius equal to the distance, to a regression model, we estimate the goodness of fit. Then, by using the maximum likelihood method, we estimate the local intrinsic dimension around the inspection point. The proposed method is shown to be comparable to conventional methods in global intrinsic dimension estimation experiments. Furthermore, we experimentally show that the proposed method outperforms a conventional local dimension estimation method.

    DOI

    Scopus

    5
    Citation
    (Scopus)
  • Double sparsity for multi-frame super resolution

    Toshiyuki Kato, Hideitsu Hino, Noboru Murata

    NEUROCOMPUTING   240   115 - 126  2017.05  [Refereed]

     View Summary

    A number of image super resolution algorithms based on the sparse coding have successfully implemented multi-frame super resolution in recent years. In order to utilize multiple low-resolution observations, both accurate image registration and sparse coding are required. Previous study on multi-frame super resolution based on sparse coding firstly apply block matching for image registration, followed by sparse coding to enhance the image resolution. In this paper, these two problems are solved by optimizing a single objective function. The proposed formulation not only has a mathematically interesting structure, called the double sparsity, but also yields comparable or improved numerical performance to conventional methods. (C) 2017 Elsevier B.V. All rights reserved.

    DOI

    Scopus

    20
    Citation
    (Scopus)
  • Time-Varying Transition Probability Matrix Estimation and Its Application to Brand Share Analysis

    Tomoaki Chiba, Hideitsu Hino, Shotaro Akaho, Noboru Murata

    PLOS ONE   12 ( 1 )  2017.01  [Refereed]

     View Summary

    In a product market or stock market, different products or stocks compete for the same consumers or purchasers. We propose a method to estimate the time-varying transition matrix of the product share using a multivariate time series of the product share. The method is based on the assumption that each of the observed time series of shares is a stationary distribution of the underlying Markov processes characterized by transition probability matrices. We estimate transition probability matrices for every observation under natural assumptions. We demonstrate, on a real-world dataset of the share of automobiles, that the proposed method can find intrinsic transition of shares. The resulting transition matrices reveal interesting phenomena, for example, the change in flows between TOYOTA group and GM group for the fiscal year where TOYOTA group's sales beat GM's sales, which is a reasonable scenario.

    DOI

    Scopus

    6
    Citation
    (Scopus)
  • Nonparametric e-Mixture Estimation

    Ken Takano, Hideitsu Hino, Shotaro Akaho, Noboru Murata

    NEURAL COMPUTATION   28 ( 12 ) 2687 - 2725  2016.12  [Refereed]

     View Summary

    This study considers the common situation in data analysis when there are few observations of the distribution of interest or the target distribution, while abundant observations are available from auxiliary distributions. In this situation, it is natural to compensate for the lack of data from the target distribution by using data sets from these auxiliary distributionsin other words, approximating the target distribution in a subspace spanned by a set of auxiliary distributions. Mixture modeling is one of the simplest ways to integrate information from the target and auxiliary distributions in order to express the target distribution as accurately as possible. There are two typical mixtures in the context of information geometry: the The is applied in a variety of research fields because of the presence of the well-known expectation-maximazation algorithm for parameter estimation, whereas the e-mixture is rarely used because of its difficulty of estimation, particularly for nonparametric models. The e-mixture, however, is a well-tempered distribution that satisfies the principle of maximum entropy. To model a target distribution with scarce observations accurately, this letter proposes a novel framework for a nonparametric modeling of the e-mixture and a geometrically inspired estimation algorithm. As numerical examples of the proposed framework, a transfer learning setup is considered. The experimental results show that this framework works well for three types of synthetic data sets, as well as an EEG real-world data set.

    DOI

    Scopus

    8
    Citation
    (Scopus)
  • Optical detection of neuron connectivity by random access two-photon microscopy

    Nasrin Shafeghat, Morteza Heidarinejad, Noboru Murata, Hideki Nakamura, Takafumi Inoue

    JOURNAL OF NEUROSCIENCE METHODS   263   48 - 56  2016.04  [Refereed]

     View Summary

    Background: Knowledge about the distribution, strength, and direction of synaptic connections within neuronal networks are crucial for understanding brain function. Electrophysiology using multiple electrodes provides a very high temporal resolution, but does not yield sufficient spatial information for resolving neuronal connection topology. Optical recording techniques using single-cell resolution have provided promise for providing spatial information. Although calcium imaging from hundreds of neurons has provided a novel view of the neural connections within the network, the kinetics of calcium responses are not fast enough to resolve each action potential event with high fidelity. Therefore, it is not possible to detect the direction of neuronal connections.
    New method: We took advantage of the fast kinetics and large dynamic range of the DiO/DPA combination of voltage sensitive dye and the fast scan speed of a custom-made random-access two-photon microscope to resolve each action potential event from multiple neurons in culture.
    Results: Long-duration recording up to 100 min from cultured hippocampal neurons yielded sufficient numbers of spike events for analyzing synaptic connections. Cross-correlation analysis of neuron pairs clearly distinguished synaptically connected neuron pairs with the connection direction.
    Comparison with existing method: The long duration recording of action potentials with voltage-sensitive dye utilized in the present study is much longer than in previous studies. Simultaneous optical voltage and calcium measurements revealed that voltage-sensitive dye is able to detect firing events more reliably than calcium indicators.
    Conclusions: This novel method reveals a new view of the functional structure of neuronal networks. (C) 2016 Elsevier B.V. All rights reserved.

    DOI

    Scopus

    7
    Citation
    (Scopus)
  • Reproducing Statistical Property of Short-term Fluctuation in Wind Power Profiles

    Seigo Furuya, Yu Fujimoto, Noboru Murata, Yasuhiro Hayashi

    10TH INTERNATIONAL RENEWABLE ENERGY STORAGE CONFERENCE, IRES 2016   99   130 - 136  2016  [Refereed]

     View Summary

    Unexpected fluctuation of wind power output will become serious problems from the viewpoint of stable supply for an electricity grid. Operating a battery system installed in the grid for mitigating the short-term fluctuation is one of the new approaches for grid stabilization. In this paper, we propose a method of generating synthetic wind power profiles with high temporal resolution for power flow simulation which aims to estimate the impact of wind power fluctuation and specify the required battery system. We numerically show the plausibility of the synthetic wind power profiles from the viewpoints of statistical properties. (C) 2016 The Authors. Published by Elsevier Ltd.

    DOI

    Scopus

    1
    Citation
    (Scopus)
  • Non-parametric e-mixture of Density Functions

    Hideitsu Hino, Ken Takano, Shotaro Akaho, Noboru Murata

    NEURAL INFORMATION PROCESSING, ICONIP 2016, PT II   9948   3 - 10  2016  [Refereed]

     View Summary

    Mixture modeling is one of the simplest ways to represent complicated probability density functions, and to integrate information from different sources. There are two typical mixtures in the context of information geometry, the m- and e-mixtures. This paper proposes a novel framework of non-parametric e-mixture modeling by using a simple estimation algorithm based on geometrical insights into the characteristics of the e-mixture. An experimental result supports the proposed framework.

    DOI

    Scopus

    1
    Citation
    (Scopus)
  • MDL Criterion for NMF with Application to Botnet Detection

    Shoma Tanaka, Yuki Kawamura, Masanori Kawakita, Noboru Murata, Jun'ichi Takeuchi

    NEURAL INFORMATION PROCESSING, ICONIP 2016, PT I   9947   570 - 578  2016  [Refereed]

     View Summary

    A method for botnet detection from traffic data of the Internet by the Non-negative Matrix Factorization (NMF) was proposed by (Yamauchi et al. 2012). This method assumes that traffic data is composed by several types of communications, and estimates the number of types in the data by the minimum description length (MDL) criterion. However, consideration on the MDL criterion was not sufficient and validity has not been guaranteed. In this paper, we refine the MDL criterion for NMF and report results of experiments for the new MDL criterion on synthetic and real data.

    DOI

    Scopus

    2
    Citation
    (Scopus)
  • Graph structure modeling for multi-neuronal spike data

    Shotaro Akaho, Sho Higuchi, Taishi Iwasaki, Hideitsu Hino, Masami Tatsuno, Noboru Murata

    INTERNATIONAL MEETING ON HIGH-DIMENSIONAL DATA-DRIVEN SCIENCE (HD3-2015)   699 ( 1 )  2016  [Refereed]

     View Summary

    We propose a method to extract connectivity between neurons for extracellularly recorded multiple spike trains. The method removes pseudo-correlation caused by propagation of information along an indirect pathway, and is also robust against the influence from unobserved neurons. The estimation algorithm consists of iterations of a simple matrix inversion, which is scalable to large data sets. The performance is examined by synthetic spike data.

    DOI

    Scopus

  • DOUBLY SPARSE STRUCTURE IN IMAGE SUPER RESOLUTION

    Toshiyuki Kato, Hideitsu Hino, Noboru Murata

    2016 IEEE 26TH INTERNATIONAL WORKSHOP ON MACHINE LEARNING FOR SIGNAL PROCESSING (MLSP)   2016-November  2016  [Refereed]

     View Summary

    There are a large number of image super resolution algorithms based on the sparse coding, and some algorithms realize multi-frame super resolution. For utilizing multiple low resolution observations, both accurate image registration and sparse coding are required. Previous study on multi-frame super resolution based on sparse coding firstly apply block matching for image registration, followed by sparse coding to enhance the image resolution. In this paper, these two problems are solved by optimizing a single objective function. The proposed formulation not only has a mathematically interesting structure called the double sparsity, but also offers improved numerical performance.

    DOI

    Scopus

    4
    Citation
    (Scopus)
  • Change-Point Detection in a Sequence of Bags-of-Data

    Kensuke Koshijima, Hideitsu Hino, Noboru Murata

    2016 32ND IEEE INTERNATIONAL CONFERENCE ON DATA ENGINEERING (ICDE)     1560 - 1561  2016  [Refereed]

     View Summary

    In this paper, the limitation that is prominent in most existing works of change-point detection methods is addressed by proposing a nonparametric, computationally efficient method. The limitation is that most works assume that each data point observed at each time step is a single multi-dimensional vector. However, there are many situations where this does not hold. Therefore, a setting where each observation is a collection of random variables, which we call a bag of data, is considered.

    DOI

    Scopus

    1
    Citation
    (Scopus)
  • An Entropy Estimator Based on Polynomial Regression with Poisson Error Structure

    Hideitsu Hino, Shotaro Akaho, Noboru Murata

    NEURAL INFORMATION PROCESSING, ICONIP 2016, PT II   9948   11 - 19  2016  [Refereed]

     View Summary

    A method for estimating Shannon differential entropy is proposed based on the second order expansion of the probability mass around the inspection point with respect to the distance from the point. Polynomial regression with Poisson error structure is utilized to estimate the values of density function. The density estimates at every given data points are averaged to obtain entropy estimators. The proposed estimator is shown to perform well through numerical experiments for various probability distributions.

    DOI

    Scopus

    1
    Citation
    (Scopus)
  • mmpp: A Package for Calculating Similarity and Distance Metrics for Simple and Marked Temporal Point Processes

    Hideitsu Hino, Ken Takano, Noboru Murata

    R JOURNAL   7 ( 2 ) 237 - 248  2015.12  [Refereed]

     View Summary

    A simple temporal point process (SPP) is an important class of time series, where the sample realization of the process is solely composed of the times at which events occur. Particular examples of point process data are neuronal spike patterns or spike trains, and a large number of distance and similarity metrics for those data have been proposed. A marked point process (MPP) is an extension of a simple temporal point process, in which a certain vector valued mark is associated with each of the temporal points in the SPP. Analyses of MPPs are of practical importance because instances of MPPs include recordings of natural disasters such as earthquakes and tornadoes. In this paper, we introduce the R package mmpp, which implements a number of distance and similarity metrics for SPPs, and also extends those metrics for dealing with MPPs.

  • Change-Point Detection in a Sequence of Bags-of-Data

    Kensuke Koshijima, Hideitsu Hino, Noboru Murata

    IEEE TRANSACTIONS ON KNOWLEDGE AND DATA ENGINEERING   27 ( 10 ) 2632 - 2644  2015.10  [Refereed]

     View Summary

    In this paper, the limitation that is prominent in most existing works of change-point detection methods is addressed by proposing a nonparametric, computationally efficient method. The limitation is that most works assume that each data point observed at each time step is a single multi-dimensional vector. However, there are many situations where this does not hold. Therefore, a setting where each observation is a collection of random variables, which we call a bag of data, is considered. After estimating the underlying distribution behind each bag of data and embedding those distributions in a metric space, the change-point score is derived by evaluating how the sequence of distributions is fluctuating in the metric space using a distance-based information estimator. Also, a procedure that adaptively determines when to raise alerts is incorporated by calculating the confidence interval of the change-point score at each time step. This avoids raising false alarms in highly noisy situations and enables detecting changes of various magnitudes. A number of experimental studies and numerical examples are provided to demonstrate the generality and the effectiveness of our approach with both synthetic and real datasets.

    DOI

    Scopus

    2
    Citation
    (Scopus)
  • Non-parametric entropy estimators based on simple linear regression

    Hideitsu Hino, Kensuke Koshijima, Noboru Murata

    COMPUTATIONAL STATISTICS & DATA ANALYSIS   89   72 - 84  2015.09  [Refereed]

     View Summary

    Estimators for differential entropy are proposed. The estimators are based on the second order expansion of the probability mass around the inspection point with respect to the distance from the point. Simple linear regression is utilized to estimate the values of density function and its second derivative at a point. After estimating the values of the probability density function at each of the given sample points, by taking the empirical average of the negative logarithm of the density estimates, two entropy estimators are derived. Other entropy estimators which directly estimate entropy by linear regression, are also proposed. The proposed four estimators are shown to perform well through numerical experiments for various probability distributions. (C) 2015 Elsevier B.V. All rights reserved.

    DOI

    Scopus

    9
    Citation
    (Scopus)
  • Multi-frame image super resolution based on sparse coding

    Toshiyuki Kato, Hideitsu Hino, Noboru Murata

    NEURAL NETWORKS   66   64 - 78  2015.06  [Refereed]

     View Summary

    An image super-resolution method from multiple observation of low-resolution images is proposed. The method is based on sub-pixel accuracy block matching for estimating relative displacements of observed images, and sparse signal representation for estimating the corresponding high-resolution image, where correspondence between high- and low-resolution images are modeled by a certain degradation process. Relative displacements of small patches of observed low-resolution images are accurately estimated by a computationally efficient block matching method. The matching scores of the block matching are used to select a subset of low-resolution patches for reconstructing a high-resolution patch, that is, an adaptive selection of informative low-resolution images is realized. The proposed method is shown to perform comparable or superior to conventional super-resolution methods through experiments using various images. (C) 2015 Elsevier Ltd. All rights reserved.

    DOI

    Scopus

    33
    Citation
    (Scopus)
  • Patchworking Multiple Pairwise Distances for Learning with Distance Matrices

    Ken Takano, Hideitsu Hino, Yuki Yoshikawa, Noboru Murata

    LATENT VARIABLE ANALYSIS AND SIGNAL SEPARATION, LVA/ICA 2015   9237   287 - 294  2015  [Refereed]

     View Summary

    A classification framework using only a set of distance matrices is proposed. The proposed algorithm can learn a classifier only from a set of distance matrices or similarity matrices, hence applicable to structured data, which do not have natural vector representation such as time series and graphs. Random forest is used to explore ideal feature representation based on the distance between points defined by a set of given distance matrices. The effectiveness of the proposed method is evaluated through experiments with point process data and graph structured data.

    DOI

    Scopus

    1
    Citation
    (Scopus)
  • Effects of a MIP start for solving weekly operational planning problem of a residential energy system

    Yoshida A, Yoshikawa J, Murata N, Amano Y

    ICOPE 2015 - International Conference on Power Engineering    2015  [Refereed]

     View Summary

    An optimal operational planning problem of residential energy system has been formulated by Mixed Integer Linear Programming (MILP). The decision variables of optimal operational planning problem are energy and mass flows, equipment's operating statuses, and energy level of storage. Many kinds of energy supply equipment are available for householders in Japan. Of course, in operational planning problem, the increase of integer variables which means equipment's on/off status, is linked to the increase of calculation time. It is important to assess the impact of introducing an energy system for a house based on the suitable planning horizon of this problem. Energy storage brings the energy in the form of hot water and electricity to the next d a y. The operational strategy of energy system including storage should be evaluated through few days toward various energy demand. The optimal planning problems become large scale because many pieces of equipment to introduce and long evaluation period are required. This paper analyzes characteristics of energy systems caused by planning horizon. Additionally, we propose a hierarchical method as heuristic method for solving large MILP problem easily, and the proposed method is tested the effectiveness. Our finding shows that the proposed search method for better feasible solution has good performance in comparison with default settings of conventional MILP solver in terms of calculation time.

    DOI

  • Economic evaluations of residential energy systems based on the prediction-operational planning-control method using time-of-use prices

    Ogata R, Yoshida A, Fujimoto Y, Murata N, Wakao S, Tanabe S.-I, Amano Y

    ECOS 2015 - 28th International Conference on Efficiency, Cost, Optimization, Simulation and Environmental Impact of Energy Systems    2015  [Refereed]

     View Summary

    Demand-side electricity saving is an important factor in the reduction of the installed capacity of power-supply facilities. In order to save electricity automatically while maintaining comfort levels, home energy management systems (HEMS) have attracted attention. These systems can control residential energy equipment cooperatively to reduce electricity consumption while considering benefits to consumers. Although many researchers have evaluated HEMS, no one has conducted a study which considers the control of various types of residential energy equipment in real time along with the uncertainties of energy demands. This paper proposes a single HEMS method which connects prediction, operational planning and control steps and enables the evaluation of operational planning methods of HEMS connected with many kinds of residential energy equipment currently in use in Japan while considering the uncertainties. The purpose of this study is to evaluate the economic potential of residential energy systems based on the proposed method with the uncertainties of energy demands and photovoltaic (PV) output under time-of-use prices. The results allowed us to establish a framework to quantitatively evaluate the operational planning methods of HEMS with the uncertainties of energy demands and PV output. In addition, the usability of the proposed method was confirmed by comparing the operational costs to those of a reference method.

  • Analytical Estimation of the Convergence Point of Populations

    Noboru Murata, Ryuei Nishii, Hideyuki Takagi, Yan Pei

    2015 IEEE CONGRESS ON EVOLUTIONARY COMPUTATION (CEC)     2619 - 2624  2015  [Refereed]

     View Summary

    We propose methods of estimating the convergence point for the moving vectors of individuals between generations or evolution paths and show that the estimated convergence point can be useful information for accelerating evolutionary computation (EC). As the first stage of this new approach, we do not combine the proposed methods with EC search in this paper, but rather evaluate how power an individual the the estimated convergence point is by comparing fitness values. Through experimental evaluations, we show that the estimated point can be a powerful elite for unimodal fitness landscapes and that clustering moving vectors according to the aimed points is the next research target for multimodal fitness landscape.

    DOI

    Scopus

    15
    Citation
    (Scopus)
  • Advantage of a home energy management systems for PV utilization connected to grid

    Yoshida A, Yoshizawa S, Fujimoto Y, Murata N, Wakao S, Tanabe S.-I, Hayashi Y, Amano Y

    ECOS 2015 - 28th International Conference on Efficiency, Cost, Optimization, Simulation and Environmental Impact of Energy Systems    2015  [Refereed]

     View Summary

    Residential houses are in the process of introducing power generators such as photovoltaic (PV) power generators and fuel cell cogeneration systems. Under current laws in Japan, surplus electricity from a residential PV system can be sold by feeding it back into the electrical grid. However, when a lot of neighboring power generators make and feed back electricity at the same time, there is an issue of an upper voltage violation of the provisions of the laws and regulations relating to the Electricity Business Act in the distribution system of the electrical grid. The issue has been solved by stopping power generators when they reach an upper voltage limit, 107V. Another solution is to acknowledge demand response signal from electrical grid operator to store electricity in residential batteries using Home Energy Management System (HEMS). The research question is that how to collaborate HEMS and Grid Energy Management System (GEMS). This paper developed an evaluation framework for the cooperative behavior between HEMS and GEMS. Using this evaluation framework, this paper demonstrated the optimal operational strategy of HEMS including PV in the case that HEMS is informed voltage profile from GEMS, and also assessed amount of PV suppression quantitatively in residential sector in practical aspect.

  • Advantage of a home energy management system for PV utilization connected to grid

    Yoshida A, Yoshizawa S, Fujimoto Y, Murata N, Wakao S, Tanabe S.-I, Hayashi Y, Amano Y

    ECOS 2015 - 28th International Conference on Efficiency, Cost, Optimization, Simulation and Environmental Impact of Energy Systems    2015  [Refereed]

     View Summary

    Residential houses are in the process of introducing power generators such as photovoltaic (PV) power generators and fuel cell cogeneration systems. Under current laws in Japan, surplus electricity from a residential PV system can be sold by feeding it back into the electrical grid. However, when a lot of neighboring power generators make and feed back electricity at the same time, there is an issue of an upper voltage violation of the provisions of the laws and regulations relating to the Electricity Business Act in the distribution system of the electrical grid. The issue has been solved by stopping power generators when they reach an upper voltage limit, 107V. Another solution is to acknowledge demand response signal from electrical grid operator to store electricity in residential batteries using Home Energy Management System (HEMS). The research question is that how to collaborate HEMS and Grid Energy Management System (GEMS). This paper developed an evaluation framework for the cooperative behavior between HEMS and GEMS. Using this evaluation framework, this paper demonstrated the optimal operational strategy of HEMS including PV in the case that HEMS is informed voltage profile from GEMS, and also assessed amount of PV suppression quantitatively in residential sector in practical aspect.

  • Prediction of Molten Steel Temperature in Steel Making Process with Uncertainty by Integrating Gray-Box Model and Bootstrap Filter

    Iftikhar Ahmad, Manabu Kano, Shinji Hasebe, Hiroshi Kitada, Noboru Murata

    JOURNAL OF CHEMICAL ENGINEERING OF JAPAN   47 ( 11 ) 827 - 834  2014.11  [Refereed]

     View Summary

    Stable operation of a continuous casting process requires precise control of molten steel temperature in a tundish (TD temp), which is a container used to feed molten steel into an ingot mold. Since TD temp is implicitly controlled by adjusting molten steel temperature in the preceding secondary refining process (RH temp), a model relating TD temp with RH temp is required. This research proposes a procedure to predict the probability distribution of TD temp by integrating a gray-box model and a bootstrap filter to cope with uncertainties of the process. The derived probability distribution is used not only to predict TD temp but also to evaluate the reliability of prediction. The proposed method was validated through its application to real operation data at a steel work, and it was confirmed that the developed model satisfied the requirements for its industrial application.

    DOI

    Scopus

    15
    Citation
    (Scopus)
  • A Nonparametric Clustering Algorithm with a Quantile-Based Likelihood Estimator

    Hideitsu Hino, Noboru Murata

    NEURAL COMPUTATION   26 ( 9 ) 2074 - 2101  2014.09  [Refereed]

     View Summary

    Clustering is a representative of unsupervised learning and one of the important approaches in exploratory data analysis. By its very nature, clustering without strong assumption on data distribution is desirable. Information-theoretic clustering is a class of clustering methods that optimize information-theoretic quantities such as entropy and mutual information. These quantities can be estimated in a nonparametric manner, and information-theoretic clustering algorithms are capable of capturing various intrinsic data structures. It is also possible to estimate information-theoretic quantities using a data set with sampling weight for each datum. Assuming the data set is sampled from a certain cluster and assigning different sampling weights depending on the clusters, the cluster-conditional information-theoretic quantities are estimated. In this letter, a simple iterative clustering algorithm is proposed based on a nonparametric estimator of the log likelihood for weighted data sets. The clustering algorithm is also derived from the principle of conditional entropy minimization with maximum entropy regularization. The proposed algorithm does not contain a tuning parameter. The algorithm is experimentally shown to be comparable to or outperform conventional nonparametric clustering methods.

    DOI

    Scopus

    4
    Citation
    (Scopus)
  • Intrinsic Graph Structure Estimation Using Graph Laplacian

    Atsushi Noda, Hideitsu Hino, Masami Tatsuno, Shotaro Akaho, Noboru Murata

    NEURAL COMPUTATION   26 ( 7 ) 1455 - 1483  2014.07  [Refereed]

     View Summary

    A graph is a mathematical representation of a set of variables where some pairs of the variables are connected by edges. Common examples of graphs are railroads, the Internet, and neural networks. It is both theoretically and practically important to estimate the intensity of direct connections between variables. In this study, a problem of estimating the intrinsic graph structure from observed data is considered. The observed data in this study are a matrix with elements representing dependency between nodes in the graph. The dependency represents more than direct connections because it includes influences of various paths. For example, each element of the observed matrix represents a co-occurrence of events at two nodes or a correlation of variables corresponding to two nodes. In this setting, spurious correlations make the estimation of direct connection difficult. To alleviate this difficulty, a digraph Laplacian is used for characterizing a graph. A generative model of this observed matrix is proposed, and a parameter estimation algorithm for the model is also introduced. The notable advantage of the proposed method is its ability to deal with directed graphs, while conventional graph structure estimation methods such as covariance selections are applicable only to undirected graphs. The algorithm is experimentally shown to be able to identify the intrinsic graph structure.

    DOI

    Scopus

    4
    Citation
    (Scopus)
  • Gray-box modeling for prediction and control of molten steel temperature in tundish

    Iftikhar Ahmad, Manabu Kano, Shinji Hasebe, Hiroshi Kitada, Noboru Murata

    JOURNAL OF PROCESS CONTROL   24 ( 4 ) 375 - 382  2014.04  [Refereed]

     View Summary

    To realize stable production in the steel industry, it is important to control molten steel temperature in a continuous casting process. The present work aims to provide a general framework of gray-box modeling and to develop a gray-box model that predicts and controls molten steel temperature in a tundish (TD temp) with high accuracy. Since the adopted first-principle model (physical model) cannot accurately describe uncertainties such as degradation of ladles, their overall heat transfer coefficient, which is a parameter in the first-principle model, is optimized for each past batch separately, then the parameter is modeled as a function of process variables through a statistical modeling method, random forests. Such a model is termed as a serial gray-box model. Prediction errors of the first-principle model or the serial gray-box model can be compensated by using another statistical model; this approach derives a parallel gray-box model or a combined gray-box model. In addition, the developed gray-box models are used to determine the optimal molten steel temperature in the Ruhrstahl-Heraeus degassing process from the target TD temp, since the continuous casting process has no manipulated variable to directly control TD temp. The proposed modeling and control strategy is validated through its application to real operation data at a steel work. The results show that the combined gray-box model achieves the best performance in prediction and control of TD temp and satisfies the requirement for its industrial application. (C) 2014 Elsevier Ltd. All rights reserved.

    DOI

    Scopus

    45
    Citation
    (Scopus)
  • K-Nearest Neighbor Approach for Forecasting Energy Demands Based on Metric Learning

    Yu Fujimoto, Taiki Sugiura, Noboru Murata

    INTERNATIONAL WORK-CONFERENCE ON TIME SERIES (ITISE 2014)     1127 - 1137  2014  [Refereed]

     View Summary

    Forecast of the energy demand is an important topic for the realization of effective energy management. In this paper, the authors focus on the K-nearest neighbor approach for forecast of the energy demand pattern, introduce an idea of distance metric learning to select appropriate K-nearest neighbors, and propose some learning frameworks for forecasting. The proposed frameworks are evaluated based on the real-world datasets from the viewpoint of selection accuracy of the true neighbors.

  • Sampling hidden parameters from oracle distribution

    Sho Sonoda, Noboru Murata

    Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)   8681   539 - 546  2014  [Refereed]

     View Summary

    A new sampling learning method for neural networks is proposed. Derived from an integral representation of neural networks, an oracle probability distribution of hidden parameters is introduced. In general rigorous sampling from the oracle distribution holds numerical difficulty, a linear-time sampling algorithm is also developed. Numerical experiments showed that when hidden parameters were initialized by the oracle distribution, following backpropagation converged faster to better parameters than when parameters were initialized by a normal distribution. © 2014 Springer International Publishing Switzerland.

    DOI

    Scopus

    6
    Citation
    (Scopus)
  • Sampling hidden parameters from oracle distribution

    Sonoda S, Murata N

    Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)   8681 LNCS   539 - 546  2014  [Refereed]

    DOI

  • An algorithm for directed graph estimation

    Hideitsu Hino, Atsushi Noda, Masami Tatsuno, Shotaro Akaho, Noboru Murata

    Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)   8681   145 - 152  2014  [Refereed]

     View Summary

    A problem of estimating the intrinsic graph structure from observed data is considered. The observed data in this study is a matrix with elements representing dependency between nodes in the graph. Each element of the observed matrix represents, for example, co-occurrence of events at two nodes, or correlation of variables corresponding to two nodes. The dependency does not represent direct connections and includes influences of various paths, and spurious correlations make the estimation of direct connection difficult. To alleviate this difficulty, digraph Laplacian is used for characterizing a graph. A generative model of an observed matrix is proposed, and a parameter estimation algorithm for the model is also proposed. The proposed method is capable of dealing with directed graphs, while conventional graph structure estimation methods from an observed matrix are only applicable to undirected graphs. Experimental result shows that the proposed algorithm is able to identify the intrinsic graph structure. © 2014 Springer International Publishing Switzerland.

    DOI

    Scopus

  • An algorithm for directed graph estimation

    Hideitsu Hino, Atsushi Noda, Masami Tatsuno, Shotaro Akaho, Noboru Murata

    Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)   8681   145 - 152  2014  [Refereed]

     View Summary

    A problem of estimating the intrinsic graph structure from observed data is considered. The observed data in this study is a matrix with elements representing dependency between nodes in the graph. Each element of the observed matrix represents, for example, co-occurrence of events at two nodes, or correlation of variables corresponding to two nodes. The dependency does not represent direct connections and includes influences of various paths, and spurious correlations make the estimation of direct connection difficult. To alleviate this difficulty, digraph Laplacian is used for characterizing a graph. A generative model of an observed matrix is proposed, and a parameter estimation algorithm for the model is also proposed. The proposed method is capable of dealing with directed graphs, while conventional graph structure estimation methods from an observed matrix are only applicable to undirected graphs. Experimental result shows that the proposed algorithm is able to identify the intrinsic graph structure. © 2014 Springer International Publishing Switzerland.

    DOI

    Scopus

  • A non-parametric maximum entropy clustering

    Hideitsu Hino, Noboru Murata

    Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)   8681   113 - 120  2014  [Refereed]

     View Summary

    Clustering is a fundamental tool for exploratory data analysis. Information theoretic clustering is based on the optimization of information theoretic quantities such as entropy and mutual information. Recently, since these quantities can be estimated in non-parametric manner, non-parametric information theoretic clustering gains much attention. Assuming the dataset is sampled from a certain cluster, and assigning different sampling weights depending on the clusters, the cluster conditional information theoretic quantities are estimated. In this paper, a simple clustering algorithm is proposed based on the principle of maximum entropy. The algorithm is experimentally shown to be comparable to or outperform conventional non-parametric clustering methods. © 2014 Springer International Publishing Switzerland.

    DOI

    Scopus

  • A non-parametric maximum entropy clustering

    Hideitsu Hino, Noboru Murata

    Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)   8681   113 - 120  2014  [Refereed]

     View Summary

    Clustering is a fundamental tool for exploratory data analysis. Information theoretic clustering is based on the optimization of information theoretic quantities such as entropy and mutual information. Recently, since these quantities can be estimated in non-parametric manner, non-parametric information theoretic clustering gains much attention. Assuming the dataset is sampled from a certain cluster, and assigning different sampling weights depending on the clusters, the cluster conditional information theoretic quantities are estimated. In this paper, a simple clustering algorithm is proposed based on the principle of maximum entropy. The algorithm is experimentally shown to be comparable to or outperform conventional non-parametric clustering methods. © 2014 Springer International Publishing Switzerland.

    DOI

    Scopus

  • Information estimators for weighted observations

    Hideitsu Hino, Noboru Murata

    NEURAL NETWORKS   46   260 - 275  2013.10  [Refereed]

     View Summary

    The Shannon information content is a valuable numerical characteristic of probability distributions. The problem of estimating the information content from an observed dataset is very important in the fields of statistics, information theory, and machine learning. The contribution of the present paper is in proposing information estimators, and showing some of their applications. When the given data are associated with weights, each datum contributes differently to the empirical average of statistics. The proposed estimators can deal with this kind of weighted data. Similar to other conventional methods, the proposed information estimator contains a parameter to be tuned, and is computationally expensive. To overcome these problems, the proposed estimator is further modified so that it is more computationally efficient and has no tuning parameter. The proposed methods are also extended so as to estimate the cross-entropy, entropy, and Kullback-Leibler divergence. Simple numerical experiments show that the information estimators work properly. Then, the estimators are applied to two specific problems, distribution-preserving data compression, and weight optimization for ensemble regression. (C) 2013 Elsevier Ltd. All rights reserved.

    DOI

    Scopus

    11
    Citation
    (Scopus)
  • Learning Ancestral Atom via Sparse Coding

    Toshimitsu Aritake, Hideitsu Hino, Noboru Murata

    IEEE JOURNAL OF SELECTED TOPICS IN SIGNAL PROCESSING   7 ( 4 ) 586 - 594  2013.08  [Refereed]

     View Summary

    Sparse signal models have been the focus of recent research. In sparse coding, signals are represented with a linear combination of a small number of elementary signals called atoms, and the collection of atoms is called a dictionary. Design of the dictionary has strong influence on the signal approximation performance. Recently, to put prior information into dictionary learning, several methods imposing a certain kind of structure on the dictionary are proposed. In this paper, like wavelet analysis, a dictionary for sparse signal representation is assumed to be generated from an ancestral atom, and a method for learning the ancestral atom is proposed. The proposed algorithm updates the ancestral atom by iterating dictionary update in unstructured dictionary space and projection of the updated dictionary onto the structured dictionary space. The algorithm allows a simple differential geometric interpretation. Numerical experiments are performed to show the characteristics and advantages of the proposed algorithm.

    DOI

    Scopus

    3
    Citation
    (Scopus)
  • Defining a pairwise similarity measure based on linearity : Application to line extraction from distorted image

    HINO Hideitsu, FUJIKI Jun, AKAHO Shotaro, MOCHIZUKI Yoshihiko, MURATA Noboru

    Technical report of IEICE. PRMU   113 ( 75 ) 29 - 34  2013.06

     View Summary

    Data clustering is a fundamental technique in many fields of information processing including image analysis. The results of data clustering depend on both the clustering algorithm and the similarity defined between pair of the data. Focusing the distribution of the data around a line connecting a pair of data points, a method of defining a similarity between the pair of data is proposed. The similarity is used for clustering the observed dataset. The method is applied to detecting line segments from distorted images taken by cameras with a fish-eye lens.

    CiNii

  • Regions of Unusually High Flexibility Occur Frequently in Human Genomic DNA

    Hajime Kimura, Dai Kageyama, Mika Furuya, Shigeru Sugiyama, Noboru Murata, Takashi Ohyama

    BIOSCIENCE BIOTECHNOLOGY AND BIOCHEMISTRY   77 ( 3 ) 612 - 617  2013.03  [Refereed]

     View Summary

    Remarkable progress has been made in genome science during the past decade, but understanding of genomes of eukaryotes is far from complete. We have created DNA flexibility maps of the human, mouse, fruit fly, and nematode chromosomes. The maps revealed that all of these chromosomes have markedly flexible DNA regions (We named them SPIKEs). SPIKEs occur more frequently in the human chromosomes than in the mouse, fruit fly, and nematode chromosomes. Markedly rigid DNA regions (rSPIKEs) are also present in these chromosomes. The ratio of the number of SPIKEs to the total number of SPIKEs and rSPIKEs correlated positively with evolutionary stage among the organisms. Repetitive DNA sequences with flexible and rigid properties contribute to the formation of SPIKEs and rSPIKEs respectively. However, non-repetitive flexible and rigid sequences appear to play a major role in SPIKE and rSPIKE formation respectively. They might be involved in the genome-folding mechanism of eukaryotes.

    DOI

  • Pairwise similarity for line extraction from distorted images

    Hideitsu Hino, Jun Fujiki, Shotaro Akaho, Yoshihiko Mochizuki, Noboru Murata

    Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)   8048 ( 2 ) 250 - 257  2013  [Refereed]

     View Summary

    Clustering a given set of data is crucial in many fields including image processing. It plays important roles in image segmentation and object detection for example. This paper proposes a framework of building a similarity matrix for a given dataset, which is then used for clustering the dataset. The similarity between two points are defined based on how other points distribute around the line connecting the two points. It can capture the degree of how the two points are placed on the same line. The similarity matrix is considered as a kernel matrix of the given dataset, and based on it, the spectral clustering is performed. Clustering with the proposed similarity matrix is shown to perform well through experiments using an artificially designed problem and a real-world problem of detecting lines from a distorted image. © 2013 Springer-Verlag.

    DOI

    Scopus

  • High-Performance Prediction of Molten Steel Temperature in Tundish through Gray-Box Model

    Toshinori Okura, Iftikhar Ahmad, Manabu Kano, Shinji Hasebe, Hiroshi Kitada, Noboru Murata

    ISIJ INTERNATIONAL   53 ( 1 ) 76 - 80  2013  [Refereed]

     View Summary

    A novel gray-box model is proposed to estimate molten steel temperature in a continuous casting process at a steel making plant by combining a first-principle model and a statistical model. The first-principle model was developed on the basis of computational fluid dynamics (CFD) simulations to simplify the model and to improve estimation accuracy. Since the derived first-principle model was not able to estimate the molten steel temperature in the tundish with sufficient accuracy, statistical models were developed to estimate the estimation errors of the first-principle model through partial least squares (PLS) and random forest (RF). As a result of comparing the three models, i.e., the first-principle model, the PLS-based gray-box model, and the RF-based gray-box model, the RF-based gray-box model achieved the best estimation performance. Thus, the molten steel temperature in the tundish can be estimated with accuracy by adding estimates of the first-principle model and those of the statistical RF model. The proposed gray-box model was applied to the real process data and the results demonstrated its advantage over other models.

    DOI

    Scopus

    19
    Citation
    (Scopus)
  • Entropy-based sliced inverse regression

    Hideitsu Hino, Keigo Wakayama, Noboru Murata

    Computational Statistics and Data Analysis   67   105 - 114  2013  [Refereed]

     View Summary

    Abstract The importance of dimension reduction has been increasing according to the growth of the size of available data in many fields. An appropriate dimension reduction method of raw data helps to reduce computational time and to expose the intrinsic structure of complex data. Sliced inverse regression is a well-known dimension reduction method for regression, which assumes an elliptical distribution for the explanatory variable, and ingeniously reduces the problem of dimension reduction to a simple eigenvalue problem. Sliced inverse regression is based on the strong assumptions on the data distribution and the form of regression function, and there are a number of methods to relax or remove these assumptions to extend the applicability of the inverse regression method. However, each method is known to have its drawbacks either theoretically or empirically. To alleviate drawbacks in the existing methods, a dimension reduction method for regression based on the notion of conditional entropy minimization is proposed. Using entropy as a measure of dispersion of data, a low dimensional subspace is estimated without assuming any specific distribution nor any regression function. The proposed method is shown to perform comparable or superior to the conventional methods through experiments using artificial and real-world datasets. © 2013 Elsevier B.V. All rights reserved.

    DOI

    Scopus

    5
    Citation
    (Scopus)
  • Design of inner and outer gray-box models to predict molten steel temperature in Tundish

    Iftikhar Ahmad, Manabu Kano, Shinji Hasebe, Hiroshi Kitada, Noboru Murata

    IFAC Proceedings Volumes (IFAC-PapersOnline)   10 ( PART 1 ) 744 - 749  2013  [Refereed]

     View Summary

    In order to realize stable production in the steel industry, it is important to control molten steel temperature in a continuous casting process. The present work aims to develop a gray-box model that predicts the molten steel temperature in the tundish (TD temp). In the proposed approach, two parameters in the first-principle model, i.e., overall heat transfer coefficients of ladle and tundish, are optimized for each past batch separately, then the relationship between the two parameters and measured process variables is modeled through random forests (RF). In this inner gray-box model, the statistical models update the physical parameters according to the operating condition. To enhance the accuracy of TD temp estimation, another RF model is developed which compensates errors of the inner gray-box. The proposed approach was validated through its application to real operation data at a steel work. © IFAC.

    DOI

    Scopus

    2
    Citation
    (Scopus)
  • A versatile clustering method for electricity consumption pattern analysis in households

    Hideitsu Hino, Haoyang Shen, Noboru Murata, Shinji Wakao, Yasuhiro Hayashi

    IEEE Transactions on Smart Grid   4 ( 2 ) 1048 - 1057  2013  [Refereed]

     View Summary

    Analysis and modeling of electric energy demand is indispensable for power planning, operation, facility investment, and urban planning. Because of recent development of renewable energy generation systems oriented for households, there is also a great demand for analysing the electricity usage and optimizing the way to install electricity generation systems for each household. In this study, employing statistical techniques, a method to model daily consumption patterns in households and a method to extract a small number of their typical patterns are presented. The electricity consumption patterns in a household is modeled by a mixture of Gaussian distributions. Then, using the symmetrized generalized Kullback-Leibler divergence as a distance measure of the distributions, typical patterns of the consumption are extracted by means of hierarchical clustering. The statistical modeling of daily consumption patterns allows us to capture essential similarities of the patterns. By experiments using a large-scale dataset including about 500 houses' consumption records in a suburban area in Japan, it is shown that the proposed method is able to extract typical consumption patterns. © 2010-2012 IEEE.

    DOI

    Scopus

    63
    Citation
    (Scopus)
  • A comparison of optimal operation of a residential fuel cell co-generation system using clustered demand patterns based on Kullback-Leibler divergence

    Akira Yoshida, Yoshiharu Amano, Noboru Murata, Koichi Ito, Takumi Hasizume

    Energies   6 ( 1 ) 374 - 399  2013  [Refereed]

     View Summary

    When evaluating residential energy systems like co-generation systems, hot water and electricity demand profiles are critical. In this paper, the authors aim to extract basic time-series demand patterns from two kinds of measured demand (electricity and domestic hot water), and also aim to reveal effective demand patterns for primary energy saving. Time-series demand data are categorized with a hierarchical clustering method using a statistical pseudo-distance, which is represented by the generalized Kullback-Leibler divergence of two Gaussian mixture distributions. The classified demand patterns are built using hierarchical clustering and then a comparison is made between the optimal operation of a polymer electrolyte membrane fuel cell co-generation system and the operation of a reference system (a conventional combination of a condensing gas boiler and electricity purchased from the grid) using the appropriately built demand profiles. Our results show that basic demand patterns are extracted by the proposed method, and the heat-to-power ratio of demand, the amount of daily demand, and demand patterns affect the primary energy saving of the co-generation system. © 2013 by the authors
    licensee MDPI, Basel, Switzerland.

    DOI

    Scopus

    10
    Citation
    (Scopus)
  • Multiple Kernel Learning with Gaussianity Measures

    Hideitsu Hino, Nima Reyhani, Noboru Murata

    NEURAL COMPUTATION   24 ( 7 ) 1853 - 1881  2012.07  [Refereed]

     View Summary

    Kernel methods are known to be effective for nonlinear multivariate analysis. One of the main issues in the practical use of kernel methods is the selection of kernel. There have been a lot of studies on kernel selection and kernel learning. Multiple kernel learning (MKL) is one of the promising kernel optimization approaches. Kernel methods are applied to various classifiers including Fisher discriminant analysis (FDA). FDA gives the Bayes optimal classification axis if the data distribution of each class in the feature space is a gaussian with a shared covariance structure. Based on this fact, an MKL framework based on the notion of gaussianity is proposed. As a concrete implementation, an empirical characteristic function is adopted to measure gaussianity in the feature space associated with a convex combination of kernel functions, and two MKL algorithms are derived. From experimental results on some data sets, we show that the proposed kernel learning followed by FDA offers strong classification power.

    DOI

    Scopus

    5
    Citation
    (Scopus)
  • Sliced Inverse Regression with Conditional Entropy Minimization

    Hideitsu Hino, Keigo Wakayama, Noboru Murata

    2012 21ST INTERNATIONAL CONFERENCE ON PATTERN RECOGNITION (ICPR 2012)     1185 - 1188  2012  [Refereed]

     View Summary

    An appropriate dimension reduction of raw data helps to reduce computational time and to reveal the intrinsic structure of complex data. In this paper, a dimension reduction method for regression is proposed. The method is based on the well-known sliced inverse regression and conditional entropy minimization. Using entropy as a measure of dispersion of data distribution, dimension reduction subspace is estimated without assuming regression function form nor data distribution, unlike conventional sliced inverse regression. The proposed method is shown to perform well compared to some conventional methods through experiments using both artificial and real-world data sets.

  • Sensitivity analysis for controlling molten steel temperature in Tundish

    Noboru Murata, Sho Sonoda, Hideitsu Hino, Hiroshi Kitada, Manabu Kano

    IFAC Proceedings Volumes (IFAC-PapersOnline)   45 ( 23 ) 270 - 271  2012  [Refereed]

     View Summary

    Controlling temperature of molten steel is crucial for product quality in continuous casting. In this paper, sensitivity analysis is carried out on a statistical model for predicting temperature in tundish, and influential operations for controlling temperature are identified. © 2012 IFAC.

    DOI

    Scopus

    2
    Citation
    (Scopus)
  • Robust Hypersurface Fitting Based on Random Sampling Approximations

    Jun Fujiki, Shotaro Akaho, Hideitsu Hino, Noboru Murata

    NEURAL INFORMATION PROCESSING, ICONIP 2012, PT III   7665 ( PART 3 ) 520 - 527  2012  [Refereed]

     View Summary

    This paper considers N -1-dimensional hypersurface fitting based on L-2 distance in N-dimensional input space. The problem is usually reduced to hyperplane fitting in higher dimension. However, because feature mapping is generally a nonlinear mapping, it does not preserve the order of lengthes, and this derives an unacceptable fitting result. To avoid it, JNLPCA is introduced. JNLPCA defines the L-2 distance in the feature space as a weighted L-2 distance to reflect the metric in the input space. In the fitting, random sampling approximation of least k-th power deviation, and least alpha-percentile of squares are introduced to make estimation robust. The proposed hypersurface fitting method is evaluated by quadratic curve fitting and quadratic curve segments extraction from artificial data and a real image.

    DOI

    Scopus

    1
    Citation
    (Scopus)
  • Nonnegative matrix factorization via generalized product rule and its application for classification

    Yu Fujimoto, Noboru Murata

    Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)   7191   263 - 271  2012  [Refereed]

     View Summary

    Nonnegative Matrix Factorization (NMF) is broadly used as a mathematical tool for processing tasks of tabulated data. In this paper, an extension of NMF based on a generalized product rule, defined with a nonlinear one-parameter function and its inverse, is proposed. From a viewpoint of subspace methods, the extended NMF constructs flexible subspaces which plays an important role in classification tasks. Experimental results on benchmark datasets show that the proposed extension improves classification accuracies. © 2012 Springer-Verlag.

    DOI

    Scopus

  • Gray-box model to control molten steel temperature in Tundish

    S. Sakashita, T. Okura, I. Ahmad, M. Kano, H. Kitada, N. Murata

    IFAC Proceedings Volumes (IFAC-PapersOnline)   45 ( 23 ) 268 - 269  2012  [Refereed]

     View Summary

    Controlling molten steel temperature in a tundish is crucial for stable and efficient production of steel products. In this research, a gray-box model is developed by combining a first-principle model and a statistical model. The first-principle model predicts the molten steel temperature in the tundish, and the statistical model predicts the prediction error of the first-principle model. The derived model was then used to derive the molten steel temperature in the Ruhrstahl-Heraeus process at the end of its operation from the target molten steel temperature in the tundish and other process variables. The result of applying the proposed model to real operation data has demonstrated its practicability. © 2012 IFAC.

    DOI

    Scopus

  • Automatic Extraction of Basic Electricity Consumption Patterns in Household

    Haoyang Shen, Hideitsu Hino, Noboru Murata, Shinji Wakao, Yasuhiro Hayashi

    INTERNATIONAL CONFERENCE ON RENEWABLE ENERGY RESEARCH AND APPLICATIONS (ICRERA)    2012  [Refereed]

     View Summary

    Electricity consumption in households varies dependent on a lot of possible reasons such as lifestyle, family configuration, and weather. It is of great importance to optimize the electricity generation system to install for each household. In our previous work, we proposed a clustering approach for extracting a small number of basic electricity consumption patterns in a household. In this study, we apply the method to a larger dataset with many households. In the previous work, we determined the number of basic patterns in a heuristic manner. In this work, we use gap statistics to automatically determine an appropriate number of basic patterns, and we obtained a reasonable result on a large-scale data.

    DOI

    Scopus

    4
    Citation
    (Scopus)
  • A tree search approach to sparse coding

    Rui Rei, João P. Pedroso, Hideitsu Hino, Noboru Murata

    Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)   7219   472 - 477  2012  [Refereed]

     View Summary

    Sparse coding is an important optimization problem with numerous applications. In this paper, we describe the problem and the commonly used pursuit methods, and propose a best-first tree search algorithm employing multiple queues for unexplored tree nodes. We assess the effectiveness of our method in an extensive computational experiment, showing its superiority over other methods even for modest computational time. © 2012 Springer-Verlag.

    DOI

    Scopus

  • A Statistical Model for Predicting the Liquid Steel Temperature in Ladle and Tundish by Bootstrap Filter

    Sho Sonoda, Noboru Murata, Hideitsu Hino, Hiroshi Kitada, Manabu Kano

    ISIJ INTERNATIONAL   52 ( 6 ) 1086 - 1091  2012  [Refereed]

     View Summary

    A statistical model for predicting the liquid steel temperature in the ladle and in the tundish is developed. Given a large data set in a steelmaking process, the proposed model predicts the temperature in a seconds with a good accuracy. The data are divided into four phases at the mediation of five temperature measurements: before tapping from the converter (CV), after throwing ferroalloys into the ladle, before and after the Ruhrstahl-Heraeus (RH) processing, and after casting into the tundish in the continuous casting (CC) machine. Based on the general state space modeling, the bootstrap filter predicts the temperature phase by phase. The particle approximation technique enables to compute general-shaped probability distributions. The proposed model gives a prediction not as a point but as a probability distribution, or a predictive distribution. It evaluates both uncertainty of the prediction and ununiformity of the temperature. It is applicable to sensitivity analysis, process scheduling and temperature control.

    DOI

    Scopus

    24
    Citation
    (Scopus)
  • A generalisation of independence in statistical models for categorical distribution

    Yu Fujimoto, Noboru Murata

    International Journal of Data Mining, Modelling and Management   4 ( 2 ) 172 - 187  2012  [Refereed]

     View Summary

    In this paper, generalised statistical independence in statistical models for categorical distributions is proposed from the viewpoint of generalised multiplication characterised by a monotonically increasing function and its inverse function, and it is implemented in naive Bayes models. This paper also proposes an idea of their estimation method which directly uses empirical marginal distributions to retain simplicity of calculation. This method is interpreted as an optimisation of a rough approximation of the Bregman divergence so that it is expected to have a kind of robust property. Effectiveness of proposed models is shown by numerical experiments on some benchmark datasets. Copyright © 2012 Inderscience Enterprises Ltd.

    DOI

    Scopus

    4
    Citation
    (Scopus)
  • A comparison of optimal operation of residential energy systems using clustered demand patterns based on kullback-leibler divergence

    Yoshida A, Amano Y, Murata N, Ito K, Hashizume T

    Proceedings of the 25th International Conference on Efficiency, Cost, Optimization and Simulation of Energy Conversion Systems and Processes, ECOS 2012   3   1 - 16  2012  [Refereed]

     View Summary

    Residential energy demand varies widely in terms of time-series behaviors, amounts consumed between families, and even within one family. Residential energy demand profiles have a high degree of uncertainty in their essentials because the demand profile is entirely based on the occupant-driven load. When evaluating residential energy systems like co-generation systems, hot water and electricity demand profiles are critical. In this paper, in order to clarify rational energy system selection guidelines and rational operation strategies, authors aim to extract basic demand time-series patterns from two kinds of measured demand (electricity and domestic hot water), measured over 26307 days of data in Japan. Authors also aim to reveal the relationship between primary energy consumption and demand patterns. Demand time-series data are categorized by means of a kind of "unsupervised" learning, which is a hierarchical clustering method using a statistical pseudo-distance. The statistical pseudo-distance is calculated from the generalized Kullback-Leibler divergence with the Gaussian mixture distribution fitted to the demand time-series data. The classified demand patterns are built using a hierarchical clustering and then a comparison is performed between the optimal operation of the two systems (a polymer electrolyte membrane fuel cell co-generation system, and a CO2 heat pump system) and the operation of a reference system (a conventional combination of a condensing gas boiler and electricity purchased from the grid) using the demand profiles appropriately built. Our results show that basic demand patterns are extracted by the proposed method. The demand patterns, the amount of daily demand and heat-to-power ratio of demand affect the primary energy reduction ratio of the polymer electrolyte membrane fuel cell co-generation system.

  • An Estimation of Generalized Bradley-Terry Models Based on the em Algorithm

    Yu Fujimoto, Hideitsu Hino, Noboru Murata

    NEURAL COMPUTATION   23 ( 6 ) 1623 - 1659  2011.06  [Refereed]

     View Summary

    The Bradley-Terry model is a statistical representation for one's preference or ranking data by using pairwise comparison results of items. For estimation of the model, several methods based on the sum of weighted Kullback-Leibler divergences have been proposed from various contexts. The purpose of this letter is to interpret an estimation mechanism of the Bradley-Terry model from the viewpoint of flatness, a fundamental notion used in information geometry. Based on this point of view, a new estimation method is proposed on a framework of the em algorithm. The proposed method is different in its objective function from that of conventional methods, especially in treating unobserved comparisons, and it is consistently interpreted in a probability simplex. An estimation method with weight adaptation is also proposed from a viewpoint of the sensitivity. Experimental results show that the proposed method works appropriately, and weight adaptation improves accuracy of the estimate.

    DOI

    Scopus

    5
    Citation
    (Scopus)
  • Speaker verification robust to talking style variation using multiple kernel learning based on conditional entropy minimization

    Tetsuji Ogawa, Hideitsu Hino, Noboru Murata, Tetsunori Kobayashi

    12TH ANNUAL CONFERENCE OF THE INTERNATIONAL SPEECH COMMUNICATION ASSOCIATION 2011 (INTERSPEECH 2011), VOLS 1-5     2752 - +  2011  [Refereed]

     View Summary

    We developed a new speaker verification system that is robust to intra-speaker variation. There is a strong likelihood that intra-speaker variations will occur due to changes in talking styles, the periods when an individual speaks, and so on. It is well known that such variation generally degrades the performance of speaker verification systems. To solve this problem, we applied multiple kernel learning (MKL) based on conditional entropy minimization, which impose the data to be compactly aggregated for each speaker class and ensure that the different speaker classes were far apart from each other. Experimental results showed that the proposed speaker verification system achieved a robust performance to intra-speaker variation derived from changes in the talking styles compared to the conventional maximum margin-based system.

  • mn SPEAKER RECOGNITION USING MULTIPLE KERNEL LEARNING BA SED ON CONDITIONA L ENTROPY MINIMIZATION

    Tetsuji Ogawa, Hideitsu Hino, Nima Reyhani, Noboru Murata, Tetsunori Kobayashi

    2011 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING     2204 - 2207  2011  [Refereed]

     View Summary

    We applied a multiple kernel learning (MKL) method based on information-theoretic optimization to speaker recognition. Most of the kernel methods applied to speaker recognition systems require a suitable kernel function and its parameters to be determined for a given data set. In contrast, MKL eliminates the need for strict determination of the kernel function and parameters by using a convex combination of element kernels. In the present paper, we describe an MKL algorithm based on conditional entropy minimization (MCEM). We experimentally verified the effectiveness of MCEM for speaker classification; this method reduced the speaker error rate as compared to conventional methods.

    DOI

    Scopus

    4
    Citation
    (Scopus)
  • Robust Hyperplane Fitting Based on k-th Power Deviation and alpha-Quantile

    Jun Fujiki, Shotaro Akaho, Hideitsu Hino, Noboru Murata

    COMPUTER ANALYSIS OF IMAGES AND PATTERNS: 14TH INTERNATIONAL CONFERENCE, CAIP 2011, PT I   6854 ( PART 1 ) 278 - 285  2011  [Refereed]

     View Summary

    In this paper, two methods for one-dimensional reduction of data by hyperplane fitting are proposed. One is least a-percentile of squares, which is an extension of least median of squares estimation and minimizes the a-percentile of squared Euclidean distance. The other is least k-th power deviation, which is an extension of least squares estimation and minimizes the k-th power deviation of squared Euclidean distance. Especially, for least k-th power deviation of 0 < k <= 1, it is proved that a useful property, called optimal sampling property, holds in one-dimensional reduction of data by hyperplane fitting. The optimal sampling property is that the global optimum for affine hyperplane fitting passes through N data points when an N-1-dimensional hyperplane is fitted to the N-dimensional data. The performance of the proposed methods is evaluated by line fitting to artificial data and a real image.

    DOI

    Scopus

    1
    Citation
    (Scopus)
  • Improved methods for dewarping images in convex mirrors in fine art: Applications to van Eyck and Parmigianino

    Yumi Usami, David G. Stork, Jun Fujiki, Hideitsu Hino, Shotaro Akaho, Noboru Murata

    COMPUTER VISION AND IMAGE ANALYSIS OF ART II   7869  2011  [Refereed]

     View Summary

    We derive and demonstrate new methods for dewarping images depicted in convex mirrors in artwork and for estimating the three-dimensional shapes of the mirrors themselves. Previous methods were based on the assumption that mirrors were spherical or paraboloidal, an assumption unlikely to hold for hand-blown glass spheres used in early Renaissance art, such as Johannes van Eyck's Portrait of Giovanni (?) Arnolfini and his wife (1434) and Robert Campin's Portrait of St. John the Baptist and Heinrich von Werl (1438). Our methods are more general than such previous methods in that we assume merely that the mirror is radially symmetric and that there are straight lines (or colinear points) in the actual source scene. We express the mirror's shape as a mathematical series and pose the image dewarping task as that of estimating the coefficients in the series expansion. Central to our method is the plumbline principle: that the optimal coefficients are those that dewarp the mirror image so as to straighten lines that correspond to straight lines in the source scene. We solve for these coefficients algebraically through principal component analysis, PCA. Our method relies on a global figure of merit to balance warping errors throughout the image and it thereby reduces a reliance on the somewhat subjective criterion used in earlier methods. Our estimation can be applied to separate image annuli, which is appropriate if the mirror shape is irregular. Once we have found the optimal image dewarping, we compute the mirror shape by solving a differential equation based on the estimated dewarping function. We demonstrate our methods on the Arnolfini mirror and reveal a dewarped image superior to those found in prior work-an image noticeably more rectilinear throughout and having a more coherent geometrical perspective and vanishing points. Moreover, we find the mirror deviated from spherical and paraboloidal shape; this implies that it would have been useless as a concave projection mirror, as has been claimed. Our dewarped image can be compared to the geometry in the full Arnolfini painting; the geometrical agreement strongly suggests that van Eyck worked from an actual room, not, as has been suggested by some art historians, a "fictive" room of his imagination. We apply our method to other mirrors depicted in art, such as Parmigianino's Self-portrait in a convex mirror and compare our results to those from earlier computer graphics simulations.

    DOI

    Scopus

  • Extraction of basic patterns of household energy consumption

    Haoyang Shen, Hideitsu Hino, Noboru Murata, Shinji Wakao

    Proceedings - 10th International Conference on Machine Learning and Applications, ICMLA 2011   2   275 - 280  2011  [Refereed]

     View Summary

    Solar power, wind power, and co-generation (combined heat and power) systems are possible candidate for household power generation. These systems have their advantages and disadvantages. To propose the optimal combination of the power generation systems, the extraction of basic patterns of energy consumption of the house is required. In this study, energy consumption patterns are modeled by mixtures of Gaussian distributions. Then, using the symmetrized Kullback-Leibler divergence as a distance measure of the distributions, the basic pattern of energy consumption is extracted by means of hierarchical clustering. By an experiment using the Annex 42 dataset, it is shown that the proposed method is able to extract typical energy consumption patterns. © 2011 IEEE.

    DOI

    Scopus

    4
    Citation
    (Scopus)
  • Calibration of radially symmetric distortion based on linearity in the calibrated image

    Jun Fujiki, Hideitsu Hino, Shotaro Akaho, Noboru Murata

    2011 IEEE INTERNATIONAL CONFERENCE ON COMPUTER VISION WORKSHOPS (ICCV WORKSHOPS)     288 - 295  2011  [Refereed]

     View Summary

    For calibration of general radially symmetric distortion of omnidirectional cameras such as fish-eye lenses, calibration parameters are usually estimated so that curved lines, which are supposed to be straight in the real-world, are mapped to straight lines in the calibrated image, which is called plumbline principle. Under the principle, the camera with radially symmetric distortion can be calibrated by at least one distorted line in a image, theoretically, and the calibrated image is equivalent to the image taken by an ideal pin-hole camera. In this paper, the method to optimize the calibration parameters by maximizing the sum of the straightness, which is invariant under translation, rotation and magnification (scaling), of distorted lines on calibrated image is proposed. The performance of the proposed method is evaluated by artificial data and a real image.

    DOI

    Scopus

    1
    Citation
    (Scopus)
  • A measure of credibility of solar power prediction

    Haoyang Shen, Hideitsu Hino, Noboru Murata, Shinji Wakao

    Proceedings - 10th International Conference on Machine Learning and Applications, ICMLA 2011   2   286 - 291  2011  [Refereed]

     View Summary

    Recently, remarkable developments of new energy technologies have been achieved against various energy problems. Photovoltaic (PV) system, one of such technologies, has an advantage of utilizing infinite and clean energy. On the contrary, it also has a disadvantage of unreliable power supply mainly caused by unstable weather. The fluctuation of the power supply of PV systems are considerably large because of rapid insulation changes and rapid weather changes, and in some cases, it seems impossible to realize high-accuracy prediction even with sophisticated prediction models. In this paper, using recently proposed estimator for the Shannon information content, a method to output a measure of credibility for prediction is proposed. With the proposed method, it is possible to judge whether the energy supply at a certain future time is unpredictably fluctuate compared to the current value or not, and it is possible to take measures against the rapid change of solar energy generation in advance. From an experimental result using solar energy supply data, we see that the proposed measure of credibility reflects the difficulty of predicting solar energy supply. © 2011 IEEE.

    DOI

    Scopus

  • A Computationally Efficient Information Estimator for Weighted Data

    Hideitsu Hino, Noboru Murata

    ARTIFICIAL NEURAL NETWORKS AND MACHINE LEARNING - ICANN 2011, PT II   6792 ( PART 2 ) 301 - 308  2011  [Refereed]

     View Summary

    The Shannon information content is a fundamental quantity and it is of great importance to estimate it from observed dataset in the field of statistics, information theory, and machine learning. In this study, an estimator for the information content using a given set of weighted data is proposed. The empirical data distribution varies depending on the weight. The notable features of the proposed estimator are its computational efficiency and its ability to deal with weighted data. The proposed estimator is extended in order to estimate cross entropy, entropy and KL divergence with weighted data. Then, the estimators are applied to classification with one-class samples, and distribution preserving data compression problems.

    DOI

    Scopus

    3
    Citation
    (Scopus)
  • Stochastic simulation of biological reactions, and its applications for studying actin polymerization

    Kazuhisa Ichikawa, Takashi Suzuki, Noboru Murata

    PHYSICAL BIOLOGY   7 ( 4 )  2010.12  [Refereed]

     View Summary

    Molecular events in biological cells occur in local subregions, where the molecules tend to be small in number. The cytoskeleton, which is important for both the structural changes of cells and their functions, is also a countable entity because of its long fibrous shape. To simulate the local environment using a computer, stochastic simulations should be run. We herein report a new method of stochastic simulation based on random walk and reaction by the collision of all molecules. The microscopic reaction rate P-r is calculated from the macroscopic rate constant k. The formula involves only local parameters embedded for each molecule. The results of the stochastic simulations of simple second-order, polymerization, Michaelis-Menten-type and other reactions agreed quite well with those of deterministic simulations when the number of molecules was sufficiently large. An analysis of the theory indicated a relationship between variance and the number of molecules in the system, and results of multiple stochastic simulation runs confirmed this relationship. We simulated Ca2+ dynamics in a cell by inward flow from a point on the cell surface and the polymerization of G-actin forming F-actin. Our results showed that this theory and method can be used to simulate spatially inhomogeneous events.

    DOI

    Scopus

    4
    Citation
    (Scopus)
  • A Conditional Entropy Minimization Criterion for Dimensionality Reduction and Multiple Kernel Learning

    Hideitsu Hino, Noboru Murata

    NEURAL COMPUTATION   22 ( 11 ) 2887 - 2923  2010.11  [Refereed]

     View Summary

    Reducing the dimensionality of high-dimensional data without losing its essential information is an important task in information processing. When class labels of training data are available, Fisher discriminant analysis (FDA) has been widely used. However, the optimality of FDA is guaranteed only in a very restricted ideal circumstance, and it is often observed that FDA does not provide a good classification surface for many real problems. This letter treats the problem of supervised dimensionality reduction from the viewpoint of information theory and proposes a framework of dimensionality reduction based on class-conditional entropy minimization. The proposed linear dimensionality-reduction technique is validated both theoretically and experimentally. Then, through kernel Fisher discriminant analysis (KFDA), the multiple kernel learning problem is treated in the proposed framework, and a novel algorithm, which iteratively optimizes the parameters of the classification function and kernel combination coefficients, is proposed. The algorithm is experimentally shown to be comparable to or outperforms KFDA for large-scale benchmark data sets, and comparable to other multiple kernel learning techniques on the yeast protein function annotation task.

    DOI

    Scopus

    20
    Citation
    (Scopus)
  • A Grouped Ranking Model for Item Preference Parameter

    Hideitsu Hino, Yu Fujimoto, Noboru Murata

    NEURAL COMPUTATION   22 ( 9 ) 2417 - 2451  2010.09  [Refereed]

     View Summary

    Given a set of rating data for a set of items, determining preference levels of items is a matter of importance. Various probability models have been proposed to solve this task. One such model is the Plackett-Luce model, which parameterizes the preference level of each item by a real value. In this letter, the Plackett-Luce model is generalized to cope with grouped ranking observations such as movie or restaurant ratings. Since it is difficult to maximize the likelihood of the proposed model directly, a feasible approximation is derived, and the em algorithm is adopted to find the model parameter by maximizing the approximate likelihood which is easily evaluated. The proposed model is extended to a mixture model, and two applications are proposed. To show the effectiveness of the proposed model, numerical experiments with real-world data are carried out.

    DOI

    Scopus

    5
    Citation
    (Scopus)
  • Self-calibration of radially symmetric distortion by model selection

    Jun Fujiki, Hideitsu Hino, Yumi Usami, Shotaro Akaho, Noboru Murata

    Proceedings - International Conference on Pattern Recognition     1812 - 1815  2010  [Refereed]

     View Summary

    For self-calibration of general radially symmetric distortion (RSD) of omnidirectional cameras such as fish-eye lenses, calibration parameters are usually estimated so that curved lines, which are supposed to be straight in the real-world, are mapped to straight lines in the calibrated image, which is assumed to be taken by an ideal pin-hole camera. In this paper, a method of calibrating RSD is introduced base on the notion of principal component analysis (PCA). In the proposed method, the distortion function, which maps a distorted image to an ideal pin-hole camera image, is assumed to be a linear combination of a certain class of basis functions, and an algorithm for solving its coefficients by using line patterns is given. Then a method of selecting good basis functions is proposed, which aims to realize appropriate calibration in practice. Experimental results for synthetic data and real images are presented to demonstrate the performance of our calibration method. © 2010 IEEE.

    DOI

    Scopus

    5
    Citation
    (Scopus)
  • Multiple kernel learning by conditional entropy minimization

    Hideitsu Hino, Nima Reyhani, Noboru Murata

    Proceedings - 9th International Conference on Machine Learning and Applications, ICMLA 2010     223 - 228  2010  [Refereed]

     View Summary

    Kernel methods have been successfully used in many practical machine learning problems. Choosing a suitable kernel is left to the practitioner. A common way to an automatic selection of optimal kernels is to learn a linear combination of element kernels. In this paper, a novel framework of multiple kernel learning is proposed based on conditional entropy minimization criterion. For the proposed framework, three multiple kernel learning algorithms are derived. The algorithms are experimentally shown to be comparable to or outperform kernel Fisher discriminant analysis and other multiple kernel learning algorithms on benchmark data sets. © 2010 IEEE.

    DOI

    Scopus

    7
    Citation
    (Scopus)
  • Estimation of a rotationally symmetric mirror shape from a frontal image of the mirror

    J. Fujiki, Y. Usami, H. Hino, S. Akaho, N. Murata

    International Conference Image and Vision Computing New Zealand    2010  [Refereed]

     View Summary

    In structure from motion problems, mirror shape reconstruction from mirror image is very interesting. In this paper, we propose a method to estimate the shape of a rotationally symmetric mirror by calibrating a mirror image which has radially symmetric distortion. In the proposed method, the shape of mirror is represented by the set of solution of a differential equation, which is derived from distortion function computed by the calibration of radially symmetric distortion of an image on a rotationally symmetric mirror. The differential equation and the boundary condition are simplified under the assumption that the image on mirror is taken by orthographic projection camera. The experimental result for a real image is presented to demonstrate the performance of our reconstruction method, and the method is applied for reconstruction of the shape of the mirror painted on the Jan van Eyck's Arnolfini Wedding which had drawn in the 15th century. © 2010 IEEE.

    DOI

    Scopus

  • A Generalization of Independence in Naive Bayes Model

    Yu Fujimoto, Noboru Murata

    INTELLIGENT DATA ENGINEERING AND AUTOMATED LEARNING - IDEAL 2010   6283   153 - +  2010  [Refereed]

     View Summary

    In this paper, generalized statistical independence is proposed from the viewpoint of generalized multiplication characterized by a monotonically increasing function and its inverse function, and it is implemented in naive Bayes models. This paper also proposes an idea of their estimation method which directly uses empirical marginal distributions to retain simplicity of calculation. Our method is interpreted as an optimization of a rough approximation of the Bregman divergence so that it is expected to have a kind of robust property. Effectiveness of our proposed models is shown by numerical experiments on some benchmark data sets.

    DOI

    Scopus

    8
    Citation
    (Scopus)
  • Bregman divergence and density integration

    Noboru Murata, Yu Fujimoto

    Journal of Math-for-Industry   JMI2009B-3   97 - 104  2009.10

    CiNii

  • Model selection and information criterion

    Noboru Murata, Hyeyoung Park

    Information Theory and Statistical Learning     333 - 354  2009  [Refereed]

     View Summary

    In this chapter, a problem of estimating model parameters from observed data is considered such as regression and function approximation, and a method of evaluating the goodness of model is introduced. Starting from so-called leave-one-out cross-validation, and investigating asymptotic statistical properties of estimated parameters, a generalized Akaike's information criterion (AIC) is derived for selecting an appropriate model from several candidates. In addition to model selection, the concept of information criteria provides an assessment of the goodness of model in various situations. Finally, an optimization method using regularization is presented as an example. © 2009 Springer US.

    DOI

    Scopus

    2
    Citation
    (Scopus)
  • ITEM-USER PREFERENCE MAPPING WITH MIXTURE MODELS Data Visualization for Item Preference

    Yu Fujimoto, Hideitsu Hino, Noboru Murata

    KDIR 2009: PROCEEDINGS OF THE INTERNATIONAL CONFERENCE ON KNOWLEDGE DISCOVERY AND INFORMATION RETRIEVAL     105 - +  2009  [Refereed]

     View Summary

    In this paper, we propose a visualization technique of a statistical relation of users and preference of items based on a mixture model. In our visualization, items are given as points in a few dimensional preference space, and user specific preferences are given as lines in the same space. The relationship between items and user preferences are intuitively interpreted via projections from points onto lines. As a primitive implementation, we introduce a mixture of the Bradley-Terry models, and visualize the relation between items and user preferences with benchmark data sets.

  • Item Preference Parameters from Grouped Ranking Observations

    Hideitsu Hino, Yu Fujimoto, Noboru Murata

    ADVANCES IN KNOWLEDGE DISCOVERY AND DATA MINING, PROCEEDINGS   5476   875 - 882  2009  [Refereed]

     View Summary

    Given a set of rating data for a set of items, determining the values of items is a matter of importance. Various probability models have been proposed to solve this task. The Plackett-Luce model is one of such models, which parametrizes the value of each item by a real valued preference parameter. In this paper, the Plackett-Luce model is generalized to cope with the grouped ranking observations such as movies or restaurants ratings. Since the maximization of the likelihood of the proposed model is computationally intractable, the lower bound of the likelihood which is easy to evaluate is derived, and the ern, algorithm is adopted to the maximization of the lower bound.

    DOI

    Scopus

    2
    Citation
    (Scopus)
  • Calibration of Radially Symmetric Distortion by Fitting Principal Component

    Hideitsu Hino, Yumi Usami, Jun Fujiki, Shotaro Akaho, Noboru Murata

    COMPUTER ANALYSIS OF IMAGES AND PATTERNS, PROCEEDINGS   5702   149 - +  2009  [Refereed]

     View Summary

    To calibrate radially symmetric distortion of omnidirectional cameras such as fish-eye lenses, calibration parameters are usually estimated so drat lines, which are supposed to be straight, in the 3D real scene; are mapped to straight lines in the calibrated image. In this paper, this problem is treated as a fitting problem of the principal component in uncalibrated images, and an estimation procedure of calibration parameters is proposed based oil the principal component analysis. Experimental results for synthetic data and real images are presented to demonstrate the performance of our calibration method.

    DOI

    Scopus

    4
    Citation
    (Scopus)
  • An Information Theoretic Perspective of the Sparse Coding

    Hideitsu Hino, Noboru Murata

    ADVANCES IN NEURAL NETWORKS - ISNN 2009, PT 1, PROCEEDINGS   5551 ( PART 1 ) 84 - 93  2009  [Refereed]

     View Summary

    The sparse coding method is formulated as an information theoretic optimization problem. The rate distortion theory leads to all objective functional which can be interpreted as an information theoretic formulation of the sparse coding. Viewing as an entropy minimization problem, the rate distortion theory and consequently the sparse coding are extended to discriminative variants. As a concrete example of this information theoretic sparse coding, a discriminative non-linear sparse coding algorithm with neural networks is proposed. Experimental results of gender classification by face images show that the discriminative sparse coding is more robust to noise, compared to the conventional method which directly uses images as inputs to a linear support vector machine.

    DOI

    Scopus

    3
    Citation
    (Scopus)
  • Robust boosting algorithm against mislabeling in multiclass problems

    Takashi Takenouchi, Shinto Eguchi, Noboru Murata, Takafumi Kanamori

    NEURAL COMPUTATION   20 ( 6 ) 1596 - 1630  2008.06  [Refereed]

     View Summary

    We discuss robustness against mislabeling in multiclass labels for classification problems and propose two algorithms of boosting, the normalized Eta-Boost.M and Eta-Boost.M, based on the Eta-divergence. Those two boosting algorithms are closely related to models of mislabeling in which the label is erroneously exchanged for others. For the two boosting algorithms, theoretical aspects supporting the robustness for mislabeling are explored. We apply the proposed two boosting methods for synthetic and real data sets to investigate the performance of these methods, focusing on robustness, and confirm the validity of the proposed methods.

    DOI

    Scopus

    18
    Citation
    (Scopus)
  • 球面上の点列に対する連接小円回帰を用いたカメラ運動の平滑化

    野田 容士, 藤木 淳, 村田 昇

    電子情報通信学会論文誌   J91-D ( 5 ) 1336 - 1348  2008.05

  • Neuromagnetic responses to chords are modified by preceding musical scale

    Asuka Otsuka, Shinya Kuriki, Noboru Murata, Toshikazu Hasegawa

    NEUROSCIENCE RESEARCH   60 ( 1 ) 50 - 55  2008.01  [Refereed]

     View Summary

    Previous psychological studies have shown that musical chords primed by Western musical scale in a tonal and modal schema are perceived in a hierarchy of stability. We investigated such priming effects on auditory magnetic responses to tonic-major and submediant-minor chords preceded by major scales and tonic-minor and submediant-major chords preceded by minor scales. Musically trained subjects participated in the experiment. During MEG recordings, subjects judged perceptual stability of the chords. The tonic chords were judged to be stable, whereas the submediant chords were judged to be unstable. Dipole moments of N1m response originating in the auditory cortex were larger in the left hemisphere for the submediant chords than for the tonic chords preceded by the major but not minor scales. No difference in the N1m or P2m moment was found for the chords presented without preceding scales. These results suggest priming effects of the tonal schema, interacting with contextual modality, on neural activity of the auditory cortex as well as perceptual stability of the chords. It is inferred that modulation of the auditory cortical activity is associated with attention induced by tonal instability and modality shift, which characterize the submediant chords. (c) 2007 Elsevier Ireland Ltd and the Japan Neuroscience Society. All rights reserved.

    DOI

    Scopus

    6
    Citation
    (Scopus)
  • A Generalized Product Rule and Weak Independence based on Bregman Divergence

    Yu Fujimoto, Noboru Murata

    WMSCI 2008: 12TH WORLD MULTI-CONFERENCE ON SYSTEMICS, CYBERNETICS AND INFORMATICS, VOL V, PROCEEDINGS   5   248 - 253  2008  [Refereed]

     View Summary

    To describe the relation between some values, arithmetic operations like multiplication and division are important and conventional tools. These arithmetic operations for probabilities are characterized by the KL divergence, and hence, they can be generalized by using the Bregman divergence instead of the KL divergence. With this idea, independence of random variables is modified by generalized product rule, and a joint probability model is proposed based on this modified weak independence. Effectiveness of weak independent models is shown by numerical experiments on toy examples, and discussed from a geometrical viewpoint.

  • Robust loss functions for boosting

    Takafumi Kanamori, Takashi Takenouchi, Shinto Eguchi, Noboru Murata

    NEURAL COMPUTATION   19 ( 8 ) 2183 - 2244  2007.08

     View Summary

    Boosting is known as a gradient descent algorithm over loss functions. It is often pointed out that the typical boosting algorithm, Adaboost, is highly affected by outliers. In this letter, loss functions for robust boosting are studied. Based on the concept of robust statistics, we propose a transformation of loss functions that makes boosting algorithms robust against extreme outliers. Next, the truncation of loss functions is applied to contamination models that describe the occurrence of mislabels near decision boundaries. Numerical experiments illustrate that the proposed loss functions derived from the contamination models are useful for handling highly noisy data in comparison with other loss functions.

    DOI

    Scopus

    39
    Citation
    (Scopus)
  • Robust loss functions for boosting

    Takafumi Kanamori, Takashi Takenouchi, Shinto Eguchi, Noboru Murata

    NEURAL COMPUTATION   19 ( 8 ) 2183 - 2244  2007.08  [Refereed]

     View Summary

    Boosting is known as a gradient descent algorithm over loss functions. It is often pointed out that the typical boosting algorithm, Adaboost, is highly affected by outliers. In this letter, loss functions for robust boosting are studied. Based on the concept of robust statistics, we propose a transformation of loss functions that makes boosting algorithms robust against extreme outliers. Next, the truncation of loss functions is applied to contamination models that describe the occurrence of mislabels near decision boundaries. Numerical experiments illustrate that the proposed loss functions derived from the contamination models are useful for handling highly noisy data in comparison with other loss functions.

    DOI

    Scopus

    39
    Citation
    (Scopus)
  • A modified EM algorithm for mixture models based on Bregman divergence

    Yu Fujimoto, Noboru Murata

    ANNALS OF THE INSTITUTE OF STATISTICAL MATHEMATICS   59 ( 1 ) 3 - 25  2007.03  [Refereed]

     View Summary

    The EM algorithm is a sophisticated method for estimating statistical models with hidden variables based on the Kullback-Leibler divergence. A natural extension of the Kullback-Leibler divergence is given by a class of Bregman divergences, which in general enjoy robustness to contamination data in statistical inference. In this paper, a modification of the EM algorithm based on the Bregman divergence is proposed for estimating finite mixture models. The proposed algorithm is geometrically interpreted as a sequence of projections induced from the Bregman divergence. Since a rigorous algorithm includes a nonlinear optimization procedure, two simplification methods for reducing computational difficulty are also discussed from a geometrical viewpoint. Numerical experiments on a toy problem are carried out to confirm appropriateness of the simplifications.

    DOI

    Scopus

    13
    Citation
    (Scopus)
  • Tutorial series on brain-inspired computing Part 6: Geometrical structure of boosting algorithm

    Takafumi Kanamori, Takashi Takenouchi, Noboru Murata

    NEW GENERATION COMPUTING   25 ( 1 ) 117 - 141  2007  [Refereed]

     View Summary

    In this article, several boosting methods are discussed, which are notable implementations of the ensemble learning. Starting from the firstly introduced "boosting by filter" which is an embodiment of the proverb "Two heads are better than one", more advanced versions of boosting methods "AdaBoost" and "U-Boost" are introduced. A geometrical structure and some statistical properties such as consistency and robustness of boosting algorithms are discussed, and then simulation studies are presented for confirming discussed behaviors of algorithms.

    DOI

    Scopus

    1
    Citation
    (Scopus)
  • Robust Estimation for Mixture of Probability Tables based on beta-likelihood

    Yu Fujimoto, Noboru Murata

    PROCEEDINGS OF THE SIXTH SIAM INTERNATIONAL CONFERENCE ON DATA MINING   2006   519 - 523  2006  [Refereed]

     View Summary

    Modeling of a large joint probability table is problematic when its variables have a large number of categories. In such a case, a mixture of simpler probability tables could be a good model. And the estimation of such a large probability table frequently has another problem of data sparseness. When constructing mixture models with sparse data, EM estimators based on the beta-likelihood are expected more appropriate than those based on the log likelihood. Experimental results show that a mixture model estimated by the beta-likelihood approximates a large joint probability table with sparse data more appropriately than EM estimators.

  • Geometrical properties of Nu support vector machines with different norms

    K Ikeda, N Murata

    NEURAL COMPUTATION   17 ( 11 ) 2508 - 2529  2005.11  [Refereed]

     View Summary

    By employing the L-1 or L-infinity norms in maximizing margins, support vector machines (SVMs) result in a linear programming problem that requires a lower computational load compared to SVMs with the L-2 norm. However, how the change of norm affects the generalization ability of SVMs has not been clarified so far except for numerical experiments. In this letter, the geometrical meaning of SVMs with the L-p norm is investigated, and the SVM solutions are shown to have rather little dependency on p.

    DOI

    Scopus

    20
    Citation
    (Scopus)
  • A Gaussian process robust regression

    N Murata, Y Kuroda

    PROGRESS OF THEORETICAL PHYSICS SUPPLEMENT   157 ( 157 ) 280 - 283  2005  [Refereed]

     View Summary

    A modified Gaussian process regression is proposed aiming at making regressors robust against outliers. The proposed method is based on U-loss, which is introduced as a natural extension of Kullback-Leibler divergence. The robustness is examined based on the influence function, and numerical experiments are conducted for contaminated data sets and it is shown that the practical performance agrees with the theoretical analysis.

  • Effects of norms on learning properties of support vector machines

    K Ikeda, N Murata

    2005 IEEE INTERNATIONAL CONFERENCE ON ACOUSTICS, SPEECH, AND SIGNAL PROCESSING, VOLS 1-5   V   241 - 244  2005  [Refereed]

     View Summary

    Support Vector Machines (SVMs) are known to have a high generalization ability, yet a heavy computational load since margin maximization results in a quadratic programming problem. It is known that this maximization task results in a pth-order programming problem if we employ the LP norm instead of the L-2 norm. In this paper, we theoretically show the effects of p on the learning properties of SVMs by clarifying its geometrical meaning.

    DOI

    Scopus

    3
    Citation
    (Scopus)
  • Information geometry of U-Boost and Bregman divergence

    N Murata, T Takenouchi, T Kanamori, S Eguchi

    NEURAL COMPUTATION   16 ( 7 ) 1437 - 1481  2004.07  [Refereed]

     View Summary

    We aim at an extension of AdaBoost to U-Boost, in the paradigm to build a stronger classification machine from a set of weak learning machines. A geometric understanding of the Bregman divergence defined by a generic convex function U leads to the U-Boost method in the framework of information geometry extended to the space of the finite measures over a label set. We propose two versions of U-Boost learning algorithms by taking account of whether the domain is restricted to the space of probability functions. In the sequential step, we observe that the two adjacent and the initial classifiers are associated with a right triangle in the scale via the Bregman divergence, called the Pythagorean relation. This leads to a mild convergence property of the U-Boost algorithm as seen in the expectation-maximization algorithm. Statistical discussions for consistency and robustness elucidate the properties of the U-Boost methods based on a stochastic assumption for training data.

    DOI

    Scopus

    134
    Citation
    (Scopus)
  • 独立性の検定を用いた,独立成分のグルーピング手法の提案

    伊東 大祐, 向井 卓也, 村田 昇

    日本生体磁気学会論文誌   16 ( 2 ) 23 - 31  2004.06

  • Improving generalization performance of natural gradient learning using optimized regularization by NIC

    H Park, N Murata, S Amari

    NEURAL COMPUTATION   16 ( 2 ) 355 - 382  2004.02  [Refereed]

     View Summary

    Natural gradient learning is known to be efficient in escaping plateau, which is a main cause of the slow learning speed of neural networks. The adaptive natural gradient learning method for practical implementation also has been developed, and its advantage in real-world problems has been confirmed. In this letter, we deal with the generalization performances of the natural gradient method. Since natural gradient learning makes parameters fit to training data quickly, the overfitting phenomenon may easily occur, which results in poor generalization performance. To solve the problem, we introduce the regularization term in natural gradient learning and propose an efficient optimizing method for the scale of regularization by using a generalized Akaike information criterion (network information criterion). We discuss the properties of the optimized regularization strength by NIC through theoretical analysis as well as computer simulations. We confirm the computational efficiency and generalization performance of the proposed method in real-world applications through computational experiments on benchmark problems.

    DOI

    Scopus

    13
    Citation
    (Scopus)
  • The most robust loss function for boosting

    T Kanamori, T Takenouchi, S Eguchi, N Murata

    NEURAL INFORMATION PROCESSING   3316   496 - 501  2004  [Refereed]

     View Summary

    Boosting algorithm is understood as the gradient descent algorithm of a loss function. It is often pointed out that the typical boosting algorithm, Adaboost, is seriously affected by the outliers. In this paper, loss functions for robust boosting are studied. Based on a concept of the robust statistics, we propose a positive-part-truncation of the loss function which makes the boosting algorithm robust against extreme outliers. Numerical experiments show that the proposed boosting algorithm is useful for highly noisy data in comparison with other competitors.

  • Nonlinear PCA/ICA for the structure from motion problem

    J Fujiki, S Akaho, N Murata

    INDEPENDENT COMPONENT ANALYSIS AND BLIND SIGNAL SEPARATION   3195   750 - 757  2004  [Refereed]

     View Summary

    Recovering both camera motion and object shape from multiple images, called structure from motion problem, is an important and essential problem in computer vision. Generally, the result of the structure from motion problem has an ambiguity represented by a three-dimensional rotation matrix. We present two kinds of specific criteria such as independence of parameters to fix the ambiguity by choosing an appropriate rotation matrix in the sense of computer vision. Once some criterion is defined, the fixing of the ambiguity is reduced to a nonlinear extension of the PCA/ICA. We examine the efficiency through synthetic experiments.

  • Learning properties of support vector machines with p-norm

    K Ikeda, N Murata

    2004 47TH MIDWEST SYMPOSIUM ON CIRCUITS AND SYSTEMS, VOL III, CONFERENCE PROCEEDINGS   3   69 - 72  2004  [Refereed]

     View Summary

    Support Vector Machines (SVMs) are a new classification technique which has a high generalization ability, yet a heavy computational load since margin maximization results in a quadratic programming problem. It is known that this maximization task results in a pth-order programming problem if we employ the p-norm instead of the Euclidean norm, that is. When p = 1, for example, it is a linear programming problem with a much lower computational load. In this article, we theoretically show that p has very little affect on the generalization performance of SVMs in practice by considering its geometrical meaning.

  • An approach of moment-based algorithm for noisy ICA models

    D Ito, N Murata

    INDEPENDENT COMPONENT ANALYSIS AND BLIND SIGNAL SEPARATION   3195   343 - 349  2004  [Refereed]

     View Summary

    Factor analysis is well known technique to uncorrelate observed signals with Gaussina noises before ICA (Independent Component Analysis) algorithms are applied. However, factor analysis is not applicable when the number of source signals are more than that of Ledermann's bound, and when the observations are contaminated by non-Gaussian noises. In this paper, an approach is proposed based on higher-order moments of signals and noises in order to overcome those constraints.

  • A robust approach to independent component analysis of signals with high-level noise measurements

    J Cao, N Murata, S Amari, A Cichocki, T Takeda

    IEEE TRANSACTIONS ON NEURAL NETWORKS   14 ( 3 ) 631 - 645  2003.05  [Refereed]

     View Summary

    In this paper, we propose a robust approach for independent component analysis (ICA) of signals that observations are contaminated with high-level additive noise and/or outliers. The source signals may contain mixtures of both sub-Gaussian and super-Gaussian components, and the number of sources is unknown. Our robust approach includes two procedures. In the first procedure, a robust prewhitening technique is used to reduce the power of additive noise, the dimensionality and the correlation among sources. A cross-validation technique is introduced to estimate the number of sources in this first procedure. In the second procedure, a nonlinear function is derived using the parameterized t-distribution density model. This nonlinear function is robust against the undue influence of outliers fundamentally. Moreover, the stability of the proposed algorithm and the robust property of misestimating the parameters (kurtosis) have been studied. By combining the t-distribution model with a family of light-tailed distributions (sub-Gaussian) model, we can separate the mixture of sub-Gaussian and super-Gaussian source components. Through the analysis of artificially synthesized data and real-world magnetoencephalographic (MEG) data, we illustrate the efficacy of this robust approach.

    DOI

  • Independent component analysis for unaveraged single-trial MEG data decomposition and single-dipole source localization

    JT Cao, N Murata, S Amari, A Cichocki, T Takeda

    NEUROCOMPUTING   49 ( “1-4” ) 255 - 277  2002.12  [Refereed]

     View Summary

    This paper presents a novel method for decomposing and localizing unaveraged single-trial magnetoencephalographic data based on the independent component analysis (ICA) approach associated with pre- and post-processing techniques. In the pre-processing stage, recorded single-trial raw data are first decomposed into uncorrelated signals with the reduction of high-power additive noise. In the stage of source separation, the decorrelated source signals are further decomposed into independent source components. In the post-processing stage, we perform a source localization procedure to seek a single-dipole map of decomposed individual source components, e.g., evoked responses. The first results of applying the proposed robust ICA approach to single-trial data with phantom and auditory evoked field tasks indicate the following. (1) A source signal is successfully extracted from unaveraged single-trial phantom data. The accuracy of dipole estimation for the decomposed source is even better than that of taking the average of total trials. (2) Not only the behavior and location of individual neuronal sources can be obtained but also the activity strength (amplitude) of evoked responses corresponding to a stimulation trial can be obtained and visualized. Moreover, the dynamics of individual neuronal sources, such as the trial-by-trial variations of the amplitude and location, can be observed. (C) 2002 Elsevier Science B.V. All rights reserved.

    DOI

    Scopus

    43
    Citation
    (Scopus)
  • On-line learning in changing environments with applications in supervised and unsupervised learning

    N Murata, M Kawanabe, A Ziehe, KR Muller, S Amari

    NEURAL NETWORKS   15 ( 4-6 ) 743 - 760  2002.06  [Refereed]

     View Summary

    An adaptive on-line algorithm extending the learning of learning idea is proposed and theoretically motivated. Relying only on gradient flow information it can be applied to learning continuous functions or distributions, even when no explicit loss function is given and the Hessian is not available. The framework is applied for unsupervised and supervised learning. Its efficiency is demonstrated for drifting and switching non-stationary blind separation tasks of acoustic signals. Furthermore applications to classification (US postal service data set) and time-series prediction in changing environments are presented. (C) 2002 Elsevier Science Ltd. All rights reserved.

    DOI

    Scopus

    47
    Citation
    (Scopus)
  • Support vector machines with different norms: motivation, formulations and results

    JP Pedroso, N Murata

    PATTERN RECOGNITION LETTERS   22 ( 12 ) 1263 - 1272  2001.10  [Refereed]

     View Summary

    We introduce two formulations for training support vector machines, based on considering the L-1 and L-infinity norms instead of the currently used L-2 norm, and maximising the margin between the separating hyperplane and each data sets using L-1 and L-infinity distances. We exploit the geometrical properties of these different norms. and propose what kind of results should be expected for them. Formulations in mathematical programming for linear problems corresponding to L-1 and L-infinity norms are also provided, for both the separable and non-separable cases. We report results obtained for some standard benchmark problems. which confirmed that the performance of all the formulations is similar. As expected, the CPU time required for machines solvable with linear programming is much shorter. (C) 2001 Elsevier Science B.V. All rights reserved.

    DOI

    Scopus

    33
    Citation
    (Scopus)
  • Multiplicative nonholonomic/Newton-like algorithm

    T Akuzawa, N Murata

    CHAOS SOLITONS & FRACTALS   12 ( 4 ) 785 - 793  2001.03  [Refereed]

     View Summary

    We construct new algorithms, which use the fourth order cumulant of stochastic variables for the cost function. The multiplicative updating rule here constructed is natural from the homogeneous nature of the Lie group and has numerous merits for the rigorous treatment of the dynamics. As one consequence, the second order convergence is shown. For the cost function, functions invariant under the componentwise scaling are chosen. By identifying points which can be transformed to each other by the scaling, we assume that the dynamics is in a coset space. In our method, a point can move toward any direction in this coset. Thus, no prewhitening is required. (C) 2001 Elsevier Science Ltd. All rights reserved.

    DOI

    Scopus

    8
    Citation
    (Scopus)
  • An approach to blind source separation based on temporal structure of speech signals

    Noboru Murata, Shiro Ikeda, Andreas Ziehe

    Neurocomputing   41 ( 1-4 ) 1 - 24  2001  [Refereed]

     View Summary

    In this paper, we introduce a new technique for blind source separation of speech signals. We focus on the temporal structure of the signals. The idea is to apply the decorrelation method proposed by Molgedey and Schuster in the time-frequency domain. Since we are applying separation algorithm on each frequency separately, we have to solve the amplitude and permutation ambiguity properly to reconstruct the separated signals. For solving the amplitude ambiguity, we use the matrix inversion and for the permutation ambiguity, we introduce a method based on the temporal structure of speech signals. We show some results of experiments with both artificially controlled data and speech data recorded in the real environment. © 2001 Elsevier Science B.V. All rights reserved.

    DOI

    Scopus

    454
    Citation
    (Scopus)
  • Sequential extraction of minor components

    TP Chen, SI Amari, N Murata

    NEURAL PROCESSING LETTERS   13 ( 3 ) 195 - 201  2001  [Refereed]

     View Summary

    Principal component analysis (PCA) and Minor component analysis (MCA) are similar but have different dynamical performances. Unexpectedly, a sequential extraction algorithm for MCA proposed by Luo and Unbehauen [11] does not work for MCA, while it works for PCA. We propose a different sequential-addition algorithm which works for MCA. We also show a conversion mechanism by which any PCA algorithms are converted to dynamically equivalent MCA algorithms and vice versa.

    DOI

    Scopus

    19
    Citation
    (Scopus)
  • Single-trial magnetoencephalographic data decomposition and localization based on independent component analysis approach

    JT Cao, N Murata, S Amari, A Cichocki, T Takeda, H Endo, N Harada

    IEICE TRANSACTIONS ON FUNDAMENTALS OF ELECTRONICS COMMUNICATIONS AND COMPUTER SCIENCES   E83A ( 9 ) 1757 - 1766  2000.09

     View Summary

    Magnetoencephalography (MEG) is a powerful and non-invasive technique for measuring human brain activity with a high temporal resolution. The motivation for studying MEG data analysis is to extract the essential features from measured data and represent them corresponding to the human brain functions. In this paper a novel MEG data analysis method based on independent component analysis (ICA) approach with pre-processing: and post-processing multistage procedures is proposed. Moreover, several kinds of ICA algorithms are investigated for analyzing MEG; tingle-trial data which is recorded in the experiment of phantom. The analyzed results are presented to illustrate the effectiveness and high performance both in source decomposition by IC:A approaches and source localization by equivalent current dipoles fitting method.

  • Single-trial magnetoencephalographic data decomposition and localization based on independent component analysis approach

    JT Cao, N Murata, S Amari, A Cichocki, T Takeda, H Endo, N Harada

    IEICE TRANSACTIONS ON FUNDAMENTALS OF ELECTRONICS COMMUNICATIONS AND COMPUTER SCIENCES   E83A ( 9 ) 1757 - 1766  2000.09  [Refereed]

     View Summary

    Magnetoencephalography (MEG) is a powerful and non-invasive technique for measuring human brain activity with a high temporal resolution. The motivation for studying MEG data analysis is to extract the essential features from measured data and represent them corresponding to the human brain functions. In this paper a novel MEG data analysis method based on independent component analysis (ICA) approach with pre-processing: and post-processing multistage procedures is proposed. Moreover, several kinds of ICA algorithms are investigated for analyzing MEG; tingle-trial data which is recorded in the experiment of phantom. The analyzed results are presented to illustrate the effectiveness and high performance both in source decomposition by IC:A approaches and source localization by equivalent current dipoles fitting method.

  • Independent component analysis algorithm for online blind source separation and blind equalization systems

    Jianting Cao, Noboru Murata, Andrew Cichocki

    Journal of Signal Processing   4 ( 2 ) 131 - 140  2000.03

    CiNii

  • Optimisation on support vector machines

    JP Pedroso, N Murata

    IJCNN 2000: PROCEEDINGS OF THE IEEE-INNS-ENNS INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS, VOL VI     399 - 404  2000  [Refereed]

     View Summary

    In this paper we deal with the optimisation problem involved in determining the maximal margin separation hyperplane in support vector machines. We consider three different formulations, based on L-2 norm distance (the standard case), L-1 norm, and L-infinity norm. We consider separation in the original space of the data (i.e., there are no kernel transformations).
    For any of these cases, we focus on the following problem: having the optimal solution for a given training data set, one is given a new training example. The purpose is to use the information about the solution of the problem without the additional example in order to speed up the new optimisation problem. We also consider the case of reoptimisation after removing an example from the data set.
    We report results obtained for some standard benchmark problems.

  • Population decoding based on an unfaithful model

    S Wu, H Nakahara, N Murata, S Amari

    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 12   12   192 - 198  2000  [Refereed]

     View Summary

    We study a population decoding paradigm in which the maximum likelihood inference is based on an unfaithful decoding model (UMLI), This is usually the case for neural population decoding because the encoding process of the brain is not exactly known, or because a simplified decoding model is preferred for saving computational cost. We consider an unfaithful decoding model which neglects the pair-wise correlation between neuronal activities, and prove that UMLI is asymptotically efficient when the neuronal correlation is uniform or of limited-range. The performance of UMLI is compared with that of the maximum likelihood inference based on a faithful model and that of the center of mass decoding method. It turns out that UMLI has advantages of decreasing the computational complexity remarkablely and maintaining a high-level decoding accuracy at the same time. The effect of correlation on the decoding accuracy is also discussed.

  • A stable and robust ICA algorithm based on t-distribution and generalized Gaussian distribution models

    Jianting Cao, Noboru Murata

    Neural Networks for Signal Processing   IX   283 - 292  1999.08

    DOI

  • Statistical analysis of learning dynamics

    N Murata, S Amari

    SIGNAL PROCESSING   74 ( 1 ) 3 - 28  1999.04  [Refereed]

     View Summary

    Learning is a flexible and effective means of extracting the stochastic structure of the environment. It provides an effective method for blind separation and deconvolution in signal processing. Two different types of learning are used, namely batch learning and on-line learning. The batch learning procedure uses all the training examples repeatedly so that its performance is compared to the statistical estimation procedure. On-line learning is more dynamical, updating the current estimate by observing a new datum one by one. On-line learning is slow in general but works well in the changing environment. The present paper gives a unified framework of statistical analysis for batch and on-line learning. The topics include the asymptotic learning curve, generalization error and training error, over-fitting and over-training, efficiency of learning, and an adaptive method of determining learning rate. (C) 1999 Elsevier Science B.V. All rights reserved.

    DOI

    Scopus

    29
    Citation
    (Scopus)
  • Stable and robust ICA algorithm based on t-distribution and generalized Gaussian distribution models

    Cao Jianting, Murata Noboru

    Neural Networks for Signal Processing - Proceedings of the IEEE Workshop   ( 99 ) 283 - 292  1999  [Refereed]

    CiNii

  • Statistical study on on-line learning

    Noboru Murata

    In David Saad editor, On-line Learning in Neural Networks, Cambridge University Press    1998.12

  • Statistical inference: learning in artificial neural networks

    HH Yang, N Murata, S Amari

    TRENDS IN COGNITIVE SCIENCES   2 ( 1 ) 4 - 10  1998.01  [Refereed]

     View Summary

    Artificial neural networks (ANNs) are widely used to model low-level neural activities and high-level cognitive functions. In this article, we review the application of statistical inference for learning in ANNs. Statistical inference provides an objective way to derive learning algorithms both for training and for evaluation of the performance of trained ANNs. Solutions to the over-fitting problem by model- selection methods, based on either conventional statistical approaches or on a Bayesian approach, are discussed. The use of supervised and unsupervised learning algorithms for ANNs are reviewed. Training a multilayer ANN by supervised learning is equivalent to nonlinear regression. The ensemble methods, bagging and arching, described here, can be applied to combine ANNs to form a new predictor with improved performance. Unsupervised learning algorithms that are derived either by the Hebbian law for bottom-up self-organization, or by global objective functions for top-down self-organization are also discussed.

    DOI

    Scopus

    9
    Citation
    (Scopus)
  • Asymptotic statistical theory of overtraining and cross-validation

    S Amari, N Murata, KR Muller, M Finke, HH Yang

    IEEE TRANSACTIONS ON NEURAL NETWORKS   8 ( 5 ) 985 - 996  1997.09  [Refereed]

     View Summary

    A statistical theory for overtraining is proposed. The analysis treats general realizable stochastic neural networks, trained with Kullback-Leibler divergence in the asymptotic case of a large number of training examples. It is shown that the asymptotic gain in the generalization error Is small if we perform early stopping, even if we have access to the optimal stopping time. Considering cross-validation stopping we answer the question: In what ratio the examples should be divided into training and cross-validation sets in order to obtain the optimum performance. Although cross-validated early stopping is useless in the asymptotic region, it surely decreases the generalization error in the nonasymptotic region. Our large scale simulations done on a CM5 are in nice agreement with our analytical findings.

    DOI

    Scopus

    281
    Citation
    (Scopus)
  • Adaptive on-line learning in changing environments

    N Murata, KR Muller, A Ziehe, SI Amari

    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 9   9   599 - 605  1997  [Refereed]

     View Summary

    An adaptive on-line algorithm extending the learning of learning idea is proposed and theoretically motivated. Relying only on gradient flow information it can be applied, to learning continuous functions or distributions, even when no explicit loss function is given and the Hessian is not available. Its efficiency is demonstrated for a non-stationary blind separation task of acoustic signals.

  • Statistical analysis of regularization constant - From Bayes, MDL and NIC points of view

    S Amari, N Murata

    BIOLOGICAL AND ARTIFICIAL COMPUTATION: FROM NEUROSCIENCE TO TECHNOLOGY   1240   284 - 293  1997  [Refereed]

     View Summary

    In order to avoid overfitting in neural learning, a regularization term is added to the loss function to be minimized. It is naturally derived from the Bayesian standpoint. The present paper studies how to determine the regularization constant from the points of view of the empirical Bayes approach, the maximum description length (MDL) approach, and the network information criterion (NIC) approach. The asymptotic statistical analysis is given to elucidate their differences. These approaches are tightly connected with the method of model selection. The superiority of the NIC is shown from this analysis.

  • Approximation bounds of three-layered neural networks -- A theorem on an integral transform with ridge functions

    Noboru Murata

    Electronics and Communications in Japan   79 ( 3 ) 23 - 33  1996.08  [Refereed]

    DOI

    Scopus

    44
    Citation
    (Scopus)
  • A numerical study on learning curves in stochastic multilayer feedforward networks

    KR Muller, M Finke, N Murata, K Schulten, S Amari

    NEURAL COMPUTATION   8 ( 5 ) 1085 - 1106  1996.07  [Refereed]

     View Summary

    The universal asymptotic scaling laws proposed by Amari et al. are studied in large scale simulations using a CM5. Small stochastic multilayer feedforward networks trained with backpropagation are investigated. In the range of a large number of training patterns t, the asymptotic generalization error scales as 1/t as predicted. For a medium range t a faster 1/t(2) scaling is observed. This effect is explained by using higher order corrections of the likelihood expansion. It is shown for small t that the scaling law changes drastically, when the network undergoes a transition from strong overfitting to effective learning.

    DOI

    Scopus

    53
    Citation
    (Scopus)
  • Approximation bounds of three-layered neural networks - A theorem on an integral transform with ridge functions

    N Murata

    ELECTRONICS AND COMMUNICATIONS IN JAPAN PART III-FUNDAMENTAL ELECTRONIC SCIENCE   79 ( 3 ) 23 - 33  1996.03  [Refereed]

     View Summary

    Neural networks have attracted attention due to their capability to perform nonlinear function approximation. In this paper, in order to better understand this capability, a new theorem on an integral transform was derived by applying ridge functions to neural networks. From the theorem, it is possible to obtain approximation bounds which clarify the quantitative relationship between the function approximation accuracy and the number of nodes in the hidden layer. The theorem indicates that the approximation accuracy depends on the smoothness of the target function. Furthermore, the theorem also shows that this type of approximation method differs from usual methods and is able to escape the so-called ''curse of dimensionality,'' in which the approximation accuracy depends strongly of the input dimension of the function and deteriorates exponentially.

  • An integral representation of functions using three-layered networks and their approximation bounds

    Noboru Murata

    Neural Networks   9 ( 6 ) 947 - 956  1996  [Refereed]

     View Summary

    Neural networks are widely known to provide a method of approximating nonlinear functions. In order to clarify its approximation ability, a new theorem on an integral transform of ridge functions is presented. By using this theorem, an approximation bound, which evaluates the quantitative relationship between the approximation accuracy and the number of elements in the hidden layer, can be obtained. This result shows that the approximation accuracy depends on the smoothness of target functions. It also shows that the approximation methods which use ridge functions are free from the 'curse of dimensionality'.

    DOI

    Scopus

    44
    Citation
    (Scopus)
  • Statistical theory of overtraining - Is cross-validation asymptotically effective?

    S Amari, N Murata, KR Muller, M Finke, H Yang

    ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS 8   8   176 - 182  1996

  • Ridge 関数による積分表現と3層ネットワークの近似誤差

    村田 昇

    電子情報通信学会論文誌   J78-A ( 9 ) 1204 - 1212  1995.09

  • NETWORK INFORMATION CRITERION - DETERMINING THE NUMBER OF HIDDEN UNITS FOR AN ARTIFICIAL NEURAL-NETWORK MODEL

    N MURATA, S YOSHIZAWA, S AMARI

    IEEE TRANSACTIONS ON NEURAL NETWORKS   5 ( 6 ) 865 - 872  1994.11  [Refereed]

     View Summary

    The problem of model selection, or determination of the number of hidden units, can be approached statistically, by generalizing Akaike's information Criterion (AIC) to be applicable to unfaithful (i.e., unrealizable) models with general loss criteria including regularization terms. The relation between the training error and the generalization error is studied in terms of the number of the training examples and the complexity of It network which reduces to the number of parameters in the ordinary statistical theory of the AIC. This relation leads to a new Network Information Criterion (NIC) which is useful for selecting the optimal network model based on a given training set.

    DOI

    Scopus

    476
    Citation
    (Scopus)
  • UNIVERSAL PROPERTIES OF LEARNING-CURVES

    S AMARI, N MURATA, K IKEDA

    COGNITIVE PROCESSING FOR VISION & VOICE     77 - 87  1994  [Refereed]

  • PREDICTION ERROR OF STOCHASTIC LEARNING MACHINE

    K IKEDA, N MURATA, S AMARI

    1994 IEEE INTERNATIONAL CONFERENCE ON NEURAL NETWORKS, VOL 1-7   2   1159 - 1162  1994  [Refereed]

  • Learning curves, model selection and complexity of neural networks

    Noboru Murata, Shuji Yoshizawa, Shun-ichi Amari

    In Stephen Jose Hanson, Jack D. Cowan, and C. Lee Giles, editors, Advances in Neural Information Processing Systems, Morgan Kaufmann Publishers, San Mateo, CA   5   607 - 614  1993

  • STATISTICAL-THEORY OF LEARNING-CURVES UNDER ENTROPIC LOSS CRITERION

    S AMARI, N MURATA

    NEURAL COMPUTATION   5 ( 1 ) 140 - 153  1993.01

     View Summary

    The present paper elucidates a universal property of learning curves, which shows how the generalization error, training error, and the complexity of the underlying stochastic machine are related and how the behavior of a stochastic machine is improved as the number of training examples increases. The error is measured by the entropic loss. It is proved that the generalization error converges to H0, the entropy of the conditional distribution of the true machine, as H0 + m*/(2t), while the training error converges as H0 - m*/(2t), where t is the number of examples and m* shows the complexity of the network. When the model is faithful, implying that the true machine is in the model, m* is reduced to m, the number of modifiable parameters. This is a universal law because it holds for any regular machine irrespective of its structure under the maximum likelihood estimator. Similar relations are obtained for the Bayes and Gibbs learning algorithms. These learning curves show the relation among the accuracy of learning, the complexity of a model, and the number of training examples.

    DOI

  • A CRITERION FOR DETERMINING THE NUMBER OF PARAMETERS IN AN ARTIFICIAL NEURAL NETWORK MODEL

    N MURATA, S YOSHIZAWA, S AMARI

    ARTIFICIAL NEURAL NETWORKS, VOLS 1 AND 2     9 - 14  1991  [Refereed]

▼display all

Books and Other Publications

  • パターン認識と機械学習 : ベイズ理論による統計的予測 上

    Bishop Christopher M, 元田 浩, 栗田 多喜夫, 樋口 知之, 松本 裕治, 村田 昇

    丸善出版  2012 ISBN: 9784621061220

  • 電気・電子・情報のための基礎数学

    村田 純一, 村田 昇

    オーム社  2011 ISBN: 9784274210877

  • パターン認識

    金森 敬文, 竹之内 高志, 村田 昇

    共立出版  2009 ISBN: 9784320019256

  • パターン認識と機械学習 : ベイズ理論による統計的予測 上

    Bishop Christopher M, 元田 浩, 栗田 多喜夫, 樋口 知之, 松本 裕治, 村田 昇

    シュプリンガー・ジャパン  2007 ISBN: 9784431100133

  • 確率と統計 : 情報学への架橋

    渡辺 澄夫, 村田 昇

    コロナ社  2005 ISBN: 9784339060775

  • 情報理論の基礎 - 情報と学習の直観的理解のために.

    村田昇

    サイエンス社  2005 ISBN: 9784781912127

  • 入門 独立成分分析.

    村田昇

    東京電機大学出版局  2004 ISBN: 4501537507

  • パターン認識と学習の統計学 - 新しい概念と手法.

    麻生英樹, 津田宏治, 村田昇

    岩波書店  2003 ISBN: 4000068466

  • 独立成分分析 - 多変量データ解析の新しい方法.

    甘利俊一, 村田昇 共編著

    サイエンス社  2002

  • パターン認識と機械学習 : ベイズ理論による統計的予測 下

    Bishop Christopher M, 元田 浩, 栗田 多喜夫, 樋口 知之, 松本 裕治, 村田 昇

    シュプリンガー・ジャパン  ISBN: 9784431100317

  • パターン認識と機械学習 : ベイズ理論による統計的予測 下

    Bishop Christopher M, 元田 浩, 栗田 多喜夫, 樋口 知之, 松本 裕治, 村田 昇

    丸善出版  ISBN: 9784621061244

▼display all

Research Projects

  • Explorations into the Neurocognitive Basis of Symbolic Processing: Focusing on the Mediation System between Form and Meaning of Human Language

    Japan Society for the Promotion of Science  Grants-in-Aid for Scientific Research

    Project Year :

    2023.04
    -
    2028.03
     

  • Analysis of transfer learning based on information geometry

    Japan Society for the Promotion of Science  Grants-in-Aid for Scientific Research

    Project Year :

    2022.04
    -
    2027.03
     

  • 脳磁図と頭蓋内脳波の時間分解MVP解析による言語の形式と意味を繋ぐシステムの解明

    日本学術振興会  科学研究費助成事業

    Project Year :

    2023.04
    -
    2026.03
     

    酒井 弘, 西本 伸志, 井上 貴文, 田中 慶太, 大関 洋平, 松本 敦, 太田 真理, 久保田 有一, 宮本 陽一, 村田 昇, 大須 理英子

  • 生命に現在の20種類の標準アミノ酸は必要か:遺伝暗号改変による理工学アプローチ

    日本学術振興会  科学研究費助成事業

    Project Year :

    2019.04
    -
    2022.03
     

    木賀 大介, 村田 昇

  • Development of machine learning method for distribution data based on information geometry

    Japan Society for the Promotion of Science  Grants-in-Aid for Scientific Research

    Project Year :

    2017.04
    -
    2022.03
     

    Akaho Shotaro

     View Summary

    An effective way to handle large amounts of data by machine learning is to reduce the data to the parameters of a probability distribution. In this project, we have been working on the development of machine learning for such data. Originally, there was a study of extending principal component analysis to distributional data, which had been developed by a project member. The significant contribution of this project was to extend it to a more flexible nonparametric framework, which was achieved through information geometry of Gaussian process regression and other methods. We have also been able to apply information geometry to neuroscience and geophysics through the application of matrix factorization.

  • Development of the MDL Principle and Its Applications

    Japan Society for the Promotion of Science  Grants-in-Aid for Scientific Research

    Project Year :

    2018.04
    -
    2021.03
     

    Takeuchi Junichi

     View Summary

    We studied development of learning theory based on the Minimum Description Length (MDL) principle and its application. Concerning the MDL principle, after reconsidering our previous results on enhancement of Barron and Cover theory (BC theory) to supervised learning, we considered the relation to deep learning. We also studied the connection between the BC theory and the stochastic complexity and its application to non-exponential families, including the mixture families and the simple contaminated Gaussian location families. As for real application, we studied MRI image reconstruction based on deep learning and data analysis for caber security. For the former topic, we proposed a high speed reconstruction method which enjoyed good image quality for MR Angiography. For the latter topic, we developed a clustering method based on the MDL principle and phylogenetic trees and showed that its performance was good by experiment using real IoT malware data.

  • Statistical Science of Bioinformatics

    Japan Society for the Promotion of Science  Grants-in-Aid for Scientific Research

    Project Year :

    2015.04
    -
    2019.03
     

    Kano Yutaka

     View Summary

    We can summarize our research results as nine categories. In particular, results on the missing data analysis and meta analysis are influential. For the results to obtain, we have organized and offered five international symposiums and a domestic one in the research period. Each of the symposiums consisted of 3 to 20 presenters. In addition, we have hosted small size colloquiums 13 times in Osaka University, where totally 15 speakers were invited; and essential discussions in the colloquiums helped to proceed and complete our research.

  • Extraction of latent structure by sparse modeling

    Japan Society for the Promotion of Science  Grants-in-Aid for Scientific Research

    Project Year :

    2013.06
    -
    2018.03
     

    Okada Masato

     View Summary

    The sparse modeling team (B01-2) sets three tasks. Task 1 is applications of Bayesian spectral decomposition method to actual data. We developed a noise variance estimation method and a fast calculation method using L1 regularization and verified its effectiveness with actual data. In task 2, we developed a basis estimation and selection method using Sp-DMD for time series data, and applied it to actual data and verified its effectiveness. In task 3, a method of evaluating the appropriateness of the basis combination using an exhaustive search was developed, and this method was applied to actual data and the effectiveness was verified. Through research on these three tasks, we developed a universal method to extract latent structures using SpM and verified its effectiveness by actual data.

  • Theoretical statistics for stochastic processes and limit theorems

    Japan Society for the Promotion of Science  Grants-in-Aid for Scientific Research

    Project Year :

    2012.04
    -
    2016.03
     

    Yoshida Nakahiro, MASUDA Hiroki, MURATA Noboru, UCHIDA Masayuki, SHIMIZU Yasutaka, FUKASAWA Masaaki, KAMATANI Kengo

     View Summary

    The quasi likelihood analysis was constructed for a stochastic regression model of volatility based on high frequency data in the finite time horizon, and an analytic criterion and a geometric criterion for non-degeneracy of the statistical random field associated with the quasi likelihood function were provided. The asymptotic mixed normality and the convergence of moments were proved. A quasi likelihood analysis was developed for a non-synchronously observed stochastic differential equation. Asymptotic expansion for a martingale with mixed normal limit was established. It is a new limit theorem beyond the frame of the present theory of asymptotic expansion for ergodic processes. The martingale expansion was applied to the p-variation. Studies of the asymptotic expansion of volatility estimators under microstructure noise have been developed. The spot volatility information criterion sVIC was proposed, and the fundamentals for developing computer software were studied.

  • Weighted data contraction based on information content

    Japan Society for the Promotion of Science  Grants-in-Aid for Scientific Research

    Project Year :

    2011
    -
    2012
     

    MURATA Noboru, HINO Hideitsu

     View Summary

    Generalizing the classical k-nearest neighbor method for entropy estimation, an computationally efficient method for estimating information contents of weighted data is proposed. For utilizing our distance-based method to various kinds of data sets, distance metric learning methods are considered in the framework of multiple kernel learning and just-in-time modeling. Validity of those proposed methods are confirmed by clustering problems and ensemble learning of real-world data.

  • Statistical regularization theory and neurophysiology

    Japan Society for the Promotion of Science  Grants-in-Aid for Scientific Research

    Project Year :

    2011
    -
    2012
     

    KANO Yutaka, KOBAYASHI Yasushi, INUI Toshiro, SHIMURA Tsuyoshi, ADACHI Kohei, MURATA Noboru, YAMAMOTO Michio

     View Summary

    We organized and hosted an international symposium “Life Science and Statistics” in Osaka University in 2011, the first year of the research. The symposium was held to aim at constructing an interface among researchers in statistics, life science, cognitive psychologyand brain sciences. At least two joint international works have been accomplished, as fruits of having the symposium; one is on a new estimation of factor analysis model and the other on a new factor rotation method. After the symposium, a new research group of statisticians and brain scientists was created to conduct an interdisciplinary study.

  • Online Change Detection with Sequential Marginal Likelihook : Particle Filter Approach

    Japan Society for the Promotion of Science  Grants-in-Aid for Scientific Research

    Project Year :

    2005
    -
    2007
     

    MATSUMOTO Takashi, MURATA Noboru

     View Summary

    1. Online change detection is referred to as detecting abrupt changes behind give data online. Explicit functional form of the target behind given data is often unavailable. In addition, target system is often nonlinear so that linear algorithms are often unsatisfactory. This project proposed two online change detection algorithms within online Bayesian framework. One uses Sequential Marginal Likelihood while the other introduces a latent variable indicating abrupt changes. Both of them are implemented via Sequential Monte Carlo methods. The algorithms are verified with several examples.
    2. Another algorithm is proposed in detecting faces in a video sequences. The algorithm is also based on an online Bayesian framework where it partly utilizes the Viola-Jones static score in a sequential manner. This is also implemented via Sequential Monte Carlo. It also checks Sequential Marginal Likelihood to detect any changes in which case the proposal distribution is reset. The algorithm is verified against real data.
    3. In order to deal with complex features of data such as discontinuity of time series and deformation of images, a new non-linear feature extraction method is investigated based on the notion of marginal entropy minimization. The validity of the method is confirmed by neumerical experiments with real face data.

  • New statistical methodology for genome diversity analysis

    Japan Society for the Promotion of Science  Grants-in-Aid for Scientific Research

    Project Year :

    2004
    -
    2007
     

    EGUCHI Shinto, FUJISAWA Hironori, HEMMI Masayuki, MATSUURA Masaaki, TAKASHI Takenouchi, MASANORI Kawakita

     View Summary

    In this project we contributed on building up a new paradigm in statistical science. In particular, we focus on developing statistical methods for conducting rational reasoning induced from genome data. The study aims at discovery of genes strongly associated with a difficult disease by statistical method, and identification of SNPs that are related with drug effect and sensitivity. For this objective we propose and implement new methodology that works these specific targets as summarized in the following.
    1. Group-Boost for the association study of gen expressions and phenotypes
    2. Common peak approach for identifying peaks as a biomarker of phenotypes.
    3. SNP identification for predicting drug effects and sensitivities.
    4. A unified research in machine learning for genome data analyses
    In these developments we are further devoting to extending conventional methods including boosting, independent/principal, component analysis, clustering to more flexible and understandable methods. This future works is expected to a wide and universal promotion in statistical science. Finally we acknowledge many researches in Jananese Foundation for Cancer Research for kind and instructive advices.

  • A Statistical Study on Bayes Statistics and Ensemble Learning

    Japan Society for the Promotion of Science  Grants-in-Aid for Scientific Research

    Project Year :

    2002
    -
    2005
     

    MURATA Noboru, IKEDA Kazushi

     View Summary

    In order to study boosting algorithms, we consider the structure of a space of general learning models which is naturally introduced by Bregman divergence. Statistical properties such as robustness against noises and outliers, asymptotic efficiency depending on the size of training samples and learning models, and Bayes optimality and consistency of convex functions which induce Bregman divergences, are discussed and clarified. Based on the above consideration, we have proposed a new generic class of boosting algorithms, which is called "U-Boost".
    Moreover, extending boosting algorithms to the density estimation, we have proposed an algorithm for regression problems. In the algorithm, Gaussian processes in reproducing kernel Hilbert spaces are used as regressors, and estimating functions based on Bregman divergences are utilized for inference.
    In our study, a close relationship between boosting algorithm and support vector machines has been exposed, therefore we have also studied on the generalization errors of support vector machines from an algebraic and geometrical viewpoint.
    For practical applications, we have coped with the following problems.
    In order to avoid an explosion of the number of parameters, which frequently occurs in estimating a huge probability table of graphical models and Bayesian networks, we have constructed a mixture model based on the concept of ensemble learning. The model consists of simple tables and has rather good generalization errors. We discussed an estimation algorithm of the model, which is an extension of the EM algorithm from a viewpoint of information geometry.
    We also worked on constructing an on-line algorithm for boosting, in order to apply the boosting to learning problems such as reinforcement learning, in which plenty of data are observed one after another. We have considered methods for reconstructing the objective function from sequentially obtained data, and compared with ordinary off-line boosting algorithms.

  • Fusion of Statistics, Neural-Net, Machine Learning

    Japan Society for the Promotion of Science  Grants-in-Aid for Scientific Research

    Project Year :

    2001
    -
    2003
     

    EGUCHI Shinto, KANAMORI Takafumi, FUJISAWA Hironori, MINAMI Mihoko, MATSUURA Masaaki, MURATA Noboru

     View Summary

    1.Challenging bioinformatics
    Our research group results to succeed the project for SNP (Single Nucleotide Polymorphism) typing with co-operative work with the genome center in Research Institute of Cancer Study. The paper has published in the journal of 'Bioinformatics' in 2004 and the portion of the work has applied to Japan Patent Office. Presently the next project of SNP haplotype blocking is promising and will attain the result in a near future.
    In this research collaboration we have a monthly research meeting, in which the investigation for proteome date is now made a rapid progress. In November, 2003 Professor P.K. Sen at North Carolina State University gave a lecture on bioinformatics and nonparametric method, which was organized by our research group.
    2.Statistical theory and Boost algorithm
    In these three years a monthly research meeting has been organized on Saturday in which Dr Murata, Dr Kanainori, Dr Takenouchi and Dr Eguchi has establish statistical understanding and several variants alternative to AdaBoost. Their two papers has published in a journal of 'Neural Computation' in 2004.
    3.Independent component analysis and principal component analysis
    The fourth International conference on ICA had held at Nara Conference center in 2003, in which Dr Murata, Dr Minami and Dr Eguchi worked as organizing committee. The work on self-organizing PCA has published in J.Machine Learning Research.

  • A Basic Study on Multimodal Biometric Person Authentication : Signature, Face and Voice

    Japan Society for the Promotion of Science  Grants-in-Aid for Scientific Research

    Project Year :

    2001
    -
    2003
     

    MATSUMOTO Takashi, MURATA Noboru

     View Summary

    The algorithms used to verify personal identity can roughly be classified into the following four categories : static, dynamic, biometric, and physical/knowledge-based. For instance, fingerprints, iris, retina, DNA, face and blood vessels are static and biometric. Algorithms classified as biometric and dynamic include lip movements, body movements and on-line signatures. Schemes that use passwords are static and knowledge-based, whereas methods using IC cards, magnetic cards, and key are physical.
    This research project investigated the following biometric schemes as the first step toward multimodal authentication :
    1. online signature
    We have proposed several different algorithms. The most recent one uses Bayesian method with MCMC (Markov Chain Monte Carlo). FRR of 0.93% and FAR of 0.73% were simultaneously achieved.
    2. face
    After segmenting a face subimage from a natural image data, nine feature points were esitiamted using Gabor wevelette features together with MCMC. A great improvements over the previous sheme has been observed.
    2. voice
    Several algorithms have been tried.
    Overall fusion of the three methods are still in its infancy and will be a topic of future project.

  • "Confirmatory" independent component analysis and independent factor analysis

    Japan Society for the Promotion of Science  Grants-in-Aid for Scientific Research

    Project Year :

    2000
    -
    2002
     

    KANO Yutaka, ICHIKAWA Masanori, MURATA Noboru, HARADA Akira

     View Summary

    (1) An international symposium on independent component analysis (ICA) and structural equation modeling (SEM) was held jointly with IMPS2001 at the Osaka University Convention Center in July 2001. We invited Professors A. Hyvarinen from Finland, P. M. Bentler from USA, Sik-Yum Lee from Hong Kong, A, Mooijaart from the Netherlands besides Japanese invited speakers. We discussed how we can incorporate the idea from confirmatory nature of SEM with the ICA field.
    (2) We showed that the model is not estimable where both specific factors and common factors influence on a dependent variable in the SEM framework and that the model can be estimated if the specific factors are mutually independent and nonnonnally distributed. The model is nothing a confirmatory ICA model.
    (3) It was shown that there are many models that are not estimable in the SEM framework but can be estimated within the ICA formulation if latent factors are nonnormal and independent. For instance, the exploratory factor analysis model with an arbitrary error structure is estimable if the errors are normally distributed.
    (4) We suggested a new goodness-of-fit test statistic of nonnormal structural models using higher-order moments.

  • 統計的因果推論と非正規性

    科学研究費助成事業(大阪大学)  科学研究費助成事業(基盤研究(C))

  • 神経回路モデルに基づく学習情報処理機構の研究

    科学研究費助成事業(東京大学)  科学研究費助成事業(一般研究(C))

  • Between ICA and SEM

▼display all

Misc

  • The global optimum of shallow neural network is attained by ridgelet transform

    Sho Sonoda, Isao Ishikawa, Masahiro Ikeda, Kei Hagihara, Yoshihiro Sawano, Takuo Matsubara, Noboru Murata

       2018.05

    Internal/External technical report, pre-print, etc.  

     View Summary

    We prove that the global minimum of the backpropagation (BP) training problem<br />
    of neural networks with an arbitrary nonlinear activation is given by the<br />
    ridgelet transform. A series of computational experiments show that there<br />
    exists an interesting similarity between the scatter plot of hidden parameters<br />
    in a shallow neural network after the BP training and the spectrum of the<br />
    ridgelet transform. By introducing a continuous model of neural networks, we<br />
    reduce the training problem to a convex optimization in an infinite dimensional<br />
    Hilbert space, and obtain the explicit expression of the global optimizer via<br />
    the ridgelet transform.

  • Toward Ridgelet Analysis of Deep Learning (Wavelet analysis and signal processing)

    Sonoda Sho, Murata Noboru

    RIMS Kokyuroku   2001   64 - 73  2016.07

    CiNii

  • グレーボックスモデルとブートストラップフィルタによる不確定な製鋼プロセスの溶鋼温度予測

    AHMAD Iftikhar, 加納学, 長谷部伸治, 北田宏, 村田昇

    化学工学会秋季大会研究発表講演要旨集(CD-ROM)   47th   ROMBUNNO.N214  2015.09

    J-GLOBAL

  • 時間推移する定常分布の潜在構造モデル化

    千葉 智暁, 日野 英逸, 赤穂 昭太郎, 村田 昇

    研究報告数理モデル化と問題解決(MPS)   2014 ( 6 ) 1 - 6  2014.12

     View Summary

    購買者が限定されているマーケットでの製品の売上や株式市場などをモデル化するには,有限な資源を複数の関係者が奪い合うような条件を想定することが必要となる.本稿では,この条件下で存在しうる,資源の移動経路の動的な潜在構造と,関係者が持つ資源の量の変動に着目することで,潜在構造と時系列の関係性を非斉時マルコフ連鎖の遷移確率行列とこれによる分布の遷移でモデル化し,この表現に基づき時間変化する潜在構造を多変量時系列から推定する手法を提案する.

    CiNii

  • Entropy Estimators Based on Simple Linear Regression

    HINO Hideitsu, KOSHIJIMA Kensuke, MURATA Noboru

      114 ( 306 ) 33 - 40  2014.11

     View Summary

    Three differential entropy estimators are proposed based on the second order expansion of the probability mass around the inspection point with respect to the distance from the point. In the second order expantion of the probability mass function, the constant term corresponds to the value of the probability density function at the inspection point. Simple linear regression is utilized to estimate the values of density function. The density estimates at every given data points are averaged to obtain entropy estimators. Another entropy estimator, which directly estimates entropy by linear regression, is also proposed. The proposed three estimators are shown to perform well through numerical experiments for various probability distributions.

    CiNii

  • S0840101 Study of Operational Policy in Residential Energy System Considering Comfort

    YOSHIDA Akira, FUJIMOTO Yu, MURATA Noboru, WAKAO Shinji, TANABE Shinichi, AMANO Yoshiharu

    Mechanical Engineering Congress, Japan   2014   "S0840101 - 1"-"S0840101-5"  2014.09

     View Summary

    The main objectives of this study are to consider both thermal comfort and energy consumption on operational planning problem of residential energy system, and to handle uncertainty of energy demand and PV output for the future as a scenario-based stochastic programming problem. The energy system consists of photovoltaic power generator, electrical-driven room air-conditioner and PEM type fuel-cell cogeneration system. We introduce mathematical optimization theory for operational planning problem, because the targeted system has many alternative operations. As a result, the optimal operational planning problem, which is extended by considering PMV and future scenarios as constraints, concludes that the residential energy system achieves 33% of energy saving ratio in the case of 1.0 of PMV.

    CiNii

  • Sampling Learning Algorithm by Oracle Distribution

    SONODA Sho, MURATA Noboru

    Technical report of IEICE. PRMU   114 ( 197 ) 137 - 142  2014.09

     View Summary

    A new sampling learning algorithm for neural networks is proposed. Based on the integral representation of neural networks, a practical algorithm for calculating an oracle probability distribution of hidden parameters is developed. The samples drawn from the oracle distribution would be good initial parameters for backpropagation. In general rigorous sampling from the oracle distribution holds numerical difficulty, a linear-time sampling algorithm is also developed. Numerical experiments showed that when hidden parameters were initialized by the oracle distribution, following backpropagation converged faster to better parameters than when parameters were initialized by a normal distribution.

    CiNii

  • Sampling Learning Algorithm by Oracle Distribution

    Sho Sonoda, Noboru Murata

    IPSJ SIG Notes. CVIM   2014 ( 24 ) 1 - 6  2014.08

     View Summary

    A new sampling learning algorithm for neural networks is proposed. Based on the integral representation of neural networks, a practical algorithm for calculating an oracle probability distribution of hidden parameters is developed. The samples drawn from the oracle distribution would be good initial parameters for backpropagation. In general rigorous sampling from the oracle distribution holds numerical difficulty, a linear-time sampling algorithm is also developed. Numerical experiments showed that when hidden parameters were initialized by the oracle distribution, following backpropagation converged faster to better parameters than when parameters were initialized by a normal distribution.

    CiNii

  • Spike Data Analysis by Weighted Directed Graph Modeling

    HIGUCHI Sho, HINO Hideitsu, TATSUNO Masami, MURATA Noboru

      114 ( 105 ) 193 - 200  2014.06

     View Summary

    With recent developments in multielectrode recording technology, neural spike data can be obtained from a brain. It is expected to understand the mechanism of information processing in a brain by analyzing these neural data. It is consider that a brain processes information by neural cooperative activity. In order to analyze neural data, the method that can take into account of neural cooperative activity is desirable. Since neural connections are asymmetric and have different connection strengths, it is required that the method is able to represent these features. In this paper, neural connections are represented by using the weighted directed graph, and a state that a lot of neurons affect each other is modeled. Then a method for estimating graph structures is proposed. Experimental results using artificial neural spike data show that the proposed method is able to estimate weighted directed graph structure.

    CiNii

  • Current Dipole Localization from EEG by Multiple Particle Filters and Model Selection

    KANEDA Yuki, SONODA Sho, HINO Hideitsu, MURATA Noboru

      114 ( 105 ) 91 - 96  2014.06

     View Summary

    In this study, the location, the moment and the number of ionic current modeled as dipoles are estimated from EEG data. The source localization problem is formulated as an inverse problem of the EEG generative model and the location and moment of dipoles are estimated by multiple particle filters. The appropriate number of dipoles is selected by a model selection criterion. The proposed method is shown to estimate accurately the locations and moments of dipoles of appropriate numbers. The proposed method is also applied to pattern reversal VEP data and shown to estimate the locations and the moments and the number of the dipoles.

    CiNii

  • Distances between Marked Point Processes and their Applications for Discriminant Analysis

    TAKANO Ken, KOBAYASHI Mei, HINO Hideitsu, MURATA Noboru

      114 ( 105 ) 83 - 89  2014.06

     View Summary

    A simple point process is a type of a random process expressed only by times at which events happen. A marked point process is an extension of a simple point process, which is expressed by times and values of events. Distance measures between simple and marked point processes are of importance for various data analyses. In this work, a method for calculating distance between time-windowed marked point processes is proposed. The proposed method is based on the extensions of conventional measure for simple point processes to marked point processes. The method combines various distance matrices of marked point processes by using the random forest algorithm taking into account the locality expressed by each distance. The proposed method is applied to seismic data analysis and shown to improve conventional methods.

    CiNii

  • C131 A study of optimal operational planning of residential PEFC system for energy demand scenarios using stochastic programming technique

    YOSHIDA Akira, OGATA Ryohei, MURATA Noboru, AMANO Yoshiharu

    National Symposium on Power and Energy Systems   2014 ( 19 ) 93 - 96  2014.06

     View Summary

    The subject of this study is to plan the operation of residential energy system with uncertain parameters based on ex-ante decision before uncertain parameters are realized. This paper applies a scenario-based stochastic programming framework to the operational planning problem having uncertain energy demand as parameters. Based on predicted energy demand scenarios, the operational strategies, which indicates the prime mover's start-stop status and the level of hot water tank, are decided. The decided operational strategies and realized values of energy demand are input to optimal control problem, which is formulated by dynamic programming. Finally we consider the operational performance of dynamic programming using the decided operational strategies based on predicted demand scenarios. As a result, the proposed stochastic programming framework decided the optimal strategies, and showed that effectiveness.

    CiNii

  • C132 Effect of demand prediction error on energy-saving performance of PEFC system for residential use

    OGATA Ryohei, YOSHIDA Akira, MURATA Noboru, AMANO Yoshiharu

    National Symposium on Power and Energy Systems   2014 ( 19 ) 97 - 100  2014.06

     View Summary

    Nowadays, PEFC-CGS is getting attention as a distributed energy system. It has high efficiency, and a lot of studies of PEFC-CGS were reported. But almost of them didn't consider energy prediction error. In this study, in order to evaluate energy-saving performance of PEFC-CGS for residential use with energy prediction error, I made PEFC-CGS control low input optimal operational plan based on energy prediction value. Consequently, operational plan to start PEFC-CGS in the morning or avoiding unneeded power runup and low load operation of PEFC-CGS can make control low of PEFC-CGS improved.

    CiNii

  • Spike Data Analysis by Weighted Directed Graph Modeling

    Sho Higuchi, Hideitsu Hino, Masami Tatsuno, Noboru Murata

    IPSJ SIG Notes   2014 ( 35 ) 1 - 8  2014.06

     View Summary

    With recent developments in multielectrode recording technology, neural spike data can be obtained from a brain. It is expected to understand the mechanism of information processing in a brain by analyzing these neural data. It is consider that a brain processes information by neural cooperative activity. In order to analyze neural data, the method that can take into account of neural cooperative activity is desirable. Since neural connections are asymmetric and have different connection strengths, it is required that the method is able to represent these features. In this paper, neural connections are represented by using the weighted directed graph, and a state that a lot of neurons affect each other is modeled. Then a method for estimating graph structures is proposed. Experimental results using artificial neural spike data show that the proposed method is able to estimate weighted directed graph structure.

    CiNii

  • Spike Data Analysis by Weighted Directed Graph Modeling

    Sho Higuchi, Hideitsu Hino, Masami Tatsuno, Noboru Murata

    IPSJ SIG technical reports   2014 ( 35 ) 1 - 8  2014.06

     View Summary

    With recent developments in multielectrode recording technology, neural spike data can be obtained from a brain. It is expected to understand the mechanism of information processing in a brain by analyzing these neural data. It is consider that a brain processes information by neural cooperative activity. In order to analyze neural data, the method that can take into account of neural cooperative activity is desirable. Since neural connections are asymmetric and have different connection strengths, it is required that the method is able to represent these features. In this paper, neural connections are represented by using the weighted directed graph, and a state that a lot of neurons affect each other is modeled. Then a method for estimating graph structures is proposed. Experimental results using artificial neural spike data show that the proposed method is able to estimate weighted directed graph structure.

    CiNii

  • Current Dipole Localization from EEG by Multiple Particle Filters and Model Selection

    Yuki Kaneda, Sho Sonoda, Hideitsu Hino, Noboru Murata

    IPSJ SIG technical reports   2014 ( 15 ) 1 - 6  2014.06

     View Summary

    In this study, the location, the moment and the number of ionic current modeled as dipoles are estimated from EEG data. The source localization problem is formulated as an inverse problem of the EEG generative model and the location and moment of dipoles are estimated by multiple particle filters. The appropriate number of dipoles is selected by a model selection criterion. The proposed method is shown to estimate accurately the locations and moments of dipoles of appropriate numbers. The proposed method is also applied to pattern reversal VEP data and shown to estimate the locations and the moments and the number of the dipoles.

    CiNii

  • Distances between Marked Point Processes and their Applications for Discriminant Analysis

    Ken Takano, Mei Kobayashi, Hideitsu Hino, Noboru Murata

    IPSJ SIG technical reports   2014 ( 14 ) 1 - 7  2014.06

     View Summary

    A simple point process is a type of a random process expressed only by times at which events happen. A marked point process is an extension of a simple point process, which is expressed by times and values of events. Distance measures between simple and marked point processes are of importance for various data analyses. In this work, a method for calculating distance between time-windowed marked point processes is proposed. The proposed method is based on the extensions of conventional measure for simple point processes to marked point processes. The method combines various distance matrices of marked point processes by using the random forest algorithm taking into account the locality expressed by each distance. The proposed method is applied to seismic data analysis and shown to improve conventional methods.

    CiNii

  • Estimation of Item Preference Parameter with User Review Date

    MOCHIZUKI Shunichi, FUJIMOTO Yu, MURATA Noboru

      113 ( 476 ) 89 - 94  2014.03

     View Summary

    There are services that recommend items such as shops or products to users based on scores or ratings that other users have given to those items. Application of the Bradley-Terry(BT) model is widely considered as a way to achieve the statistical representation of items based on ratings that users have given to items. The BT model evaluates the true preference parameters of items through the statistical representation. In this paper, a divide and conquer optimization algorithm that enables estimating preference parameters in a feasible computational time when the number of items scales up to the order of ten-thousands is proposed. A number of experimental studies using both synthetic and real datasets are provided to demonstrate the effectiveness of this method.

    CiNii

  • Graph Structure Modeling for Spike Data Analysis

    HIGUCHI Sho, NODA Atsushi, HINO Hideitsu, MURATA Noboru

    IEICE technical report. Neurocomputing   113 ( 111 ) 145 - 150  2013.06

     View Summary

    Information in the brain is processed by neural cooperative activity. Estimation of neural graph structures is important to understand the mechanism of information processing in the brain. Since neural connections are asymmetric, directed graphs are appropriate to represent neural graph structures. In this paper, directed graphs are represented using the digraph Laplacian, and information transition on graphs is modeled by an exponential map of the digraph Laplacian. Moreover, a parameter estimation method is proposed. At last, the proposed method is experimentally shown to be able to estimate directed graph structures using artificial neural spike data.

    CiNii

  • スパースコーディングを用いたマルチフレーム超解像

    加藤利幸, 日野英逸, 村田昇

    研究報告コンピュータビジョンとイメージメディア(CVIM)   2013 ( 3 ) 1 - 9  2013.05

     View Summary

    複数の低解像度の観測画像から高解像度画像の推定を行う,マルチフレーム超解像の一手法を提案する.提案手法は高解像度画像を推定するために,信号のスパース表現を用いる.スパース表現を用いる超解像手法の従来手法の多くはシングルフレーム超解像の手法である.また,スパース表現を用いるマルチフレーム超解像の従来手法は,画像間のサブピクセル精度の位置関係を考慮しないものであったため,複数枚の画像を効率的に利用できなかった.提案手法は低解像度画像に対応する辞書を,画像の劣化過程に従って作成し,その際にサブピクセル精度の位置ずれを取り入れることでこの問題を解決した.観測された低解像度画像の位置関係は,サブピクセル精度のブロックマッチングを用いて求める.超解像に用いる低解像度パッチのサブセットはマッチングの結果として得られる類似度に応じて選択する.提案手法の優れている点は,入力画像の中の一部の画像しか超解像に利用できない場合は,その条件に応じて出来る限り良い推定画像を作成することができる点である.特に一つのみの低解像度パッチが選択される場合,提案手法はシングルフレームの超解像手法としても実行される.実画像を使用した実験により,提案手法は従来のシングルフレームやマルチフレームの超解像手法と同等もしくはより良い結果を示すことを確認した.

    CiNii

  • 溶鋼温度推定を目的としたグレイボックスモデルの比較

    AHMAD Iftikhar, 加納学, 長谷部伸治, 北田宏, 村田昇

    材料とプロセス(CD-ROM)   26 ( 1 ) ROMBUNNO.TO17  2013.03

    J-GLOBAL

  • カーネル層別逆回帰のためのモデル選択手法

    日野英逸, 越島健介, 村田昇

    研究報告数理モデル化と問題解決(MPS)   2012 ( 6 ) 1 - 6  2012.11

     View Summary

    データの次元を適切に削減することは,計算コスト及び記憶領域の削減,データの本質的構造の把握に繋がる.特に回帰問題においては,データが拘束されている部分空間を推定する sufficient dimension reduction の問題が盛んに研究されている.その代表例である層別逆回帰は,説明変数が楕円分布に従う場合は,応答変数の値によって層別した説明変数の中心ベクトルを用いて線型次元削減行列を推定することの正当性が理論的に保証されている.層別逆回帰の拡張として,カーネルトリックを利用することで非線型次元削減部分空間を推定する手法も提案されているが,利用するカーネル関数を適切に選択する必要がある.本研究では,カーネル層別逆回帰に利用するカーネル関数の選択手法を提案する.カーネル関数を定めることで,付随する特徴空間における説明変数の分布が決定されることに着目し,説明変数が特徴空間で正規分布に従うようにカーネルを選択する.正規分布は楕円分布の一種であり,これにより層別逆回帰における仮定が満たされる.特徴空間における分布の正規性を特性関数によって評価し,カーネル関数の凸結合の結合係数を最適化するアルゴリズムを導出する.幾つかの実データを用いた実験から,提案手法の有用性を示す.

    CiNii

  • カーネル層別逆回帰のためのモデル選択手法

    日野英逸, 越島健介, 村田昇

    研究報告バイオ情報学(BIO)   2012 ( 6 ) 1 - 6  2012.11

     View Summary

    データの次元を適切に削減することは,計算コスト及び記憶領域の削減,データの本質的構造の把握に繋がる.特に回帰問題においては,データが拘束されている部分空間を推定する sufficient dimension reduction の問題が盛んに研究されている.その代表例である層別逆回帰は,説明変数が楕円分布に従う場合は,応答変数の値によって層別した説明変数の中心ベクトルを用いて線型次元削減行列を推定することの正当性が理論的に保証されている.層別逆回帰の拡張として,カーネルトリックを利用することで非線型次元削減部分空間を推定する手法も提案されているが,利用するカーネル関数を適切に選択する必要がある.本研究では,カーネル層別逆回帰に利用するカーネル関数の選択手法を提案する.カーネル関数を定めることで,付随する特徴空間における説明変数の分布が決定されることに着目し,説明変数が特徴空間で正規分布に従うようにカーネルを選択する.正規分布は楕円分布の一種であり,これにより層別逆回帰における仮定が満たされる.特徴空間における分布の正規性を特性関数によって評価し,カーネル関数の凸結合の結合係数を最適化するアルゴリズムを導出する.幾つかの実データを用いた実験から,提案手法の有用性を示す.

    CiNii

  • ランダムウォークに基づいたグラフ構造モデリング

    野田淳史, 石田諒, 日野英逸, 龍野正実, 赤穂昭太郎, 村田昇

    研究報告バイオ情報学(BIO)   2012 ( 21 ) 1 - 6  2012.11

     View Summary

    グラフ構造とは,データの変数間の関係を表す構造であり,鉄道路線,インターネット,神経回路網などが代表例として挙げられる.これらのグラフの変数間の依存関係が得られた際に,変数間の繋がりの強度を推定する問題は実用上重要であるが,ある変数間の依存関係には複数の変数から受けた影響が含まれているため,直接的な繋がりは分かりにくい.そこで本稿では.繋がりの有無のみを表現した行列でグラフを簡潔に表現し,そのグラフ上で情報が遷移する様子をモデル化することで,変数間の依存関係を近似する.モデルには,遷移の回数を表すパラメータとグラフ構造を表すパラメータが含まれており,それらを推定するアルゴリズムを提案する.最後に,提案手法の優位性を実験的に示す.

    CiNii

  • Ancestral Atom Learning for Dictionary Generation in Sparse Coding

    ARITAKE Toshimitsu, HINO Hideitsu, MURATA Noboru

      112 ( 279 ) 343 - 350  2012.10

     View Summary

    Sparse Coding is a methodology to represent signals with combinations of only a small number of basis vectors. In sparse coding, designing dictionary is a fundamental problem. An approach for desigining dictionary which adapts observed signals is learning from observed signals. In this paper, like wavelet analysis, a dictionary for sparse signal representation is assumed to be generated from single vector called ancestral atom, and a method for learning the ancestral atom is proposed. Experimental results of ancestral atom learning by proposed method with both artificial and real-world seismic signal are shown to exhibit characteristics and advantages of the proposed algorithm.

    CiNii

  • Predictability of Approximate Accuracy for Asymptotic Expansion

    NOMURA Ryosuke, HINO Hideitsu, MURATA Noboru, YOSHIDA Nakahiro

      112 ( 279 ) 319 - 325  2012.10

     View Summary

    The pricing of derivatives is an important problem in finance. Compared to Monte Carlo method, which is high accuracy but computationally expensive, asymptotic expansion which approximates probability distributions deterministically is a hopeful method. In this study, it is aimed to evaluate the availability of asymptotic expansion for various stochastic differential equation models. By estimating error rate between asymptotic expansion and Monte Carlo method, it is experimentally showed that it is possible to determine the criterion to use asymptotic expansion for the pricing of derivatives.

    CiNii

  • スパース表現の数理とその応用 (パターン認識・メディア理解)

    日野 英逸, 村田 昇

    電子情報通信学会技術研究報告 : 信学技報   112 ( 197 ) 133 - 142  2012.09

     View Summary

    スパースコーディングは生物の一次視覚野の情報処理を数学的にモデル化したものであり,与えられた画像を少数の基底の線型結合で表現する手法である.観測信号のスパース表現は,工学的にも効率的な情報の保持・伝達,あるいはノイズに対して頑健な情報表現を実現する手法として注目を集めている.本稿では,スパースコーディングを始めとする種々の行列分解手法の数理的側面を,その確率モデルを介して統一的に論じる.また,スパースコーディングの代表的なアルゴリズムと幾つかの応用を紹介する.

    CiNii

  • スパース表現の数理とその応用 (情報論的学習理論と機械学習)

    日野 英逸, 村田 昇

    電子情報通信学会技術研究報告 : 信学技報   112 ( 198 ) 133 - 142  2012.09

     View Summary

    スパースコーディングは生物の一次視覚野の情報処理を数学的にモデル化したものであり,与えられた画像を少数の基底の線型結合で表現する手法である.観測信号のスパース表現は,工学的にも効率的な情報の保持・伝達,あるいはノイズに対して頑健な情報表現を実現する手法として注目を集めている.本稿では,スパースコーディングを始めとする種々の行列分解手法の数理的側面を,その確率モデルを介して統一的に論じる.また,スパースコーディングの代表的なアルゴリズムと幾つかの応用を紹介する.

    CiNii

  • スパース表現の数理とその応用

    日野英逸, 村田昇

    研究報告コンピュータビジョンとイメージメディア(CVIM)   2012 ( 20 ) 1 - 10  2012.08

     View Summary

    スパースコーディングは生物の一次視覚野の情報処理を数学的にモデル化したものであり,与えられた画像を少数の基底の線型結合で表現する手法である.観測信号のスパース表現は,工学的にも効率的な情報の保持・伝達,あるいはノイズに対して頑健な情報表現を実現する手法として注目を集めている.本稿では,スパースコーディングを始めとする種々の行列分解手法の数理的側面を,その確率モデルを介して統一的に論じる.また,スパースコーディングの代表的なアルゴリズムと幾つかの応用を紹介する.

    CiNii

  • タンディッシュ内溶鋼温度制御用グレイボックスモデルの開発

    阪下翔太, 加納学, 北田宏, 村田昇

    計測自動制御学会制御部門大会(CD-ROM)   12th   ROMBUNNO.3GATSU14NICHI,DAI2SHITSU,16:00-17:40,SAKASHITA  2012.03

    J-GLOBAL

  • Estimation of Recombination Rates by Using Dirichlet Process and Variational Bayes

    YOMOGIDA Yuna, MURATA Noboru, INOUE Masato

      111 ( 275 ) 181 - 185  2011.11

     View Summary

    After completing sequence of the human genome, many studies have been done about connection between genotype and phenotype. Particularly, it is essential for realizing personalized medicine to discover patterns of genetic variation and their connection to drug sensitivity and disease, and haplotype block is efficient for this. Therefore it is important to find recombination hot spots to estimate haplotype block. We estimate recombination rates between each SNP to find recombination hot spots. EM algorithm has been used for this problem, but there are some defects; EM algorithm estimater may converge to local minimum and it is difficult to determine optimal number of components. In this report, we propose estimation using Dirichlet Process (DP) and Variational Bayes (VB) and tried determining number of components automatically.

    CiNii

  • Multiple Kernel Learning Based on Gaussanity

    HINO Hideitsu, MURATA Noboru

      111 ( 275 ) 99 - 104  2011.11

     View Summary

    Multiple Kernel Learning (MKL) is one of a kernel optimization approaches for kernel methods, which is extensively studied recently. Kernel methods are applied to various classifiers including Fisher's linear discriminant analysis (FDA), which is known to give an optimal classification surface if the distributions of two class data in feature space are Gaussians sharing the same covariance matrix. Based on this fact, an MKL method based on the measure of Gaussianity is proposed. Empirical characteristic function is adopted as a measure of the Gaussianity in the feature space associated with kernel functions, and empirical kernel mapping is adopted as a technical mean to avoid the problem of dealing with infinite dimensionality associated with kernel functions. By experimental results using a number of benchmark data sets, the usefulness of the proposed kernel learning method is evaluated.

    CiNii

  • Confidence estimation of solar irradiance forecast with Just-In-Time Modeling

    TERAZONO Takahiro, WAKAO Shinji, SHEN Haoyang, HINO Hideitsu, MURATA Noboru

      2011 ( 15 ) 37 - 42  2011.11

    CiNii

  • 較正画像における直線度の最大化に基づく放射対称歪曲の較正

    藤木淳, 赤穂昭太郎, 日野英逸, 村田昇

    画像の認識・理解シンポジウム(MIRU2011)論文集   2011   1264 - 1271  2011.07

    CiNii

  • 最適抽出可能性に基づく1次元低い超平面や超曲面のあてはめ~ランダムサンプリングは大域的最適解の夢をみるか?~

    藤木淳, 赤穂昭太郎, 日野英逸, 村田昇

    画像の認識・理解シンポジウム(MIRU2011)論文集   2011   80 - 87  2011.07

    CiNii

  • hypersurface fitting based on random sampling

    FUJIKI Jun, AKAHO Shotaro, HINO Hideitsu, MURATA Noboru

    IEICE technical report   111 ( 48 ) 69 - 74  2011.05

     View Summary

    In this paper, N-1-dimensional hypersurface fitting for N-dimensional data is investigated. Generally, this fitting problem is resolved into M-1-dimensional hyperplane fitting in M-dimensional feature space by considering appropriate feature mapping. Because the distance in original space and is approximated by the weighted distance in feature space, the hypersurface fitting problem based on the Euclidean distance in the original space is equivalent to the hyperplane fitting problem based on the weighted Euclidean distance in the feature space. On the other hand, the hyperplane fitting to reduce one dimension of data by minimizing the weighted sum of k-th power deviation of Euclidean distance has the optimal sampling property when 0&lt;k≦1. Then there exists the global optimum of the M-1-dimensional hyperplane fitting in M-dimensional space which passes M data points. In this paper, we propose the approximation of the fitting problem based on the random sampling by using the optimal sampling property, which is called least k-th power deviations. and we also propose the extension of LMedS, which is called least α-percentile of squares. Then we discussed the relation among the proposed methods and RANSAC, which is also based on the random sampling.

    CiNii

  • 取鍋およびタンディッシュ内溶鋼温度推定モデルの構築

    大倉才昇, 加納学, 北田宏, 村田昇

    材料とプロセス(CD-ROM)   24 ( 1 ) ROMBUNNO.TO8  2011.03

    J-GLOBAL

  • ブートストラップフィルタによる溶鋼温度分布の予測と制御

    園田翔, 村田昇, 日野英逸, 進藤史裕, 北田宏, 加納学

    材料とプロセス(CD-ROM)   24 ( 1 ) ROMBUNNO.TO9  2011.03

    J-GLOBAL

  • High performance prediction of molten steel temperature through gray-box model

    Toshinori Okura, Manabu Kano, Shinji Hasebe, Hiroshi Kitada, Noboru Murata

    Computing and Systems Technology Division - Core Programming Topic at the 2011 AIChE Annual Meeting   2   1153 - 1154  2011  [Refereed]

  • Quantile Based Information Estimation for Weighted Observation and its Applications

    HINO Hideitsu, MIURA Kazuki, MURATA Noboru

    IEICE technical report   110 ( 265 ) 159 - 168  2010.10

     View Summary

    Shannon's measure of information is a measure of value for the observed event. Information is a very important and fundamental quantity in the field of statistics, information theory and machine learning, and its estimation from a set of given data is of great importance. In this manuscript, based on the notion of quantiles, a novel estimator for information using a set of given weighted data is proposed. The proposed method is extended to cross entropy and entropy estimation of the weighted data. The proposed estimators are applied the problem of learning a classifier from one-class data, and the problem of change point detection.

    CiNii

  • Calibration of radially symmetric distortion on stereographic projection plane

    FUJIKI Jun, HINO Hideitsu, USAMI Yumi, AKAHO Shotaro, MURATA Noboru

    IEICE technical report   109 ( 470 ) 377 - 382  2010.03

     View Summary

    For calibration of general radially symmetric distortion of omnidirectional cameras such as fish-eye lenses, calibration parameters are usually estimated so that curved lines, which are supposed to be straight in the real-world, are mapped to straight lines in the calibrated image, which is called plumbline principle. Under the principle, the calibrated image is the image taken by an ideal pin-hole camera. The authors proposed a calibration method based on the plumbline principle, which consists of linear algorithm, however, the method is only for the camera of which field of view is less than 180 degree. Then in this paper, the authors propose a calibration method based on the stereographical plumbline principle, which is an extension of plumbline principle to apply for the camera of which field of view is over 180 degree. The proposed algorithm only consists of linear algorithm.

    CiNii

  • Towards robust Jacobian kernel PCA

    FUJIKI JUN, AKAHO SHOTARO, HINO HIDEITSU, MURATA NOBORU

    IPSJ SIG Notes. CVIM   2009 ( 29 ) 191 - 196  2009.03

     View Summary

    Conventional kernel principal component analysis finds the principal subspace of the data by means of least squares estimation with respect to the metric defined in the feature space, on the other hand, Jacobian kernel principle component analysis (JKPCA) finds the principal subspace based on the metric in the input space by weighted least squares associated with the Jacobian of the feature map. For some applications such as image processing, it is more natural to utilize the metric in the input space, however, the weighted least squares estimation is sometimes too sensitive to outliers. To overcome this drawback of JKPCA and make it robust to outliers, an outlier detection method based on random sampling consensus (RANSAC) and χ^2 test is discussed, and its validity is confirmed by numerical experiments.

    CiNii

  • Towards robust Jacobian kernel PCA

    FUJIKI JUN, AKAHO SHOTARO, HINO HIDEITSU, MURATA NOBORU

    IEICE technical report   108 ( 484 ) 191 - 196  2009.03

     View Summary

    Conventional kernel principal component analysis finds the principal subspace of the data by means of least squares estimation with respect to the metric defined in the feature space, on the other hand, Jacobian kernel principle component analysis (JKPCA) finds the principal subspace based on the metric in the input space by weighted least squares associated with the Jacobian of the feature map. For some applications such as image processing, it is more natural to utilize the metric in the input space, however, the weighted least squares estimation is sometimes too sensitive to outliers. To overcome this drawback of JKPCA and make it robust to outliers, an outlier detection method based on random sampling consensus (RANSAC) and χ^2 test is discussed, and its validity is confirmed by numerical experiments.

    CiNii

  • Calibration of radially symmetric distortion by fitting principle curve

    FUJIKI Jun, AKAHO Shotaro, HINO Hideitsu, MURATA Noboru

    IEICE technical report   108 ( 363 ) 13 - 18  2008.12

     View Summary

    To calibrate radially symmetric distortion of pinhole camera and/or omnidirectional cameras such as fish-eye lenses, calibration parameters are usually estimated so that lines, which are suppored to be straight in the real-world, are mapped to straight lines in the calibrated image. In this paper, this problem is treated as a fitting problem of principle curves in uncalibrated images in the framework of kernel principle component analysis, and an estimation procedure of calibration parameters is proposed based on Jacobian kernel principle component analysis.

    CiNii

  • D-1-9 GENERALIZATION OF INDEPENDENCY BASED ON THE BREGMAN DIVERGENCE

    Fujimoto Yu, Murata Noboru

    Proceedings of the IEICE General Conference   2008 ( 1 ) 9 - 9  2008.03

    CiNii

  • F-050 Evaluation of Parameter Estimation Methods based on Divergence corrected with GES

    Fujimoto Yu, Murata Noboru

      6 ( 2 ) 461 - 464  2007.08

    CiNii

  • LF_001 Estimation of e-mixture models

    Fujimoto Yu, Murata Noboru

      5   93 - 96  2006.08

    CiNii

  • Buliding a New Mutation Model based on Bayesian Networks and estimation by a modified Belief Propagation algorithm

    DOI Yoshinori, MURATA Noboru

    IPSJ SIG technical reports   2006 ( 64 ) 69 - 74  2006.06

     View Summary

    It is thought that ancient DNA repair proteins were more active than present ones, and obtaining the ancient DNA repair proteins would be helpful for hereditary diseases such as cancer. We propose a mutation model using Bayesian Networks and a modified the belief propagation algorithm, which is applied to estimate an ancient protein from multiple proteins that are present. Also, we applied the cross-validation to derive an optimum parameter. We virtually generated multiple proteins from an ancient protein following a specific mutation probability, and evaluated our model by estimating the ancient protein by the proposed model.

    CiNii

  • Extraction of independent features in the hidden layer of Neural Networks

    MATSUKURA Kentaro, MURATA Noboru

    IEICE technical report   106 ( 102 ) 63 - 67  2006.06

     View Summary

    Neural Networks (NN) are simplified models of neural processing in a human brain. Its applications varies in wide areas of machine learning. Each output of the, unit consisting the hidden layer in a NN, is often similar when learning the network. On the other hand, nonlinear sparse coding can be performed by expressing the relation between the input and the output in the hidden layer. We can expect to extract information of the I/O from each unit of the hidden layer by this performance. We propose a method to extract independent features from the hidden layer in NNs and investigated it by a simple experiment.

    CiNii

  • On-line boosting method based on EM algorithm

    NISHINO Ryotaro, MURATA Noboru

    IEICE technical report   106 ( 102 ) 25 - 30  2006.06

     View Summary

    When the Adaboost algorithm is used in case that a datum is generated one after another, in general, all the learners have to be trainned again or a new learner has to be added for the new sample. In this paper, an on-line boosting algorithm is considered to improve the accuracy without increasing learners. In the proposed algorithm, dependence of data for each learner is estimated by the EM algorithm. Moreover, advantages of the proposed method is confirmed by some numerical experiments.

    CiNii

  • Learning Go using neural networks and generalization to unknown situation

    OGATA Yasuhiro, MURATA Noboru

    IEICE technical report   106 ( 102 ) 7 - 12  2006.06

     View Summary

    The complexity of a Go game makes it hard to determine a preferable step. We used a three-layered perception to generate an agent predicting the best step in each stage of the game. Simply applying Neural Networks by inputting all phases would end up in too many parameters, requiring plenty of training data and learning time. We propose a method limiting the situation to some special phases, reducing the number of parameters and computation time.

    CiNii

  • Buliding a New Mutation Model based on Bayesian Networks and estimation by a modified Belief Propagation algorithm

    DOI Yoshinori, MURATA Noboru

    IEICE technical report   106 ( 101 ) 65 - 70  2006.06

     View Summary

    It is thought that ancient DNA repair proteins were more active than present ones, and obtaining the ancient DNA repair proteins would be helpful for hereditary diseases such as cancer. We propose a mutation model using Bayesian Networks and a modified the belief propagation algorithm, which is applied to estimate an ancient protein from multiple proteins that are present. Also, we applied the cross-validation to derive an optimum parameter. We virtually generated multiple proteins from an ancient protein following a specific mutation probability, and evaluated our model by estimating the ancient protein by the proposed model.

    CiNii

  • Predicting sleep stages based on time-dependent hidden Markov model

    YOSHIMURA Masako, MURATA Noboru

    IEICE technical report   106 ( 101 ) 7 - 12  2006.06

     View Summary

    Recently, many people suffer from an unpleasant awakening and a lack of sleep. Everyone might also have the experience that there is a day with a pleasant awakening or that with an unpleasant, though sleeping time at the same level. A pleasant awakening of day can be fulfilled when waking on the sleep stage is low, which is called REM (Rapid Eye Movement) sleep. Sleep stages, which show the depth of sleep, have the feature continuity and periodic, and a biometric data at sleep such as heart rate and body movement are the candidate for indexing sleep stages. So in this paper, estimating the depth of sleep, using time-dependent hidden Markov model, which is a state transition probability depending on time, is proposed.

    CiNii

  • Basics of Sparse Coding and Its Application to Image Processing

    MURATA Noboru

    IEICE technical report   105 ( 673 ) 155 - 162  2006.03

     View Summary

    Sparse coding is an algorithm for modeling primitive visual processing in biological systems, and it provides a method of constructing proper basis functions based on statistical and information theoretic criteria. Images are effectively reconstructed as a linear combination of a small number of bases which are chosen from the basis functions trained by the algorithm. In this manuscript, we introduce the idea of sparse coding by considering how images can be compressed when images are decomposed by a linear combination of bases and represented by their coefficients.

    CiNii

  • Basics of Sparse Coding and Its Application to Image Processing

    MURATA Noboru

    IPSJ SIG Notes. CVIM   2006 ( 25 ) 155 - 162  2006.03

     View Summary

    Sparse coding is an algorithm for modeling primitive visual processing in biological systems, and it provides a method of constructing proper basis functions based on statistical and information theoretic criteria. Images are effectively reconstructed as a linear combination of a small number of bases which are chosen from the basis functions trained by the algorithm. In this manuscript, we introduce the idea of sparse coding by considering how images can be compressed when images are decomposed by a linear combination of bases and represented by their coefficients.

    CiNii

  • 7. Boosting a Learning Algorithm : Are "Three Heads" Better than One?(<Special Section>Secret Arts for Handling Probabilities : A Classic Yet New Research Paradigm on Probabilistic/Statistical Models)

    MURATA Noboru, KANAMORI Takafumi, TAKENOUCHI Takashi

    The Journal of the Institute of Electronics, Information and Communication Engineers   88 ( 9 ) 724 - 729  2005.09

    CiNii

  • β尤度に基づく混合した確率表の推定 (テーマ:特集「ベイジアンネットワーク」および一般)

    藤本 悠, 村田 昇

    人工知能基本問題研究会   60   13 - 18  2005.08

    CiNii

  • LH-004 Mixture of Independent Tables Estimated with Small Samples

    Fujimoto Yu, Murata Noboru

      4   129 - 130  2005.08

    CiNii

  • ブースティングとそのロバスト化 (記録値の統計的推測と関連する統計学)

    竹之内 高志, 金森 敬文, 村田 昇

    数理解析研究所講究録   1439   111 - 127  2005.07

    CiNii

  • Smoothing of camera motions via small circle fitting for the sequence of spherical points

    MIMURA Junichi, MURATA Noboru, FUJIKI Jun

    Technical report of IEICE. PRMU   105 ( 119 ) 47 - 52  2005.06

     View Summary

    Shape reconstruction methods such as the factorization method allow us to reconstruct camera motions and 3D-shape of the object from a sequence of 2D-images. The reconstructed motions, however, tend to be unnatural affected by fluctuations of the camera position and noises, because such reconstruction methods basically do not assume the continuity of the camera motions. In this paper, a new method for smoothing an estimated camera motion is proposed, in which each camera position is regarded as a point on a unit sphere, and a small circle is fitted for representing a smoothed trajectory of the camera motion on the sphere. The validity of the proposed method is experimentally confirmed by using real images.

    CiNii

  • Welcome to a World of Learning Theory

    MURATA Noboru

      44 ( 5 ) 291 - 292  2005.05

    CiNii

  • D-14-7 SELECTION OF ICA ALGORITHM USING THE SEPARATED SIGNALS

    Kono Shuntaro, Murata Noboru

    Proceedings of the IEICE General Conference   2005 ( 1 ) 124 - 124  2005.03

    CiNii

  • D-12-104 FEATURE EXTRACTION OF COMPOSERS USING SOURCE CODING

    Yamashita Tetsuo, Fujimoto Yu, Murata Noboru

    Proceedings of the IEICE General Conference   2005 ( 2 ) 254 - 254  2005.03

    CiNii

  • Geometrical Structure of Learning Algorithms

    MURATA Noboru

    IEICE technical report. Neurocomputing   104 ( 349 ) 51 - 56  2004.10

     View Summary

    In the machine learning, it is quite important how to model target problems, and also how to construct efficient algorithms in order to obtain optimal solutions. The concept of information geometry, in which geometrical structures of stochastic models are considered based on differential geometry, helps us to understand the mechanism of learning algorithms. In this article, geometrical understandings of learning algorithms are discussed, such as EM algorithm, boosting, bagging.

    CiNii

  • An Improvisational Music Generation System using Bayesian Network

    FUJIMOTO Yu, MURATA Noboru

    IEICE technical report. Neurocomputing   104 ( 348 ) 43 - 48  2004.10

     View Summary

    An uncertainty of human behavior is expected to be described by Bayesian networks. An improvisational performance of jazz is a remarkable example of such an uncertainty. In jazz, the uncertainty depending on musical sense and experience of a performer causes an audience's interest. Our purpose is expressing the uncertainty of jazz, and constructing a system which generates an improvisational music. This paper shows a problem and a solution of probabilistic modeling by constructing an improvisational music generation system.

    CiNii

  • G-022 The Improvisational Music Generation System using Bayesian Network : Model Estimation with Small Samples

    Fujimoto Yu, Murata Noboru

      3 ( 2 ) 395 - 396  2004.08

    CiNii

  • Robust Boosting and Loss Functions

    KANAMORI Takafumi, TAKENOUCHI Takashi, EGUCHI Shinto, MURATA Noboru

    IEICE technical report. Neurocomputing   104 ( 225 ) 1 - 6  2004.07

     View Summary

    Boosting algorithm is understood as the gradient descent algorithm of a loss function. It is often pointed out that the typical boosting algorithm, Adaboost, is seriously affected by the outliers. In this paper, loss functions for robust boosting are studied. Based on a concept of the robust statistics, we propose a positive-part-truncation of the loss function which makes the boosting algorithm robust against extreme outliers. Next the truncation of the loss function is applied to the contamination models which describe the occurrence of mislabels near the decision boundary. Numerical experiments illustrate that the proposed boosting algorithm derived from the contamination model is useful for highly noisy data in comparison with other competitors.

    CiNii

  • D-2-1 LEARNING PROPERTIES OF SUPPORT VECTOR MACHINES WITH L_p NORM

    Ikeda Kazushi, Murata Noboru, Onishi Takaharu, Aoishi Tsutomu

    Proceedings of the IEICE General Conference   2004 ( 1 ) 11 - 11  2004.03

    CiNii

  • An Introduction to Information Geometry

    MURATA Noboru

    IEICE technical report. Neurocomputing   103 ( 227 ) 1 - 6  2003.07

     View Summary

    The information geometry is a method for investigating mechanisms of statistical inference, statistical test, learning and so on, based on the geometrical properties of the space of probability distributions. In this manuscript, basic ideas of the information geometry are discussed under Pythagorean proposition derived from the KL-divergence, without using differential geometrical concepts.

    CiNii

  • 推定量を組み合わせる-バギングとブースティング, パターン認識と学習の統計学-新しい概念と手法

    村田昇

    統計科学のフロンティア    2003

    CiNii

  • Geometrical Understanding of Boosting Algorithms

    MURATA Noboru

    Technical report of IEICE. PRMU   102 ( 379 ) 37 - 42  2002.10

     View Summary

    Ensemble learning algorithms are widely noticed as attractive methods for avoiding the explosion of computational costs and the decline of generalization abilities in huge learning machines. In this article the AdaBoost algorithm, one of the boosting algorithms which construct a strong machine from many weak machines trained by reweighted samples, is focused and its mechanism is considered based on the geometrical structure of the space of learning models.

    CiNii

  • Neural Network

    MURATA Noboru, Noboru Murata

    Journal of Japanese Society for Artificial Intelligence   17 ( 2 ) 243 - 245  2002.03

    CiNii

  • Cultivation of the Religious Sentiment-Based on the Prayer for Venerato Vitae

    Murata Noboru

    Journal of the Nippon Buddhist Education Research Association   7   15 - 51  1999.03

    CiNii

  • Independent Componet Analysis

    Murata Noboru

    Proceedings of the Society Conference of IEICE     266 - 267  1999

    CiNii

  • MEG data analysus using ICA

    IKEDA Shiro, MURATA Noboru

    Technical report of IEICE. HIP   98 ( 131 ) 29 - 36  1998.06

     View Summary

    MEG (Magnetoencephalograph) is one of the promissing ways to analyze the activity of the brain. One of the problems in analizing MEG is the noise. Since the signals from the brain is extremely small compared to the earth magnetism (10^-9), it is important to remove the noise. Usually MEG is recorded in a shielded room, and also some external sonsors are used to estimate the background noise (reference channels) and a digital band passed filter and averaging are applied to the data. In this technical report, we applied ICA (Independent Component Analysis) to the data. The algorithm for ICA is the one propose by Molgedey and Schuster. As the result, we can remove the earth magnetism and the noise from electir power supply. Using the extracted source signal, we estimated the location of the dipole with very high accuracy. We used the data from Shimadzu.

    CiNii

  • 7p-YD-6 A statistical theory of learning

    Murata Noboru

    Meeting abstracts of the Physical Society of Japan   52 ( 2 ) 788 - 788  1997.09

    CiNii

  • EDUARD SPRANGER : Sein Leben

    Murata Noboru

    Memoirs of the Faculty of Education, Shiga University. Cultural science, social science and educational science   41   21 - 51  1991

    CiNii

  • Uber die Didaktik Fr. Paulsens

    Murata Noboru

    Memoirs of the Faculty of Education, Shiga University. Cultural science, social science and educational science   40   105 - 125  1990

    CiNii

▼display all

 

Syllabus

▼display all

 

Sub-affiliation

  • Faculty of Science and Engineering   Graduate School of Advanced Science and Engineering

Research Institute

  • 2023
    -
    2024

    Center for Data Science   Concurrent Researcher

  • 2022
    -
    2024

    Waseda Research Institute for Science and Engineering   Concurrent Researcher

  • 2022
    -
    2024

    Research Organization for Open Innovation Strategy   Concurrent Researcher