Brain waves, or electroencephalograms (EEGs), are applicable to user verification. We devise a two-factor system so that impersonators who hold identification numbers in fraudulence are detectable. In the first step, a subject either authentic or false tries to input a digit of a ten-key in the personal identification number by a P300 speller. The P300 speller is a brain-computer interface that detects positive voltage jump when a subject identifies specific digits on a display visually. By considering the performance of the P300 speller, we allow an error of one digit out of the four digits. On the other hand, we keep suspicion even for the case of perfect four digits because of the possibility of impersonation by a stolen case. Following the P300 spelling, we apply a verification of subjects by brain waves. Averaging of detected P300 waveforms after band-pass filtering takes the role of feature extraction. Then, a support vector machine applied to the averaged waveforms decides whether the subject is authentic or false. Thus, the total system does not entail the complexity of multimodality. For this system, we measured average error rates for 20 subjects. Experiments showed the false rejection rate of 3.9% at the false acceptance rate of 0% for the 4-digit number case. These pair values are successfully low even by using brain waves that usually contain many artifacts. Additionally, experiments on a diabetes patient before and after an insulin injection are also conducted. The result shows that the appropriate injection control maintains no difference from ordinary subjects. In concluding remarks, we consider methods to increase subjects and digits for applications in a larger society.

In this paper, we propose a method to utilize low-frequency brain signals for continuous authentication of users. During such monitoring, the users to be authenticated can work without interruption. This style of authentication is expected to complement traditional methods based on passwords, which can be easily forgotten or stolen. For brain signal-based continuous authentication, we measured oxyhemoglobin changes in the brain through near-infrared spectroscopy (NIRS). There are two cases of NIRS measurement: a rest case, and a keyboard typing task case. In both cases, the brain signals were found to show specific patterns in the range around 1.5 Hz. Identified personality was used to prevent impersonators. For the detection of impostors, we first carried out a principal component analysis (PCA) of the logarithmic power spectra of the NIRS signals. Small eigenvalues were discarded so that excessive learning of system parameters can be avoided. The processed spectral data were utilized to obtain an average weight vector for support vector machines (SVMs). The average weight vector was applied to the spectral data to emphasize characteristic patterns in low-frequency regions. This process generated separable clusters for each subject's NIRS signals. In the test phase, unknown subject's NIRS signals were measured and preprocessed. Following this, we carried out continuous authentication by computing the Mahalanobis distance to the registered cluster set. For both the rest and task cases of the NIRS, the authentication accuracy of our proposed method was greater than 99% at the equal error rate (EER). Dynamic authentication of this sort using brain signals can offer a viable method for reducing excessive dependence on traditional password-based methods. (C) 2015 Elsevier B.V. All rights reserved.

Video data captured and uploaded under volatile conditions is accumulating into a flood of unstructured content that is hard to manage. In this paper, we present a set of algorithms that generate numeric or soft labels to structure automatically and produce video similarity ran kings. Exemplar frames are extracted from each video during the labeling process. We propose five types of learning algorithms based on the following three classes: affinity propagation, k-means or harmonic competition, and pairwise nearest-neighbor. For redundancy reduction, we also provide an algorithm that prunes excessive exemplars. The five learning algorithms produced creditable order-aware exemplar sets for the target videos. Because the content and lengths of the videos differ in terms of temporal order, we provide new methods to analyze the similarities between exemplar sets. The m-distance similarity measure is the core concept used for the global and local alignments performed on the obtained exemplar sets. Based on this comparison mechanism, we identified high-precision recall curves for all five methods. In terms of learning speed, the k-means and pairwise nearest-neighbor classes are recommendable. To facilitate similar-video retrieval, we developed a graphical user interface that accepts videos downloaded from the Web. By replacing a procedure in the software, the proposed similar-video retrieval system can accommodate more elaborate frame comparison features.

Learning algorithms that harmonize standardized video similarity tools and an integrated system are presented. The learning algorithms extract exemplars reflecting time courses of video frames. There were five types of such clustering methods. Among them, this paper chooses a method called time-partition pairwise nearest-neighbor because of its reduced complexity. On the similarity comparison among videos whose lengths vary, the M-distance that can absorb the difference of the exemplar cardinalities is utilized both for global and local matching. Given the order-aware clustering and the M-distance comparison, system designers can build a basic similar-video retrieval system. This paper promotes further enhancement on the exemplar similarity that matches the video signature tools for the multimedia content description interface by ISO/IEC. This development showed the ability of the similarity ranking together with the detection of plagiarism of video scenes. Precision-recall curves showed a high performance in this experiment.

New learning algorithms and systems for retrieving similar videos are presented. Each query is a video itself. For each video, a set of exemplars is machine-learned by new algorithms. Two methods were tried. The first and main one is the time-bound affinity propagation. The second is the harmonic competition which approximates the first. In the similar-video retrieval, the number of exemplar frames is variable according to the length and contents of videos. Therefore, each exemplar possesses responsible frames. By considering this property, we give a novel similarity measure which contains the Levenshtein distance (L-distance) as its special case. This new measure, the M-distance, is applicable to both of global and local alignments for exemplars. Experimental results in view of precision-recall curves show creditable scores in the region of interest.

A new approach to continuous authentication is presented. The method is based on a combination of statistical decision machines for brain signals. Functional Near Infra Red Spectroscopy (NIRS) is used to measure brain oxyhemoglobin changes for each subject to be authenticated. Such biosignal authentication is expected to be a viable complementary method to traditional static security systems. The designed system is based on a discriminant function which utilizes the average weight vector of one-versus-one support vector machines for NIRS spectra. By computing a histogram of Mahalanobis distances, high separability among subjects was recognized. This experimental result guarantees the utility of brain NIRS signals to the continuous authentication.

A new icon spotting method for designing a user-friendly GUI is described. Here, each icon can represent continuous and discrete vector data which are possibly high-dimensional. An important issue is icon-margin adjustment or uniforming while the relative positioning is maintained. For generating such GUI, multidimensional scaling, kernel principal component analysis (KPCA) and regularization were combined. This method was applied to a set of city locations and a big data set of web-registered job hunter profiles. The former is used to check to see location errors. There were only little mis-allocations. The latter is a set of high dimensional and sparsely discrete-valued big data in the real world. Through these experiments, it was recognized that the presented method, which combines multidimensional scaling, KPCA and the regularization, is applicable to a wide class of jammed big data for generating a user-friendly GUI. © Springer International Publishing Switzerland 2013.

Multimodal signals emanated from human users are applied to operations of bipedal humanoids. Distinctive features of the designed system include recognition and conversion of sensibilities as patterns contained in the biosignals. The total recognition system is a combination of Bayesian networks, hidden Markov models, independent component analysis, and support vector machines. Input biosignals are electro-encephalogram, brain near infra-red spectroscopy, neural spike trains, and body motions (gestures). After the recognition of biosignals, user intentions issued as patterns are transduced to different electronic tasks. In addition to such an ability of sensory conversion, this mechanism has a merit of enhancing the independence of target machines. The combined recognizer allows system designers to use conventional PCs alone. With all of such advantages, a bipedal humanoid, which is selected as a representative of contemporary electronic appliances, is operated by the human biosignals. Operations by inner language patterns are also realized.

Mapping tools applicable to big data of composite elements are designed based on a machine learning approach. The central method adopted is the multi-dimensional scaling (MDS). The data set is mapped onto a continuous surface such as a sphere. For checking to see the effectiveness of this method, preliminary experiments on the local optimality were conducted. Supported by those results, the main target for the application in this paper is the design for a spherical GUI (Graphical User Interface) which presents "must-watch" and "no-need" program clusters in TV big data. This GUI shows a certain genre of programs at around the North Pole. Programs having an opposite genre placed at around the South Pole. Since all-recording systems of TV programs are within the realm of home appliances, this GUI can be expected to be one of necessary tools for a video culture.

Fast estimation algorithms of Hidden Markov Models (HMMs), or alpha-HMMs, are presented. Such novel algorithms inherit speedup properties of the alpha-EM algorithm. Since the alpha-EM algorithm includes the traditional log-EM algorithm as its special case, the alpha-HMM also includes the traditional log-HMM as its special case. This generalization appears as the utilization of the past information which is the main device of the speedup. Since the memorization of the past information requires only little increase of computational load and memory, the iteration speedup directly appears as that of CPU time. Experimental results are given.

Fast estimation algorithms for Hidden Markov models (HMMs) for given data are presented. These algorithms start from the alpha-EM algorithm which includes the traditional log-EM as its proper subset. Since existing or traditional HMMs are the outcome of the log-EM, it had been expected that the alpha-HMM would exist. In this paper, it is shown that this foresight is true by using methods of the iteration index shift and likelihood ratio expansion. In each iteration, new update equations utilize one-step past terms which are computed and stored during the previous maximization step. Therefore, iteration speedup directly appears as that of CPU time. Since the new method is theoretically based on the alpha-EM, all of its properties are inherited. There are eight types of alpha-HMMs derived. They are discrete, continuous, semi-continuous and discrete-continuous alpha-HMMs, and both for single and multiple sequences. Using the properties of the alpha-EM algorithm, the speedup property is theoretically analyzed. Experimental results including real world data are given.

New algorithms for joint learning of independent component analysis and graphical high-order correlation (GC-ICA: Graphically Correlated ICA) are presented. The presented method has a fixed point style or of the FastICA, however, it comprises independent but correlated subparts. Correlations by teacher signals are also allowed. In spite of such inclusion of the dependency, the presented algorithm shows fast convergence. The converged set of bases has reduced indeterminacy on the ordering. This is equivalent to a self-organization of bases. This method can be used to analyze multiple images simultaneously. Examples are given on images from 3D-stereo videos shots. The correlation of bases on left and right eye views is shown for the first time here. Further speedup using the strategy of the RapidICA is possible.

Heterogeneous bio-signals including human motions, brain NIRS and neural spike trains are utilized for operating biped humanoids. The Bayesian network comprising Hidden Markov Models and Support Vector Machines is designed for the signal integration. By this method, the system complexity is reduced so that that total operation is within the scope of PCs. The designed system is capable of transducing original sensory meaning to another. This leads to prosthesis, rehabilitation and gaming. In addition to the supervised mode, the humanoid can act autonomously for its own designed tasks.

A fast learning algorithm for Hidden Markov Models is derived starting from convex divergence optimization. This method utilizes the alpha-logarithm as a surrogate function for the traditional logarithm to process the likelihood ratio. This enables the utilization of a stronger curvature than the logarithm. This paper's method includes the ordinary Baum-Welch re-estimation algorithm as a proper subset. The presented algorithm shows fast learning by utilizing time-shifted information during the progress of iterations. The computational complexity of this algorithm, which directly affects the CPU time, remains almost the same as the logarithmic one since only stored results are utilized for the speedup. Software implementation and speed are examined in the test data. The results showed that the presented method is creditable.

A faster algorithm of ICA is devised. This method utilizes multi-step past information with the existing fixed-point method. The use of past information comes from the idea of surrogate optimization. The speed and implemented software are checked for both simulated and read data. The presented ICA, named RapidICA, is applied to image-to-image retrieval without indices. Images judged to be similar reflected user's opinion well.

Human-humanoid symbiosis by using brain signals is presented. Humans issue two types of brain signals. One is non-invasive NIRS giving oxygenated hemoglobin concentration change and tissue oxygeneration index. The other is a set of neural spike trains (measured on macaques for safety compliance). In addition to such brain signals, human motions are combined so that rich in carbo information is provided for the operation of a humanoid which is a representative of in silico information processing appliances. The total system contains a recognition engine of an HMM/SVM-embedded Bayesian network so that the in carbo signals are integrated, recognized and converted to operate the humanoid. This well-folded system has made it possible to operate the humanoid by thinking alone using a conventional PC. The designed system's ability of transducing sensory information is expected to lead to amusement systems, rehabilitation and prostheses.

Sensibility-aware image retrieval methods axe, presented and their performances are compared. Three systems are discussed in this paper: PCA/ICA-based method called RIM (Retrieval-aware IMage format), JPEG, and JPEC2000. In each case, a query is an image per se. Similar images are retrieved to this query. The RIM method is judged to be the best settlement in view of the retrieval performance and the response speed according a carefully designed set of opinion tests. An integrated retrieval system for image collections from the network and databases which contain RIM, JPEG and JPEG2000 is realized and evaluated lastly. Source codes of the RIM method is opened.

Sensibility-aware image retrieval methods are presented and their performances are compared. Three systems are discussed in this paper: PCA/ICA-based method called RIM (Retrieval-aware IMage format), JPEG, and JPEG2000. In each case, a query is an image per se. Similar images are retrieved to this query. The RIM method is judged to be the best settlement in view of the retrieval performance and the response speed according a carefully designed set of opinion tests. An integrated retrieval system for image collections from the network and databases which contain RIM, JPEG and JPEG2000 is realized and evaluated lastly. Source codes of the RIM method is opened. © 2009 Springer Berlin Heidelberg.

Protein folding classification is a meaningful step to improve analysis of the whole structures. We have designed committee Support Vector Machines (committee SVMs) and their array (committee SVM array) for the prediction of the folding classes. Learning and test data are amino acid sequences drawn from SCOP (Structure Classification Of Protein database). The classification category is compatible with the SCOP. SVMs and committee SVMs are designed in an one-versus-others style both for chemical data and sliding window patterns (spectrum kernels). This generates the committee SVM array. Classification performances are measured in view of the Receiver Operating Characteristic curves (ROC). Superiority of the committee SVM array to existing prediction methods is obtained through extensive experiments to compute the ROCs.

Joint recognition of bio-signals emanated from human(s) is discussed. The bio-signals in this paper include camera-captured gestures and brain signals of hemoglobin change Delta O(2)H(b). The recognition of the integrated data is applied to the operation of a biped humanoid. Hidden Markov Models (HMMs) and Support Vector Machines (SVMs) undertake the first stage recognition of individual signal. These subsystems are regarded as soft command issuers. Then, such low-level commands are integrated by a Bayesian Network (BN). Therefore, the total system is a novel HMM/SVM-embedded BN. Using this new recognition system, human operators can control the biped humanoid through the network by realizing more motion classes than methods of HMM-alone, SVM-alone and BN-alone.

Methods to integrate multimodal beliefs by Bayesian Networks (BNs) comprising Hidden Markov Models (HMMs) and Support Vector Machines (SVMs) are presented. The integrated system is applied to the operation of ambulating PCs (biped humanoids) across the network. New features in this paper are twofold. First, the HMM/SVM-embedded BN for the multimodal belief integration is newly presented. Its subsystem also has a new structure such as a committee SVM array. Another new fearure is with the applications. Body and brain signals are applied to the ambulating PC operation by using the recognition of multimodal signal patterns. The body signals here are human gestures. Brain signals are either HbO(2) of NIBS or neural spike trains. As for such ambulating PC operation, the total system shows better performance than HAIM and BN systems alone.

Image-to-Image retrieval (I2I) accompanied by data compression are presented. Given a query image, the presented retrieval system computes PCA and/or ICA bases by extracting source information. On the data compression in the sense of rate-distortion, this method outperforms JPEG which uses DCT. Besides, PCA and ICA bases reflect each image's edge and texture information. Therefore, these bases can be used to define similarities combined with color information. Such similarity :measures are tested to be compatible with human sensibility and association, or human "kansei", via a set of opinion tests. Designed systems are proven to be effective and productive to such similar-image retrieval. In addition to static database, images collected through the Internet and WWW which are based on Text-to-Image retrieval (T2I) are tested. Free annotations attached to such images are often irresponsible and erroneous. Therefore, filtering on so collected images is necessary. This paper's I2I system can filter out such wrong images without sorting by human tasks. All source codes and object codes are open and can be downloaded form the authors' web site.

A method to combine a Bayesian Network (BN) and Hidden Markov Models (HMMs) is presented. This compound system is applied to robot operations. The addressed problem and presented methods are novel with the following features: (1) BN and HMMs make a total decision system by accepting evidences from HMMs to the BN. (2) The HMM-embedded BN is applied to the human motion recognition for the biped humanoid operation. (3) Besides the motion recognition, the image recognition is incorporated by adding a BN subsystem. Thus, the total HMM-embedded BN can be regarded as an integrator of heterogeneous commands. (4) The human operator and the biped humanoid can be located on the other side of the network each other. (5) The piped humanoid follows various commands of human motions without falling down by showing better sophistication and operation success than HMM-alone and BN-alone systems. In addition to the above, an information supply to the BN from brain signals is realized through a combination with a Support Vector Machine (SVM). Copyright 2008 ACM.

New methods for joint compression and Image-to-Image retrieval (I2I retrieval) are presented. The novelty exists in the usage of computationally learned image bases besides color distributions. The bases are obtained by the Principal Component Analysis and/or the Independent Component Analysis. On the image compression, PCA and ICA outperform the JPEG's DCT. This superiority holds even if the bases and superposition coefficients are quantized and encoded. On the I2I retrieval, the precision-recall curve is used to measure the performance. It is found that adding the basis information always increases the baseline ability of the color information. Besides the retrieval evaluation, a unified image format called RIM (Retrieval-aware IMage format) for effective packing of codewords including bases is specified. Furthermore, an image search viewer called Wisvi (Waseda Image Search Vlewer) is developed and exploited. A ss-version of all source codes can be down-loaded from a web site given in the text.

A retrieval-aware image format (rim format) is developed for the usage in the similar-image retrieval. The format is based on PCA and ICA which can compress source images with an equivalent or often better rate-distortion than JPEG. Besides the data compression, the learned PCA/ICA bases are utilized in the similar-image retrieval since they reflect each source image's local patterns. Following the format presentation, an image search viewer for network environments (Wisvi; Waseda image search viewer) is presented. Therein, each query is an image per se. The Wisvi system based on the "rim" method successfully finds similar-images from non-uniform network environments. Experiments support that the PCA/ICA methods are viable to the joint compression and retrieval of digital images. Interested test users can download a beta-version of the tool for the joint image compression and retrieval from a web site specified in this paper.

Similar-image retrieval systems are newly presented and examined. The systems use ICA bases (independent component analysis bases) or PCA bases (principal component analysis bases). These bases can contain source image's information, however, the indeterminacy. of ordering and amplitude on the bases exists due to the PCA and ICA problem formulation per se. But, this paper successfully avoids this difficulty by using weighted inner products of similar bases. A set of opinion test is carried out on 18 systems according to the combination of {similarity measures (ICA, PCA, color histogram), color spaces (RGB, YIQ, HSV), filtering (with, without)}. The color histogram method is a traditional method. The opinion test shows that the presented method of {ICA, HSV, without filtering} is the best. Runners-up are {ICA, HSV or RGB or YIQ, with filtering}. The traditional method is judged to be much inferior. Thus, this paper's method is found quite effective to the similar-image retrieval from large databases. (c) 2005 Elsevier Ltd. All rights reserved.

Similar-image retrieval systems are presented and evaluated. The new systems directly use image bases via ICA (Independent Component Analysis) and PCA (Principal Component Analysis). These bases can extract source image's information which is viable to define similarity measures. But, the indeterminacy on amplitude and permutation exists. In this paper, similarity measures which can absorb such indeterminacy are presented. Then, carefully designed opinion tests are carried out to compare the new systems' ability with existing ones. The compatibility of color spaces such as RGB, YIQ, and HSV is also examined. By these massive tests, {ICA., HSV} is judged the best. The resulting system is thus proved to he highly competent at the similar-image retrieval.

Network communication architecture for cooperative humanoids are designed and realized. The humanoids are operated to walk by two legs. The network is a LAN whose nodes are one master PC and other controller PCs. The master PC works as a central control machine which contains a blackboard, an image processor, and mediators for the controller PCs. The controller PC includes a collaborator and a motion controller. The collaborator communicates with the mediator and the image processor. The motion controller regulates the two-legged humanoid. Under such a combination of computing and communication systems, the walking humanoids carry a bulky box without falling down. The total job is successful if position and angle errors of the humanoids in front of the object are within an admissible range. Thus, this paper successfully adds evidences that walking humanoids can cooperate. The network communications were essential.

Independent component analysis (ICA) is applied to image coding. There, new design methods for ICA bases are presented. The new feature of this learning algorithm includes the weak guidance, or decreasing supervisory information. The weak guidance reduces the permutation indeterminacy which is unavoidable in usual ICA algorithms. In view of the image compression, this effect corresponds to the generation of image bases honoring the space frequency's neighborhood and 2-D ordering. Following the presentation of this learning algorithm, experiments are performed to obtain serviceable ICA bases. Finally, image compression and restoration are demonstrated to show the eligibility for "image. ipeg." Other applications such as image retrieval are also commented.

A new method for E. coli DNA segment classification on promoters and non-promoters is presented. The algorithm is based on the Independent Component Analysis (ICA). Since the DNA segments are composed of discrete symbols, this paper contains two major steps: (1) Position-dependent transformation of DNA segments to real number sequences, and (2) Applications of the ICA to the E. coli promoter recognition. These steps are related to each other Therefore, algorithmic explanations are given in detail while referring mutually. The automatic precision of 93.7% is obtained. Since the presented method allows threshold adjustments, twilight-zone data can be further cross-checked individually so that false negatives are reduced.

A network environment that unifies the human movement, animation and humanoid is generated. Since the degrees of freedom are different among these entities, raw human movements are recognized and labeled using the hidden Markov model. This is a class of gesture recognition which extracts necessary information transmitted to the animation software and to the humanoid. The total environment enables the surrogate of the human movement by the animation character and the humanoid. Thus, the humanoid can work as a moving computer acting as a remotely located human in the ubiquitous computing environment.

A new likelihood maximization algorithm called the alpha-EM algorithm (alpha-Expectation-Maximization algorithm) is presented. This algorithm outperforms the traditional or logarithmic EM algorithm in terms of convergence speed for an appropriate range of the design parameter alpha. The log-EM algorithm is a special case corresponding to alpha = -1. The main idea behind the alpha-EM algorithm is to search for an effective surrogate function or a minorizer for the maximization of the observed data's likelihood ratio. The surrogate function adopted in this paper is based upon the alpha-logarithm which is related to the convex divergence. The convergence speed of the alpha-EM algorithm is theoretically analyzed through alpha-dependent update. matrices and illustrated by numerical simulations. Finally, general guidelines for using the alpha-logarithmic methods are given. The choice of alternative surrogate functions is also discussed.

A new class of learning algorithms for independent component analysis (ICA) is presented. Starting from theoretical discussions on convex divergence, this information measure is minimized to derive new ICA algorithms. Since the convex divergence includes logarithmic information measures as special cases, the presented method comprises faster algorithms than existing logarithmic ones. Another important feature of this paper's ICA algorithm is to accept supervisory information. This ability is utilized to reduce the permutation indeterminacy which is inherent in usual ICA. By this method, the most important activation pattern can be found as the top one. The total algorithm is tested through applications to brain map distillation from functional MRI data. The derived algorithm is faster than logarithmic ones with little additional memory requirement, and can find task related brain maps successfully via conventional personal computer.

Iterative optimization of convex divergence is discussed. The convex divergence is used as a measure of independence for ICA algorithms. An additional method to incorporate supervisory information to reduce the ICA's permutation indeterminacy is also given. Speed of the algorithm is examined using a set of simulated data and brain fMRI data.

Likelihood optimization methods for learning algorithms are generalized and faster algorithms are provided. The idea is to transfer the optimization to a general class of convex divergences between two probability density functions. The first part explains why such optimization transfer is significant. The second part contains derivation of the generalized ICA (Independent Component Analysis). Experiments on brain fMRI maps are reported. The third part discusses this optimization transfer in the generalized EM algorithm (Expectation-Maximization). Hierarchical descendants to this algorithm such as vector quantization and self-organization are also explained.

Likelihood optimization methods for learning algorithms are generalized and faster algorithms are provided. The idea is to transfer the optimization to a general class of convex divergences between two probability density functions. The first part explains why such optimization transfer is significant. The second part contains derivation of the generalized ICA (Independent Component Analysis). Experiments on brain fMRI maps are reported. The third part discusses this optimization transfer in the generalized EM algorithm (Expectation-Maximization). Hierarchical descendants to this algorithm such as vector quantization and self-organization are also explained.

This paper gives a method to control or organize itself an activation pattern of fMRI maps obtained by ICA (independent component analysis). The presented method uses an additional term to the convex divergence's gradient. The following merits are observed: (i) Prior knowledge can be effectively used so that obtained activation patterns properly reflect the task on the subject. (ii) Difficulty of finding the appropriate activation pattern due to the permutation can be avoided. Experiments on brain fMRI maps for visual cortices are tried and reported.

A class of ICA algorithms (Independent Component Analysis) using a minimization of the convex divergence is presented. This is called the f-ICA. This algorithm is a super class of the minimum mutual information ICA and our own alpha-ICA. The following properties are obtained in this paper:
(i) The f-ICA can be implemented by both momentum and turbo methods. Their combination is also possible.
(ii) Formerly presented alpha-ICA can claim an equivalent form to the f-ICA if the design parameter alpha is chosen appropriately.
(iii) The f-ICA is much faster than the minimum mutual information ICA.
(iv) Additional complexity required to the divergence ICA is light. Therefore, this algorithm is applicable to large amount of data via conventional personal computers.
(v) Detection of human brain areas that strongly respond to moving objects is reported in this paper.

The convex divergence is used as a surrogate function for obtaining a class of ICA algorithms (Independent Component Analysis) called the f-ICA. The convex divergence is a super class of alpha-divergence, which is a further upper family of Kullback-Leibler divergence or mutual information. Therefore, the f-ICA contains the alpha-ICA and the minimum mutual information ICA. In addition to theoretical interest of generalization, the f-ICA contains a subset faster than the minimum mutual information ICA. It is found that this speed control is equivalent to the alpha-ICA. Finally, applications to brain fMRI map's distillation is presented.

The alpha-logarithm extends the logarithm as the special case of alpha = -1. Usage of alpha-related information measures based upon this extended logarithm is expected to be effective to speedup of convergence, i.e., on the improvement of learning aptitude. In this paper, two typical cases are investigated. One is the alpha-EM algorithm (alpha-Expectation-Maximization algorithm) which is derived from the alpha-log-likelihood ratio. The other is the alpha-ICA (alpha-Independent Component Analysis) which is formulated as minimizing the alpha-mutual information. In the derivation of both algorithms, the alpha-divergence plays the main role. For the alpha-EM algorithm, the reason for the speedup is explained using Hessian and Jacobian matrices for learning. For the alpha-ICA learning, methods of exploiting the past and future information are presented. Examples are shown on single-loop alpha-EM's and sample-based alpha-ICA's. In all cases, effective speedups are observed. Thus, this paper's examples together with formerly reported ones are evidences that the speed improvement by the alpha-logarithm is a general property beyond individual problems.

The alpha-EM algorithm is a super-class of the traditional EM algorithm. This algorithm is derived by computing the likelihood ratio of incomplete data through an extended logarithm; namely, the alpha-logarithm. The case of alpha = -1 corresponds to the logarithm. The number a adjusts eigenvalues of update matrices by reflecting the optimization function's second-order properties with respect to the estimation parameter. This property shows merits on speedup of convergence. In the text, derivation of the algorithm is given first. Then, convergence and speedup properties are discussed. Finally, applicability of the alpha-EM algorithm and examples are shown.

Multiple descent cost competitive learning is applied to data-compressed texture generation for 3D image processing and graphics. This learning method organizes itself by generating two types of feature maps: the grouping feature map and the weight vector feature map, which can both change regional shapes. This merit makes it possible for users to generate data-compressed image morphing. The resulting textures can be used to create virtual 3D objects. Examples are given of generating emotional expressions. The theoretical relationship between the α-EM (expectation maximization) algorithm and the multiple descent cost competitive learning algorithm is also clarified.

The α-EM (expectation maximization) algorithm is a super-class of the traditional log-EM algorithm. The case of α=-1 corresponds to the. log-EM algorithm. For the stable region of α&gt
-1, the α-EM algorithm outperforms the traditional method in terms of the learning speed measured by iterations and CPU time. Both the α-EM algorithm and the log-EM algorithm try to maximize the conditional expectation on the tentative complete data. On the other hand, there is an extension of the traditional EM algorithm which includes direct maximization on the incomplete-data likelihood-which is the true performance measure. This is the ECME (expectation and conditional maximization or either) algorithm. Thus, this paper describes the α-version of the ECME first. Then, a speed evaluation is made on this α-ECME algorithm using the extended Fisher information matrix. Examples of unsupervised and supervised learning are given. The α-ECME algorithm is more meritorious than the plain α-EM or α-ECM (expectation and conditional maximization) algorithms in terms of the iteration count. If the CPU time is of ultimate importance, the plain α-EM algorithm and the α-ECME algorithm are comparable.

Multiple descent cost competition is a composition of learning phases for minimizing a given measure of total performance, i.e., cost. If these phases are heterogeneous toward each other, the total learning algorithm shows a variety of extraordinary abilities; especially in regards to multimedia information processing. In the first phase of descent cost learning, elements of source data are grouped. Simultaneously, a weight vector for minimal learning, (i.e., a winner), is found. Then, the winner and its partners are updated for further cost reduction. Therefore, two classes of self-organizing feature maps are generated. One is called a grouping feature map, which partitions the source data. The other is an ordinary weight vector feature map. The grouping feature map, together with the winners, retains most of the source data information. This feature map is able to assist in a high quality approximation of the original data. Traditional weight vector feature maps lack this ability. Another important capacity of the grouping feature map is that it can change its shape. Thus, the grouping pattern can accept external directions in order to metamorphose. In the text, the total algorithm of the multiple descent cost competition is explained first. In that section, image processing concepts are introduced in order to assist in the description of this algorithm. Then, a still image is first data-compressed (DC). Next, a restored image is morphed using the grouping feature map by receiving directions given by an external intelligence. Next, an interpolation of frames is applied in order to complete animation coding (AC). Thus, multiple descent cost competition bridges "DC to AC." Examples of multimedia processing on virtual digital movies are given.

Recently established alpha-EM algorithm contains the conventional EM algorithm as a special case. But, the Cramer-Rao bound can be proved to be the same for all alpha. On the learning speed, however, the alpha-EM algorithm has a range of alpha that outperforms the traditional EM algorithm. There is a hierarchy in a class of unsupervised learning emanated from the alpha-EM algorithm to deterministic vector quantization. Vector quantization is a special case of competitive learning which can be interpreted as a hard-max version of the EM algorithm. This paper gives a unified theory starting from the alpha-EM algorithm terminated by vector quantization. Performance comparison on various subclasses are given. Relationships to harmonic competition, multiple descent cost competition and self-organization are also discussed.

A class of non-logarithmic likelihood ratio is considered and is applied to learning of neural networks including hierarchical experts. Such a likelihood ratio is based on an ct-logarithm which contains the usual logarithm as a special case. This generalized logarithm is defined through a discussion of the a-divergence which includes the Kullback-Leibler number as a special case. It is found that the usage of such a generalized logarithm on the likelihood ratio is equivalent to a prior probability weight. Then, this prior weighting is derived for learning on:neural networks of expert mixtures. Both of gradient ascent maximization and EM learning are discussed. The prior weighting is understood as speed-up and stabilization on the learning.

Starting from Renyi's alpha-divergence, a class of generalized EM algorithms called the alpha-EM algorithms or the WEM algorithms are derived. Merits of this generalization are found on speedup of learning, i.e acceleration of convergence. Discussions include novel alpha-versions of logarithm, efficient scores, information matrices and the Cramer-Rao bound. The speedup is examined on Gaussian mixture learning systems.

A class of extended logarithms is used to derive alpha-weighted EM (alpha-weighted Expectation and Maximization) algorithms. These extended EM algorithms (WEM's, alpha-EM's) have been anticipated to outperform the traditional (logarithmic) EM algorithm on the speed. The traditional approach falls into a special case of the new WEM. In this paper, general theoretical discussions are given first. Then, clear-cut evidences that show faster convergence than the ordinary EM approach are given on the case of mixture-of-expert neural networks. This process takes three steps. The first step is to show concrete algorithms. Then, the convergence is theoretically checked. Thirdly, experiments on the mixture-of-expert learning are tried to show the superiority of the WEM. Besides the supervised learning, unsupervised case on a Gaussian mixture is also examined. Faster convergence of the WEM is observed again.

The alpha-divergence is utilized to derive a generalized expectation and maximization algorithm (EM algorithm). This algorithm has a wide range of applications. In this paper, neural network learning for mixture probabilities is focused. The alpha-EM algorithm includes the existing EM algorithm as a special case since that corresponds to alpha = -1. The parameter alpha specifies a probability weight for the learning. This number affects learning speed and local optimality. In the discussions of update equations of neural nets, extensions of basic statistics such as Fisher's efficient score, his measure of information and Cramer-Rao's inequality are also given. Besides, this paper unveils another new idea. It is found that the cyclic EM structure can be used as a building block to generate a learning systolic array. Attaching monitors to this systolic array makes it possible to create a functionally distributed learning system.

The expectation and maximization algorithm (EM algorithm) is generalized so that the learning proceeds according to adjustable weights in terms of probability measures The presented method, the weighted EM algorithm, or the or-EM algorithm includes the existing EM algorithm as a special case. It is further found that this learning structure can work systolically. It is also possible to add monitors to interact with lower systolic subsystems. This is made possible by attaching building blocks of the weighted (or plain) EM learning. Derivation of the whole algorithm is based on generalized divergences. In addition to the discussions on the learning, extensions of basic statistical properties such as Fisher's efficient score, his measure of information and Cramer-Rao's inequality are given. These appear in update equations of the generalized expectation learning. Experiments show that the presented generalized version contains cases that outperform traditional learning methods.

Harmonic competition is a learning strategy based upon winner-take-all or winner-take-quota with respect to a composite of heterogeneous subcosts. This learning is unsupervised and organizes itself, The subcosts may conflict with each other, Thus, the total learning system realizes a self-organizing multiple criteria optimization, The subcosts are combined additively and multiplicatively using adjusting parameters, For such a total cost, a general successive learning algorithm is derived first, Then, specific problems in the Euclidian space are addressed, Vector quantization with various constraints and traveling salesperson problems are selected as test problems, The former is a typical class of problems where the number of neurons is less than that of the data, The latter is an opposite case, Duality exists in these two classes, In both cases? the combination parameters of the subcosts show wide dynamic ranges in the course of learning, It is possible, however, to decide the parameter control from the structure of the total cost, This method finds a preferred solution from the Pareto optimal set of the multiple object optimization, Controlled mutations motivated by genetic algorithms are proved to be effective in finding near-optimal solutions, All results show significance of the additional constraints and the effectiveness of the dynamic parameter control.

ISBN: 1061-5369/93

1992年度電子情報通信学会論文賞

Vector quantization with optimized grouping of elements is studied. The presented vector quantization allows optimal or suboptimal grouping of source data. Thus, the algorithms herein are called variable region vector quantization. The optimization yielding the data subgroups can also be interpreted as the connection weight adjustmen. The presented methods are still executable on conventional SISD computers. However, the adaptation of the variable region vector quantization to SIMD and MIMD computation via PDP (Parallel Distributed Processing) approach motivates new computational concepts and tools. Here, a fine-grain MIMD computer is emulated and used for the variable region vector quantizer design. Experimental results on digital speech and images are given.

Telecomm. Advancement Promotion Award