Misinformation posted on social media during COVID-19 is one main example of infodemic data. This phenomenon was prominent in China when COVID-19 happened at the beginning. While a lot of data can be collected from various social media platforms, publicly available infodemic detection data remains rare and is not easy to construct manually. Therefore, instead of developing techniques for infodemic detection, this paper aims at constructing a Chinese infodemic dataset, “infodemic 2019”, by collecting widely spread Chinese infodemic during the COVID-19 outbreak. Each record is labeled as true, false or questionable. After a four-time adjustment, the original imbalanced dataset is converted into a balanced dataset by exploring the properties of the collected records. The final labels achieve high intercoder reliability with healthcare workers’ annotations and the high-frequency words show a strong relationship between the proposed dataset and pandemic diseases. Finally, numerical experiments are carried out with RNN, CNN and fastText. All of them achieve reasonable performance and present baselines for future works.

During the outbreak of the COVID-19 (2019 coronavirus disease), misinformation related to the virus spread rapidly online and have led to serious difficulties in controlling the disease. The term infodemic is coined to outline the bad effect from the extensive dissemination of misinformation during the outbreak. With regards to this phenomenon, the World Health Organization emphasized the need to fight against infodemic and asked all countries not only to make efforts in slowing down the spread of the COVID-19 but also in countering the risk caused by infodemic. Due to its negative impact, this paper analyzes infodemic on Chinese social media at the initial stage of the COVID-19 outbreak and presents a 4P framework standing for the four features of Chinese infodemic: Prevention Attention, Problem Orientation, Patterns Interaction and Points Globalization. Furthermore, a selective review of existing datasets in the neural networks domain is synthesized based on the 4P framework. Finally, research directions, including recommendations, about constructing a large-scale dataset for Chinese infodemic automatic detection are proposed.

Data imbalance occurs on most real-world classification problems and decreases the performance of classifiers. An oversampling method addresses the imbalance problem by generating synthetic samples to balance the data distribution. However, many of the existing oversampling methods have potential problems in generating wrong and unnecessary synthetic samples, which makes the learning tasks difficult. This paper proposes a new segmented oversampling method for imbalanced data classification. The input space is first partitioned into several linearly separable local partitions along the potential separation boundary by introducing a bottom-up, minimal-spanning-tree-based clustering method; an oversampling method is then applied within each local linear partition to prevent the generation of wrong and unnecessary synthetic samples; a quasi-linear support vector machine is finally used to realize the classification by taking advantages of the local linear partitions. Simulation results on different real-world datasets show that the proposed segmented oversampling method is effective for imbalanced data classifications. (C) 2017 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

This paper proposes a two-step method to construct a nonlinear classifier consisting of multiple local linear classifiers interpolated with a basis function. In the first step, a geometry-based approach is first introduced to detect local linear partitions and build local linear classifiers. A coarse nonlinear classifier can then be constructed by interpolating the local linear classifiers. In the second step, a support vector machine (SVM) formulation is used to further implicitly optimize the linear parameters of the nonlinear classifier. In this way, the nonlinear classifier is constructed in exactly the same way as a standard SVM, using a special data-dependent quasi-linear kernel composed of the information of the local linear partitions. Numerical experiments on several real-world datasets demonstrate the effectiveness of the proposed classifier and show that, in cases where traditional nonlinear SVMs run into overfitting problems, the proposed classifier is effective in improving the classification performance. (C) 2017 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

The multiple linear model is used successfully to extend the linear model to nonlinear problems. However, the conventional multilinear models fail to learn the global structure of a training data set because the local linear models are independent of each other. Furthermore, the local linear transformations are learned in the original space. Therefore, the performance of multilinear methods is strongly dependent on the results of partition. This paper presents a kernel approach for the implementation of the local linear discriminant analysis for face recognition problems. In the original space, we utilize a set of local linear transformations with interpolation to approximate an optimal global nonlinear transformation. Based on the local linear models in the original space, we derive an explicit kernel mapping to map the training data into a high-dimensional transformed space. The optimal transformation is learned globally in the transformed space. Experimental results show that the proposed method is more robust to the partition results than the conventional multilinear methods. Compared with the general nonlinear kernels that utilize a black-box mapping, our proposed method can reduce the negative effects caused by the potential overfitting problem. (c) 2016 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

This paper proposes a deep quasi-linear kernel for support vector machines (SVMs). The deep quasi-linear kernel can be constructed by using a pre-trained deep neural network. To realize this goal, a multilayer gated bilinear classifier is first designed to mimic the functionality of the pre-trained deep neural network, by generating the gate control signals using the deep neural network. Then, a deep quasi-linear kernel is derived by applying an SVM formulation to the multilayer gated bilinear classifier. In this way, we are able to further implicitly optimize the parameters of the multilayer gated bilinear classifier, which are a set of duplicate but independent parameters of the pre-trained deep neural network, by using an SVM optimization. Experimental results on different data sets show that SVMs with the proposed deep quasi-linear kernel have an ability to take advantage of the pre-trained deep neural networks and outperform SVMs with RBF kernels.

The three dimensional (3D) reconstruction of a medical image sequence can provide intuitive morphologies of a target and help doctors to make more reliable diagnosis and give a proper treatment plan. This paper aims to reconstruct the surface of a renal corpuscle from the microscope renal biopsy image sequence. First, the contours of renal corpuscle in all slices are extracted automatically by using a context-based segmentation method with a coarse registration. Then, a new coevolutionary-based strategy is proposed to realize a fine registration. Finally, a Gauss-Seidel iteration method is introduced to achieve a non-rigid registration. Benefiting from the registrations, a smooth surface of the target can be reconstructed easily. Experimental results prove that the proposed method can effectively register the contours and give an acceptable surface for medical doctors.

The quasi-linear ARX radial basis function network (RBFN) model has shown good approximation ability and usefulness in nonlinear system identification and control. It has an easy-to-use structure, good generalization and strong tolerance to input noise. In this paper, we propose a self-organizing quasi-linear ARX RBFN (QARX-RBFN) model by introducing a self-organizing scheme to the quasi-linear ARX RBFN model. Based on the active firing rate and the mutual information of RBF nodes, the RBF nodes in the quasi-linear ARX RBFN model can be added or removed, so as to automatically optimize the structure of the quasi-linear ARX RBFN model for a given system. This significantly improves the performance of the model. Numerical simulations on both identification and control of nonlinear dynamical system confirm the effectiveness of the proposed self-organizing QARX-RBFN model.

In this paper, a new switching mechanism is proposed based on the state of dynamic tracking error so that more information will be provided -not only the error but also a one up to pth differential error will be available as the switching variable. The switching index is based on the Lyapunov stability theory. Thus the switching mechanism can work more effectively and efficiently. A simplified quasi-ARX neural-network (QARXNN) model presented by a state-dependent parameter estimation (SDPE) is used to derive the controller formulation to deal with its computational complexity. The switching works inside the model by utilizing the linear and nonlinear parts of an SDPE. First, a QARXNN is used as an estimator to estimate an SDPE. Second, by using SDPE, the state of dynamic tracking error is calculated to derive the switching index. Additionally, the switching formula can use an SDPE as the switching variable more easily. Finally, numerical simulations reveal that the proposed control gives satisfactory tracking and disturbance-rejection performances. Experimental results demonstrate its effectiveness. (c) 2015 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

In the multi-label classification issue, some implicit constraints and dependencies are always existed among labels. Exploring the correlation information among different labels is important for many applications. It not only can enhance the classifier performance but also can help to interpret the classification results for some specific applications. This paper presents an improved multi-label classification method based on local label constraints and classifier chains for solving multi-label tasks with large number of labels. Firstly, in order to exploit local label constraints in multi-label problem with large number of labels, clustering approach is utilized to segment training label set into several subsets. Secondly, for each label subset, local tree-structure constraints among different labels are mined based on mutual information metric. Thirdly, based on the mined local tree-structure label constraints, a variant of classifier chain strategy is implemented to enhance the multi-label learning system. Experiment results on five multi-label benchmark datasets show that the proposed method is a competitive approach for solving multi-label classification tasks with large number of labels.

Automatic registration is an important task to prepare aligned 2D images for 3D structure visualization, and it is a challenging problem especially for the microscope images. This paper proposes a novel coevolution-based coarse-to-fine registration method, aiming to align the regions of interest (ROIs) in the image sequence. Firstly, a coarse registration for whole images is executed by a scale-invariant feature transform (SIFT) based method, which can facilitate the segmentation of ROIs. Secondly, a fine registration for the segmented ROIs is done by a genetic algorithm (GA) with a novel coevolutionary strategy. Experimental results demonstrate the good performance of the proposed method and it is also successfully applied to the renal biopsy image sequence.

The control system is a key technology to extract maximum energy from the incident wind. By regulating aerodynamic control, it is possible to adapt the changes in wind speed by controlling shaft speed. Thus, the turbine generator can track maximum power extracted from wind. In this paper, we propose a Lyapunov based switching control under quasi-linear ARX neural network (QARXNN) model to track maximum power of wind energy conversion system. The switching index is used to measure the stability of nonlinear controller and selects linear or nonlinear controller in order to ensure the stability. Interestingly, a simple switching law can be built utilizing the parameters of model directly. Finally, we have compared the proposed algorithm of switching controller with another algorithm. The results show that the proposed algorithm has better control performance.

Kernel functions based machine learning algorithms have been extensively studied over the past decades with successful applications in a variety of real-world tasks. In this paper, we formulate a kernel level composition method to embed multiple local classifiers (kernels) into one kernel function, so as to obtain a more flexible data-dependent kernel. Since such composite kernels are composed by multiple local classifiers interpolated with several localizing gating functions, a specific learning process is also introduced in this paper to pre-determine their parameters. Experimental results are provided to validate two major perspectives of this paper. Firstly, the introduced learning process is effective to detect local information, which is essential for the parameter pre-determination of the localizing gating functions. Secondly, the proposed composite kernel has a capacity to improve classification performance.

In this paper, we introduce a data-dependent kernel called deep quasi-linear kernel, which can directly gain a profit from a pre-trained feedforward deep network. Firstly, a multi-layer gated bilinear classifier is formulated to mimic the functionality of a feed-forward neural network. The only difference between them is that the activation values of hidden units in the multi-layer gated bilinear classifier are dependent on a pre-trained neural network rather than a pre-defined activation function. Secondly, we demonstrate the equivalence between the multi-layer gated bilinear classifier and an SVM with a deep quasi-linear kernel. By deriving a kernel composition function, traditional optimization algorithms for a kernel SVM can be directly implemented to implicitly optimize the parameters of the multi-layer gated bilinear classifier. Experimental results on different data sets show that our proposed classifier obtains an ability to outperform both an SVM with a RBF kernel and the pre-trained feedforward deep network.

By itself, a wind turbine is already a fairly complex system with highly nonlinear dynamics. Changes in wind speed can affect the dynamic parameters of wind turbines, thus rendering the parameters uncertain. However, we can identify the dynamics of the wind energy conversion system (WECS) online by a quasi-ARX neural network (QARXNN) model. A QARXNN presents a problem in searching for the coefficients of the regression vector (input vector). A multilayer perceptron neural network (MLPNN) is an embedded system that provides the unknown parameters used to parameterize the input vector. Fascinatingly, the coefficients of the input vector from prediction model can be set as controller parameters directly. The stability of the closed-loop controller is guaranteed by the switching of the linear and nonlinear parts of the parameters. The dynamic of WECS is derived with given parameters, and then a wind speed signal created by a random model is fed to the system causing uncertainty parameters and reducing the power that can be absorbed from wind. By using a minimum variance controller, the maximum power is tracked from WECS. From the simulation results, it is observed that the proposed controller is effective in tracking the maximum power of WECS. (c) 2015 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

A renal biopsy is a procedure to get a small piece of kidney for microscopic examination. With the development of tissue sectioning and medical imaging techniques, microscope renal biopsy image sequences are consequently obtained for computer-aided diagnosis. This paper proposes a new context-based segmentation algorithm for acquired image sequence, in which an improved genetic algorithm (GA) patching method is developed to segment different size target. To guarantee the correctness of first image segmentation and facilitate the use of context information, a boundary fusion operation and a simplified scale-invariant feature transform (SIFT)-based registration are presented respectively. The experimental results show the proposed segmentation algorithm is effective and accurate for renal biopsy image sequence.

This paper presents a new strategy to build multi tree hierarchical structure SVM which can get a more efficient and accuracy classification model for multiclass problems. Base on the theory of Binary Tree SVM (BTS), we proposed an improvement algorithm which extend binary tree structure to a multi tree structure, In the multi tree hierarchical structure, similarity clustering method was proposed to cluster classes to groups in each non-leaf node. In order to get a multi node division, one-against-all (OAA) was applied to train those groups rather than classes. The proposed method can avoid data imbalanced problem occurred in OAA, also the classification area of classifier in the upper layer is larger than classifier in lower layer. Compared with other several well-known methods, experiments on many data sets demonstrate that our method can reduce the number of classifiers in the testing phase and get a higher accuracy.

This paper proposes an improved SVM based multi-label classification method by using relationship among labels. Following a traditional multi-label solution, binary relevance (BR) method is first used to decompose the multi-label classification problem into multiple binary classification sub-problems, each of which is solved by an SVM classifier. By using Platt's sigmoid technique, each SVM classifier gives probability output for the following correction. A probability model is introduced to estimate the relationship among labels. The extracted label relationship is then applied to correct the outputs of SVM classifiers, in which a dynamic weight strategy is further introduced. Numerical experiments on widely used benchmark datasets show that the proposed method can improve the accuracy of multi-label classification when compared with traditional BR method and some other conventional multi-label classification methods.

This paper proposes a geometric way to construct a quasi-linear kernel by which a quasi-linear support vector machine (SVM) is performed. A quasi-linear SVM is a SVM with quasi-linear kernel, in which the nonlinear separation boundary is approximated by using multi-local linear boundaries with interpolation. However, the local linearity extraction for the composition of quasi-linear kernel is still an open problem. In this paper, according to the geometric theory, a method based on piecewise linear classifier is proposed to extract the local linearity in a more precise and efficient way. We firstly construct a function set including multiple linear functions and each of those functions reflects one part of linearity of the whole nonlinear separation boundary. Then the obtained local linearity is added as prior information into the composition of quasi-linear kernel. Experimental results on synthetic data sets and real world data sets show that our proposed method is effective to improve classification performances.

This paper presents a Transductive Support Vector Machine (TSVM) with quasi-linear kernel based on a clustering assumption for semi-supervised classification. Since the potential separating boundary is located in low density area between classes, a modified density clustering method by considering label information is firstly introduced to extract the information of potential separating boundary in low density region between different classes. Then the information is used to compose a quasi-linear kernel for the TSVM. The optimization of TSVM is further speeded up by developing a pair wise label switching method on minimal sets. Experiment results on benchmark datasets show that the proposed

An improved Elman neural network (IENN) controller with particle swarm optimization (PSO) is presented for nonlinear systems. The proposed controller is composed of a quasi-ARX neural network (QARXNN) prediction model and a switching mechanism. The switching mechanism is used to guarantee that the prediction model works well. The primary controller is designed based on IENN using the backpropagation (BP) learning algorithm with PSO. PSO is used to adjust the learning rates in the BP process for improving the learning capability. The adaptive learning rates of the controller are investigated via the Lyapunov stability theorem. The proposed controller performance is verified through numerical simulation. The method is compared with the fuzzy switching and 0/1 switching methods to show its effectiveness in terms of stability, accuracy, and robustness. (C) 2014 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

This paper proposes a so called quasi-linear support vector machine (SVM), which is an SVM with a composite quasi-linear kernel. In the quasi-linear SVM model, the nonlinear separation hyperplane is approximated by multiple local linear models with interpolation. Instead of building multiple local SVM models separately, the quasi-linear SVM realizes the multi local linear model approach in the kernel level. That is, it is built exactly in the same way as a single SVM model, by composing a quasi-linear kernel. A guided partitioning method is proposed to obtain the local partitions for the composition of quasi-linear kernel function. Experiment results on artificial data and benchmark datasets show that the proposed method is effective and improves classification performances.

In this article, a fuzzy adaptive controller approach is presented for nonlinear systems. The proposed quasi-ARX neural network based on Lyapunov learning algorithm is used to update its weight for prediction model as well as to modify fuzzy adaptive controller. The improving performances of the Lyapunov learning algorithm are stable in the learning process of the controller and able to increase the accuracy of the controller as well as fast convergence of error. The simulations are intended to show the effectiveness of the proposed method. © 2013 ISAROB.

Differential evolution (DE) is an effective and efficient evolutionary algorithm in continuous space. The setting of control parameters is highly relevant with the convergence efficiency, and varies with different optimization problems even at different stages of evolution. Self-adapting control parameters for finding global optima is a long-term target in evolutionary field. This paper proposes a two-layered DE (TLDE) with self-adaptive control parameters combined with niching method based mutation strategy. The TLDE consists of two DE layers: a bottom DE layer for the basic evolution procedure, and a top DE layer for control parameter adaptation. Both layers follow the procedure of DE. Moreover, to mitigate the common phenomenon of premature convergence in DE, a clearing niching method is brought out in finding efficient mutation individuals to maintain diversity during the evolution and stabilize the evolution system. The performance is validated by a comprehensive set of twenty benchmark functions in parameter optimization and competitive results are presented.

Support Vector Data Description (SVDD) inherits properties of Support Vector Machines (SVM) and has become a prominent One Class Classifier (OCC). Same to standard SVM, its O(n(3)) time and O(n(2)) space complexities, where n is the number of training samples, have become major limitations in cases of large training datasets. As a simple and effective method, reducing the size of training dataset through reserving only samples mostly relevant to learned classifier, can be adopted to overcome the limitations. A trained SVDD enclosed decision boundary always locates on edge area of data distribution and is decided by a small subset of Support Vectors(SVs). Therefore, in this paper, we present a method based on edge detection such that edge samples mostly relevant to decision boundary can be preserved. And clustering techniques are also be applied to keep centroids representing the global distribution properties so as to avoid over-outside of decision boundary. To restrict the influences of noises, each training pattern is assigned with a weight. Experiments on real and artificial data sets prove that the classifier trained on reconstruction training set consisting of edge points and centroids can preserve performance with much faster training speed.

Clustering, one of the important data mining techniques, has two main processing methods on data-based similarity clustering and space-based density grid clustering. The latter has more advantage than the former on larger and multiple shape and density dataset. However, due to a global partition of existing grid-based methods, they will perform worse when there is a big difference on the density of clusters. In this paper, we propose a novel algorithm that can produces appropriate grid space in different density regions by simulating cell division process. The time complexity of the algorithm is O(n) in which n is number of points in dataset. The proposed algorithm will be applied on popular chameleon datasets and our synthetic datasets with big density difference. The results show our algorithm is effective on any multi-density situation and has scalability on space optimization problems.

This paper focuses on semi-supervised classification problem by using Transductive Support Vector Machine. Traditional TSVM for semi-supervised classification firstly train an SVM model with labeled data. Then use the model to predict unlabeled data and optimize unlabeled data prediction to retrain the SVM. TSVM always uses a predefined kernel and fixed parameters during the optimization procedure and they also suffers potential over-fitting problem. In this paper we introduce proposed quasi-linear kernel to the TSVM. An SVM with quasi-linear kernel realizes an approximate nonlinear separation boundary by multi-local linear boundaries with interpolation. By applying quasi-linear kernel to semi-supervised classification it can avoid potential over-fitting and provide more accurate unlabeled data prediction. After unlabeled data prediction optimization, the quasi-linear kernel can be further adjusted considering the potential boundary data distribution as prior knowledge. We also introduce a minimal set method for optimizing unlabeled data prediction. The minimal set method follows the clustering assumption of semi-supervised learning. The pairwise label switching is allowed between minimal sets. It can speed up optimization procedure and reduce influence from label constrain in TSVM. Experiment results on benchmark gene datasets show that the proposed method is effective and improves classification performances.

This paper proposes a self-organizing maps (SOM) based kernel composition method for the quasi-linear support vector machine (SVM). The quasi-linear SVM is SVM model with quasi-linear kernel, in which the nonlinear separation hyperplane is approximated by multiple local linear models with interpolation. The basic idea underlying the proposed method is to use clustering and projection properties of SOM to partition the input space and construct a SOM based quasi-linear kernel. By effectively extracting the distribution information using SOM, the quasi-linear SVM with the SOM-based quasi-linear kernel is expected to have better performance in the cases of high-noise and high-dimension. Experiment results on synthetic datasets and real world datasets show the effectiveness of the proposed method.

Assembling a large genome using next generation sequencing reads requires large computer memory and a long execution time. To reduce these requirements, a memory and time efficient assembler is presented from applying FM-index in JR-Assembler, called FMJ-Assembler, where FM stand for FMR-index derived from the FM-index and BWT and J for jumping extension. The FMJ-Assembler uses expanded FMindex and BWT to compress data of reads to save memory and jumping extension method make it faster in CPU time. An extensive comparison of the FMJ-Assembler with current assemblers shows that the FMJ-Assembler achieves a better or comparable overall assembly quality and requires lower memory use and less CPU time. All these advantages of the FMJ-Assembler indicate that the FMJ-Assembler will be an efficient assembly method in next generation sequencing technology.

A nonlinear model-predictive control (NMPC) is demonstrated for nonlinear systems using an improved fuzzy switching law. The proposed moving average filter fuzzy switching law (MAFFSL) is composed of a quasi-ARX radial basis function neural network (RBFNN) prediction model and a fuzzy switching law. An adaptive controller is designed based on a NMPC. a MAFFSL is constructed based on the system switching criterion function which is better than the (ON/OFF) switching law and a RBFNN is used to replace the neural network (NN) in the quasi-ARX black box model which is understood in terms of parameters and is not an absolute black box model, in comparison with NN. The proposed controller performance is verified through numerical simulations to demonstrate the effectiveness of the proposed method.

In this paper, a novel approach is proposed as a new fast Support Vector Machine(SVM) basing on sequential minimal optimization(SMO), minimum enclosing ball(MEB) approach and active set strategy. The combination with these 3 techniques largely accelerates the training process of SVM, attains fewer support vectors(SVs) as well as obtains a acceptable accuracy comparing to original SVM. From simulation results, it is stated that the proposed method will be a good alternative for classification of large scale data.

A quasi-ARX (quasi-linear ARX) neural network (QARXNN) model is able to demonstrate its ability for identification and prediction highly nonlinear system. The model is simplified by a linear correlation between the input vector and its nonlinear coefficients. The coefficients are used to parameterize the input vector performed by an embedded system called as state dependent parameter estimation (SDPE), which is executed by multi layer parceptron neural network (MLPNN). SDPE consists of the linear and nonlinear parts. The controller law is derived via SDPE of the linear and nonlinear parts through switching mechanism. The dynamic tracking controller error is derived then the stability analysis of the closed-loop controller is performed based Lyapunov theorem. Linear based adaptive robust control and nonlinear based adaptive robust control is performed with the switching of the linear and nonlinear parts parameters based Lyapunov theorem to guarantee bounded and convergence error.

Pulse coupled neural network (PCNN) is a new type of artificial neural network specific for image processing applications. It is a single layer, two dimensional network with neurons which have I : I correspondence to the pixels of an input image. It is convenient to process the intensities and spatial locations of image pixels simultaneously by applying a PCNN. Therefore, we propose a modified PCNN with anisotropic synaptic weight matrix for image edge detection from the aspect of intensity similarities of pixels to their neighborhoods. By applying the anisotropic synaptic weight matrix, the interconnections are only established between the central neuron and the neighboring neurons corresponding to pixels with similar intensity values in a 3 by 3 neighborhood. Neurons corresponding to edge pixels and non-edge pixels will receive different input signal from the neighboring neurons. By setting appropriate threshold conditions, image step edges can be detected effectively. Comparing with conventional PCNN based edge detection methods, the proposed modified PCNN is much easier to control, and the optimal result can be achieved instantly after all neurons pulsed. Furthermore, the proposed method is shown to be able to distinguish the isolated pixels from step edge pixels better than derivative edge detectors.

In a standard support vector machine (SVM), the training process has O(n3) time and O(n2) space complexities, where n is the size of the training dataset. For very large datasets, it is thus computationally infeasible. Reducing the size of training dataset is naturally considered as a method to solve this problem. SVM classifiers are constructed by using the training samples called support vectors (SVs) that lie close to the separation boundary. Thus, removing the other samples that are not relevant to SVs might have no effect on building the separation boundary. In other words, we need to reserve the samples that are likely to be SVs. Therefore, a method based on edge detection techniques is proposed to extract such samples near the separation boundary. In order to avoid overfitting, we also use a clustering algorithm to keep the distribution properties of the training dataset. The samples selected by the edge detector and the centroids of clusters are used to reconstruct the training dataset. In the proposed approach, the edge detection technique helps us to extract the local properties around the separation boundary and the clustering algorithm preserves the properties of the entire data. The reconstructed training dataset with a smaller number of samples can make the training process very fast without degrading the classification accuracy. (c) 2013 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

Several researchers have shown that multiple classifier systems can result in effective solutions to difficult real-world classification tasks. However, most of these approaches are easily influenced by noise, and the training datasets for local classifiers get easily imbalanced. One of the main reasons for this is that it is hard to guarantee that the centers of the subsets are close to the separation hyperplane, so that it is difficult to evenly distribute the samples in the two sides of the hyperplane. In order to solve this problem, we redefine the description of classifier modeling problem as a task of piecewise approximation of the separation hyperplane. On the basis of this description, we propose a novel multiple support vector machine (SVM) classifier system. Its main contribution is a novel construction approach to the subtraining datasets. The proposed approach partitions the area close to the separation hyperplane into some subsets to construct the subtraining datasets. The subtraining datasets describe the subtasks for identifying segments of the separation hyperplane. Local SVMs are trained to solve the respective subtasks. Finally, the decisions of these local SVMs are appropriately combined on the basis of a probabilistic interpretation to obtain the final classification decision. The effectiveness of this approach is demonstrated through comparisons with some well-known approaches on both synthetic and real-world datasets. © 2013 Institute of Electrical Engineers of Japan.

This work exploits the idea on how to search parameter estimation and increase its convergence speed for the Liner Time Invariant (LTI) system. The convergence speed of parameter estimation is the one problem and plays an important role in the adaptive controller to increase performance. The well-known algorithm is the recursive least square algorithm. However, the speed of convergence is still low and is influenced by the number of sampling, which is represented by the limited availability for the information vector. We offer a new method to increase the convergence speed by applying Quasi-ARX model. Quasi-ARX model performs two steps identification process by presenting parameter estimation as a function over time. The first, parameters estimation of macro-part sub-model are searched by the least square error, and the second is to sharpen the searching by performing backpropagation learning of multi layer parceptron network.

This paper focuses on imbalanced dataset classification problem by using SVM and oversampling method. Traditional oversampling method increases the occurrence of over-lapping between classes, which leads to poor generalization of SVM classification. To solve this problem this paper proposes a combined method of quasi-linear SVM and assembled SMOTE. The quasi-linear SVM is an SVM with quasi-linear kernel function. It realizes an approximate nonlinear separation boundary by mulit-local linear boundaries with interpolation. The assembled SMOTE implements oversampling with considering of the data distribution information and avoids occurrence of overlapping between classes. Firstly, a partition method based on Minimal Spanning Tree is proposed to obtain local linear partitions, each of which can be separated with one linear separation boundary. Secondly, using the information of local linear partitions, the assembled SMOTE generates synthetic minority class samples. Finally, the quasi-linear SVM realizes a classification of oversampled datasets in the same way as a standard SVM by using a composite quasi-linear kernel function. Experiment results on artificial data and benchmark datasets show that the proposed method is effective and improves classification performances. © 2013 IEEE.

Multi-label classification is an extension of traditional classification problem in which each instance is associated with a set of labels. For some multi-label classification tasks, labels are usually overlapped and correlated, and some implicit constraint rules are existed among the labels. This paper presents an improved multi-label classification method based on label ranking strategy and label constraints. Firstly, oneagainst-all decomposition technique is used to divide a multilabel classification task into multiple independent binary classification sub-problems. One binary SVM classifier is trained for each label. Secondly, based on training data, label constraint rules are mined by association rule learning method. Thirdly, a correction model based on label constraints is used to correct the probabilistic outputs of SVM classifiers for label ranking. Experiment results on three well-known multi-label benchmark datasets show that the proposed method outperforms some conventional multi-label classification methods.

In this paper, an SVM-based approach is proposed for stock market trend prediction. The proposed approach consists of two parts: feature selection and prediction model. In the feature selection part, a correlation-based SVM filter is applied to rank and select a good subset of financial indexes. And the stock indicators are evaluated based on the ranking. In the prediction model part, a so called quasi-linear SVM is applied to predict stock market movement direction in term of historical data series by using the selected subset of financial indexes as the weighted inputs. The quasi-linear SVM is an SVM with a composite quasi-linear kernel function, which approximates a nonlinear separating boundary by multi-local linear classifiers with interpolation. Experimental results on Taiwan stock market datasets demonstrate that the proposed SVM-based stock market trend prediction method produces better generalization performance over the conventional methods in terms of the hit ratio. Moreover, the experimental results also show that the proposed SVM-based stock market trend prediction system can find out a good subset and evaluate stock indicators which provide useful information for investors. © 2013 IEEE.

This paper presents a fuzzy adaptive controller applied to a non linear system modeled under a Quasi-linear ARX Neural Network, with stability proof by using the Lyapunov approach. This work exploits the new idea to use Lyapunov function to train multi-input multi-output neural network on the core-part sub-model. The proposed controller is designed between a non linear controller and linear controller based on fuzzy switching algorithm. Finally improving performances of the Lyapunov learning algorithm are stable in the learning process, fast convergence of error, and able to increase the accuracy of the controller.

According to the new economic growth theory, a new method of computing the contribution rate of scientific and technological (S&T) progress to economic growth based on the Cobb-Douglas production function and the Solow residual value method is proposed in this paper. This method includes three steps: Firstly, according to their levels of S&T progress, fuzzy soft clustering of thirty one Chinese regions is performed to obtain the membership degree of these places to the categories. Secondly, to calculate the contribution rates that different categories of levels of S&T progress contribute to economic growth. Thirdly, to multiply the obtained contribution rate of each category by the membership degree of the place belonging to this category, from which the contribution rate of S&T progress to economic growth in each place is obtained. Finally, this method is used to calculate the contribution rates of S&T progress to economic growth in thirty one Chinese regions during the period from 1998 to 2007. Last but not least, some reasonable suggestions and conclusions are proposed by analyzing the computing results. (c) 2011 Elsevier Ltd. All rights reserved.

Economic contribution rate of education (ECRE) is the key factor of education economics. This article selected China, South Korea, United States and other countries for a total of 15 samples, and put the data of the same period under the framework of soft computing, to simulate the production chain of "education-potential human capital-actual human capital-economic growth". The basic idea is: Firstly, 15 countries are softly categorized according to the level of science and technology (S&T) progress. Secondly, potential human capital and actual human capital establish the internal correlation (fuzzy mapping) in the same classification, and we conceptualize actual human capital as one production factor, joined with the other two production factors, fixed asset and land, to set up the fuzzy mapping to economic growth., and then calculate economic contribution rate of education of China and foreign by two fuzzy mapping of them. Thirdly, this paper analyzes the present state and differences in the development of education between China and foreign according to different ECRE, and offers proposals for promoting the education in China. (C) 2012 Elsevier B.V. All rights reserved.

In this paper, a fuzzy switching adaptive control approach is presented for nonlinear systems. The proposed fuzzy switching adaptive control law is composed of a quasi-ARX radial basis function network (RBFN) prediction model and a fuzzy switching mechanism. The quasi-ARX RBFN prediction model consists of two parts: a linear part used for a linear controller to ensure boundedness of the input and output signals; and an RBFN nonlinear part used to improve control accuracy. By using the fuzzy switching scheme between the linear and nonlinear controllers to replace the 0/1 switching, it can realize a better balance between stability and accuracy. Theoretical analysis and simulation results show the effectiveness of the proposed control method on the stability, accuracy, and robustness. (c) 2012 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

The quasi-ARX neurofuzzy (Q-ARX-NF) model has shown great approximation ability and usefulness in nonlinear system identification and control. It owns an ARX-like linear structure, and the coefficients are expressed by an incorporated neurofuzzy (InNF) network. However, the Q-ARX-NF model suffers from curse-of-dimensionality problem, because the number of fuzzy rules in the InNF network increases exponentially with input space dimension. It may result in high computational complexity and over-fitting. In this paper, the curse-of-dimensionality is solved in two ways. Firstly, a support vector regression (SVR) based approach is used to reduce computational complexity by a dual form of quadratic programming (QP) optimization, where the solution is independent of input dimensions. Secondly, genetic algorithm (GA) based input selection is applied with a novel fitness evaluation function, and a parsimonious model structure is generated with only important inputs for the InNF network. Mathematical and real system simulations are carried out to demonstrate the effectiveness of the proposed method.

Hierarchical multi-label classification (HMC) is a variant of classification where instances may belong to multiple classes at the same time and these classes are organized in a hierarchy. Gene function prediction is a complicated HMC problem with large class number and usually strongly imbalanced class distributions. This paper proposes an improved HMC method based on over-sampling and hierarchy constraint for solving the gene function prediction problem. The HMC task is transferred into a set of binary support vector machine (SVM) classification tasks. Then, two measures are implemented to enhance the HMC performance by introducing the hierarchy constraint into learning procedures. Firstly, for imbalanced classes, a hierarchical synthetic minority over-sampling technique (SMOTE) is proposed as over-sampling preprocessing to improve the SVM learning performance. Secondly, an improved True Path Rule (TPR) ensemble approach is introduced to combine the results of binary probabilistic SVM classifications. It can improve the classification results and guarantee the hierarchy constraint of classes. Experiment results on four benchmark FunCat Yeast datasets show that the proposed method significantly outperforms the basic TPR method and the Flat ensemble method. (C) 2012 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

to understand the behavior of smartphone online application software will be helpful to predict whether the software application would be adopted by the users and to guide the providers to enhance the functions of the software. A wide range of criteria are used to assess smartphone software quality, but most of these criteria have interdependent or interactive characteristics, which can make it difficult to effectively analyze and improve smartphone use intention. The purpose of this study is to address this issue using a hybrid MCDM (multiple criteria decision-making) approach that includes the DEMATEL (decision-making trial and evaluation laboratory), DANP (the DEMATEL-based analytic network process) methods to achieve an optimal solution. By exploring the influential interrelationships between criteria related to mobile communication industry's and related value-added service content providers' reference in the respect of operation. This approach can be used to solve interdependence and feedback problems, allowing for greater satisfaction of the actual needs of mobile communication industries.

This paper proposes a multiinput and multioutput (MIMO) quasi-autoregressive eXogenous (ARX) model and amultivariable-decoupling proportional integral differential (PID) controller for MIMO nonlinear systems based on the proposed model. The proposed MIMO quasi-ARX model improves the performance of ordinary quasi-ARX model. The proposed controller consists of a traditional PID controller with a decoupling compensator and a feed-forward compensator for the nonlinear dynamics based on the MIMO quasi-ARX model. Then an adaptive control algorithm is presented using the MIMO quasi-ARX radial basis function network (RBFN) prediction model and some stability analysis of control system is shown. Simulation results show the effectiveness of the proposed control method.

In recent years, support vector regression (SVR) has attracted much attention for nonlinear system identification. It can solve nonlinear problems in the form of linear expressions within the linearly transformed space. Commonly, the convenient kernel trick is applied, which leads to implicit nonlinear mapping by replacing the inner product with a positive definite kernel function. However, only a limited number of kernel functions have been found to work well for the real applications. Moreover, it has been pointed that the implicit nonlinear kernel mapping is not always good, since it may faces the potential over-fitting for some complex and noised learning task. In this paper, explicit nonlinear mapping is learnt by means of the quasi-ARX modeling, and the associated inner product kernel, which is named quasi-linear kernel, is formulated with nonlinearity tunable between the linear and nonlinear kernel functions. Numerical and real systems are simulated to show effectiveness of the quasi-linear kernel, and the proposed identification method is also applied to microarray missing value imputation problem.

This paper proposes a hierarchical multi-label classification method based on SVM with composite kernel for solving gene function prediction. The hierarchical multi-label classification problem is resolved into a set of binary classification tasks. A composite kernel based SVM (ck-SVM) is introduced to deal with the binary classification tasks. In estimation procedure of ck-SVM, a supervised clustering with over-sampling strategy is introduced for solving imbalance dataset learning problem and improve classification performance. Experimental results on benchmark datasets demonstrate that the proposed method improves the classification performance efficiently.

This paper presents a method for nonlinear discriminant analysis utilizing a composite kernel which is derived from a combination of local linear models with interpolation. The underlying idea is to decompose a complex nonlinear problem into a set of simpler local linear problems. Combining with the theory of nonlinear classification based on kernels, the local linear models with interpolation can be formulated as a composite kernel based discriminant analysis form. In face recognition, linear discriminant analysis (LDA) has been widely adopted owing to its efficiency, but it fails to solve nonlinear problems. Conventional kernel based approaches such as generalized discriminant analysis (GDA) has been successfully applied to extend LDA to nonlinear pattern recognition tasks. However, selecting an appropriate kernel function is usually difficult. Utilizing an implicit kernel mapping may face potential over-training problems for some complex and noised tasks. Our proposed method gives an alternative solution for nonlinear discriminant analysis while the conventional linear and nonlinear approaches are difficult to achieve a satisfactory results. Experiments on both synthetic data and face data set show the effectiveness of the proposed methods.

This paper present a neural predictive controller (NPC) based on improved quasi-ARX neural network (IQARXNN) for nonlinear dynamical systems. The IQARXNN is used as a model identifier with switching algorithm and switching stability analysis. The primary controller is designed based on a modified Elman neural network (MENN) controller using back-propagation (BP) learning algorithm with modified particle swarm optimization (MPSO) to adjust the learning rates in the BP process to improve the learning capability. The adaptive learning rates of the controller are investigated via Lyapunov stability theorem, which are respectively used to guarantee the convergences of the predictive controller. Performance of the proposed MENN controller with MPSO is verified by simulation results to show the effectiveness of the proposed method both on stability and accuracy.

In this note, we present the modeling of nonlinear dynamical systems with Quasi-ARX neural network using Lyapunov algorithm in learning process. This work exploits the idea on learning algorithm in nonlinear kernel part of Quasi-ARX model to improve stability and fast convergence of error. The proposed algorithm is then employed to model and predict a classical nonlinear system with input dead zone and nonlinear dynamic systems, exhibiting the effectiveness of proposed algorithm. Based on the result of simulation, the proposed algorithm can make the error in process learning become fast convergence, ultimately bounded, and the error distributed uniformly.

This paper presents an automated approach to extracting product data from commercial web pages. Our web mining method involves the following two phrases: First, it analyzes the data information located at the leaf node of DOM tree structure of the web page, generates the semantic information vector for other nodes of the DOM tree and find maximum repeat semantic vector pattern. Second, it identifies the product data region and data records, builds a product object template by using semantic tree matching technique and uses it to extract all product data from the web page. The main contribution of this study is in developing a fully automated approach to extract product data from the commercial sites without any user's assistance. Experiment results show that the proposed technique is highly effective. © Springer-Verlag 2012.

Sequencing by hybridization is a promising cost-effective technology for high-throughput DNA sequencing via microarray chips. However, due to the effects of spectrum errors rooted in experimental conditions, an accurate and fast reconstruction of original sequences has become a challenging problem. In the last decade, a variety of analyses and designs have been tried to overcome this problem, where different strategies have different trade-offs in speed and accuracy. Motivated by the idea that the errors could be identified by analyzing the interrelation of spectrum elements, this paper presents a constructive heuristic algorithm, featuring an accurate reconstruction guided by a set of well-defined criteria and rules. Instead of directly reconstructing the original sequence, the new algorithm first builds several accurate short fragments, which are then carefully assembled into a whole sequence. The experiments on benchmark instance sets demonstrate that the proposed method can reconstruct long DNA sequences with higher accuracy than current approaches in the literature.

Glomeruli extraction is a vital step in computer-aided diagnosis systems of kidney disease. Since there are not only glomeruli but also other tissues in an image, when detecting the edges of glomeruli, lot of noises caused by other tissues will be detected at the same time. These noises cause discontinuous edges of glomeruli when some operation, such as labeling, is applied to denoise. According to this characteristic, this article proposes a contour extraction method based on genetic algorithm (GA) for edge patching. First, a Canny operator is applied to obtain the edges of glomeruli with noises. Then labeling and other operations such as dilation, thinning and cross-point deletion are applied to markedly remove the noises. After the above operations, GA is finally used to search for optimal patching segments to join the discontinuous edges together and a closed curve with highest fitness would be able to form the contour of glomeruli. Experimental results show that the proposed method performs well for the renal biopsy images. (C) 2011 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

Polynomial NARX (nonlinear autoregressive with exogenous) model identification has received considerable attention in last three decades. However, in a high-order nonlinear system, it is very difficult to obtain the model structure directly even with state-of-art algorithms, because the number of candidate monomial terms is huge and increases drastically as the model order increases. Motivated by this fact, in this research, the identification is performed in two steps: firstly a prescreening process is carried out to select a reasonable number of important monomial terms based on two kinds of the importance indices. Then, in the reduced searching space with only the selected important terms, multi-objective evolutionary algorithm (MOEA) is applied to determine a set of significant terms to be included in the polynomial model with the help of independent validation data. In this way, the whole identification algorithm is implemented efficiently. Numerical simulations are carried out to demonstrate the effectiveness of the proposed identification method. (C) 2011 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

In this paper, quasi-ARX wavelet network (Q-ARX-WN) is proposed for nonlinear system identification. There are mainly two contributions are clarified. Firstly, compared with conventional wavelet networks (WNs), it is equipped with a linear structure, where WN is incorporated to interpret parameters of the linear ARX structure, thus Q-ARX-WN prediction model could be constructed and it is easy-to-use in nonlinear control. Secondly, guidelines for network construction are well considered due to the introduction of WNs, and Q-ARX-WN could be represented in a linear-in-parameter way. Therefore, linear support vector regression (SVR) based identification scheme may be introduced for the robust performance. Moreover, in adaptive control procedure, only linear parameters are needed to be adjusted when sudden changes have happened on the nonlinear system, thus the controller can track reference signal quickly. The effectiveness and robustness of the proposed nonlinear system identification method are validated by applying it to identify a real data system and a mathematical example, and an example of nonlinear system control is given to show usefulness of the proposed model.

The central criterion of feature selection is that good feature sets contain features that are highly correlated with the output, yet uncorrelated with each other. Based on this criterion, we address the problem of feature selection through correlation-based feature clustering and support vector machine (SVM) based feature ranking. Correlation-based clustering is proposed to group features into some clusters based on the correlation between two features. As a result, a feature is highly correlated to any other feature in the same cluster but uncorrelated to the features in other clusters. From each cluster, we select a feature as the delegate based on its influence quantities on the output. The influence quantities are measured by the feature sensitivity in the SVM. The proposed approach can identify relevant features and eliminate redundancy among them effectively. The effectiveness of the proposed approach is demonstrated through comparisons with other methods using real-world data with different dimensions. (C) 2011 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

Nonlinear polynomial NARX model identification often faces the problem of huge pool of candidate terms, which makes the evolutionary optimization based identification algorithm work with low efficiency. This paper proposes an efficient identification scheme with pre-processing to reduce the searching space effectively. Both the input selection and term selection are implemented to truncate the candidate pool with the help of correlation based orthogonal forward selection (COFS) algorithm and simplified orthogonal least square (OLS) algorithm, respectively. Then multi objective evolutionary algorithm (MOEA) is used to identify the polynomial model in a relative small searching space.

paper proposes a niching EDA based on clustering and fitness sharing for solving off-lattice protein folding model. Firstly, AP clustering is used to adaptively partition the population into niches during a run of EDA. A cluster is seen as a niche, the niche number and the niche radiuses may vary obviously for different generations. Secondly, fitness sharing strategy is used for reserving the diversity of population to enhance the performance of EDA. Experiment results on the artificial and real protein sequences show that the proposed niching EDA is effective to solve the off-lattice AB model and outperforms other evolutionary-based methods.

Nonlinear polynomial NARX model identification often faces the problem of huge pool of candidate terms, which makes the evolutionary optimization based identification algorithm work with low efficiency. This paper proposes an efficient identification scheme with pre-processing to reduce the searching space effectively. Both the input selection and term selection are implemented to truncate the candidate pool with the help of correlation based orthogonal forward selection (COFS) algorithm and simplified orthogonal least square (OLS) algorithm, respectively. Then multi objective evolutionary algorithm (MOEA) is used to identify the polynomial model in a relative small searching space.

This paper proposes a multi-input and multi-output (MIMO) quasi-ARX model and a multivariable decoupling proportional-integral-differential (PID) controller for MIMO nonlinear systems based on the proposed model. The proposed controller consists of a traditional PID controller with a decoupling compensator and a feed-forward compensator for the nonlinear dynamics from the MIMO quasi-ARX model. Then an adaptive control algorithm is presented using the MIMO quasi-ARX Radial Basis Function Network (RBFN) prediction model. Simulation results show the effectiveness of the proposed control method.

Quasi-ARX neurofuzzy (Q-ARX-NF) models have shown great approximation ability and usefulness in nonlinear system identification and control. However, the incorporated neurofuzzy networks suffer from the curse-of-dimensionality problem, which may result in high computational complexity and over-fitting. In this paper, support vector regressor (SVR) based identification approach is used to reduce computational complexity with the help of transfonning the original problem into Lagrange space, which is only sensitive to the number of data samples. Furthermore, to improve the generalization capability, a parsimonious model structure is obtained by eliminating insignifuicant input variables for the incorporated neurofuzzy network, which is implemented by genetic algorithm (GA) based input selection method with a novel fitness evaluation function. Two numerical simulations are tested to show the effectiveness of the proposed method.

Missing value imputation for microarray data is important for gene expression analysis algorithms, such as clustering, classification and network design. A number of algorithms have been proposed to solve this problem, but most of them are only limited in linear analysis methods, such as including the estimation in the linear combination of other no-missing-value genes. It may result from the fact that microarray data. often comprises of huge size of genes with only a small number of observations, and nonlinear regression techniques are prone to overfitting. In this paper, a quasi-linear SVR model is proposed to improve the linear approaches, and it can be explained in a piecewise linear interpolation way. Two real datasets are tested and experimental results show that the quasi-linear approach for missing value imputation outperforms both the linear and nonlinear approaches.

In this paper, a multi-label classification method based on label ranking and delicate boundary Support Vector Machine (SVM) is proposed for solving the functional genomics applications. Firstly, an improved probabilistic SVM with delicate decision boundary is used as scoring approach to obtain a proper label rank. Secondly, an instance-dependent thresholding strategy is proposed to decide classification results. A d-folds validation approach is utilised to determine a set of target thresholds for all training samples as teachers, then an appropriate instance-dependent threshold for each testing instance is obtained by applying k-Nearest Neighbours (KNN) strategy on this teacher threshold set. © 2010 Inderscience Enterprises Ltd.

Protein structure prediction (PSP) is one of the most important problems in computational biology. This paper proposes a novel hybrid estimation of distribution algorithm (EDA) to solve the PSP problem on HP model. First, a composite fitness function containing the information cif folding structure core (I-I-core) is introduced to replace the traditional fitness function of HP model. The proposed fitness function is expected to select better individuals for the probabilistic model of EDA. Second, local search with guided operators is utilized to refine the found solutions for improving the efficiency of EDA. Third, an improved backtracking-based repairing method is proposed to repair invalid individuals sampled by the probabilistic model of EDA. It can significantly reduce the number of backtracking searching operation and the computational cost for a long-sequence protein. Experimental results demonstrate that the proposed method outperform the basic EDA method. At the same time, it is very competitive with other existing algorithms for the PSP problem on lattice HP models. (C) 2010 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

Multi-label classification problem is an extension of traditional multi-class classification problem in which the classes are not mutually exclusive and each sample may belong to several classes simultaneously. Such problems occur in many important applications. Some researches indicate that the performance of classifier can be improved by introducing the information of multi-label training samples into learning procedure effectively. In this paper, we propose a novel method based on SVM with delicate decision boundary. For the basic overlapping problem of two labels, characteristics of double-label samples are utilized to obtain the range of overlapping sample space decided by two binary SVM classifier separating surfaces. And a bias model with delicate decision boundary is built for samples in overlapping sample space to improve the classification accuracy. Experimental results on the benchmark datasets of Yeast and Scene show that our proposed method improves the classification accuracy efficiently, compared with the basic binary SVM method and some existing well-known methods.

Sequencing by hybridization is a promising cost-effective technology for high-throughput DNA sequencing via microarray chips. However, due to the effects of spectrum errors rooted from experimental conditions, a fast and accurate reconstruction of original sequences has become a challenging problem. In the last decade, a variety of analyses and designs have been tried to overcome this problem, where different strategies have different tradeoffs in speed and accuracy. Motivated by the idea that the errors could be identified by analyzing the interrelation of spectrum elements, this paper presents a new constructive heuristic algorithm, featuring an accurate reconstruction guided by a set of well-defined criteria and rules. The experiments on benchmark instance sets demonstrate that the proposed method can reconstruct long DNA sequences more accurately than current approaches in the literature. © 2010 IEEE.

In this paper, we propose a Hierarchical Frequency Sensitive Competitive Learning (HFSCL) method to achieve Color Quantization (CQ). In HFSCL, the appropriate number of quantized colors and the palette can be obtained by an adaptive procedure following a binary tree structure with nodes and layers. Starting from the root node that contains all colors in an image until all nodes are examined by split conditions, a binary tree will be generated. In each node of the tree, a Frequency Sensitive Competitive Learning (FSCL) network is used to achieve two-way division. To avoid over-split, merging condition is defined to merge the clusters that are close enough to each other at each layer. Experimental results show that the proposed HFSCL has desired ability for CQ.

Estimation of Distribution Algorithms (EDAs) still suffer from the drawback of premature convergence for solving the optimization problems with irregular and complex multimodal landscapes. In this paper, we propose an adaptive niching EDA based on Affinity Propagation (AP) clustering analysis. The AP clustering is used to adaptively partition the niches and mine searching information from the evolution process. The obtained information is successfully utilized to improve the EDA performance by a balance niching searching strategy. Two different categories of optimization problems are used to evaluate the proposed adaptive niching EDA. The first is the continuous EDA based on single Gaussian probabilistic model to solve two benchmark functional multimodal optimization problems. The second is a real complicated discrete EDA optimization problem, the protein 3-D HP model based on k-order Markov probabilistic model. The experiment studies demonstrate that the proposed adaptive niching EDA is an efficient method.

In this paper, an improved multi-label classification is proposed based on label ranking and delicate decision boundary SVM. Firstly, an improved probabilistic SVM with delicate decision boundary is used as the scoring method to obtain a proper label rank. It can improve the probabilistic label rank by introducing the information of overlapped training samples into learning procedure. Secondly, a threshold selection related with input instance and label rank is proposed to decide the classification results. It can estimate an appropriate threshold for each testing instance according to the characteristics of instance and label rank. Experimental results on four multi-label benchmark datasets show that the proposed method improves the performance of classification efficiently, compared with binary SVM method and some existing well-known methods.

Brain-Computer Interface (BCI) is a system provides an alternative communication and control channel between the human brain and computer. In Motor Imagery-based (MI) BCI system, Common Spatial Pattern (CSP) is frequently used for extracting discriminative patterns from the electroencephalogram (EEG). There are many studies have proven that the performance of CSP has a very important relation with the choice of operational frequency band. As the fact that the CSP features at different frequency bands contain discriminative and complementary information for classification, this paper proposes a new frequency band selection method to nd the best frequency band set on which subject-speci cs CSP are complementary for MI classi cation. Compared to the performance offered by the existing method based on frequency band partition, the proposed algorithm can yield error rate reductions of 49.70% for the same BCI competition dataset.

This paper presents a novel approach for designing adaptive controller of nonlinear dynamical systems based on an improved quasi-ARX neural network prediction model. The improved quasi-ARX neural network prediction model has two parts: the linear part is used for stability and the nonlinear part is used to satisfy accuracy requirement. Then, we can obtain a linear controller and a nonlinear controller based on the characteristic of the improved quasi-ARX neural network prediction model. A fuzzy switching algorithm is designed between the two controllers. Theory analysis and simulations are given to show the effectiveness of the proposed method both on stability and accuracy.

In this paper, we propose a hierarchical frequency sensitive competitive learning (HFSCL) method to achieve color quantization (CQ). In HFSCL, the appropriate number of quantized colors and the palette can be obtained by an adaptive procedure following a binary tree structure with nodes and layers. Starting from the root node that contains all colors in an image until all nodes are examined by split conditions, a binary tree will be generated. In each node of the tree, a frequency sensitive competitive learning (FSCL) network is used to achieve two-way division. To avoid over-split, merging condition is defined to merge the clusters that are close enough to each other at each layer. Experimental results show that HFSCL has the desired ability for CQ. © 2010 IEEE.

Multi-label classification is an extension of traditional classification problem in which each instance is associated with a set of labels. Recent research has shown that the ranking approach is an effective way to solve this problem. In the multi-labeled sets, classes are often related to each other. Some implicit constraint rules are existed among the labels. So we present a novel multi-label ranking algorithm inspired by the pairwise constraint rules mined from the training set to enhance the existing method. In this method, one-against-all decomposition technique is used firstly to divide a multi-label problem into binary class sub-problems. A rank list is generated by combining the probabilistic outputs of each binary Support Vector Machine (SVM) classifier. Label constraint rules are learned by minimizing the ranking loss. Experimental performance evaluation on well-known multi-label benchmark datasets show that our method improves the classification accuracy efficiently, compared with some existed methods.

Estimation of Distribution Algorithms (EDAs) still suffer from the drawback of premature convergence for solving the optimization problems with irregular and complex multimodal landscapes. In this paper, we propose an adaptive niching EDA based on Affinity Propagation (AP) clustering analysis. The AP clustering is used to adaptively partition the niches and mine searching information from the evolution process. The obtained information is successfully utilized to improve the EDA performance by a balance niching searching strategy. Two different categories of optimization problems are used to evaluate the proposed adaptive niching EDA. The first is the continuous EDA based on single Gaussian probabilistic model to solve two benchmark functional multimodal optimization problems. The second is a real complicated discrete EDA optimization problem, the protein 3-D HP model based on k-order Markov probabilistic model. The experiment studies demonstrate that the proposed adaptive niching EDA is an efficient method.

This paper presents a novel image edge detection method based on a simplified pulse coupled neural network with anisotropic interconnections (PCNNAI) by applying an anisotropic linking mechanism. PCNNAI utilizes the anisotropic linking mechanism to create an adaptive synaptic weight matrix to achieve the anisotropic interconnection model among neurons. Therefore, the neurons corresponding to edge and non-edge pixels will receive different feedback signal from neighborhood. Due to the PCNN structure the edges will be detected by different internal activity of edge neurons and non-edge neurons. Comparing with conventional PCNN edge detection methods, PCNNAI simplifies the system structure and the outputs are controllable, meanwhile PCNNAI also achieves more accurate results than the classical image edge detectors. Experimental results show that PCNNAI is effective at image edge detection. © 2010 IEEE.

This paper proposes an improved Hierarchical Multi-label Classification (HMC) method for solving the gene function prediction. The HMC task is transferred into a series of binary SVM classification tasks. By introducing the hierarchy constraint into learning procedures, two measures with incorporating prior information are implemented to improve the HMC performance. Firstly, for imbalanced functional classes, a hierarchical SMOTE is proposed as over-sampling preprocessing to improve the SVM learning performance. Secondly, an improved True Path Rule consistency approach is introduced to ensemble the results of binary probabilistic SVM classifications. It can improve the classification results and guarantee the hierarchy constraint of classes. © 2010 IEEE.

This paper proposes a switching scheme for salt and pepper noise reduction by combining a noise detection algorithm based on a simplified pulse coupled neural network (PCNN) with a simple adaptive median filter. The simplified PCNN utilizes an adaptive synaptic weight matrix created by anisotropic linking mechanism to achieve anisotropic linking model, that is the interconnections between neurons with large absolute difference in intensity will be interrupted. Therefore, the neurons corresponding to noise corrupted pixels will receive smaller feedback signal from the neighborhood and generate smaller internal activities compare with the ones corresponding to noise free pixels. The impulse will be detected by setting an appropriate dynamic threshold. After the PCNN based noise detection scheme, the pixels contaminated by salt and pepper noise will be restored by a simple adaptive median filter. Experimental results prove that the proposed switching median filter outperform over the conventional methods in both noise reduction and detail preserving. © 2010 IEEE.

This paper proposes a local linear multi-SVM method based on composite kernel for solving classification tasks in gene function prediction. The proposed method realizes a nonlinear separating boundary by estimating a series of piecewise linear boundaries. Firstly, according to the distribution information of training data, a guided partitioning approach composed of separating boundary detection and clustering technique is used to obtain local subsets, and each subset is utilized to capture prior knowledge of corresponding local linear boundary. Secondly, a composite kernel is introduced to realize the local linear multi-SVM model. Instead of building multiple local SVM models separately, the prior knowledge of local subsets is used to construct a composite kernel, then the local linear multi-SVM model is realized by using the composite kernel exactly in the same way as a single SVM model. Experimental results on benchmark datasets demonstrate that the proposed method improves the classification performance efficiently. © 2010 IEEE.

In this paper, a switching adaptive control method is presented for nonlinear system. The proposed switching adaptive control law is based on a quasi-ARX Radial Basis Function Network model and a switching mechanism. A d-difference operator is used to relax the globally assumption of nonlinear system. Theory analysis and simulations show the effectiveness of the proposed control method on stability and accuracy.

Image thresholding is a useful method in many image processing and computer vision applications. Otsu method is one of popular thresholding methods and has been frequently used as a classical technique in various applications. A premise of this method is that the probability of edge region is supposed to be zero. However, the premise is not proper in some situations such as the images are corrupted by noises. To solve this problem, a curvilinear thresholding method (CTM) is proposed based on the traditional Otsu method. We give a recursive algorithm of line thresholding method (LTM), which is a particular case of CTM, to verify our ideas. In addition, Otsu based method has another weakness that is it gives satisfactory results only when the numbers of pixels in each class are close to each other. Otherwise the threshold will be biased. This paper introduces a two-dimensional (2D) histogram projection method to correct the Otsu threshold. A. fast algorithm for searching the valley of one-dimensional (1D) projected histogram is also given based on wavelet transform. Experimental results show that the proposed method performs much better than the traditional Otsu method.

In this research we design a network administrator assistance system based on traffic measurement and fuzzy c-means (FCM) clustering analysis method. Network traffic measurement is an essential tool for monitoring and controlling communication network. It can provide valuable information about network traffic-load patterns and performances. The proposed system utilizes the FCM method to analyze users' network behaviors and traffic-load patterns based on traffic measurement data of IP network. Analysis results can be used as assistance for administrator to determine efficient controlling and configuring parameters of network management systems. The system is applied in Dali University campus network, and it is effective in practice.

The protein structure prediction (PSP) problem is one of the most important problems in computational biology. This paper proposes a novel Estimation of Distribution Algorithms (EDAs) based method to solve the PSP problem on HP model. Firstly, a composite fitness function containing the information of folding structure core formation is introduced to replace the traditional fitness function of RP model. It can help to select more optimum individuals for probabilistic model of EDAs algorithm. And a set of guided operators are used to increase the diversity of population and the likelihood of escaping from local optima. Secondly, an improved backtracking repairing algorithm is proposed to repair invalid individuals sampled by the probabilistic model of EDAs for the long sequence protein instances. A detection procedure of feasibility is added to avoid entering invalid closed areas when selecting directions for the residues. Thus, it can significant reduce the number of backtracking operation and the computational cost for long sequence protein. Experimental results demonstrate that the proposed method outperform the basic EDAs method. At the same time, it is very competitive with the other existing algorithms for the PSP problem on lattice HP models.

Glomerulus is the filtering unit of the kidney. In the computer aided diagnosis system designed for kidney disease, glomerulus extraction is an important step for analyzing kidney-tissue image. Against the disadvantages of traditional methods, this paper proposes a glomerulus extraction method using genetic algorithm for edge patching. Firstly, Canny edge detector is applied to get discontinuous edges of glomerulus. After labeling to remove the noises, genetic algorithm is used to search for optimal patching segments to join those edges together. Lastly, the edges and the patching segments with high fitness would be able to form the whole edge of the glomerulus. Experiments and comparisons indicate the proposed method can extract the glomerulus from kidney-tissue image both fast and accurately.

In a standard support vector machine (SVM), the training process has O(n(3)) time and O(n(2)) space complexities, where it is the size of training dataset. Thus, it is computationally infeasible for very large datasets. Reducing the size of training dataset is naturally considered to solve this problem. SVM classifiers depend on only support vectors (SVs) that lie close to the separation boundary. Therefore, we need to reserve the samples that are likely to be SVs. In this paper, we propose a method based on the edge detection technique to detect these samples. To preserve the entire distribution properties, we also use a clustering algorithm such as K-means to calculate the centroids of clusters. The samples selected by edge detector and the centroids of clusters are used to reconstruct the training dataset. The reconstructed training dataset with a smaller size makes the training process much faster, but without degrading the classification accuracies.

In this paper, an automatic segmentation method based on self-organizing feature map (SOFM) neural network (NN) is presented for color images. First, a binary tree clustering procedure is used to cluster the colors in an image. In each node of the tree, a SOFM NN is used as a classifier which is fed by image color values. The output neurons of the SOFM NN define the color classes for each node. In our method, the number of color classes for each node is two. For each node of the tree, Hotelling transform based splitting condition is used to define if the current color classes should be split. To speed up the entire algorithm, a nearest neighbor interpolation is used to get the small training set for SOFM NN. Once the colors in an image are clustered, it is easy to segment a target by analyzing the color feature in an image. The method is independent of the color scheme, so it is applicable to any type of color images. Our experimental results show the validity of the proposed method.

Protein structure prediction (PSP) is one of the most important problems in computational biology And it also is a very difficult optimization task, especially for long sequence instances. This paper proposes a novel clustering based niching EDA for HP model folding problem. The EDA individuals are clustered by the affinity propagation clustering method before submitting them to niching clearing. A cluster can be seen. as a niche in clearing procedure. The niche clearing radius can be adaptively determined by clustering. And an approach based on Boltzmann scheme is proposed to determine the niche capacity according to the adaptive clearing radius and niche fitness. Experimental results demonstrate that the proposed method outperforms the basic EDAs method. At the same time, it is very competitive with other existing algorithms for the PSP problem on lattice HP models.

A two-step identification method for nonlinear polynomial model using Evolutionary Algorithm (EA) Is proposed in this paper, and the method has the ability to select a parsimonious structure from a very large pool of model terms. In a nonlinear polynomial model, the number of candidate monomial terms increases drastically as the order of polynomial model increases, and it is impossible to obtain the accurate model structure directly even with state-of-art algorithms. The proposed method firstly carries out a pre-screening process to select a reasonable number of important monomial terms based on the importance index. In the next step, EA is applied to determine a set of significant terms to be included in the polynomial model. In this way, the whole identification algorithm is implemented very efficiently. Numerical simulations are carried out to demonstrate the effectiveness of the proposed identification method.

When a linear model is used for controlling nonlinear systems solely, it can't satisfy accuracy requirement. Whereas, although a neural network can deal with the accuracy problem, it may lead to instability. In this paper, an adaptive controller is proposed for nonlinear dynamical systems based on linear model and quasi-ARX neural network model. A switching algorithm is designed between the linear and nonlinear models. Theory analysis and simulations are given to show the effectiveness of the proposed method both on stability and accuracy.

Feature selection is a process to select a subset of original features. It can improve the efficiency and accuracy by removing redundant and irrelevant terms. Feature selection is commonly used in machine learning, and has been wildly applied in many fields, we propose a new feature selection method. This is an integrative hybrid method. It first uses Affinity Propagation and SVM sensitivity analysis to generate feature subset, and then use forward selection and backward elimination method to optimize the feature subset based on feature ranking. Besides, we apply this feature selection method to solve a new problem, Human resource selection. The data is acquired by questionnaire survey. The simulation results show that the proposed feature selection method is effective, it not only reduced human resource features but also increased the classification performance.

Pattern theorem in financial time-series is one of the most important technical analysis methods in financial prediction. Recent researches have achieved big progresses in identifying and recognizing time-series patterns. And most of the recent works on time-series deal with this task by using static approaches and mainly focus on the recognition accuracy, but considering that recognition of patterns in financial time-series, especially for stock time-series, are always time-consuming rather than pattern recognition in other fields, a dynamic recognition approach is more preferable so that investment on stock pattern become executable. In this paper we propose a dynamic approach for extracting and recognizing the patterns in stock-series. In our approach a slide window with flexible length is defined for extracting feature vertexes in stock-series, and in addition, a dynamic perceptual important point (PIP) locating method is proposed based on the PIP locating method for avoiding the computation expense problem and an artificial neural network (ANN) approach is involved for pattern recognition and window length identification.

Universal Learning Networks (ULNs) provide a generalized framework for many kinds of structures in neural networks with supervised learning. Multi-Branch Neural Networks (MBNNs) which use the Framework of ULNs have already been shown to have better representation ability in feedforward neural networks (FNNs). The multi-branch structure of MBNNs can he easily extended to recurrent neural networks (RNNs) because the characteristics of ULNs include the connection of multiple branches with arbitrary time delays. In this paper, therefore. RNNs with multi-branch structure are proposed and are shown to have better representation ability than conventional RNNs. RNNs can represent dynamical systems and are useful for time series prediction. The performance evaluation of RNNs with multi-branch structure was carried out using a benchmark of time series prediction. Simulation results showed that RNNs with multi-branch structure could obtain better performance than conventional RNNs, and also showed that they could improve the representation ability even if they are smaller-sized networks (C) 2009 Wiley Periodicals, Inc. Electron Comm Jpn 91(9): 37-44, 2008; Published Online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ecj.10157

A method of class association rule mining from incomplete databases is proposed using Genetic Network Programming (GNP). GNP is one of the evolutionary optimization techniques, which uses the directed graph structure. An incomplete database includes missing data in some tuples, however, the proposed method can extract important rules using these tuples, and users can define the conditions of important rules flexibly. Generally, it is not easy for Aprior-like methods to extract important rules from incomplete database, so we have estimated the performances of the rule extraction and classification of the proposed method using incomplete data set. The results showed that the accuracy of classification of the proposed method is favorable even if some tuples include missing data.

An efficient algorithm for building a classifier is proposed based on an important association rule mining using genetic network programming (GNP). The proposed method measures the significance of the association via the chi-squared test. Users can define the conditions of important association rules for building a classifier flexibly. The definition can include not only the minimum threshold chi-squared value, but also the number of attributes in the association rules. Therefore, all the extracted important rules can be used for classification directly. GNP is one of the evolutionary optimization techniques, which uses the directed graph structure as genes. Instead of generating a large number of candidate rules, Our method can obtain a sufficient number of important association rules for classification. In addition, our method suits association rule mining from dense databases such as medical datasets, where many frequently occurring items are found in each tuple. In this paper, we describe an algorithm for classification using important association rules extracted by GNP with acquisition mechanisms and present some experimental results of medical datasets. (C) 2008 Wiley Periodicals, Inc. Electron Comm Jpn, 91(2): 46-54, 2008; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10. 1002/ecj.10022

According to Hebb's cell assembly theory, the brain has the capability of function localization. On the other hand, it is suggested that in the brain there are three different learning paradigms: supervised, unsupervised, and reinforcement learning, which are related deeply to the three parts of brain: cerebellum, cerebral cortex, and basal ganglia, respectively. Inspired by the above knowledge of the brain in this paper we present a brainlike learning system consisting of three parts: supervised learning (SL) part, unsupervised learning (UL) part, and reinforcement learning (RL) part. The SL part is a main part learning inputoutput mapping
the UL part is a competitive network dividing input space into subspaces and realizes the capability of function localization by controlling firing strength of neurons in the SL part based on input patterns
the RL part is a reinforcement learning scheme, which optimizes system performance by adjusting the parameters in the UL part. Numerical simulations have been carried out and the simulation results confirm the effectiveness of the proposed brainlike learning system. © 2007 Wiley Periodicals, Inc.

Neural networks (NNs) can solve only simple problems if the network size is too small, but increasing the network size is costly in terms of memory space and calculation time. Thus, we have studied how to construct a network structure with high performance and low cost in space and time. One solution is a multibranch structure. Conventional NNs use the single-branch structure for connections, while the multibranch structure has multiple branches between nodes. In this paper, a new method which enables the multibranch NNs to have the localized property is proposed. It is well known that RBF networks have the localized property, which makes it possible to approximate functions faster than sigmoidal NNs. By using the multibranch structure having the localized property of RBF networks, NNs can obtain superior performance while maintaining lower costs in space and time. Simulation results of function approximations and a classification problem are presented to illustrate the effectiveness of multibranch NNs. (c) 2007 Wiley Periodicals, Inc.

The presence of similar patterns in regulatory sequences may aid users in identifying co-regulated genes or inferring regulatory modules. By modelling pattern occurrences in regulatory regions with Poisson statistics, this paper presents a log likelihood ratio statistics-based distance measure to calculate pair-wise similarities between regulatory sequences. We employed it within three clustering algorithms: hierarchical clustering, Self-Organising Map, and a self-adaptive neural network. The results indicate that, in comparison to traditional clustering algorithms, the incorporation of the log likelihood ratio statistics-based distance into the learning process may offer considerable improvements in the process of regulatory sequence-based classification of genes. © 2008 Inderscience Enterprises Ltd.

This paper presents an approach to the multiple sequence alignment (MSA) problem by applying genetic algorithms with a reserve selection mechanism. MSA is one of the most fundamental operations in bioinformatics, which plays an important part in predicting the structure, function and evolution of biological sequences. In order to solve the MSA problem efficiently, genetic algorithms (GAs) were applied. As the number and length of sequences increase, however, GAs are usually suffered from premature convergence where they are easily trapped into local optima. In this paper, we employ the reserve selection that is a new selection scheme to avoid premature convergence in GAs. Empirical studies demonstrate that genetic algorithms with reserve selection (GARS) could bring about a rise in the quality of multiple sequence alignment when compared with standard GAs.

Deceptive problems are a class of challenging problems for conventional genetic algorithms (GAs), which usually mislead the search to some local optima rather than the global optimum. This paper presents an improved genetic algorithm with reserve selection to solve deceptive problems. The concept "potential" of individuals is introduced as a new criterion for selecting individuals for reproduction, where some individuals with low fitness are also let survive only if they have high potentials. An operator called adaptation is further employed to release the potentials for approaching the global optimum. Case studies are done in two deceptive problems, demonstrating the effectiveness of the proposed algorithm.

An efficient algorithm for mining important association rule from multi-relational database using distributed mining ideas. Most existing data mining approaches look for rules in a single data table. However, most databases are multi-relational. In this paper, we present a novel distributed data-mining method to mine important rules in multiple tables (relations) and combine the method with genetic algorithm to enhance the mining efficiency. Genetic algorithm is in charge of finding antecedent rules and aggregate of transaction set that produces the corresponding rule from the chief attributes. Apriori and statistic method is in charge of mining consequent rules from the rest relational attributes of other tables according to the corresponding transaction set producing the antecedent rule in a distributed way. Our method has several advantages over most exiting data mining approaches. First, it can process multi-relational database efficiently. Second, rules produced have finer pattern. Finally, we adopt a new concept of extended association rules that contain more import and underlying information.

This paper presents a novel support vector regression (SVR) network for financial time series prediction. The SVR network consists of two layers of SVR: transformation layer and prediction layer. The SVRs in the transformation layer forms a modular network; but distinguished with conventional modular networks, the partition of the SVR modular network is based on the output domain that has much smaller dimension. Then the transformed outputs from the transformation layer are used as the inputs for the SVR in prediction layer. The whole SVR network gives an online prediction of financial time series. Simulation results on the prediction of currency exchange rate between US dollar and Japanese Yen show the feasibility and the effectiveness of the proposed method.

One of the central problems of functional genomics is gene classification. Microarray data are currently a major source of information about the functionality of genes. Various mathematical techniques, such as neural networks (NNs), self-organizing map (SOM) and several statistical methods, have been applied to classify the data in attempts to extract the underlying knowledge. As for conventional classification, the problem mainly addressed so far has been how to classify the multi-label gene data and how to deal with the imbalance problem. In this paper, we proposed an improved support vector machine (SVM) classifier with soft decision boundary. This boundary is a classification boundary based on belief degrees of data. The boundary can reflect the distribution of data, especially in the mutual part between classes and the excursion caused by the data imbalance.

propose an efficient data mining system for making quick response to users and providing a friendly interface. When data tuples have higher relationship, it could contain long frequent itemsets. If apriori algorithm mines all frequent itemsets in those tuples, its candidate itemsets will become very huge and it has to scan database huge times. Meanwhile, the number of rules mined by the apriori algorithm is huge. Our method avoids mining rules through huge candidate itemsets, just mines maximal frequent itemsets and scans the database for the frequent itemsets users are interested in. First, use GA to mine the maximal frequent itemsets and show them to users. Second, let users pick up one to deduce the association rules. Final, scan the database for the real support and confidence and show them to users. So, our method can not only save many times scanning the database and make quick response to users, but provide a friendly interface that let users select his interesting rules to mine.

Glomerulus extraction is an important step for automatic analysis of the kidney diseases in the computer-aided diagnosis system. A method based on searching the best fitting curve is proposed based on the characteristics of the renal biopsy images in microscope. This method can solve the problem of the large defect of the enhanced boundary, which lead to unsuccessful extraction. Firstly, a parametric equation is constructed based on the cubic spline interpolation function to draw the closed fitting curve. Secondly, the different scale binary images can be obtained by adjusting the parameters of the LOG filter. Finally, after labeling to remove the noises and thinning, a genetic algorithm is used to search the best fitting curve for glomerulus boundary. Experimental results indicate the correctness and effectiveness of this method.

Image segmentation, which is the first essential and fundamental issue in the image analysis and pattern recognition, is a classical difficult problem in image processing. In the computer-aided diagnosis system of the renal biopsy images in microscope, the correct segmentation of glomerulus is an important step for automatic analysis. Complex characteristics of renal biopsy images lead to the difficulty in boundary features description. A kind of feature operator based on the definition of the cavum boundary is proposed in this paper. According to this operator, a nonlinear thresholding surface can be constructed by neural network, and the appropriate surface can be selected to enhance the cavum boundary by the fault tolerance analysis. After denoising, the segmentation results can be obtained. Experimental results indicate that this method can enhance the boundary and suppress noises at the same time; it can obtain good segmented results and has a fine adaptability to various sample images.

Image segmentation plays an important role in image analysis and computer vision system. Among all segmentation techniques, the automatic thresholding methods are widely used because of their advantages of simple implement and time saving. Otsu method is one of thresholding methods and frequently used in various fields. Two-dimensional (2D) Otsu method behaves well in segmenting images of low signal-to-noise ratio than one-dimensional (1D). But it gives satisfactory results only when the numbers of pixels in each class are close to each other. Otherwise, it gives the improper results. In this paper, 2D histogram projection is used to correct the Otsu threshold. The 1D histograms are acquired by 2D histogram projection in x and y axes and a fast algorithm for searching the extrema of the projected histogram is proposed based on the wavelet transform in this paper. Experimental results show that the proposed method performs better than the traditional Otsu method for our renal biopsy samples. © 2008 IEEE.

To analyze the nuclei distributed in the glomeruli of the renal biopsy images automatically, the problem of missing or wrong extraction of them caused by the complex characteristics of the renal biopsy images must be solved. This paper introduces a dynamic thresholding method using eigenvalue feedback strategy based on multi-channel information. Firstly, the nonlinear thresholding surface adjusted by area eigenvalue feedback strategy is used to R-channel images of RGB color space and C-channel images of CMYK color space respectively to obtain the binary images. Through fusing the two images, the undetermined regions, which consist of the genuine and spurious nucleus, can be marked. And the reality degree is calculated to distinguish which one is genuine. Secondly, an area proportionality eigenvalue is considered in feedback strategy to divide the nuclei clusters into single ones. Then the nuclei extraction can be realized. The experimental results demonstrate the effectiveness of this method. © 2008 IEEE.

The discrete particle swarm optimization (DPSO) is a kind of particle swarm optimization (PSO) algorithm to find optimal solutions for discrete problems. This paper proposes an improved DPSO based on cooperative swarms, which partition the search space into lower dimensional subspaces. The k-means split scheme and regular split scheme are applied to split the solution vector into swarms. Then the swarms optimize the different components of the solution vector cooperatively. Some strategies are further used to improve the accuracy and convergence. Application of the proposed cooperative swarms based DPSO (CDPSO) on the traveling salesman problem (TSP) shows a significant improvement over conventional DPSOs. © 2008 IEEE.

Universal Learning Networks (ULNs) provide a generalized framework to many kinds of structures of neural networks with supervised learning. Multi-Branch Neural Networks (MBNNs) which use the framework of ULNs have been already shown that they have better representation ability in feedforward neural networks (FNNs). Multi-Branch structure of MBNNs can be easily extended to recurrent neural networks (RNNs) because the characteristics of ULNs include the connection of multiple branches with arbitrary time delays. In this paper, therefore, RNNs with Multi-Branch structure are proposed and they show that their representation ability is better than conventional RNNs. RNNs can represent dynamical systems and are useful for time series prediction. The performance evaluation of RNNs with Multi-Branch structure was carried out using a benchmark of time series prediction. Simulation results showed that RNNs with Multi-Branch structure could obtain better performance than conventional RNNs, and also showed that they could improve the representation ability even if they are smaller sized networks.

This paper proposes a graph-based evolutionary algorithm called Genetic Network Programming (GNP). Our goal is to develop GNP, which can deal with dynamic environments efficiently and effectively, based on the distinguished expression ability of the graph (network) structure. The characteristics of GNP are as follows. 1) GNP programs are composed of a number of nodes which execute simple judgment/processing, and these nodes are connected by directed links to each other. 2) The graph structure enables GNP to re-use nodes, thus the structure can be very compact. 3) The node transition of GNP is executed according to its node connections without any terminal nodes, thus the pst history of the node transition affects the current node to be used and this characteristic works as an implicit memory function. These structural characteristics are useful for dealing with dynamic environments. Furthermore, we propose an extended algorithm, "GNP with Reinforcement Learning (GNP-RL)" which combines evolution and reinforcement learning in order to create effective graph structures and obtain better results in dynamic environments. In this paper, we applied GNP to the problem of determining agents' behavior to evaluate its effectiveness. Tileworld was used as the simulation environment. The results show some advantages for GNP over conventional methods. © 2007 by the Massachusetts Institute of Technology.

Elevator Group Supervisory Control System (EGSCS) is a very large scale stochastic dynamic optimization problem. Due to its vast state space, significant uncertainty and numerous resource constraints such as finite car capacities and registered hall/car calls, it is hard to manage EGSCS using conventional control methods. Recently, many solutions for EGSCS using Artificial Intelligence (AI) technologies have been reported. Genetic Network Programming (GNP), which is proposed as a new evolutionary computation method several years ago, is also proved to be efficient when applied to EGSCS problem. In this paper, we propose an extended algorithm for EGSCS by introducing Reinforcement Learning (RL) into GNP framework, and an improvement of the EGSCS' performances is expected since the efficiency of GNP with RL has been clarified in some other studies like tile-world problem. Simulation tests using traffic flows in a typical office building have been made, and the results show an actual improvement of the EGSCS' performances comparing to the algorithms using original GNP and conventional control methods. Furthermore, as a further study, an importance weight optimization algorithm is employed based on GNP with RL and its efficiency is also verified with the better performances.

Stability is one of the most important subjects in control systems. As for the stability of nonlinear dynamical systems, Lyapunov's direct method and linearized stability analysis method have been widely used. But, it is generally recognized that finding an appropriate Lyapunov function is fairly difficult especially for the nonlinear dynamical systems, and also it is not so easy for the linearized stability analysis to find the locally asymptotically stable region. Therefore, it is crucial and highly motivated to develop a new stability analysis method, which is easy to use and can easily study the locally asymptotically stable region at least approximately, if not exactly. On the other hand, as for the calculation of the higher order derivative, Universal Learning Networks ( ULNs) are equipped with a systematic mechanism that calculates their first and second order derivatives exactly.
So, in this paper, an approximate stability analysis method based on h approximation is proposed in order to overcome the above problems and its application to a nonlinear dynamical control system is discussed. The proposed method studies the stability of the original trajectory by investigating whether the perturbed trajectory can approach the original trajectory or not. The above investigation is carried out approximately by using the higher order derivatives of ULNs.
In summarizing the proposed method, firstly, the absolute values of the first order derivatives of any nodes of the trajectory with respect to any initial disturbances are calculated by using ULNs. If they approach zero at time infinity, then the trajectory is locally asymptotically stable. This is an alternative linearized stability analysis method for nonlinear trajectories without calculating Jacobians directly. In the method, the stability analysis of time-varying systems with multi-branches having any sample delays is possible, because the systems are modeled by ULNs. Secondly, the locally asymptotically stable region, where asymptotical stability is secured approximately, is obtained by finding the area where the first order terms of Taylor expansion are dominant compared to the second order terms with h approximation assuming that the higher order terms more than the third order are negligibly small in the area.
Simulations of an inverted pendulum balancing system are carried out. From the results of the simulations, it is clarified that the stability of the inverted pendulum control system is easily analyzed by the proposed method in terms of studying the locally asymptotically stable region. (c) 2005 Elsevier B. V. All rights reserved.

A new evolutionary method named 'Genetic Network Programming, GNP' has been proposed. GNP represents its solutions as directed graph structures which have some useful features inherently. For example, GNP has the implicit memory function which memorizes the past action sequences of agents, and GNP can re-use nodes repeatedly in the network flow, so very compact graph structures can be made. In this paper, the stock trading model using GNP with Candle Chart is proposed and its effectiveness is comfirmed by trading simulations.

According to Hebb's Cell assembly theory, the brain has the capability of function localization. On the other hand, it is suggested that the brain has three different learning paradigms: supervised, unsupervised and reinforcement learning. Inspired by the above knowledge of brain, we present a hierarchical learning system consisting of three parts: supervised learning (SL) part, unsupervised learning (UL) part and reinforcement learning (RL) part. The SL part is a main part learning input-output mapping; the UL part realizes the function localization of learning system by controlling firing strength of neurons in SL part based on input patterns; the RL part optimizes system performance by adjusting parameters in UL part. Simulation results confirm the effectiveness of the proposed hierarchical learning system.

This, palter investigates how genetic algorithms (GAs) can be improved to solve large-scale and complex problems more efficiently. First of all we review premature convergence, one of, the challenges confronted with when applying GAs to real-world problems. Next, some of the methods now available to prevent, premature convergence and their intrinsic defects are discussed. A qualitative analysis is then done oil the cause of premature convergence that is the loss of building blocks hosted in less-fit, individuals during the course of evolution. Thus, we propose a new improver - GAS with Reserve Selection (GARS), where a reserved area is set up to save, potential building, blocks and a selection mechanism based on individual uniqueness is employed to activate the potentials. Finally, case studies are done in a few standard problems well known in the literature, where the experimental results demonstrate the effectiveness and robustness of GARS in suppressing premature convergence, and also an enhancement is found in global optimization capacity.

This paper provides a deep insight into the performance of genetic algorithms with reserve selection (GARS), and investigates how parameters can be regulated to solve optimization problems more efficiently. First of all, we briefly present GARS, an improved genetic algorithm with a reserve selection mechanism which helps to avoid premature convergence. The comparable results to state-of-the-art techniques such as fitness scaling and sharing demonstrate both the effectiveness and the robustness of GARS in global optimization. Next, two strategies named Static RS and Dynamic RS are proposed for tuning the parameter reserve size to optimize the performance of GARS. Empirical studies conducted in several cases indicate that the optimal reserve size is problem dependent.

The Elevator Group Supervisory Control Systems (EGSCS) are the control systems that systematically manage three or more elevators in order to efficiently transport the passengers in buildings. Double-deck elevators, where two cages are connected with each other, are expected to be the next, generation elevator systems. Meanwhile, Destination Floor Guidance Systems (DFGS) are also expected in Double-Deck Elevator Systems (DDES). With these, the passengers could be served at two consecutive floors and could input, their destinations at elevator halls instead of conventional systems without DFGS. Such systems become more, complex than the traditional systems and require new control methods Genetic Network Programming (GNP), a graph-based evolutionary method, has been applied to EGSCS and its advantages are shown in some previous papers. GNP can obtain the strategy of a new hall call assignment to the optimal elevator because it, performs crossover and mutation operations to judgment nodes and processing nodes. In studies so far, the passenger's arrival has been assumed to take Exponential distribution for many years. In this paper, we have applied Erlang distribution and Binomial distribution in order to study how the passenger's arrival distribution affects EGSCS. We have found that the passenger's arrival distribution has great influence on EGSCS. It has been also clarified that GNP makes good performances under different conditions.

Particle swarm optimization (PSO) is a kind of evolutionary algorithm to find optimal (or near optimal) solutions for numerical and qualitative problems. Recently, a new variation on the traditional PSO algorithm, called cooperative particle swarm optimization (CPSO), has been proposed, employing cooperative behavior to significantly improve the performance of the original algorithm. However, a standard CPSO is focused only on continuous problems. In this paper, we present a new approach based on the CPSO to solve combination optimization problems by introducing dynamic splitting schemes. Reverse operation and simulated annealing techniques are further used to prevent the algorithm from being trapped in local minima. Finally, Traveling salesman problem (TSP) is applied to show the effectiveness of the proposed PSO. © 2007 SICE.

Our brain has three different learning paradigms: supervised, unsupervised and reinforcement learning. And it is suggested that those learning paradigms relate deeply to the cerebellum, cerebral cortex and basal ganglia in the brain, respectively. Inspired by these knowledge of brain, we present a brain-like learning system with those three different learning algorithms. The proposed system consists of three parts: the supervised learning (SL) part, the unsupervised learning (UL) part and the reinforcement learning (RL) part. The SL part, corresponding to the cerebellum of brain, learns an input-output mapping by supervised learning. The UL part, corresponding to the cerebral cortex of brain, is a competitive learning network, and divides an input space to subspaces by unsupervised learning. The RL part, corresponding to the basal ganglia of brain, optimizes the model performance by reinforcement learning. Numerical simulations show that the proposed brain-like learning system optimizes its performance automatically and has superior performance to an ordinary neural network.

Many methods of generating behavior sequences of agents by evolution have been reported. A new evolutionary computation method named Genetic Network Programming (GNP) has been also developed recently along with these trends. The aim of this paper is to build an artificial model to realize functional localization of GNP considering the fact that the functional localization of the brain is realized in such a way that a different part of the brain corresponds to a different function. In this paper, it is especially stated that the switching function for functional localization can be realized using GNP with Importance Index (GNP IMX).

An efficient algorithm for building a classifier is proposed based on an important association rule mining using Genetic Network Programming (GNP). The proposed method measures the significance of the association via the chi-squared test. Users can define the conditions of important association rules for building a classifier flexibly. The definition can include not only the minimum threshold chi-squared value, but also the number of attributes in the association rules. Therefore, all the extracted important rules can be used for classification directly. GNP is one of the evolutionary optimization techniques, which uses the directed graph structure as genes. Instead of generating a large number of candidate rules, our method can obtain a sufficient number of important association rules for classification. In addition, our method suits association rule mining from dense databases such as medical datasets, where many frequently occurring items are found in each tuple. In this paper, we describe an algorithm for classification using important association rules extracted by GNP with acquisition mechanisms and present some experimental results of medical datasets.

The way of propagating and control of stochastic signals through Universal Learning Networks (ULNs) and its applications are proposed. ULNs have been already developed to form a superset of neural networks and have been applied as a universal framework for modeling and control of non-linear large-scale complex systems.
However, the ULNs cannot deal with stochastic variables. Deterministic signals can be propagated through a ULN, but the ULN does not provide any stochastic characteristics of the signals propagating through it.
The proposed method named Probabilistic Universal Learning Networks (PrULNs) can process stochastic variables and can train network parameters so that the signals behave with the pre-specified stochastic properties.
As examples of applications of the proposed method, control and identification of non-linear dynamic systems with noises are studied, and it is shown that the method are useful for dealing with the control and identification of the non-linear stochastic systems contaminated with noises. (c) 2005 Elsevier Ltd. All rights reserved.

Recently, Genetic Network Programming (GNP) has been proposed as one of the evolutionary algorithms. It represents its solutions as directed graph structures and the distinguished abilities have been shown. However, when we apply GNP to complex problems like the real world one, GNP must have robustness against the changes of environments and evolve quickly. Therefore, we introduced Automatically Generated Macro Nodes (AGMs) to GNP (GNP with AGMs). Actually GNP with AGMs has shown higher performances than the conventional GNP in terms of the fitness and the speed of evolution. In this paper, a new mechanism, AGMs with variable size, is introduced to GNP. Conventional AGMs have the fixed number of nodes and they evolve using only genetic operations, while a new method allows AGM to add nodes by necessity and delete nodes which do not contribute to the evolution of the AGM. The proposed GNP with AGMs of variable size is expected to evolve effectively and efficiently when it is applied to agent systems and also expected to make better behavior sequences of agents more easily than the conventional GNP algorithm. In the simulations, the proposed and conventional methods are applied to a tileworld problem and they are compared with each other. From the results, GNP with AGMs of variable size shows better fitness than GNP with AGMs of fixed size and the conventional GNP when adapting ten different environments.

Multiagent Systems with Symbiotic Learning and Evolution (Masbiole) has been proposed and studied, which is a new methodology of Multiagent Systems (MAS) based on symbiosis in the ecosystem. Masbiole employs a method of symbiotic learning and evolution where agents can learn or evolve according to their symbiotic relations toward others, i.e., considering the benefits/losses of both itself and an opponent. As a result, Masbiole can escape from Nash Equilibria and obtain better performances than conventional MAS where agents consider only their own benefits. This paper focuses on the evolutionary model of Masbiole, and its characteristics are examined especially with an emphasis on the behaviors of agents obtained by symbiotic evolution. In the simulations, two ideas suitable for the effective analysis of such behaviors are introduced; "Match Type Tile-world (MTT)" and "Genetic Network Programming (GNP)". MTT is a virtual model where tile-world is improved so that agents can behave considering their symbiotic relations. GNP is a newly developed evolutionary computation which has the directed graph type gene structure and enables to analyze the decision making mechanism of agents easily. Simulation results show that Masbiole can obtain various kinds of behaviors and better performances than conventional MAS in MTT by evolution.

A method of association rule mining using Genetic Network Programming (GNP) is proposed to improve the performance of association rule extraction from dense database. Rule extraction is done without identifying frequent itemsets used in Apriori-like methods. Association rules are represented as the connections of nodes in GNP. The proposed mechanisms calculate measurements of association rules directly from a database using GNP, and measure the significance of the association via the chi-squared test. The proposed system evolves itself by an evolutionary method and obtains candidates of association rules by genetic operations. Extracted association rules are stored in a pool all together through generations and reflected in genetic operators as acquired information. In this paper, we describe an algorithm capable of finding important association rules using GNP with sophisticated rule acquisition mechanisms and present some experimental results.

This paper proposes an improved model for the application of support vector machine (SVM) to achieve the real-world data classification. Being different from traditional SVM classifiers, the new model takes the thought about fuzzy theory into account. And a fuzzy decision-making function is also built to replace the sign function in the prediction stage of classification process. In the prediction part, the method proposed uses the decision value as the independent variable of fuzzy decision-making function to classify test data set into different classes, but not only the sign of which. This flexible design of decision-making model more approaches to the properties of real-world conditions in which interaction and noise influence exist around the boundary between different clusters. So many misclassified cases can be modified when these sets are considered as fuzzy ones. In addition, a boundary offset is also introduced to modify the excursion produced by the imbalance of real-world dataset. Then an improved and more robust performance will be presented by using this adjustable fuzzy decision-making SVM model in simulations.

Inspired by Hebb's cell assembly theory about how the brain worked, we have developed a function localization neural network (FLNN). The main part of a FLNN is structurally the same as an ordinary feedforward neural network, but it is considered to consist of several overlapping modules, which are switched according to input patterns. A FLNN constructed in this way has been shown to have better representation ability than an ordinary neural network. However, BP training algorithm for such FLNN is very easy to get stuck at a local minimum. In this paper, we mainly discuss the methods for improving BP training of the FLNN by utilizing the structural property of the network. Two methods are proposed. Numerical simulations are used to show the effectiveness of the improved BP training methods.

A new graph-based evolutionary algorithm called "Genetic Network Programming, GNP" has been proposed. The solutions of GNP are represented as graph structures, which can improve the expression ability and performance. In addition, GNP with Reinforcement Learning (GNP-RL) has been proposed to search for solutions efficiently. GNP-RL can use current information (state and reward) and change its programs during task execution. Thus, it has an advantage over evolution-based algorithms in case much information can be obtained during task execution. The GNP we proposed in the previous research deals with discrete information, but in this paper, we extend the conventional GNP-RL which can deal with numerical information. The proposed method is applied to the controller of Khepera simulator and its performance is evaluated.

An improved support vector machine (SVM) classifier model for classifying the real-world lopsided data is proposed. The most obvious differences between the model proposed and conventional SVM classifiers are the designs of decision-making functions and the introduction of an offset parameter. With considering about the vagueness of the real-world data sets, a fuzzy decision-making function is designed to take the place of the traditional sign function in the prediction part of SVM classifier. Because of the existence of the interaction and noises influence around the boundary between different clusters, this flexible design of decision-making model which is more similar to the real-world situations can present better performances. In addition, in this paper we mainly discuss an offset parameter introduced to modify the boundary excursion caused by the imbalance of the real-world datasets. Because noises in the real-world can also influence the separation boundary, a weighted harmonic mean (WHM) method is used to modify the offset parameter. Due to these improvements, more robust performances are presented in our simulations.

Neural networks (NNs) can solve only a simple problem if the network size is too small. On the other hand, if the network size increases, it costs a lot in terms of memory space andcalculation time. Therefore, we have studied how to construct the network structure with high performances and low costs in space and time. A solution is a multi-branch structure. Conventional NNs use the single-branch for the connections, while the multi-branch structurehas multibranches between nodes. In this paper, a new method which enables the multi-branch NNs to have functional localization is proposed. Neural networks with Branch Control adjust signals propagating through branches between the intermediate layer and output layer depending on the inputs of the network. Therefore, a branch could be cut depending on input values. Simulation results of function approximations and a classification problem illustrated the effectiveness of multi-branch NNs with functional localization.

Genetic Network Programming (GNP) has been proposed as a new method of evolutionary computations and its basic characteristics are studied. GNP is constructed using graph structures whose gene consists of directed graphs, so it is possible to search solutions effectively especially for the dynamic real world problems due to the implicit memory function of its structure. Also GNP can easily implement a priori knowledge into its structure as functional nodes. There has been no example where GNP is applied to real world systems until now. In this paper, Elevator Group Supervisory Control System (EGSCS) which is a typical real world system is studied using GNP. From the simulations, the availability of GNP to EGSCS is confirmed and it is clarified that the proposed method can show better performances than other conventional methods.

Neural networks (NNs) can solve only a simple problem if the network size is too small, on the other hand, if the network size increases, it costs a lot in terms of memory space and calculation time. So, we have studied how to construct the network structure with high performances and low costs in space and time. A solution is a multi-branch structure. Conventional NNs use the single-branch for the connections, while the multi-branch structure has multi-branches between nodes. In this paper, a new method which enables the multi-branch NNs to have localized property is proposed. It is well known that RBF networks have localized property that makes it possible to approximate functions faster than sigmoidal NNs. By using the multi-branch structure having localized property of RBF networks, NNs could obtain high performances keeping the lower costs in space and time. Simulation results of function approximations and a classification problem illustrated the effectiveness of multi-branch NNs.

Various stock prices predicting and sell-buy strategy models have been so far proposed. They are classified as the fundamental analysis using the achievements of the companies and the trend of business, etc., and the technical analysis which carries out the numerical analysis of the movement of stock prices. On the other hand, as one of the methods for data mining which finds out the regularity from a vast quantity of stock price data, Genetic Algorithm (GA) has been so far applied widely. As a concrete example, the optimal values of parameters of stock indices like various moving averages and rates of deviation, etc. is computed by GA, and there have been developed various methods for predicting stock prices and determinig sell-buy strategy based on it. However, it is hard to determine which is the most effective index by the conventional GA. Moreover, the most effective one depends on the brands. So in this paper, a stock price prediction and sell-buy strategy model which searches for the optimal combination of various indices in the technical analysis has been proposed using Genetic Network programming and its effectiveness is confirmed by simulations.

According to the knowledge of brain science, it is suggested that there exists cerebral functional localization, which means that a specific part of the cerebrum is activated depending on various kinds of information human receives. The aim of this paper is to build an artificial model to realize functional localization based on Genetic Network Programming (GNP), a new evolutionary computation method recently developed. GNP has a directed graph structure suitable for realizing functional localization. We studied the basic characteristics of the proposed system by making GNP work in a functionally localized way.

According to Hebb's cell assembly theory, the brain has the capability of function localization. On the other hand, it is suggested that the brain has three different learning paradigms: supervised, unsupervised and reinforcement learning. Inspired by the above knowledge of brain, we present a self-organizing function localization neural network (FLNN), that contains supervised, unsupervised and reinforcement learning paradigms. In this paper, we concentrate our discussion mainly on applying a simplified reinforcement learning called evaluative feedback to optimization of the self-organizing FLNN. Numerical simulations show that the self-organizing FLNN has superior performance to an ordinary artificial neural network (ANN).

According to Hebb's cell assembly theory, the brain has the capability of function localization. On the other hand, it is suggested that the brain has three different learning paradigms: supervised, unsupervised and reinforcement learning. Inspired by the above knowledge of brain, we present a self-organizing function localization neural network (FLNN), that contains supervised, unsupervised and reinforcement learning paradigms. In this paper, we concentrate our discussion mainly on applying a simplified reinforcement learning called evaluative feedback to optimization of the self-organizing FLNN. Numerical simulations show that the self-organizing FLNN has superior performance to an ordinary artificial neural network (ANN).

A new evolutionary algorithm named "Genetic Network Programming, GNP" has been proposed. GNP represents its solutions as directed graphs, which realizes better expression ability than GA and GP which use string and tree structures, respectively. The aim of developing GNP is to deal with dynamic environments efficiently by using the distinguished expression ability and the inherently equipped functions of the network structure. However, since GNP is based on evolution, the programs cannot be changed until one generation ends. In this paper, we propose the extended algorithm, "GNP with Evolution and Learning" which combines evolution and reinforcement learning in order to adapt to dynamic environments quickly. The tileworld is used as a simulation environment and the results show some advantages of the proposed method.

A new evolutionary algorithm named "Genetic Network Programming, GNP" has been proposed. GNP represents its solutions as directed graphs, which realizes better expression ability than GA and GP which use string and tree structures, respectively. The aim of developing GNP is to deal with dynamic environments efficiently by using the distinguished expression ability and the inherently equipped functions of the network structure. However, since GNP is based on evolution, the programs cannot be changed until one generation ends. In this paper, we propose the extended algorithm, "GNP with Evolution and Learning" which combines evolution and reinforcement learning in order to adapt to dynamic environments quickly. The tileworld is used as a simulation environment and the results show some advantages of the proposed method.

Genetic Network Programming (GNP) extended from other evolutionary computations such as Genetic Algorithm (GA) and Genetic Programming (GP) has network structures as gene. Previously, the program size of conventional GNP was fixed and GNP programs have not introduced the concept of sub-routines, although GA and GP paid attention to sub-routines. In this paper, a new method where GNP with Automatically Generated Macro Nodes (GNP with AGMs) composed of a number of nodes is proposed for improving the performance of GNP. These AGMs also have network structures and are evolved like main GNP. In addition to that, AGMs have multiple inputs and outputs that have not been introduced in the past. In the simulations, comparisons between GNP program only and GNP with AGMs are carried out using the tile world. Simulation results shows that the proposed method brings better results compared with traditional GNP. And it is clarified from simulations that the node transition rules obtained by AGMs show the generalized rules able to deal with unknown environments.

In this paper, a new robust control method and its application to a photovoltaic (PV) supplied, separately excited DC motor loaded with a constant torque is discussed. The robust controller is designed against the load torque changes by using the first and second ordered derivatives of the universal learning networks (ULNs). These derivatives are calculated using the forward propagation algorithm, which is considered as an extended version of real time recurrent learning (RTRL). In this application, two ULN's are used: The first is the ULN identifier trained offline to emulate the dynamic performance of the DC motor system. The second is the ULN controller, which is trained online not only to make the motor speed follow a selected reference signal, but also to make the overall system operate at the maximum power point of the PV source. To investigate the effectiveness of the proposed robust control method, the simulation is carried out at four different values of the robustness coefficient in two different stages: The training stage, in which the simulation is done for a constant load torque. And the control stage, in which the controller performance is obtained when the load torque is changed. The simulation results showed that the robustness of the control system is improved although the motor load torque at the control stage is different from that at the training stage. (C) 2004 Elsevier Ltd. All rights reserved.

This paper presents a self-organizing function localization neural network (FLNN) inspired by Hebb's cell assembly theory about how the brain worked. The proposed self-organizing FLNN consists of two parts: main part and control part. The main part is an ordinary 3-layered feedforward neural network, but each hidden neuron contains a signal from the control part, controlling its firing strength. The control part consists of a SOM network whose outputs are associated with the hidden neurons of the main part. Trained with an unsupervised learning, SOM control part extracts structural features of input-output spaces and controls the firing strength of hidden neurons in the main part. Such self-organizing FLNN realizes capabilities of function localization and learning. Numerical simulations show that the self-organizing FLNN has superior performance than an ordinary neural network.

Quasi-ARX neural networks (NN) are NN based nonlinear models that not only have linear structures similar to linear ARX models, but also have useful interpretation in part of their parameters. However when applying an ordinary backpropagation (BP) for the training, it has potential risk that the BP algorithm is stuck at a local minimum, which results in a poorly trained model. In this paper, a homotopy continuation method is introduced to improve the convergence performance of BP training. The idea is to start the BP training with the criterion function for linear ARX model, which is gradually deformed first into one for quasi-ARX NN model with linear node functions, and then into the actual one for quasi-ARX NN with sigmoid node functions. By building the deformation into a usual recursive procedure for BP training of quasi-ARX NN model with adaptable node functions so that the proposed homotopy based BP algorithm is able to achieve improved convergence performance without much increase in the computation load. Numerical simulation results show that the proposed homotopy based BP has better performance than an ordinary BP.

In this paper, a functions localized network with branch gates (FLN-bg) is studied, which consists of a basic network and a branch gate network. The branch gate network is used to determine which intermediate nodes of the basic network should be connected to the output node with a gate coefficient ranging from 0 to 1. This determination will adjust the outputs of the intermediate nodes of the basic network depending on the values of the inputs of the network in order to realize a functions localized network. FLN-bg is applied to function approximation problems and a two-spiral problem. The simulation results show that FLN-bg exhibits better performance than conventional neural networks with comparable complexity. (C) 2003 Elsevier Ltd. All rights reserved.

Recently, various attempts relevant to Multi Agent Systems (MAS) which is one of the most promising systems based on Distributed Artificial Intelligence have been studied to control large and complicated systems efficiently. In these trends of MAS, Multi Agent Systems with Symbiotic Learning and Evolution named Masbiole has been proposed. In Masbiole, symbiotic phenomena among creatures are considered in the process of learning and evolution of MAS. So we can expect more flexible and sophisticated solutions than conventional MAS. In this paper, we apply Masbiole to Iterative Prisoner's Dilemma Games (IPD Games) using Genetic Network Programming (GNP) which is a newly developed evolutionary computation method for constituting agents. Some characteristics of Masbiole using GNP in IPD Games are clarified.

Neural networks have attracted much interest in system identification and control communities because they can learn any nonlinear mapping. However, from a user's point of view, when neural networks are used as models for controller design, they do not have structures of easy use. This paper introduces a new neural network based prediction model for control of nonlinear systems. Distinctive features of the new model to conventional neural-network based ones are that it has not only meaningful interpretation on part of its parameters but also is linear for the input variables. The former feature makes parameter estimation easier and the latter allows us to derive a nonlinear controller directly from the identified prediction model. The modeling and the parameter estimation are described in detail. The usefulness of the new model is demonstrated by using numerical simulations.

Recently, systems are becoming more complex and larger than ever, so numerous attempts have been made to introduce biological features into artificial systems, because many biological systems in the nature exist as one of the most complex systems.<br>Multi agent system with symbiotic learning and evolution have been recently proposed. It is named Masbiole. In this paper, Masbiole is reviewed and the method for evolving multi agent systems is proposed. From simulations on a multi objective knapsack problem, it has been clarified that Masbiole has better performance than that of conventional multi objective genetic algorithms.

Genetic Network Programming (GNP) is a kind of evolutionary methods, which evolves arbitrary directed graph programs. Previously, the program size of GNP was fixed. In the paper, a new method is proposed, where the program size is adaptively changed depending on the frequency of the use of nodes. To control and to decide a program size are important and difficult problems in Evolutionary Computation, especially, a well-known crossover operator tends to cause bloat. We introduce two additional operators, add operator and delete operator, that can change the number of each kind of nodes based on whether a node function is important in the environment or not. Simulation results shows that the proposed method brings about extremely better results compared with ordinary fixed size GNP. © 2003, The Institute of Electrical Engineers of Japan. All rights reserved.

A new evolutionary model with the network structure named Genetic Network Programming (GNP) has been proposed recently. GNP, that is, an expansion of GA and GP, represents solutions as a network structure and evolves it by using “offline learning (selection, mutation, crossover)”. GNP can memorize the past action sequences in the network flow, so it can deal with Partially Observable Markov Decision Process (POMDP) well. In this paper, in order to improve the ability of GNP, Q learning (an off-policy TD control algorithm) that is one of the famous online methods is introduced for online learning of GNP. Q learning is suitable for GNP because (1) in reinforcement learning, the rewards an agent will get in the future can be estimated, (2) TD control doesn't need much memory and can learn quickly, and (3) off-policy is suitable in order to search for an optimal solution independently of the policy. Finally, in the simulations, online learning of GNP is applied to a player for “Prisoner's dilemma game“ and its ability for online adaptation is confirmed. © 2003, The Institute of Electrical Engineers of Japan. All rights reserved.

Recently, many methods of evolutionary computation such as Genetic Algorithm(GA) and Genetic Programming(GP) have been developed as a basic tool for modeling and optimizing complex systems. GA has the genome of string structure, while the genome in GP is of tree structure. In this paper, a new evolutionary method named Genetic Network Programming(GNP), whose genome has network structure is applied to multiagent sysytems. Hetero Multiagent Sysytems with GNP are studied, where each agent of the same group has its own GNP program in order to build the adaptive agents against changing environments. Specifically, the comparison between Hetero Multiagent Systems and conventional Homo Multiagent Sysytems is carried out in simulations on ants behaviors. © 2003, The Institute of Electrical Engineers of Japan. All rights reserved.

A variety of neuron models combine the neural inputs through their summation and sigmoidal functions. Such structure of neural networks leads to shortcomings such as a large number of neurons in hidden layers and huge training data required. We introduce a kind of multiplication neuron which multiplies their inputs instead of summing to overcome the above problems. A hybrid universal learning network constructed by the combination of multiplication units arid summation units is proposed and trained for several well known benchmark problems. Different combinations of the above two are tried. It is clarified that multiplication is an essential computational element in many cases and the combination of the multiplication units with summation units in different layers in the networks improved the performance of the network. © 2003, The Institute of Electrical Engineers of Japan. All rights reserved.

Genetic algorithms are often well suited for optimization problems because of their parallel searching and evolutionary ability. Crossover and mutation are believed to be the main exploration operators. In this paper, we focus on how crossover and mutation work in binary-coded genetic algorithm and investigate their effects on bit's frequency of population. According to the analysis of equilibrium of crossover, we can see the bit-based simulated crossover (BSC) is strong crossover method. Furthermore, to increase robustness of binary-coded genetic algorithm, multi-generation inheritance evolutionary strategy(MGIS) was proposed. Simulation results demonstrate the effectiveness of the proposed method. © 2003, The Institute of Electrical Engineers of Japan. All rights reserved.

In this paper, RasID is enhanced and evaluated systematically. RasID is an abbreviation of Random Search with Intensification and Diversification. RasID can search for a global minimum based on a probabilitiy density function, which can be modified adaptively using information based on success and failure of the past searching. As a result, RasID performs intensified search and diversified search iteratively to find a global minimum.<br>The improvements of RasID are, (1) the probability density function is modified so that each variable can have its own unique density function, and (2) multiple candidates for solutions are created in the search to enhance the efficiency of the searching.<br>Modified RasID is compared systematically with typical optimization methods such as Evolutionary. Programming and Fast Evolutionary Programming introduced recently using 23 different complibated functions. From simulation results, it has been clarified that the performance of Modified RasID is comparable to the ones of EP and FEP in spite of using one individual.

A new evolutionary computation method named Genetic Network Programming (GNP) was proposed recently. In this paper, an online learning method for GNP is proposed. This method uses Q learning to improve its state transition rules so that it can make GNP adapt to the dynamic environments efficiently.

This paper introduces a new method for applying multilayer perceptron (MLP) network to control of nonlinear systems. The MLP network is not used directly as a nonlinear controller, but used indirectly via an ARX-like macro-model. The ARX-like model incorporating MLP network is constructed in such a way that it has similar linear properties to a linear ARX model. The nonlinear controller is then designed in the same way as designing a linear controller based on a linear ARX model. Numerical simulations are carried to demonstrate the effectiveness of the new method.

This paper proposes a class of quasi-ARMAX models for non-linear systems. Similar to ordinary non-linear ARMAX models, the quasi-ARMAX models are flexible black-box models, but they have various linearity properties similar to those of linear ARMAX models. A modelling scheme is introduced to construct models consisting of two parts: a macro-part and a kernel-part. By using Taylor expansion and other mathematical transformation techniques, it is first constructed as a class of quasi-ARMAX interfaces (macro-parts) that have various linearity properties but contain some complicated coefficients. MIMO neurofuzzy models (kernel-parts) are then introduced to represent the complicated coefficients. It is shown that the proposed quasi-ARMAX models have both good approximation ability and some easy-to-use properties. The proposed models have been successfully applied to prediction, fault detection and adaptive control of non-linear systems.

This paper presents a homotopy approach to improving PEM identification of ARMAX model. PEM estimates of ARMAX model parameters are determined as the global minimum of criterion function, which is however not always unimodal because of the MA noise model part. An optimization-based PEM identification algorithm has a potential risk to be stuck at a local minimum that results in a poorly identified model. A homotopy continuation method is introduced to solve this problem. The idea is to start the estimation with the criterion function for PEM identification of the ARX model, which is gradually deformed into the actual one for PEM identification of the ARMAX model as the algorithm iterates. By building the deformation into the usual recursive procedure for the ARMAX identification and introducing a scheme to control the solution continuously staying in the global minima of the deformed criterion functions, the homotopy-based PEM identification algorithm is implemented in such a way that it has very good convergence performance, with only little increase in computation load compared to the usual PEM algorithm. (C) 2001 Elsevier Science Ltd. All rights reserved.

Recently, many people are involved in the research of complex systems which are made of a large number of elements interacting with each other.<br>In this paper, a new modeling method of mutual interaction of the complicated systems is proposed by using Spatial Distribution Universal Learning Networks (SdULNs) and Fuzzy inference algorithms.<br>In addition, the proposed model is applied to the modeling of ecosystems which are described as the Lotka-Volterra equation.<br>From simulations where apriori information on ecosystems is given, it has been cleared that the proposed method can present more flexible emergent dynamics than the conventional Lotka-Volterra equation.

A new control method of nonlinear dynamic systems is proposed based on the impulse responses of universal learning networks (ULNs). ULNs form a superset of neural networks. They consist of a number of interconnected nodes where the nodes may have any continuously differentiable nonlinear functions in them and each pair of nodes can be connected by multiple branches with arbitrary time delays. A generalized learning algorithm is derived for the ULNs, in which both the first order derivatives (gradients) and the higher order derivatives are incorporated. One of the distinguished features of the proposed control method is that the impulse response of the systems is considered as an extended part of the criterion function and it can be calculated by using the higher order derivatives of ULNs. By using the impulse response as the criterion function, nonlinear dynamics with not only quick response but also quick damping and small steady state error can be more easily obtained than the conventional nonlinear control systems with quadratic form criterion functions of state and control variables.

This paper discusses the problem of applying sigmoidal neural network to prediction and control of nonlinear dynamical systems. Instead of directly using neural networks as nonlinear models, we first develop a shield based on application specific knowledge, and then embed sigmoidal neural network model in the shield. An embedded sigmoidal neural network model obtained in this way not only has a structure favorable for certain applications such as controller design, but also has useful interpretation on part of model parameters. Corresponding to the meaningful part and the meaningless part of model parameters, a hierarchical training algorithm consisting of two learning loops is introduced to train the model, which has good performance on solving local minimum problems. The usefulness of the proposed prediction model is demonstrated by applying it to prediction and control of a simulated nonlinear system.

This paper discusses the problem of applying sigmoidal neural networks to identification of nonlinear dynamical systems. When using sigmoidal neural networks directly as nonlinear models, one often meets problems such as model parameters lack of physical meaning, sensitivity to noise in model training. In this paper, we introduce an embedded sigmoidal neural network model, in which the neural network is not used directly as a model, but is embedded in a shield such that part of the model parameters become meaningful. Corresponding to the meaningful part and the meaningless part of model parameters, a hierarchical training algorithm consisting of two learning loops is then introduced to train the model. Simulation results show that such a dual loop learning algorithm can solve the noise sensitivity and local minimum problems to some extent.

This paper studies how the generalization ability of models of dynamical systems can be improved by taking advantage of the second order derivatives of the outputs with respect to the external inputs. The proposed method can be regarded as a direct implementation of the well-known regularization technique using the higher order derivatives of the Universal Learning Networks (ULNs). ULNs consist of a number of interconnected nodes where the nodes may have any continuously differentiable nonlinear functions in them and each pair of nodes can be connected by multiple branches with arbitrary time delays. A generalized learning algorithm has been derived for the ULNs, in which both the first order derivatives (gradients) and the higher order derivatives are incorporated. First, the method for computing the second order derivatives of ULNs is discussed. Then, a new method for implementing the regularization term is presented. Finally, simulation studies on identification of a nonlinear dynamical system with noises are carried out to demonstrate the effectiveness of the proposed method. Simulation results show that the proposed method can improve the generalization ability of neural networks significantly, especially in terms that (1) the robust network can be obtained even when the branches of trained ULNs are destructed, and (2) the obtained performance does not depend on the initial parameter values. © 2001 Elsevier Science Ltd. All rights reserved.

Generalization ability of neural networks is the most important criterion to determine whether one algorithm is powerful or not. Many new algorithms have been devised to enhance the generalization ability of neural networks[1][2]. In this paper a new algorithm using the Gram-Schmidt orthogonalization algorithm [3] to the outputs of nodes in the hidden layers is proposed with the aim to reduce the interference among the nodes in the hidden layers, which is much more efficient than the regularizers methods. Simulation results confirm the above assertion.

Probabilistic Universal Learning Networks(PrULNs)are proposed which are learning networks with capability of dealing with stochastic signals. PrULNs are extension of Universal Learning Networks(ULNs). ULNs form a superset of neural networks and were proposed to provide a universal framework for modeling and control of nonlinear large-scale complex systems. A generalized learning algorithm has been devised for ULNs which can also be used in a unified manner for almost all kinds of learning networks. However, the ULNs can not deal with stochastic variables. Specific value of a stochastic signal can be propagated through a ULN, but the ULN does not provide any stochastic characteristic of the signals propagating through it. The PrULNs proposed here are equipped with machinery to calculate stochastic properties of signals and to train network parameters so that the signals behave with the prespecified stochastic propertied. The PrULNs will contribute to the solution of the following problems : (1)improving the generalization capability of the learning networks, (2)more sophisticated stochastic control than the conventional stochastic control, (3)designing problem for the complex systems such as chaotic systems.

Universal learning networks (ULN's) and robust control system design are discussed, ULN's provide a generalized framework to model and control complex systems. They consist of a number of inter-connected nodes where the nodes may have any continuously differentiable nonlinear functions in them and each pair of nodes can be connected by multiple branches with arbitrary time delays. Therefore, physical systems which can be described by differential or difference equations and also their controllers can be modeled in a unified way. So, ULN's constitute a superset of neural networks or fuzzy neural networks. In order to optimize the systems, a generalized learning algorithm is derived for the ULN's, in which both the first order derivatives (gradients) and the higher order derivatives are incorporated. The derivatives are calculated by using forward or backward propagation schemes. These algorithms for calculating the derivatives are extended versions of back propagation through time (BPTT) and real time recurrent learning (RTRL) by Williams in the sense that generalized nonlinear functions and higher order derivatives are dealt with. As an application of ULN's, the higher order derivative, one of the distinguished features of ULN's, is applied to realizing a robust control system in this paper, In addition, it is shown that the higher order derivatives are effective tools to realize sophisticated control of nonlinear systems. Other features of ULN's such as multiple branches with arbitrary time delays and using a priori information will be discussed in other papers.

A new chaos control method is proposed which is useful for taking advantage of chaos and avoiding it, The proposed method is based on the following facts: 1) chaotic phenomena can be generated and eliminated by controlling maximum Lyapunov exponent of systems and 2) maximum Lyapunov exponent can be formulated and calculated by using higher order derivatives of Universal Learning Networks (ULN's). ULN's consist of a number of inter-connected nodes where the nodes may have any continuously differentiable nonlinear functions in them and each pair of nodes can be connected by multiple branches with arbitrary time delays. A, generalized learning algorithm has been derived for the ULN's, in which both the first-order derivatives (gradients) and the higher order derivatives are incorporated. In simulations, parameters of ULN's with bounded node outputs are adjusted for maximum Lyapunov component to approach the target value. And, it has been shown that a fully connected ULN with three sigmoidal function nodes is able to generate and eliminate chaotic behaviors by adjusting the parameters.

In this paper, Universal Learning Networks with Branch Control (BrcULNs) are proposed, which consist of basic networks and branch control networks. The branch control network can be used to determine which branches of the basic network should be connected or disconnected. This determination depends on the inputs or the network flows of the basic network. Therefore, by using the BrcULNs, locally functions distributed networks can be realized depending on the values of the inputs of the network or the information of the network flows. The proposed network is applied to some function approximation problems. The simulation results show that the BrcULNs exhibit better performance than the conventional networks with comparable complexity.

Universal Learning Networks (ULNs) are proposed and their application to chaos control is discussed. ULNs provide a generalized framework to model and control complex systems. They consist of a number of inter-connected nodes where the nodes may have any continuously differentiable nonlinear functions in them and each pair of nodes can be connected by multiple branches with arbitrary time delays. Therefore, physical systems, which can be described by differential or difference equations and also their controllers, can be modeled in a unified way, and so ULNs may form a super set of neural networks and fuzzy neural networks. In order to optimize the ULNs, a generalized learning algorithm is derived, in which both the first order derivatives (gradients) and the higher order derivatives are incorporated. The derivatives are calculated by using forward or backward propagation schemes. These algorithms for calculating the derivatives are extended versions of Back Propagation Through Time (BPTT) and Real Time Recurrent Learning (RTRL) of Williams in the sense that generalized node functions, generalized network connections with multi-branch of arbitrary time delays, generalized criterion functions and higher order derivatives can be deal with. As an application of ULNs, a chaos control method using maximum Lyapunov exponent of ULNs is proposed. Maximum Lyapunov exponent of ULNs can be formulated by using higher order derivatives of ULNs, and the parameters of ULNs can be adjusted so that the maximum Lyapunov exponent approaches the target value. From the simulation results, it has been shown that a fully connected ULN with three nodes is able to display chaotic behaviors. © 2000 Elsevier Science Ltd.

This paper presents a case study for the overlapped multi-neural-network (OMNN). An overlapped multi-neural-network, structurally, is the same as an ordinary feedforward neural network, but it is considered as one consisting of several subnets. Ail subnets have the same input-output units, but some different hidden units. Input-output spaces are partitioned into several parts, each of which corresponds to one subnet of OMNN. Numerical simulations show that such an OR INN has superior performance in that It has better presentation ability than an ordinary neural network and better generalization ability than a non-overlapped multi-neural-network.

Recently many researches on group robot systems have been studied, where a number of robots behave in a group like birds' or ants. It is generally known that each robot has a limited intellectual power, but the robots can behave more intellectually in a group because they can interact each other. One of the most famous researches in these fields is Boids which is the artificial model of the birds behavior in the computer software. And there have been reported the multi-agent robot systems which can do many kinds of tasks efficiently by training the rules between environments and actions using reinforced learning. This paper also proposes a multi-agent system where a criterion function is defined regarding the behavior of the multi-agent system and parameters of mutual interaction of the agents are trained in order to optimize the above criterion function. From simulations, it has been shown that emergent behaviors of the agents can be developed by appropriately adjusting the parameters.

This paper deals with the problems of robust fault detection using Kullback discrimination information (KDI) in presence of nonlinear undermodeling. The systems to be diagnosed are assumed to contain certain unknown nonlinear elements. The fault detection is performed by applying the KDI to a linear ARMAX model with model uncertainty, in which error due to nonlinear undermodeling is described using a group of fuzzy models with adjustable parameters. The estimate of modeling error is considered in the KDI analysis and thresholding decision for robustness realization. The effectiveness of the proposed robust fault detection scheme is examined through numerical simulations.

According to recent knowledge of brain science, it is suggested that there exists functions distribution, which means that specific parts exist in the brain for realizing specific functions. This paper introduces a new brain-like model called Learning Petri Network (LPN) that has the capability of functions distribution and learning, The idea is to use Petri net to realize the functions distribution and to incorporate the learning and representing ability of neural network into the Petri net. The obtained LPN can be used in the same way as a neural network to model and control dynamic systems, while it is distinctive to a neural network in that it has the capability of functions distribution, An application of the LPN to nonlinear crane control systems is discussed. It is shown via numerical simulations that the proposed LPN controller has superior performance to the commonly-used neural network one.

This research is concerned with fault detection of nonlinear systems using Kullback discrimination information (KDI) as an index. A hybrid quasi-ARMAX model is proposed, which combines a linear ARMAX model and a multi-ARX-model based on interpolation. In the case where the faults occur on the ARMAX model part, a KDI-based "robust" fault detection is performed, in which multi-ARX-model part is treated as error due to nonlinear undermodeling. Ill other cases, the model is transformed into several local ARMAX models and fault detection is performed by using the KDI to discriminate each identified local model. In this paper, we mainly concentrate our discussion on the latter cases. Copyright (C) 1998 IFAC.

This paper presents a modeling scheme for nonlinear black-bet systems based on Universal Leaning Networks (ULN). The ULN, a superset of all kinds of neural networks, consists of two kinds of elements: nodes and branches corresponding to equations and their relations in traditional description of dynamic systems. Following the idea of ULN, a nonlinear black-bet, system is first represented by a set of related unknown equations, and then treated as the ULN with nodes and branches. Each unknown node function in the ULN is re-parameterized by using an adaptive fuzzy model. One of distinctive features of the black-bet model constructed in this way is that it can incorporate prier knowledge obtained from input-output data into its modeling and thus its parameters to be trained have explicit meanings useful for estimation and application.

This paper presents an adaptive control scheme for nonlinear black-bet systems based on the we of Universal Learning Networks (ULN). A ULN nonlinear controller is constructed in a similar way to linear stochastic control theory. In the obtained ULN controller, some node functions are known, while others are unknown. Each unknown node function is reparameterized using an adaptive fuzzy model. A robust adaptive algorithm is developed to adjust the unknown parameters in the controller. The effectiveness of the proposed control scheme is examined via numerical simulations.

This paper presents a novel random searching scheme called RasID for neural networks training. The idea is to introduce a sophisticated probability density function (PDF) for generating search vector. The PDF provides two parameters for realizing intensified search in the area where it is likely to find good solutions locally or diversified search in order to escape from a local minimum based on the success-failure of the past search. Gradient information is wed to improve the search performance. The proposed scheme Is applied to layered neural networks training and Is benchmarked against other deterministic and nondeterministic methods.

This paper proposes a hybrid quasi-ARMAX modeling and identification scheme for nonlinear systems. The idea is to incorporate a group of certain nonlinear nonparametric models (NNMs) into a linear ARMAX structure. Particular effort is made to find a, better compromise to the trade-off between the model flexibility and the simplicity for estimation by using knowledge information efficiently. As the result, we obtain a model equipped with a linear ARMAX structure, flexibility and simplicity The effectiveness and usefulness of the proposed hybrid model are examined by applying it to identification and control of nonlinear systems.

Many methods of generating behavior sequences of agents by evolution have been reported. A new evolutionary computation method named Genetic Network Programming (GNP) has also been developed recently along with these trends. In this paper, a new method for evolving GNP considering Breadth and Depth is proposed. The performance of the proposed method is shown from simulations using garbage collector problem.

Function localization and layer structure are two basic features of complex systems. In this research, we developed two hierarchical function localized brain-like systems : one is Self-organizing function localized learning system with supervised learning, unsupervised learning and reinforcement learning : the other is function localized genetic network programming. And the developed systems are applied to prediction, control and classification of complex systems.

Genetic Algorithm (GA), a method which imitates nature and living things, was proposed as a model which explains the adaptive process of the system of nature. In addition, Genetic Programming (GP) was proposed to deal with knowledge expression, programs, concept trees and so on. Many algorithms of these kinds of evolutionary computations have been developed and applied to real world problems. However, these methods represent their solutions using strings or tree structures, so the abilities of representing solutions and the evolution are not enough in terms of the system modeling and the optimization. Therefore, Genetic Network Programming was proposed and its effectiveness was confirmed. In this research, the following extensions and the real world applications of GNP are studied.1. Extensions of GNPCombination of learning and evolution, function localized GNP, GNP with Macro nodes, GNP with Symbiotic learning and evolution, Variable-size GNP2. Applications of GNPElevator Group Supervisory Control Systems, Data Mining, Stock Trading Model In GNP, judgment nodes and processing nodes are connected with directed links with each other. The graph structure contributes to reusing nodes, good expression ability, simplicity of understanding the algorithms and good performance of evolution. In addition, unlike Finite Automata, GNP uses only the necessary information at the current time to judge the situation, so GNP can be evolved under Partially Observable Markov Decision Process. As a result, the applicable field of GNP can be extended

Genetic Algorithm (GA), a method which imitates nature and living things, was proposed as a model which explains the adaptive process of the system of nature. In addition, Genetic Programming (GP) was proposed to deal with knowledge expression, programs, concept trees and so on. Many algorithms of these kinds of evolutionary computations have been developed and applied to real world problems. However, these methods represent their solutions using strings or tree structures, so the abilities of representing solutions and the evolution are not enough in terms of the system modeling and the optimization. Therefore, Genetic Network Programming was proposed and its effectiveness was confirmed. In this research, the following extensions and the real world applications of GNP are studied.
1. Extensions of GNP
Combination of learning and evolution, function localized GNP, GNP with Macro nodes, GNP with Symbiotic learning and evolution, Variable-size GNP
2. Applications of GNP
Elevator Group Supervisory Control Systems, Data Mining, Stock Trading Model In GNP, judgment nodes and processing nodes are connected with directed links with each other. The graph structure contributes to reusing nodes, good expression ability, simplicity of understanding the algorithms and good performance of evolution. In addition, unlike Finite Automata, GNP uses only the necessary information at the current time to judge the situation, so GNP can be evolved under Partially Observable Markov Decision Process. As a result, the applicable field of GNP can be extended.

Intelligent system consisting of multi-individuals interacting between each other have been developed and the learning and evolution of the system have been studied. It is shown that the proposed system has better performance than conventional methods.1.Intelligent agents : Genetic Network Programming (GNP) has been developed as intelligent agents, which represents solutions using directed graph structures, and showed better performance than the conventional method in creating agent behavior.2.Interaction between agents : In order to create intelligent interaction between agents composed of GNP, Multi-Agent Systems with Symbiotic Learning and Evolution (Masbiole) has been developed, which applied symbiotic relations in the ecosystem such as Mutualism, Predation, Competition and Altruism. From the results of the simulations using Tileworld problem, it is clarified that the proposed method shows complex interaction between agents by introducing the symbiotic strategies.3.Learning and Evolution in Multi-agent systems : Generally, living things has been developing through evolution and learning. Evolution has been done for a long period of time, and learning is done based on trial-and-error during lifetime of each individual. Based on this idea, a GNP algorithm has been developed by -using evolution and reinforcement learning. This method can automatically create programs which show better performance than the GNP based on evolution only and the conventional evolutionary computation (Genetic Programming) because the proposed method can improve the programs during task execution (online learning) in addition to the evolution executed after task execution

Intelligent system consisting of multi-individuals interacting between each other have been developed and the learning and evolution of the system have been studied. It is shown that the proposed system has better performance than conventional methods.1.Intelligent agents : Genetic Network Programming (GNP) has been developed as intelligent agents, which represents solutions using directed graph structures, and showed better performance than the conventional method in creating agent behavior.2.Interaction between agents : In order to create intelligent interaction between agents composed of GNP, Multi-Agent Systems with Symbiotic Learning and Evolution (Masbiole) has been developed, which applied symbiotic relations in the ecosystem such as Mutualism, Predation, Competition and Altruism. From the results of the simulations using Tileworld problem, it is clarified that the proposed method shows complex interaction between agents by introducing the symbiotic strategies.3.Learning and Evolution in Multi-agent systems : Generally, living things has been developing through evolution and learning. Evolution has been done for a long period of time, and learning is done based on trial-and-error during lifetime of each individual. Based on this idea, a GNP algorithm has been developed by -using evolution and reinforcement learning. This method can automatically create programs which show better performance than the GNP based on evolution only and the conventional evolutionary computation (Genetic Programming) because the proposed method can improve the programs during task execution (online learning) in addition to the evolution executed after task execution

Since the first proposal of a neuron model by Mc Culloch and Pitts in the 1940's, especially after the revitalization of artificial neural networks in 1980's, a variety of neural networks have been devised and are now applied in many fields. The vast majority of neural networks in use are those networks whose parameters or weights are tuned by gradiant-based supervised learning. This category includes feedfoward networks or multilayer parceptrons, various types of recurrent neural networks, radial basis function networks, fuzzy neural networks, and some networks with special architectures, such as time delay neural networks. These networks seemingly have different architectures and are trained by distinguishable training algorithms. In essence, however, they can be unified in a single framework in regard to both their architectures and learning algorithms. Universal Learning Networks (ULN's) have been proposed, as the name indicates, to provide a universal framework for the class of neural networks and moreover to model and control complex systems because most of the general complex systems in the real world can be modeled by the networks whose nodes represent the processing elements, and the branch between the nodes can describe the relation among the processes. Unification of a variety of network architectures which can describe the complex systems and unification of their learning algorithms are an objective of ULN's. This provides a consistent viewpoint for the various kinds of networks

Autonomous systems with internal models requires learning algorithms based on variety of trials, methods for extracting general knowledge out of trained results, and hierarchical control structure and its switching mechanisms. To meet these requisites, the following have been studied.As a tool for learning based on variety of trials, the following have been devised : a control mechanism of chaotic behaviors which can produce non-deterministic trials, an optimization method which controls the search range as required, genetic symbiosis algorithm that finds various(sub-)optimal solutions with desirable features, and an optimization procedure that tunes its own design parameters. Their validity has been examined by examples.To achieve extraction of general knowledge, the following have been proposed : neural networks with input gates and a method to merge nodes in Radial Basis Function networks. These enables us to represent the knowledge in a generally applicable way which has a fewer number of inputs(conditions). This has been verfied through learning of behaviors of autonomous moving agents.Neural networks with node gates have been also devised which can realize hierarchical control structure as well as its switching mechanism at the same time. This has been confirmed by examples of nonlinear system control. In addition, in learning of behaviors of autonomous moving agents, a mechanism has been added that extract rules applicable to general environments to are. This makes it possible for the agents to adapt new environment more quickly, which has been demonstrated in examples

Neural networks have recently attracted much interest in system control community because they learn any nonlinear mapping. However, from a user's point of view, neural networks are not user friendly, That is, they are not easy-to-use ; more specifically they do not have structures favorable to the applications of system control and fault diagnosis. To solve these problems, the following studies have been carried out.1. A modeling scheme has been developed, which consists of two parts : a macro-net part and kernel-net part. The macro-net part is a user-friendly interface constructed using application specific knowledge and the nature of network structure. The kernel-net part is a flexible multi-input-multi-output (HIMO) nonlinear model such as neural networks and neurofuzzy networks.2. An optimization scheme has been developed. The scheme consists of two learning loops. It has been studied to increase robustness of the algorithm to local minima and over-fitting, by using such as homotopy and hierarchical techniques.3. Applications of the proposed modeling scheme to controller design and fault detection of nonlinear systems have been studied. Some new approaches are proposed and confirmed through numerical simulations

　準線形サポートベクターマシン（SVM）では、問題毎に最適なカーネルを機械学習により構築を行っているが、従来のSVMと同じように訓練では、O(n^3)の計算量とO(n^2)の保存空間が必要であり、訓練データが増えると、訓練するための計算量と保存空間が膨大となり、小規模な訓練データを持つ問題へ適用しかできないという課題がある。準線形SVMを大規模な訓練データを有する問題、いわゆるビッグデータ問題に適用できるようにするため、訓練方法の改良が必要となる。そこで、本研究では、1) 準線形SVMの訓練問題を最小包含球（Minimum Enclosing Ball:MEB）を求める問題に変換し、効率的な近似アルゴリズムでMEBを求めることによって、準線形SVMの訓練を高速化すること；2) 逐次最小問題最適化法（Sequential Minimal Optimization: SMO）を利用したMEB求める効率的なアルゴリズムを開発し、メモリー空間が膨大となる問題を解決することを目指している。今年度では、前年度の基本構成の続き、「Active Setの導入」と「２次微分情報の利用」で、より効率的な逐次最小問題最適化法（SMO）の開発の試みをした。

　準線形サポートベクターマシン（SVM）では、問題毎に最適なカーネルを機械学習により構築を行っているが、従来のSVMと同じように訓練では、O(n^3)の計算量とO(n^2)の保存空間が必要であり、訓練データが増えると、訓練するための計算量と保存空間が膨大となり、小規模な訓練データを持つ問題へ適用しかできないという課題がある。本研究では、準線形SVMを大規模な訓練データを有する問題に適用できるようにするため、高速化訓練法の開発を行う。本年度では、具体的に、１）準線形SVMの訓練問題を最小包含球（MEB）の求める問題に変換し、効率的な近似アルゴリズムでMEBを求めることにより準線形SVMの高速訓練法を検討した。２）逐次最小問題最適化法を利用したMEB求めるアルゴリズムを開発し、保存空間が膨大となる問題を解決する試みをした。

　進化的手法は、全局的な探索手法として工学的に幅広く応用されているが、対象問題の解空間の情報を考慮しないため、膨大な解空間が有する複雑な最適化問題には、適用しにくい問題点がある。そこで、本研究では、対象問題の解空間の情報を自動的に抽出し活用しながら適応型進化的手法の探索を誘導するように構築を行う。本年度では、進化的手法一種である遺伝的アルゴリズム（Genetic Algorithm, GA）を改良し、腎臓の糸球体の抽出に適用した。①Cannyオペレータによる抽出した腎臓の切片画像の部分目標エージと②既に隣の切片画像から抽出された目標エージの情報をＧＡのコーディングにおいて有効に活用したハイブリッドGAによるエージ抽出法を提案した。提案法では計算時間の縮短と抽出精度の向上を同時に実現したことを実データによる確認した。

　進化的手法は、生物の遺伝子の複製、選択淘汰のメカニズムをモデルとしたもので、全局的な探索手法として工学的に幅広く応用されているが、対象問題の解空間の情報を考慮しないため、膨大な解空間が有し、候補解の評価に時間が掛かるという複雑な最適化問題には、適用しにくい問題点がある。そこで、ゲノムの解析や機械翻訳などの応用を想定した適応型進化的システムの知的構築を行う。すなわち、進化的手法を適応化することだけでなく、対象問題の解空間の情報を自動的に抽出し活用しながら適応型進化的手法の探索を誘導するように構築を行う。　本年度では、進化的手法一種である差分進化（Differential Evolution, DE）を改良し、様々な考察を行った。　①TOP階DEとBOTTOM階DEという二つのDEからなる２階構造を有した差分進化について考察した。TOP階のSimplified DEは解空間の情報を抽出し、これを活用しながらBOTTOM階の差分進化DEの進化パラメータを自動的に適応することを実現する。また、BOTTOM階の差分進化DEはダブルセットの進化パラメータを有し、並列的に進化することにより進化パラメータの評価を行う。　②ニッチング差分進化について考察した。初めに、Clearing Niching Methodにより差分進化の集団全体の適応度を向上すると同時に集団個体の多様性を維持する。次に、個体の適応度や探索空間の探索履歴情報などを取り入れ、ボルツマン(Boltzmann)選択機構で、集団全体の適応度向上と個体の多様性維持という二つの対立の目的を有する操作を適応的に調整行う。これにより差分進化をロバスト的・効率的にする。　③２０個の最適化問題からなるベンチマークによる数値実験を行い、探索結果や探索成功率などを指標として、改良した差分進化法の基本性能を考察し、提案法の有効性を明らかにした。

本研究では、準線形学習ネットワークとして利用しやすい線形回帰構造を有する準線形サポートベクターマシン（SVM）の構築と応用を行っている。具体的に、①　準線形SVMの構築について、非線形システムを効率的に近似できる補間によるマルチ局所線形モデルを構築し、SVM学習法を適用することによって、線形カーネルと非線形カーネルの間に調整できる準線形カーネルを持つ準線形SVMの構築を構築する。クラスタリングなどの技術で、対象システムの入力空間の局所的線形特性を持つ分割情報を抽出し、これらの情報に基づいて、応用対象ごとに最適なカーネルを合成し、その合成カーネルでSVMの学習を通して準線形SVMの知的構成と機械学習による体系的な学習を実現する。一例として、制御系設計などに利用しやすい構造を持つ準線形SVMを構築した。まず、応用対象の物理の法則などの数式で表せる先見情報だけでなく、応用に望ましいネットワーク構造（例えば、制御系設計を容易にするための制御入力変数に関し線形なるようなネットワーク構造）という利用しやすい構造(Macro-Net)を取り入れた利用しやすい準線形回帰モデルを構成し、これを基底関数（RBF）の補間によるマルチ局所線形化し、さらにSVMの学習を適用することによって、利用しやすい構造を有する準線形SVMを構築した。②　応用について、準線形SVMの線形特性を生かした非線形ダイナミカルシステムの適応制御を例として行った。線形予測器と非線形予測器を同時に同定するための準線形モデリング技術、安定な線形適応制御技術と高精度の非線形適応制御技術、制御系の安定性を確保しながら制御精度を向上するスイッチング制御技術の開発を行った。具体的に、制御系の安定性を示す指標を導入したスイッチング機構により、制御系の安定性に余裕があるときには、非線形適応制御系を中心にして制御精度を向上させ、制御系の安定性が欠如したときには、線形適応制御系を中心にして制御系の安定性を向上させた。このように安定な線形適応制御と高精度の非線形適応制御を自動的に切換えることによって、準線形SVMという一つのモデルで制御系の安定性を保証しながら制御精度を向上することを実現した。

確率モデルを基づいた分布推定アルゴリズム(EDA)は近年一種の新しい進化論的な手法として注目されている。本研究では、EDAを複雑な最適化問題へ適用するために、ロバストな適応的EDA進化システムを構築する。まず、AP(Affinity Propagation)クラスターリング法で集団を分割し適応的ニッチを構成する。次に、個体の適応度や探索空間の探索履歴情報などを取り入れ、ボルツマン(Boltzmann)選択機構で適応的に個体の選択を行うような適応的EDAを構築した。さらに、この適応的EDAをベースにして、EDAの集団全体の適応度を向上すると共に、集団における個体の多様性も保つように工夫をし、多様性の維持する進化を陰進化、適応度の向上する進化を陽進化として、二つの対立・統一な進化メカニズムを有する陰陽進化システムへの拡張を行った。また、提案法をHPモデルによるタンパク質構造の予測問題や準ARXニューラルネットワークと非線形多項式モデルのパラメータ推定問題に適用し、その有効性を確認した。

遺伝的ネットワークプログラミング（Genetic Network Programming：GNP)は遺伝的アルゴリズム（GA)や遺伝的プログラミング（GP)とは異なり、有向グラフを遺伝子とする新しい進化型計算アルゴリズムである。本研究では、この新しいGNPを活用して、従来のニューラルネットワーク、相関ルール、決定木・回帰木とは異なる新しい進化型のデータマイニング手法の基本技術について研究を行い、GNPを用いた興味深い相関ルールの抽出法を提案した。提案法では、相関ルールの指標はGNPの構造的な特徴を利用して算出される。ルール抽出は世代継続的に行われるため抽出された相関ルールはライブラリに蓄積される。抽出された相関ルールに関する情報は、抽出を継続中のGNPの個体評価および進化操作時に用いられると新しい進化方法を用いており、より効率的に興味深い相関ルールの抽出を行うことができる。

本研究では、連続システムの機能局在と階層構成に焦点を絞って研究し、教師あり学習・教師なし学習・強化学習を複合したbrain-like学習システムを構築した。　我々の脳が持つ学習方法は、人工知能の分野では教師あり学習、教師なし学習、強化学習という３つのアルゴリズムに分類される。脳内では、これらの異なる学習方法が単体ではなく、適切に組み合わされて使用しており、それによって我々は非常に高度な情報しょりを実現していると言われている。これらの知見から、教師あり学習、教師なし学習、強化学習をそれぞれ適用した、３つ部分からなる知的学習システムを開発した。　（１）教師あり学習部分（SL part): SL partは問題を解くための主要な部分であり、構造は通常の3層階層型ANNと同じであるが、中間層の各ノードの発火強度がUL partからの信号によって制御される。　（２）教師なし学習部分（UL part):　UL partは競合学習ネットワークで構成され、入力空間の分割と、SL partの中間層ノードの発火強度の制御を行う。　（３）強化学習部分（RL part)：　RL partは、強化学習によってUL partの出力を調整することで、SL partの性能を最適化するための発火強度制御信号を自動的に決定する。

　遺伝的ネットワークプログラミング（Genetic Network Programming：GNP)は遺伝的アルゴリズム（GA)や遺伝的プログラミング（GP)とは異なり、有向グラフを遺伝子とする新しい進化型計算アルゴリズムである。本研究では、この新しいGNPを活用して、従来のニューラルネットワーク、相関ルール、決定木・回帰木とは異なる新しい進化型のデータマイニング手法の基本技術について研究を行った。　GNPは判定ノードと処理ノードがネットワーク状に結合しており、判定ノードでは環境からの情報を判定し多分岐し、処理ノードでは環境に対して出力を行なうノードである。GNPの特徴は、「１」有限状態機械と異なり必要な情報を必要なときに取り込むことが出来るため、部分マルコフ決定過程のプロセスをモデル化できる；「２」判定ノードおよび処理ノードを重複して活用できるため、コンパクトな構成が可能になる；「３」有向グラフによるネットワーク構成のため、GNPの内部に過去のノード遷移の履歴を記憶することができる等である。このGNPをデータマイニングに活用し、GNPの判定ノードと処理ノードの進化による柔軟な組み合わせにより、データの中の普遍的な知識が効率的に抽出可能なことに着目して、ルール抽出手法を開発し、エレベータ群管理システムの呼び割付けや株価指数の予測などへ展開も行った。

ハーバート・A・サイモンが名著“システムの科学”で述べているように複雑なシステムの特徴は、各種の機能がシステム内に機能局在と全体システムが上位・下位のサブシステムよりなる階層構造である。本研究では、連続および離散複雑システムの共通の特質である機能局在と階層構造を連続と離散両システムに統一した概念で学習と進化により知的に構成について研究を行った。（１）連続系複雑システムの実現の基本要素として、自己組織化機能局在型ニューラルネットワーク（Function Localization Neural network: FLNN) を提案し、次の技術について研究を行った。　　・FLNNで構成された複数個のモジュールネットワークを動的にオーバーレッピングすることにより連続系複雑システムの機能局在を実現する技術；　　・基本ネットと制御ネットの階層構成により連続系複雑システムを構築する技術；　　・機能局在・階層構成の連続系複雑システムを知的に構築するための教師あり学習、教師なし学習および強化学習からなるbrain-like学習技術。（２）離散系複雑システムの実現の基本要素として、提案しているネットワークを遺伝子とする進化型計算アルゴリズム（Genetic Network Programming: GNP)に関して、次の技術について研究を行った。　　・GNPの中にオーバーラップする複数のモジュールGNPを内蔵し、これらが相互作用する機能局在型の離散システムを構築する技術。

脳科学の知見によれば、脳では様々な状況の経験を積むことによって同じような状況に遭遇したときにそれまでの経験と照らし合わせて最善の行動を選べる。これは脳における「学習」という。また、脳では音楽、運動などの個々の知性が働く時、脳の異なる部位が活動する現象があり、これは脳における「機能局在」が実現されているという。1949年、心理学者Hebbがセルアセンブリ(cell assembly)という脳構成理論を提案した。脳神経回路では、異なる回路間でのニューロンの重複(neurons overlapping)し、機能的シナプス結合の変化による回路自体の動的な変化(connection dynamics)される。 脳の原理を工学的に展開する研究について、1980年代から90年代にかけて人工的ニューラルネットワークとその学習アルゴリズム、記憶方式、カオスとの関連等の研究が進められているが、殆どの研究は脳の「学習」機能だけに注目し、脳の「機能局在」を実現した研究は少ない。そこで本研究では、上記のHebbのセルアセンブリ理論を参考にした学習能力と機能局在性を両方有する自己組織化機能局在ニューラルネットワーク（ＦＬＮＮ）を提案している。提案した自己組織化ＦＬＮＮは二つ部分、メイン部とコントロール部で構成される。メイン部は通常の３層フィードフォワードニューラルネットワークであるが、中間層の各ニューロンの発火強度はコントロール部からの信号によって制御される。コントロール部は自己組織化マップ（ＳＯＭ）ネットワークであり、その出力はメイン部の中間層ニューロンと関連付けられている。ＳＯＭによるコントロール部は入出力空間の構造的特徴を抽出し、メイン部の中間層ニューロンの発火強度を制御する。これにより自己組織化ＦＬＮＮは学習能力だけでなく機能局在化能力も実現されている。また、数値シミュレーションを通して提案法の有効性を確認している。