Many recent sequential recommendation methods have aimed to enhance the modeling of users' diverse interests to provide more accurate and varied recommendations. Despite great success, existing models possess a limitation in their capacity to solely perceive items within a user behavior sequence. Such an approach may potentially constrain their ability to effectively discover users' multiple interests. In this paper, we present a novel approach that addresses this gap by harnessing global item transitions to uncover users' potential interests, subsequently modeling them as local subsequences. To achieve this objective, we represent global item transitions as a directed graph, constructed by amalgamating behavior sequences from the training set. This graph‐based representation allows the design of a specialized module capable of aggregating global item transitional information and uncovering local subsequences specific to individual users. Consequently, downstream sequence models can focus on pertinent item transitions, disregarding unrelated ones. We evaluate the effectiveness of our method on four public data sets, outperforming existing alternatives or delivering competitive results. © 2023 Institute of Electrical Engineer of Japan and Wiley Periodicals LLC.

Pedestrians are essential component in traffic, and with the advent of intelligent transportation systems, conflicts over right‐of‐way between pedestrians and other traffic participants have become prevalent. Predicting pedestrian intentions and motions is crucial for autonomous driving and intelligent transportation. However, pedestrian prediction faces challenges such as the variability of pedestrian intentions and the flexibility of their motions, as well as the need for highly generalization capability and lightweight modeling on autonomous vehicles with limited computing power. To face these challenge, this paper proposes a Social Artificial Potential Field (Social‐APF) method for pedestrian trajectory prediction. The method combines target intention feature and surrounding obstacles features. We test the performance of Social‐APF on several public datasets. The results demonstrate that the Social‐APF method outperforms state‐of‐the‐art approaches on the public datasets in both lightweight and model accuracy. © 2023 Institute of Electrical Engineer of Japan and Wiley Periodicals LLC.

Misinformation posted on social media during COVID-19 is one main example of infodemic data. This phenomenon was prominent in China when COVID-19 happened at the beginning. While a lot of data can be collected from various social media platforms, publicly available infodemic detection data remains rare and is not easy to construct manually. Therefore, instead of developing techniques for infodemic detection, this paper aims at constructing a Chinese infodemic dataset, “infodemic 2019”, by collecting widely spread Chinese infodemic during the COVID-19 outbreak. Each record is labeled as true, false or questionable. After a four-time adjustment, the original imbalanced dataset is converted into a balanced dataset by exploring the properties of the collected records. The final labels achieve high intercoder reliability with healthcare workers’ annotations and the high-frequency words show a strong relationship between the proposed dataset and pandemic diseases. Finally, numerical experiments are carried out with RNN, CNN and fastText. All of them achieve reasonable performance and present baselines for future works.

During the outbreak of the COVID-19 (2019 coronavirus disease), misinformation related to the virus spread rapidly online and have led to serious difficulties in controlling the disease. The term infodemic is coined to outline the bad effect from the extensive dissemination of misinformation during the outbreak. With regards to this phenomenon, the World Health Organization emphasized the need to fight against infodemic and asked all countries not only to make efforts in slowing down the spread of the COVID-19 but also in countering the risk caused by infodemic. Due to its negative impact, this paper analyzes infodemic on Chinese social media at the initial stage of the COVID-19 outbreak and presents a 4P framework standing for the four features of Chinese infodemic: Prevention Attention, Problem Orientation, Patterns Interaction and Points Globalization. Furthermore, a selective review of existing datasets in the neural networks domain is synthesized based on the 4P framework. Finally, research directions, including recommendations, about constructing a large-scale dataset for Chinese infodemic automatic detection are proposed.

In this paper, aiming at the cumbersome solution of control law in neural network predictive control algorithm, a quasi-linear neural network identification and predictive control algorithm is proposed. The recurrent neural network is embedded into the quasi-linear model, which can be viewed as a quasi-ARX model macroscopically. In the quasi-linear recurrent neural network predictive control, the solution of the control law only need one-step derivation, which can greatly simplify the solution process of control law. At the same time, the quasi-linear recurrent neural network can effectively restrain the over-fitting problem in the identification process. Theoretical analysis and simulations are given to prove the simplicity and effectiveness of the proposed computing method.

Genetic Algorithms are commonly used to generate high-quality solutions to combinational optimization problems. However, the execution time can become a limiting factor for large and complex problems. In this paper, we propose a parallel Genetic Algorithm consisting of an island model at the upper level and a fine-grained model at the lower level. This design is highly consistent with the CUDA framework in order to get the maximum speedup without compromising to solutions' quality. As several parameters control the performance of the hybrid method, we test them by a flexible flow shop scheduling problem and analyze their influence. Finally, numerical experiments show that our approach cannot only obtain competitive results but also reduces execution time by setting a medium size selection diameter, a relatively large island size and a wide range size migration interval.

Data imbalance occurs on most real-world classification problems and decreases the performance of classifiers. An oversampling method addresses the imbalance problem by generating synthetic samples to balance the data distribution. However, many of the existing oversampling methods have potential problems in generating wrong and unnecessary synthetic samples, which makes the learning tasks difficult. This paper proposes a new segmented oversampling method for imbalanced data classification. The input space is first partitioned into several linearly separable local partitions along the potential separation boundary by introducing a bottom-up, minimal-spanning-tree-based clustering method; an oversampling method is then applied within each local linear partition to prevent the generation of wrong and unnecessary synthetic samples; a quasi-linear support vector machine is finally used to realize the classification by taking advantages of the local linear partitions. Simulation results on different real-world datasets show that the proposed segmented oversampling method is effective for imbalanced data classifications. (C) 2017 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

This paper proposes a two-step method to construct a nonlinear classifier consisting of multiple local linear classifiers interpolated with a basis function. In the first step, a geometry-based approach is first introduced to detect local linear partitions and build local linear classifiers. A coarse nonlinear classifier can then be constructed by interpolating the local linear classifiers. In the second step, a support vector machine (SVM) formulation is used to further implicitly optimize the linear parameters of the nonlinear classifier. In this way, the nonlinear classifier is constructed in exactly the same way as a standard SVM, using a special data-dependent quasi-linear kernel composed of the information of the local linear partitions. Numerical experiments on several real-world datasets demonstrate the effectiveness of the proposed classifier and show that, in cases where traditional nonlinear SVMs run into overfitting problems, the proposed classifier is effective in improving the classification performance. (C) 2017 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

Quasi-linear autoregressive with exogenous inputs (Quasi-ARX) models have received considerable attention for their usefulness in nonlinear system identification and control. In this paper, identification methods of quasi-ARX type models are reviewed and categorized in three main groups, and a two-step learning approach is proposed as an extension of the parameter-classified methods to identify the quasi-ARX radial basis function network (RBFN) model. Firstly, a clustering method is utilized to provide statistical properties of the dataset for determining the parameters nonlinear to the model, which are interpreted meaningfully in the sense of interpolation parameters of a local linear model. Secondly, support vector regression is used to estimate the parameters linear to the model
meanwhile, an explicit kernel mapping is given in terms of the nonlinear parameter identification procedure, in which the model is transformed from the nonlinear-in-nature to the linear-in-parameter. Numerical and real cases are carried out finally to demonstrate the effectiveness and generalization ability of the proposed method.

The multiple linear model is used successfully to extend the linear model to nonlinear problems. However, the conventional multilinear models fail to learn the global structure of a training data set because the local linear models are independent of each other. Furthermore, the local linear transformations are learned in the original space. Therefore, the performance of multilinear methods is strongly dependent on the results of partition. This paper presents a kernel approach for the implementation of the local linear discriminant analysis for face recognition problems. In the original space, we utilize a set of local linear transformations with interpolation to approximate an optimal global nonlinear transformation. Based on the local linear models in the original space, we derive an explicit kernel mapping to map the training data into a high-dimensional transformed space. The optimal transformation is learned globally in the transformed space. Experimental results show that the proposed method is more robust to the partition results than the conventional multilinear methods. Compared with the general nonlinear kernels that utilize a black-box mapping, our proposed method can reduce the negative effects caused by the potential overfitting problem. (c) 2016 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

This paper proposes a deep quasi-linear kernel for support vector machines (SVMs). The deep quasi-linear kernel can be constructed by using a pre-trained deep neural network. To realize this goal, a multilayer gated bilinear classifier is first designed to mimic the functionality of the pre-trained deep neural network, by generating the gate control signals using the deep neural network. Then, a deep quasi-linear kernel is derived by applying an SVM formulation to the multilayer gated bilinear classifier. In this way, we are able to further implicitly optimize the parameters of the multilayer gated bilinear classifier, which are a set of duplicate but independent parameters of the pre-trained deep neural network, by using an SVM optimization. Experimental results on different data sets show that SVMs with the proposed deep quasi-linear kernel have an ability to take advantage of the pre-trained deep neural networks and outperform SVMs with RBF kernels.

The three dimensional (3D) reconstruction of a medical image sequence can provide intuitive morphologies of a target and help doctors to make more reliable diagnosis and give a proper treatment plan. This paper aims to reconstruct the surface of a renal corpuscle from the microscope renal biopsy image sequence. First, the contours of renal corpuscle in all slices are extracted automatically by using a context-based segmentation method with a coarse registration. Then, a new coevolutionary-based strategy is proposed to realize a fine registration. Finally, a Gauss-Seidel iteration method is introduced to achieve a non-rigid registration. Benefiting from the registrations, a smooth surface of the target can be reconstructed easily. Experimental results prove that the proposed method can effectively register the contours and give an acceptable surface for medical doctors.

The quasi-linear ARX radial basis function network (RBFN) model has shown good approximation ability and usefulness in nonlinear system identification and control. It has an easy-to-use structure, good generalization and strong tolerance to input noise. In this paper, we propose a self-organizing quasi-linear ARX RBFN (QARX-RBFN) model by introducing a self-organizing scheme to the quasi-linear ARX RBFN model. Based on the active firing rate and the mutual information of RBF nodes, the RBF nodes in the quasi-linear ARX RBFN model can be added or removed, so as to automatically optimize the structure of the quasi-linear ARX RBFN model for a given system. This significantly improves the performance of the model. Numerical simulations on both identification and control of nonlinear dynamical system confirm the effectiveness of the proposed self-organizing QARX-RBFN model.

In this paper, a new switching mechanism is proposed based on the state of dynamic tracking error so that more information will be provided -not only the error but also a one up to pth differential error will be available as the switching variable. The switching index is based on the Lyapunov stability theory. Thus the switching mechanism can work more effectively and efficiently. A simplified quasi-ARX neural-network (QARXNN) model presented by a state-dependent parameter estimation (SDPE) is used to derive the controller formulation to deal with its computational complexity. The switching works inside the model by utilizing the linear and nonlinear parts of an SDPE. First, a QARXNN is used as an estimator to estimate an SDPE. Second, by using SDPE, the state of dynamic tracking error is calculated to derive the switching index. Additionally, the switching formula can use an SDPE as the switching variable more easily. Finally, numerical simulations reveal that the proposed control gives satisfactory tracking and disturbance-rejection performances. Experimental results demonstrate its effectiveness. (c) 2015 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

In the multi-label classification issue, some implicit constraints and dependencies are always existed among labels. Exploring the correlation information among different labels is important for many applications. It not only can enhance the classifier performance but also can help to interpret the classification results for some specific applications. This paper presents an improved multi-label classification method based on local label constraints and classifier chains for solving multi-label tasks with large number of labels. Firstly, in order to exploit local label constraints in multi-label problem with large number of labels, clustering approach is utilized to segment training label set into several subsets. Secondly, for each label subset, local tree-structure constraints among different labels are mined based on mutual information metric. Thirdly, based on the mined local tree-structure label constraints, a variant of classifier chain strategy is implemented to enhance the multi-label learning system. Experiment results on five multi-label benchmark datasets show that the proposed method is a competitive approach for solving multi-label classification tasks with large number of labels.

Automatic registration is an important task to prepare aligned 2D images for 3D structure visualization, and it is a challenging problem especially for the microscope images. This paper proposes a novel coevolution-based coarse-to-fine registration method, aiming to align the regions of interest (ROIs) in the image sequence. Firstly, a coarse registration for whole images is executed by a scale-invariant feature transform (SIFT) based method, which can facilitate the segmentation of ROIs. Secondly, a fine registration for the segmented ROIs is done by a genetic algorithm (GA) with a novel coevolutionary strategy. Experimental results demonstrate the good performance of the proposed method and it is also successfully applied to the renal biopsy image sequence.

The control system is a key technology to extract maximum energy from the incident wind. By regulating aerodynamic control, it is possible to adapt the changes in wind speed by controlling shaft speed. Thus, the turbine generator can track maximum power extracted from wind. In this paper, we propose a Lyapunov based switching control under quasi-linear ARX neural network (QARXNN) model to track maximum power of wind energy conversion system. The switching index is used to measure the stability of nonlinear controller and selects linear or nonlinear controller in order to ensure the stability. Interestingly, a simple switching law can be built utilizing the parameters of model directly. Finally, we have compared the proposed algorithm of switching controller with another algorithm. The results show that the proposed algorithm has better control performance.

Kernel functions based machine learning algorithms have been extensively studied over the past decades with successful applications in a variety of real-world tasks. In this paper, we formulate a kernel level composition method to embed multiple local classifiers (kernels) into one kernel function, so as to obtain a more flexible data-dependent kernel. Since such composite kernels are composed by multiple local classifiers interpolated with several localizing gating functions, a specific learning process is also introduced in this paper to pre-determine their parameters. Experimental results are provided to validate two major perspectives of this paper. Firstly, the introduced learning process is effective to detect local information, which is essential for the parameter pre-determination of the localizing gating functions. Secondly, the proposed composite kernel has a capacity to improve classification performance.

In this paper, we introduce a data-dependent kernel called deep quasi-linear kernel, which can directly gain a profit from a pre-trained feedforward deep network. Firstly, a multi-layer gated bilinear classifier is formulated to mimic the functionality of a feed-forward neural network. The only difference between them is that the activation values of hidden units in the multi-layer gated bilinear classifier are dependent on a pre-trained neural network rather than a pre-defined activation function. Secondly, we demonstrate the equivalence between the multi-layer gated bilinear classifier and an SVM with a deep quasi-linear kernel. By deriving a kernel composition function, traditional optimization algorithms for a kernel SVM can be directly implemented to implicitly optimize the parameters of the multi-layer gated bilinear classifier. Experimental results on different data sets show that our proposed classifier obtains an ability to outperform both an SVM with a RBF kernel and the pre-trained feedforward deep network.

The quasi-linear ARX radial basis function network (QARX-RBFN) model has shown good approximation ability and usefulness in nonlinear system identification and control. It owns an ARX-like structure, easy design, good generalization and strong tolerance to input noise. However, the QARX-RBFN model still needs to improve the prediction accuracy by optimizing its structure. In this paper, a novel self-organizing QARX-RBFN (SOQARX-RBFN) model is proposed to solve this problem. The proposed SOQARX-RBFN model consists of simultaneously network construction and parameter optimization. It offers two important advantages. Firstly, the hidden neurons in the SOQARX-RBFN model can be added or removed, based on the neuron activity and mutual information (MI), to achieve the appropriate network complexity and maintain overall computational efficiency for identification. Secondly, the model performance can be significantly improved through the structure optimization. Additionally, the convergence of the SOQARX-RBFN model is analyzed, and the proposed approach is applied to identify and control the nonlinear dynamical systems. Mathematical system simulations are carried out to demonstrate the effectiveness of the proposed method.

By itself, a wind turbine is already a fairly complex system with highly nonlinear dynamics. Changes in wind speed can affect the dynamic parameters of wind turbines, thus rendering the parameters uncertain. However, we can identify the dynamics of the wind energy conversion system (WECS) online by a quasi-ARX neural network (QARXNN) model. A QARXNN presents a problem in searching for the coefficients of the regression vector (input vector). A multilayer perceptron neural network (MLPNN) is an embedded system that provides the unknown parameters used to parameterize the input vector. Fascinatingly, the coefficients of the input vector from prediction model can be set as controller parameters directly. The stability of the closed-loop controller is guaranteed by the switching of the linear and nonlinear parts of the parameters. The dynamic of WECS is derived with given parameters, and then a wind speed signal created by a random model is fed to the system causing uncertainty parameters and reducing the power that can be absorbed from wind. By using a minimum variance controller, the maximum power is tracked from WECS. From the simulation results, it is observed that the proposed controller is effective in tracking the maximum power of WECS. (c) 2015 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

A renal biopsy is a procedure to get a small piece of kidney for microscopic examination. With the development of tissue sectioning and medical imaging techniques, microscope renal biopsy image sequences are consequently obtained for computer-aided diagnosis. This paper proposes a new context-based segmentation algorithm for acquired image sequence, in which an improved genetic algorithm (GA) patching method is developed to segment different size target. To guarantee the correctness of first image segmentation and facilitate the use of context information, a boundary fusion operation and a simplified scale-invariant feature transform (SIFT)-based registration are presented respectively. The experimental results show the proposed segmentation algorithm is effective and accurate for renal biopsy image sequence.

This paper proposes a general nonlinear adaptive predictor using a class of neurofuzzy models. The obtained predictor may be seen as a linear predictor network consisting of a global linear predictor and several local linear predictors with interpolation. It has distinctive features as well as good prediction ability: its parameters have explicit meanings useful for initial values setting: it may be transformed into a form linear for the variables synthesized in control systems, making deriving a control law straightforward.

This paper presents a new strategy to build multi tree hierarchical structure SVM which can get a more efficient and accuracy classification model for multiclass problems. Base on the theory of Binary Tree SVM (BTS), we proposed an improvement algorithm which extend binary tree structure to a multi tree structure, In the multi tree hierarchical structure, similarity clustering method was proposed to cluster classes to groups in each non-leaf node. In order to get a multi node division, one-against-all (OAA) was applied to train those groups rather than classes. The proposed method can avoid data imbalanced problem occurred in OAA, also the classification area of classifier in the upper layer is larger than classifier in lower layer. Compared with other several well-known methods, experiments on many data sets demonstrate that our method can reduce the number of classifiers in the testing phase and get a higher accuracy.

This paper proposes an improved SVM based multi-label classification method by using relationship among labels. Following a traditional multi-label solution, binary relevance (BR) method is first used to decompose the multi-label classification problem into multiple binary classification sub-problems, each of which is solved by an SVM classifier. By using Platt's sigmoid technique, each SVM classifier gives probability output for the following correction. A probability model is introduced to estimate the relationship among labels. The extracted label relationship is then applied to correct the outputs of SVM classifiers, in which a dynamic weight strategy is further introduced. Numerical experiments on widely used benchmark datasets show that the proposed method can improve the accuracy of multi-label classification when compared with traditional BR method and some other conventional multi-label classification methods.

This paper proposes a geometric way to construct a quasi-linear kernel by which a quasi-linear support vector machine (SVM) is performed. A quasi-linear SVM is a SVM with quasi-linear kernel, in which the nonlinear separation boundary is approximated by using multi-local linear boundaries with interpolation. However, the local linearity extraction for the composition of quasi-linear kernel is still an open problem. In this paper, according to the geometric theory, a method based on piecewise linear classifier is proposed to extract the local linearity in a more precise and efficient way. We firstly construct a function set including multiple linear functions and each of those functions reflects one part of linearity of the whole nonlinear separation boundary. Then the obtained local linearity is added as prior information into the composition of quasi-linear kernel. Experimental results on synthetic data sets and real world data sets show that our proposed method is effective to improve classification performances.

This paper presents a Transductive Support Vector Machine (TSVM) with quasi-linear kernel based on a clustering assumption for semi-supervised classification. Since the potential separating boundary is located in low density area between classes, a modified density clustering method by considering label information is firstly introduced to extract the information of potential separating boundary in low density region between different classes. Then the information is used to compose a quasi-linear kernel for the TSVM. The optimization of TSVM is further speeded up by developing a pair wise label switching method on minimal sets. Experiment results on benchmark datasets show that the proposed

An improved Elman neural network (IENN) controller with particle swarm optimization (PSO) is presented for nonlinear systems. The proposed controller is composed of a quasi-ARX neural network (QARXNN) prediction model and a switching mechanism. The switching mechanism is used to guarantee that the prediction model works well. The primary controller is designed based on IENN using the backpropagation (BP) learning algorithm with PSO. PSO is used to adjust the learning rates in the BP process for improving the learning capability. The adaptive learning rates of the controller are investigated via the Lyapunov stability theorem. The proposed controller performance is verified through numerical simulation. The method is compared with the fuzzy switching and 0/1 switching methods to show its effectiveness in terms of stability, accuracy, and robustness. (C) 2014 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

This paper proposes a so called quasi-linear support vector machine (SVM), which is an SVM with a composite quasi-linear kernel. In the quasi-linear SVM model, the nonlinear separation hyperplane is approximated by multiple local linear models with interpolation. Instead of building multiple local SVM models separately, the quasi-linear SVM realizes the multi local linear model approach in the kernel level. That is, it is built exactly in the same way as a single SVM model, by composing a quasi-linear kernel. A guided partitioning method is proposed to obtain the local partitions for the composition of quasi-linear kernel function. Experiment results on artificial data and benchmark datasets show that the proposed method is effective and improves classification performances.

In this article, a fuzzy adaptive controller approach is presented for nonlinear systems. The proposed quasi-ARX neural network based on Lyapunov learning algorithm is used to update its weight for prediction model as well as to modify fuzzy adaptive controller. The improving performances of the Lyapunov learning algorithm are stable in the learning process of the controller and able to increase the accuracy of the controller as well as fast convergence of error. The simulations are intended to show the effectiveness of the proposed method. © 2013 ISAROB.

Differential evolution (DE) is an effective and efficient evolutionary algorithm in continuous space. The setting of control parameters is highly relevant with the convergence efficiency, and varies with different optimization problems even at different stages of evolution. Self-adapting control parameters for finding global optima is a long-term target in evolutionary field. This paper proposes a two-layered DE (TLDE) with self-adaptive control parameters combined with niching method based mutation strategy. The TLDE consists of two DE layers: a bottom DE layer for the basic evolution procedure, and a top DE layer for control parameter adaptation. Both layers follow the procedure of DE. Moreover, to mitigate the common phenomenon of premature convergence in DE, a clearing niching method is brought out in finding efficient mutation individuals to maintain diversity during the evolution and stabilize the evolution system. The performance is validated by a comprehensive set of twenty benchmark functions in parameter optimization and competitive results are presented.

Support Vector Data Description (SVDD) inherits properties of Support Vector Machines (SVM) and has become a prominent One Class Classifier (OCC). Same to standard SVM, its O(n(3)) time and O(n(2)) space complexities, where n is the number of training samples, have become major limitations in cases of large training datasets. As a simple and effective method, reducing the size of training dataset through reserving only samples mostly relevant to learned classifier, can be adopted to overcome the limitations. A trained SVDD enclosed decision boundary always locates on edge area of data distribution and is decided by a small subset of Support Vectors(SVs). Therefore, in this paper, we present a method based on edge detection such that edge samples mostly relevant to decision boundary can be preserved. And clustering techniques are also be applied to keep centroids representing the global distribution properties so as to avoid over-outside of decision boundary. To restrict the influences of noises, each training pattern is assigned with a weight. Experiments on real and artificial data sets prove that the classifier trained on reconstruction training set consisting of edge points and centroids can preserve performance with much faster training speed.

Clustering, one of the important data mining techniques, has two main processing methods on data-based similarity clustering and space-based density grid clustering. The latter has more advantage than the former on larger and multiple shape and density dataset. However, due to a global partition of existing grid-based methods, they will perform worse when there is a big difference on the density of clusters. In this paper, we propose a novel algorithm that can produces appropriate grid space in different density regions by simulating cell division process. The time complexity of the algorithm is O(n) in which n is number of points in dataset. The proposed algorithm will be applied on popular chameleon datasets and our synthetic datasets with big density difference. The results show our algorithm is effective on any multi-density situation and has scalability on space optimization problems.

This paper focuses on semi-supervised classification problem by using Transductive Support Vector Machine. Traditional TSVM for semi-supervised classification firstly train an SVM model with labeled data. Then use the model to predict unlabeled data and optimize unlabeled data prediction to retrain the SVM. TSVM always uses a predefined kernel and fixed parameters during the optimization procedure and they also suffers potential over-fitting problem. In this paper we introduce proposed quasi-linear kernel to the TSVM. An SVM with quasi-linear kernel realizes an approximate nonlinear separation boundary by multi-local linear boundaries with interpolation. By applying quasi-linear kernel to semi-supervised classification it can avoid potential over-fitting and provide more accurate unlabeled data prediction. After unlabeled data prediction optimization, the quasi-linear kernel can be further adjusted considering the potential boundary data distribution as prior knowledge. We also introduce a minimal set method for optimizing unlabeled data prediction. The minimal set method follows the clustering assumption of semi-supervised learning. The pairwise label switching is allowed between minimal sets. It can speed up optimization procedure and reduce influence from label constrain in TSVM. Experiment results on benchmark gene datasets show that the proposed method is effective and improves classification performances.

This paper proposes a self-organizing maps (SOM) based kernel composition method for the quasi-linear support vector machine (SVM). The quasi-linear SVM is SVM model with quasi-linear kernel, in which the nonlinear separation hyperplane is approximated by multiple local linear models with interpolation. The basic idea underlying the proposed method is to use clustering and projection properties of SOM to partition the input space and construct a SOM based quasi-linear kernel. By effectively extracting the distribution information using SOM, the quasi-linear SVM with the SOM-based quasi-linear kernel is expected to have better performance in the cases of high-noise and high-dimension. Experiment results on synthetic datasets and real world datasets show the effectiveness of the proposed method.

Assembling a large genome using next generation sequencing reads requires large computer memory and a long execution time. To reduce these requirements, a memory and time efficient assembler is presented from applying FM-index in JR-Assembler, called FMJ-Assembler, where FM stand for FMR-index derived from the FM-index and BWT and J for jumping extension. The FMJ-Assembler uses expanded FMindex and BWT to compress data of reads to save memory and jumping extension method make it faster in CPU time. An extensive comparison of the FMJ-Assembler with current assemblers shows that the FMJ-Assembler achieves a better or comparable overall assembly quality and requires lower memory use and less CPU time. All these advantages of the FMJ-Assembler indicate that the FMJ-Assembler will be an efficient assembly method in next generation sequencing technology.

A nonlinear model-predictive control (NMPC) is demonstrated for nonlinear systems using an improved fuzzy switching law. The proposed moving average filter fuzzy switching law (MAFFSL) is composed of a quasi-ARX radial basis function neural network (RBFNN) prediction model and a fuzzy switching law. An adaptive controller is designed based on a NMPC. a MAFFSL is constructed based on the system switching criterion function which is better than the (ON/OFF) switching law and a RBFNN is used to replace the neural network (NN) in the quasi-ARX black box model which is understood in terms of parameters and is not an absolute black box model, in comparison with NN. The proposed controller performance is verified through numerical simulations to demonstrate the effectiveness of the proposed method.

In this paper, a novel approach is proposed as a new fast Support Vector Machine(SVM) basing on sequential minimal optimization(SMO), minimum enclosing ball(MEB) approach and active set strategy. The combination with these 3 techniques largely accelerates the training process of SVM, attains fewer support vectors(SVs) as well as obtains a acceptable accuracy comparing to original SVM. From simulation results, it is stated that the proposed method will be a good alternative for classification of large scale data.

A quasi-ARX (quasi-linear ARX) neural network (QARXNN) model is able to demonstrate its ability for identification and prediction highly nonlinear system. The model is simplified by a linear correlation between the input vector and its nonlinear coefficients. The coefficients are used to parameterize the input vector performed by an embedded system called as state dependent parameter estimation (SDPE), which is executed by multi layer parceptron neural network (MLPNN). SDPE consists of the linear and nonlinear parts. The controller law is derived via SDPE of the linear and nonlinear parts through switching mechanism. The dynamic tracking controller error is derived then the stability analysis of the closed-loop controller is performed based Lyapunov theorem. Linear based adaptive robust control and nonlinear based adaptive robust control is performed with the switching of the linear and nonlinear parts parameters based Lyapunov theorem to guarantee bounded and convergence error.

The method of rule extraction based on differential evolution (DE-Rule) is proposed in this paper. The style of rule is IF-THEN. Connection word is AND in the antecedent of rule. The consequence of rule is class label. The above rule is encoded and expressed to individual in population of differential evolution. Then the population is iterated through differential mutation and binomial crossover. Optimal rule set is obtained after decoding the optimal individual. Finally, DE-Rule, RS-Rule and ANN-GA-Cascades-Rule are used to recognize oil reservoir (dry layer, water layer, inferiority layer and oil layer) in the Jianghan oil field. Data of well oilsk81 is training data and data of well oilsk83 is testing data. The results show that performance of accuracy and interpretability in DE-Rule is the best.

Pulse coupled neural network (PCNN) is a new type of artificial neural network specific for image processing applications. It is a single layer, two dimensional network with neurons which have I : I correspondence to the pixels of an input image. It is convenient to process the intensities and spatial locations of image pixels simultaneously by applying a PCNN. Therefore, we propose a modified PCNN with anisotropic synaptic weight matrix for image edge detection from the aspect of intensity similarities of pixels to their neighborhoods. By applying the anisotropic synaptic weight matrix, the interconnections are only established between the central neuron and the neighboring neurons corresponding to pixels with similar intensity values in a 3 by 3 neighborhood. Neurons corresponding to edge pixels and non-edge pixels will receive different input signal from the neighboring neurons. By setting appropriate threshold conditions, image step edges can be detected effectively. Comparing with conventional PCNN based edge detection methods, the proposed modified PCNN is much easier to control, and the optimal result can be achieved instantly after all neurons pulsed. Furthermore, the proposed method is shown to be able to distinguish the isolated pixels from step edge pixels better than derivative edge detectors.

In a standard support vector machine (SVM), the training process has O(n3) time and O(n2) space complexities, where n is the size of the training dataset. For very large datasets, it is thus computationally infeasible. Reducing the size of training dataset is naturally considered as a method to solve this problem. SVM classifiers are constructed by using the training samples called support vectors (SVs) that lie close to the separation boundary. Thus, removing the other samples that are not relevant to SVs might have no effect on building the separation boundary. In other words, we need to reserve the samples that are likely to be SVs. Therefore, a method based on edge detection techniques is proposed to extract such samples near the separation boundary. In order to avoid overfitting, we also use a clustering algorithm to keep the distribution properties of the training dataset. The samples selected by the edge detector and the centroids of clusters are used to reconstruct the training dataset. In the proposed approach, the edge detection technique helps us to extract the local properties around the separation boundary and the clustering algorithm preserves the properties of the entire data. The reconstructed training dataset with a smaller number of samples can make the training process very fast without degrading the classification accuracy. (c) 2013 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

Several researchers have shown that multiple classifier systems can result in effective solutions to difficult real-world classification tasks. However, most of these approaches are easily influenced by noise, and the training datasets for local classifiers get easily imbalanced. One of the main reasons for this is that it is hard to guarantee that the centers of the subsets are close to the separation hyperplane, so that it is difficult to evenly distribute the samples in the two sides of the hyperplane. In order to solve this problem, we redefine the description of classifier modeling problem as a task of piecewise approximation of the separation hyperplane. On the basis of this description, we propose a novel multiple support vector machine (SVM) classifier system. Its main contribution is a novel construction approach to the subtraining datasets. The proposed approach partitions the area close to the separation hyperplane into some subsets to construct the subtraining datasets. The subtraining datasets describe the subtasks for identifying segments of the separation hyperplane. Local SVMs are trained to solve the respective subtasks. Finally, the decisions of these local SVMs are appropriately combined on the basis of a probabilistic interpretation to obtain the final classification decision. The effectiveness of this approach is demonstrated through comparisons with some well-known approaches on both synthetic and real-world datasets. © 2013 Institute of Electrical Engineers of Japan.

This work exploits the idea on how to search parameter estimation and increase its convergence speed for the Liner Time Invariant (LTI) system. The convergence speed of parameter estimation is the one problem and plays an important role in the adaptive controller to increase performance. The well-known algorithm is the recursive least square algorithm. However, the speed of convergence is still low and is influenced by the number of sampling, which is represented by the limited availability for the information vector. We offer a new method to increase the convergence speed by applying Quasi-ARX model. Quasi-ARX model performs two steps identification process by presenting parameter estimation as a function over time. The first, parameters estimation of macro-part sub-model are searched by the least square error, and the second is to sharpen the searching by performing backpropagation learning of multi layer parceptron network.

This paper focuses on imbalanced dataset classification problem by using SVM and oversampling method. Traditional oversampling method increases the occurrence of over-lapping between classes, which leads to poor generalization of SVM classification. To solve this problem this paper proposes a combined method of quasi-linear SVM and assembled SMOTE. The quasi-linear SVM is an SVM with quasi-linear kernel function. It realizes an approximate nonlinear separation boundary by mulit-local linear boundaries with interpolation. The assembled SMOTE implements oversampling with considering of the data distribution information and avoids occurrence of overlapping between classes. Firstly, a partition method based on Minimal Spanning Tree is proposed to obtain local linear partitions, each of which can be separated with one linear separation boundary. Secondly, using the information of local linear partitions, the assembled SMOTE generates synthetic minority class samples. Finally, the quasi-linear SVM realizes a classification of oversampled datasets in the same way as a standard SVM by using a composite quasi-linear kernel function. Experiment results on artificial data and benchmark datasets show that the proposed method is effective and improves classification performances. © 2013 IEEE.

Multi-label classification is an extension of traditional classification problem in which each instance is associated with a set of labels. For some multi-label classification tasks, labels are usually overlapped and correlated, and some implicit constraint rules are existed among the labels. This paper presents an improved multi-label classification method based on label ranking strategy and label constraints. Firstly, oneagainst-all decomposition technique is used to divide a multilabel classification task into multiple independent binary classification sub-problems. One binary SVM classifier is trained for each label. Secondly, based on training data, label constraint rules are mined by association rule learning method. Thirdly, a correction model based on label constraints is used to correct the probabilistic outputs of SVM classifiers for label ranking. Experiment results on three well-known multi-label benchmark datasets show that the proposed method outperforms some conventional multi-label classification methods.

In this paper, an SVM-based approach is proposed for stock market trend prediction. The proposed approach consists of two parts: feature selection and prediction model. In the feature selection part, a correlation-based SVM filter is applied to rank and select a good subset of financial indexes. And the stock indicators are evaluated based on the ranking. In the prediction model part, a so called quasi-linear SVM is applied to predict stock market movement direction in term of historical data series by using the selected subset of financial indexes as the weighted inputs. The quasi-linear SVM is an SVM with a composite quasi-linear kernel function, which approximates a nonlinear separating boundary by multi-local linear classifiers with interpolation. Experimental results on Taiwan stock market datasets demonstrate that the proposed SVM-based stock market trend prediction method produces better generalization performance over the conventional methods in terms of the hit ratio. Moreover, the experimental results also show that the proposed SVM-based stock market trend prediction system can find out a good subset and evaluate stock indicators which provide useful information for investors. © 2013 IEEE.

This paper presents a fuzzy adaptive controller applied to a non linear system modeled under a Quasi-linear ARX Neural Network, with stability proof by using the Lyapunov approach. This work exploits the new idea to use Lyapunov function to train multi-input multi-output neural network on the core-part sub-model. The proposed controller is designed between a non linear controller and linear controller based on fuzzy switching algorithm. Finally improving performances of the Lyapunov learning algorithm are stable in the learning process, fast convergence of error, and able to increase the accuracy of the controller.

According to the new economic growth theory, a new method of computing the contribution rate of scientific and technological (S&T) progress to economic growth based on the Cobb-Douglas production function and the Solow residual value method is proposed in this paper. This method includes three steps: Firstly, according to their levels of S&T progress, fuzzy soft clustering of thirty one Chinese regions is performed to obtain the membership degree of these places to the categories. Secondly, to calculate the contribution rates that different categories of levels of S&T progress contribute to economic growth. Thirdly, to multiply the obtained contribution rate of each category by the membership degree of the place belonging to this category, from which the contribution rate of S&T progress to economic growth in each place is obtained. Finally, this method is used to calculate the contribution rates of S&T progress to economic growth in thirty one Chinese regions during the period from 1998 to 2007. Last but not least, some reasonable suggestions and conclusions are proposed by analyzing the computing results. (c) 2011 Elsevier Ltd. All rights reserved.

Economic contribution rate of education (ECRE) is the key factor of education economics. This article selected China, South Korea, United States and other countries for a total of 15 samples, and put the data of the same period under the framework of soft computing, to simulate the production chain of "education-potential human capital-actual human capital-economic growth". The basic idea is: Firstly, 15 countries are softly categorized according to the level of science and technology (S&T) progress. Secondly, potential human capital and actual human capital establish the internal correlation (fuzzy mapping) in the same classification, and we conceptualize actual human capital as one production factor, joined with the other two production factors, fixed asset and land, to set up the fuzzy mapping to economic growth., and then calculate economic contribution rate of education of China and foreign by two fuzzy mapping of them. Thirdly, this paper analyzes the present state and differences in the development of education between China and foreign according to different ECRE, and offers proposals for promoting the education in China. (C) 2012 Elsevier B.V. All rights reserved.

In this paper, a fuzzy switching adaptive control approach is presented for nonlinear systems. The proposed fuzzy switching adaptive control law is composed of a quasi-ARX radial basis function network (RBFN) prediction model and a fuzzy switching mechanism. The quasi-ARX RBFN prediction model consists of two parts: a linear part used for a linear controller to ensure boundedness of the input and output signals; and an RBFN nonlinear part used to improve control accuracy. By using the fuzzy switching scheme between the linear and nonlinear controllers to replace the 0/1 switching, it can realize a better balance between stability and accuracy. Theoretical analysis and simulation results show the effectiveness of the proposed control method on the stability, accuracy, and robustness. (c) 2012 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

The quasi-ARX neurofuzzy (Q-ARX-NF) model has shown great approximation ability and usefulness in nonlinear system identification and control. It owns an ARX-like linear structure, and the coefficients are expressed by an incorporated neurofuzzy (InNF) network. However, the Q-ARX-NF model suffers from curse-of-dimensionality problem, because the number of fuzzy rules in the InNF network increases exponentially with input space dimension. It may result in high computational complexity and over-fitting. In this paper, the curse-of-dimensionality is solved in two ways. Firstly, a support vector regression (SVR) based approach is used to reduce computational complexity by a dual form of quadratic programming (QP) optimization, where the solution is independent of input dimensions. Secondly, genetic algorithm (GA) based input selection is applied with a novel fitness evaluation function, and a parsimonious model structure is generated with only important inputs for the InNF network. Mathematical and real system simulations are carried out to demonstrate the effectiveness of the proposed method.

Hierarchical multi-label classification (HMC) is a variant of classification where instances may belong to multiple classes at the same time and these classes are organized in a hierarchy. Gene function prediction is a complicated HMC problem with large class number and usually strongly imbalanced class distributions. This paper proposes an improved HMC method based on over-sampling and hierarchy constraint for solving the gene function prediction problem. The HMC task is transferred into a set of binary support vector machine (SVM) classification tasks. Then, two measures are implemented to enhance the HMC performance by introducing the hierarchy constraint into learning procedures. Firstly, for imbalanced classes, a hierarchical synthetic minority over-sampling technique (SMOTE) is proposed as over-sampling preprocessing to improve the SVM learning performance. Secondly, an improved True Path Rule (TPR) ensemble approach is introduced to combine the results of binary probabilistic SVM classifications. It can improve the classification results and guarantee the hierarchy constraint of classes. Experiment results on four benchmark FunCat Yeast datasets show that the proposed method significantly outperforms the basic TPR method and the Flat ensemble method. (C) 2012 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

to understand the behavior of smartphone online application software will be helpful to predict whether the software application would be adopted by the users and to guide the providers to enhance the functions of the software. A wide range of criteria are used to assess smartphone software quality, but most of these criteria have interdependent or interactive characteristics, which can make it difficult to effectively analyze and improve smartphone use intention. The purpose of this study is to address this issue using a hybrid MCDM (multiple criteria decision-making) approach that includes the DEMATEL (decision-making trial and evaluation laboratory), DANP (the DEMATEL-based analytic network process) methods to achieve an optimal solution. By exploring the influential interrelationships between criteria related to mobile communication industry's and related value-added service content providers' reference in the respect of operation. This approach can be used to solve interdependence and feedback problems, allowing for greater satisfaction of the actual needs of mobile communication industries.

This paper proposes a multiinput and multioutput (MIMO) quasi-autoregressive eXogenous (ARX) model and amultivariable-decoupling proportional integral differential (PID) controller for MIMO nonlinear systems based on the proposed model. The proposed MIMO quasi-ARX model improves the performance of ordinary quasi-ARX model. The proposed controller consists of a traditional PID controller with a decoupling compensator and a feed-forward compensator for the nonlinear dynamics based on the MIMO quasi-ARX model. Then an adaptive control algorithm is presented using the MIMO quasi-ARX radial basis function network (RBFN) prediction model and some stability analysis of control system is shown. Simulation results show the effectiveness of the proposed control method.

In recent years, support vector regression (SVR) has attracted much attention for nonlinear system identification. It can solve nonlinear problems in the form of linear expressions within the linearly transformed space. Commonly, the convenient kernel trick is applied, which leads to implicit nonlinear mapping by replacing the inner product with a positive definite kernel function. However, only a limited number of kernel functions have been found to work well for the real applications. Moreover, it has been pointed that the implicit nonlinear kernel mapping is not always good, since it may faces the potential over-fitting for some complex and noised learning task. In this paper, explicit nonlinear mapping is learnt by means of the quasi-ARX modeling, and the associated inner product kernel, which is named quasi-linear kernel, is formulated with nonlinearity tunable between the linear and nonlinear kernel functions. Numerical and real systems are simulated to show effectiveness of the quasi-linear kernel, and the proposed identification method is also applied to microarray missing value imputation problem.

This paper proposes a hierarchical multi-label classification method based on SVM with composite kernel for solving gene function prediction. The hierarchical multi-label classification problem is resolved into a set of binary classification tasks. A composite kernel based SVM (ck-SVM) is introduced to deal with the binary classification tasks. In estimation procedure of ck-SVM, a supervised clustering with over-sampling strategy is introduced for solving imbalance dataset learning problem and improve classification performance. Experimental results on benchmark datasets demonstrate that the proposed method improves the classification performance efficiently.

This paper presents a method for nonlinear discriminant analysis utilizing a composite kernel which is derived from a combination of local linear models with interpolation. The underlying idea is to decompose a complex nonlinear problem into a set of simpler local linear problems. Combining with the theory of nonlinear classification based on kernels, the local linear models with interpolation can be formulated as a composite kernel based discriminant analysis form. In face recognition, linear discriminant analysis (LDA) has been widely adopted owing to its efficiency, but it fails to solve nonlinear problems. Conventional kernel based approaches such as generalized discriminant analysis (GDA) has been successfully applied to extend LDA to nonlinear pattern recognition tasks. However, selecting an appropriate kernel function is usually difficult. Utilizing an implicit kernel mapping may face potential over-training problems for some complex and noised tasks. Our proposed method gives an alternative solution for nonlinear discriminant analysis while the conventional linear and nonlinear approaches are difficult to achieve a satisfactory results. Experiments on both synthetic data and face data set show the effectiveness of the proposed methods.

This paper present a neural predictive controller (NPC) based on improved quasi-ARX neural network (IQARXNN) for nonlinear dynamical systems. The IQARXNN is used as a model identifier with switching algorithm and switching stability analysis. The primary controller is designed based on a modified Elman neural network (MENN) controller using back-propagation (BP) learning algorithm with modified particle swarm optimization (MPSO) to adjust the learning rates in the BP process to improve the learning capability. The adaptive learning rates of the controller are investigated via Lyapunov stability theorem, which are respectively used to guarantee the convergences of the predictive controller. Performance of the proposed MENN controller with MPSO is verified by simulation results to show the effectiveness of the proposed method both on stability and accuracy.

In this note, we present the modeling of nonlinear dynamical systems with Quasi-ARX neural network using Lyapunov algorithm in learning process. This work exploits the idea on learning algorithm in nonlinear kernel part of Quasi-ARX model to improve stability and fast convergence of error. The proposed algorithm is then employed to model and predict a classical nonlinear system with input dead zone and nonlinear dynamic systems, exhibiting the effectiveness of proposed algorithm. Based on the result of simulation, the proposed algorithm can make the error in process learning become fast convergence, ultimately bounded, and the error distributed uniformly.

This paper presents an automated approach to extracting product data from commercial web pages. Our web mining method involves the following two phrases: First, it analyzes the data information located at the leaf node of DOM tree structure of the web page, generates the semantic information vector for other nodes of the DOM tree and find maximum repeat semantic vector pattern. Second, it identifies the product data region and data records, builds a product object template by using semantic tree matching technique and uses it to extract all product data from the web page. The main contribution of this study is in developing a fully automated approach to extract product data from the commercial sites without any user's assistance. Experiment results show that the proposed technique is highly effective. © Springer-Verlag 2012.

Sequencing by hybridization is a promising cost-effective technology for high-throughput DNA sequencing via microarray chips. However, due to the effects of spectrum errors rooted in experimental conditions, an accurate and fast reconstruction of original sequences has become a challenging problem. In the last decade, a variety of analyses and designs have been tried to overcome this problem, where different strategies have different trade-offs in speed and accuracy. Motivated by the idea that the errors could be identified by analyzing the interrelation of spectrum elements, this paper presents a constructive heuristic algorithm, featuring an accurate reconstruction guided by a set of well-defined criteria and rules. Instead of directly reconstructing the original sequence, the new algorithm first builds several accurate short fragments, which are then carefully assembled into a whole sequence. The experiments on benchmark instance sets demonstrate that the proposed method can reconstruct long DNA sequences with higher accuracy than current approaches in the literature.

Glomeruli extraction is a vital step in computer-aided diagnosis systems of kidney disease. Since there are not only glomeruli but also other tissues in an image, when detecting the edges of glomeruli, lot of noises caused by other tissues will be detected at the same time. These noises cause discontinuous edges of glomeruli when some operation, such as labeling, is applied to denoise. According to this characteristic, this article proposes a contour extraction method based on genetic algorithm (GA) for edge patching. First, a Canny operator is applied to obtain the edges of glomeruli with noises. Then labeling and other operations such as dilation, thinning and cross-point deletion are applied to markedly remove the noises. After the above operations, GA is finally used to search for optimal patching segments to join the discontinuous edges together and a closed curve with highest fitness would be able to form the contour of glomeruli. Experimental results show that the proposed method performs well for the renal biopsy images. (C) 2011 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

Polynomial NARX (nonlinear autoregressive with exogenous) model identification has received considerable attention in last three decades. However, in a high-order nonlinear system, it is very difficult to obtain the model structure directly even with state-of-art algorithms, because the number of candidate monomial terms is huge and increases drastically as the model order increases. Motivated by this fact, in this research, the identification is performed in two steps: firstly a prescreening process is carried out to select a reasonable number of important monomial terms based on two kinds of the importance indices. Then, in the reduced searching space with only the selected important terms, multi-objective evolutionary algorithm (MOEA) is applied to determine a set of significant terms to be included in the polynomial model with the help of independent validation data. In this way, the whole identification algorithm is implemented efficiently. Numerical simulations are carried out to demonstrate the effectiveness of the proposed identification method. (C) 2011 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

In this paper, quasi-ARX wavelet network (Q-ARX-WN) is proposed for nonlinear system identification. There are mainly two contributions are clarified. Firstly, compared with conventional wavelet networks (WNs), it is equipped with a linear structure, where WN is incorporated to interpret parameters of the linear ARX structure, thus Q-ARX-WN prediction model could be constructed and it is easy-to-use in nonlinear control. Secondly, guidelines for network construction are well considered due to the introduction of WNs, and Q-ARX-WN could be represented in a linear-in-parameter way. Therefore, linear support vector regression (SVR) based identification scheme may be introduced for the robust performance. Moreover, in adaptive control procedure, only linear parameters are needed to be adjusted when sudden changes have happened on the nonlinear system, thus the controller can track reference signal quickly. The effectiveness and robustness of the proposed nonlinear system identification method are validated by applying it to identify a real data system and a mathematical example, and an example of nonlinear system control is given to show usefulness of the proposed model.

The central criterion of feature selection is that good feature sets contain features that are highly correlated with the output, yet uncorrelated with each other. Based on this criterion, we address the problem of feature selection through correlation-based feature clustering and support vector machine (SVM) based feature ranking. Correlation-based clustering is proposed to group features into some clusters based on the correlation between two features. As a result, a feature is highly correlated to any other feature in the same cluster but uncorrelated to the features in other clusters. From each cluster, we select a feature as the delegate based on its influence quantities on the output. The influence quantities are measured by the feature sensitivity in the SVM. The proposed approach can identify relevant features and eliminate redundancy among them effectively. The effectiveness of the proposed approach is demonstrated through comparisons with other methods using real-world data with different dimensions. (C) 2011 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

Nonlinear polynomial NARX model identification often faces the problem of huge pool of candidate terms, which makes the evolutionary optimization based identification algorithm work with low efficiency. This paper proposes an efficient identification scheme with pre-processing to reduce the searching space effectively. Both the input selection and term selection are implemented to truncate the candidate pool with the help of correlation based orthogonal forward selection (COFS) algorithm and simplified orthogonal least square (OLS) algorithm, respectively. Then multi objective evolutionary algorithm (MOEA) is used to identify the polynomial model in a relative small searching space.

paper proposes a niching EDA based on clustering and fitness sharing for solving off-lattice protein folding model. Firstly, AP clustering is used to adaptively partition the population into niches during a run of EDA. A cluster is seen as a niche, the niche number and the niche radiuses may vary obviously for different generations. Secondly, fitness sharing strategy is used for reserving the diversity of population to enhance the performance of EDA. Experiment results on the artificial and real protein sequences show that the proposed niching EDA is effective to solve the off-lattice AB model and outperforms other evolutionary-based methods.

Nonlinear polynomial NARX model identification often faces the problem of huge pool of candidate terms, which makes the evolutionary optimization based identification algorithm work with low efficiency. This paper proposes an efficient identification scheme with pre-processing to reduce the searching space effectively. Both the input selection and term selection are implemented to truncate the candidate pool with the help of correlation based orthogonal forward selection (COFS) algorithm and simplified orthogonal least square (OLS) algorithm, respectively. Then multi objective evolutionary algorithm (MOEA) is used to identify the polynomial model in a relative small searching space.

This paper proposes a multi-input and multi-output (MIMO) quasi-ARX model and a multivariable decoupling proportional-integral-differential (PID) controller for MIMO nonlinear systems based on the proposed model. The proposed controller consists of a traditional PID controller with a decoupling compensator and a feed-forward compensator for the nonlinear dynamics from the MIMO quasi-ARX model. Then an adaptive control algorithm is presented using the MIMO quasi-ARX Radial Basis Function Network (RBFN) prediction model. Simulation results show the effectiveness of the proposed control method.

Quasi-ARX neurofuzzy (Q-ARX-NF) models have shown great approximation ability and usefulness in nonlinear system identification and control. However, the incorporated neurofuzzy networks suffer from the curse-of-dimensionality problem, which may result in high computational complexity and over-fitting. In this paper, support vector regressor (SVR) based identification approach is used to reduce computational complexity with the help of transfonning the original problem into Lagrange space, which is only sensitive to the number of data samples. Furthermore, to improve the generalization capability, a parsimonious model structure is obtained by eliminating insignifuicant input variables for the incorporated neurofuzzy network, which is implemented by genetic algorithm (GA) based input selection method with a novel fitness evaluation function. Two numerical simulations are tested to show the effectiveness of the proposed method.

Missing value imputation for microarray data is important for gene expression analysis algorithms, such as clustering, classification and network design. A number of algorithms have been proposed to solve this problem, but most of them are only limited in linear analysis methods, such as including the estimation in the linear combination of other no-missing-value genes. It may result from the fact that microarray data. often comprises of huge size of genes with only a small number of observations, and nonlinear regression techniques are prone to overfitting. In this paper, a quasi-linear SVR model is proposed to improve the linear approaches, and it can be explained in a piecewise linear interpolation way. Two real datasets are tested and experimental results show that the quasi-linear approach for missing value imputation outperforms both the linear and nonlinear approaches.

In this paper, a multi-label classification method based on label ranking and delicate boundary Support Vector Machine (SVM) is proposed for solving the functional genomics applications. Firstly, an improved probabilistic SVM with delicate decision boundary is used as scoring approach to obtain a proper label rank. Secondly, an instance-dependent thresholding strategy is proposed to decide classification results. A d-folds validation approach is utilised to determine a set of target thresholds for all training samples as teachers, then an appropriate instance-dependent threshold for each testing instance is obtained by applying k-Nearest Neighbours (KNN) strategy on this teacher threshold set. © 2010 Inderscience Enterprises Ltd.

Protein structure prediction (PSP) is one of the most important problems in computational biology. This paper proposes a novel hybrid estimation of distribution algorithm (EDA) to solve the PSP problem on HP model. First, a composite fitness function containing the information cif folding structure core (I-I-core) is introduced to replace the traditional fitness function of HP model. The proposed fitness function is expected to select better individuals for the probabilistic model of EDA. Second, local search with guided operators is utilized to refine the found solutions for improving the efficiency of EDA. Third, an improved backtracking-based repairing method is proposed to repair invalid individuals sampled by the probabilistic model of EDA. It can significantly reduce the number of backtracking searching operation and the computational cost for a long-sequence protein. Experimental results demonstrate that the proposed method outperform the basic EDA method. At the same time, it is very competitive with other existing algorithms for the PSP problem on lattice HP models. (C) 2010 Institute of Electrical Engineers of Japan. Published by John Wiley & Sons, Inc.

Multi-label classification problem is an extension of traditional multi-class classification problem in which the classes are not mutually exclusive and each sample may belong to several classes simultaneously. Such problems occur in many important applications. Some researches indicate that the performance of classifier can be improved by introducing the information of multi-label training samples into learning procedure effectively. In this paper, we propose a novel method based on SVM with delicate decision boundary. For the basic overlapping problem of two labels, characteristics of double-label samples are utilized to obtain the range of overlapping sample space decided by two binary SVM classifier separating surfaces. And a bias model with delicate decision boundary is built for samples in overlapping sample space to improve the classification accuracy. Experimental results on the benchmark datasets of Yeast and Scene show that our proposed method improves the classification accuracy efficiently, compared with the basic binary SVM method and some existing well-known methods.

This paper proposes an edge detection method by Canny operator with a feedback strategy for glomeruli extraction. As we know, the effect of the Canny operator is determined by three parameters: high threshold, low threshold and standard deviation. To obtain the appropriate parameters for each image, Otsu method is used to set high and low thresholds of Canny operator firstly. And then, to select the optimal standard deviation, a feedback strategy is developed. After parameter selection, Canny operator is applied to our renal biopsy images and experimental results show that some samples can achieve successful extraction and the others result in the discontinuous edges of glomeruli. In the case of the latter, the endpoints in an image should be located and connected to form a whole edge of glomerulus. The experiments have produced the promising results for our samples. © 2010 Copyright SPIE - The International Society for Optical Engineering.

Sequencing by hybridization is a promising cost-effective technology for high-throughput DNA sequencing via microarray chips. However, due to the effects of spectrum errors rooted from experimental conditions, a fast and accurate reconstruction of original sequences has become a challenging problem. In the last decade, a variety of analyses and designs have been tried to overcome this problem, where different strategies have different tradeoffs in speed and accuracy. Motivated by the idea that the errors could be identified by analyzing the interrelation of spectrum elements, this paper presents a new constructive heuristic algorithm, featuring an accurate reconstruction guided by a set of well-defined criteria and rules. The experiments on benchmark instance sets demonstrate that the proposed method can reconstruct long DNA sequences more accurately than current approaches in the literature. © 2010 IEEE.

In this paper, we propose a Hierarchical Frequency Sensitive Competitive Learning (HFSCL) method to achieve Color Quantization (CQ). In HFSCL, the appropriate number of quantized colors and the palette can be obtained by an adaptive procedure following a binary tree structure with nodes and layers. Starting from the root node that contains all colors in an image until all nodes are examined by split conditions, a binary tree will be generated. In each node of the tree, a Frequency Sensitive Competitive Learning (FSCL) network is used to achieve two-way division. To avoid over-split, merging condition is defined to merge the clusters that are close enough to each other at each layer. Experimental results show that the proposed HFSCL has desired ability for CQ.

Estimation of Distribution Algorithms (EDAs) still suffer from the drawback of premature convergence for solving the optimization problems with irregular and complex multimodal landscapes. In this paper, we propose an adaptive niching EDA based on Affinity Propagation (AP) clustering analysis. The AP clustering is used to adaptively partition the niches and mine searching information from the evolution process. The obtained information is successfully utilized to improve the EDA performance by a balance niching searching strategy. Two different categories of optimization problems are used to evaluate the proposed adaptive niching EDA. The first is the continuous EDA based on single Gaussian probabilistic model to solve two benchmark functional multimodal optimization problems. The second is a real complicated discrete EDA optimization problem, the protein 3-D HP model based on k-order Markov probabilistic model. The experiment studies demonstrate that the proposed adaptive niching EDA is an efficient method.

In this paper, an improved multi-label classification is proposed based on label ranking and delicate decision boundary SVM. Firstly, an improved probabilistic SVM with delicate decision boundary is used as the scoring method to obtain a proper label rank. It can improve the probabilistic label rank by introducing the information of overlapped training samples into learning procedure. Secondly, a threshold selection related with input instance and label rank is proposed to decide the classification results. It can estimate an appropriate threshold for each testing instance according to the characteristics of instance and label rank. Experimental results on four multi-label benchmark datasets show that the proposed method improves the performance of classification efficiently, compared with binary SVM method and some existing well-known methods.

Brain-Computer Interface (BCI) is a system provides an alternative communication and control channel between the human brain and computer. In Motor Imagery-based (MI) BCI system, Common Spatial Pattern (CSP) is frequently used for extracting discriminative patterns from the electroencephalogram (EEG). There are many studies have proven that the performance of CSP has a very important relation with the choice of operational frequency band. As the fact that the CSP features at different frequency bands contain discriminative and complementary information for classification, this paper proposes a new frequency band selection method to nd the best frequency band set on which subject-speci cs CSP are complementary for MI classi cation. Compared to the performance offered by the existing method based on frequency band partition, the proposed algorithm can yield error rate reductions of 49.70% for the same BCI competition dataset.

This paper presents a novel approach for designing adaptive controller of nonlinear dynamical systems based on an improved quasi-ARX neural network prediction model. The improved quasi-ARX neural network prediction model has two parts: the linear part is used for stability and the nonlinear part is used to satisfy accuracy requirement. Then, we can obtain a linear controller and a nonlinear controller based on the characteristic of the improved quasi-ARX neural network prediction model. A fuzzy switching algorithm is designed between the two controllers. Theory analysis and simulations are given to show the effectiveness of the proposed method both on stability and accuracy.

In this paper, we propose a hierarchical frequency sensitive competitive learning (HFSCL) method to achieve color quantization (CQ). In HFSCL, the appropriate number of quantized colors and the palette can be obtained by an adaptive procedure following a binary tree structure with nodes and layers. Starting from the root node that contains all colors in an image until all nodes are examined by split conditions, a binary tree will be generated. In each node of the tree, a frequency sensitive competitive learning (FSCL) network is used to achieve two-way division. To avoid over-split, merging condition is defined to merge the clusters that are close enough to each other at each layer. Experimental results show that HFSCL has the desired ability for CQ. © 2010 IEEE.

Multi-label classification is an extension of traditional classification problem in which each instance is associated with a set of labels. Recent research has shown that the ranking approach is an effective way to solve this problem. In the multi-labeled sets, classes are often related to each other. Some implicit constraint rules are existed among the labels. So we present a novel multi-label ranking algorithm inspired by the pairwise constraint rules mined from the training set to enhance the existing method. In this method, one-against-all decomposition technique is used firstly to divide a multi-label problem into binary class sub-problems. A rank list is generated by combining the probabilistic outputs of each binary Support Vector Machine (SVM) classifier. Label constraint rules are learned by minimizing the ranking loss. Experimental performance evaluation on well-known multi-label benchmark datasets show that our method improves the classification accuracy efficiently, compared with some existed methods.

Estimation of Distribution Algorithms (EDAs) still suffer from the drawback of premature convergence for solving the optimization problems with irregular and complex multimodal landscapes. In this paper, we propose an adaptive niching EDA based on Affinity Propagation (AP) clustering analysis. The AP clustering is used to adaptively partition the niches and mine searching information from the evolution process. The obtained information is successfully utilized to improve the EDA performance by a balance niching searching strategy. Two different categories of optimization problems are used to evaluate the proposed adaptive niching EDA. The first is the continuous EDA based on single Gaussian probabilistic model to solve two benchmark functional multimodal optimization problems. The second is a real complicated discrete EDA optimization problem, the protein 3-D HP model based on k-order Markov probabilistic model. The experiment studies demonstrate that the proposed adaptive niching EDA is an efficient method.

This paper presents a novel image edge detection method based on a simplified pulse coupled neural network with anisotropic interconnections (PCNNAI) by applying an anisotropic linking mechanism. PCNNAI utilizes the anisotropic linking mechanism to create an adaptive synaptic weight matrix to achieve the anisotropic interconnection model among neurons. Therefore, the neurons corresponding to edge and non-edge pixels will receive different feedback signal from neighborhood. Due to the PCNN structure the edges will be detected by different internal activity of edge neurons and non-edge neurons. Comparing with conventional PCNN edge detection methods, PCNNAI simplifies the system structure and the outputs are controllable, meanwhile PCNNAI also achieves more accurate results than the classical image edge detectors. Experimental results show that PCNNAI is effective at image edge detection. © 2010 IEEE.

This paper proposes an improved Hierarchical Multi-label Classification (HMC) method for solving the gene function prediction. The HMC task is transferred into a series of binary SVM classification tasks. By introducing the hierarchy constraint into learning procedures, two measures with incorporating prior information are implemented to improve the HMC performance. Firstly, for imbalanced functional classes, a hierarchical SMOTE is proposed as over-sampling preprocessing to improve the SVM learning performance. Secondly, an improved True Path Rule consistency approach is introduced to ensemble the results of binary probabilistic SVM classifications. It can improve the classification results and guarantee the hierarchy constraint of classes. © 2010 IEEE.

This paper proposes a switching scheme for salt and pepper noise reduction by combining a noise detection algorithm based on a simplified pulse coupled neural network (PCNN) with a simple adaptive median filter. The simplified PCNN utilizes an adaptive synaptic weight matrix created by anisotropic linking mechanism to achieve anisotropic linking model, that is the interconnections between neurons with large absolute difference in intensity will be interrupted. Therefore, the neurons corresponding to noise corrupted pixels will receive smaller feedback signal from the neighborhood and generate smaller internal activities compare with the ones corresponding to noise free pixels. The impulse will be detected by setting an appropriate dynamic threshold. After the PCNN based noise detection scheme, the pixels contaminated by salt and pepper noise will be restored by a simple adaptive median filter. Experimental results prove that the proposed switching median filter outperform over the conventional methods in both noise reduction and detail preserving. © 2010 IEEE.

This paper proposes a local linear multi-SVM method based on composite kernel for solving classification tasks in gene function prediction. The proposed method realizes a nonlinear separating boundary by estimating a series of piecewise linear boundaries. Firstly, according to the distribution information of training data, a guided partitioning approach composed of separating boundary detection and clustering technique is used to obtain local subsets, and each subset is utilized to capture prior knowledge of corresponding local linear boundary. Secondly, a composite kernel is introduced to realize the local linear multi-SVM model. Instead of building multiple local SVM models separately, the prior knowledge of local subsets is used to construct a composite kernel, then the local linear multi-SVM model is realized by using the composite kernel exactly in the same way as a single SVM model. Experimental results on benchmark datasets demonstrate that the proposed method improves the classification performance efficiently. © 2010 IEEE.

In this paper, a switching adaptive control method is presented for nonlinear system. The proposed switching adaptive control law is based on a quasi-ARX Radial Basis Function Network model and a switching mechanism. A d-difference operator is used to relax the globally assumption of nonlinear system. Theory analysis and simulations show the effectiveness of the proposed control method on stability and accuracy.

Image thresholding is a useful method in many image processing and computer vision applications. Otsu method is one of popular thresholding methods and has been frequently used as a classical technique in various applications. A premise of this method is that the probability of edge region is supposed to be zero. However, the premise is not proper in some situations such as the images are corrupted by noises. To solve this problem, a curvilinear thresholding method (CTM) is proposed based on the traditional Otsu method. We give a recursive algorithm of line thresholding method (LTM), which is a particular case of CTM, to verify our ideas. In addition, Otsu based method has another weakness that is it gives satisfactory results only when the numbers of pixels in each class are close to each other. Otherwise the threshold will be biased. This paper introduces a two-dimensional (2D) histogram projection method to correct the Otsu threshold. A. fast algorithm for searching the valley of one-dimensional (1D) projected histogram is also given based on wavelet transform. Experimental results show that the proposed method performs much better than the traditional Otsu method.

In this research we design a network administrator assistance system based on traffic measurement and fuzzy c-means (FCM) clustering analysis method. Network traffic measurement is an essential tool for monitoring and controlling communication network. It can provide valuable information about network traffic-load patterns and performances. The proposed system utilizes the FCM method to analyze users' network behaviors and traffic-load patterns based on traffic measurement data of IP network. Analysis results can be used as assistance for administrator to determine efficient controlling and configuring parameters of network management systems. The system is applied in Dali University campus network, and it is effective in practice.

The protein structure prediction (PSP) problem is one of the most important problems in computational biology. This paper proposes a novel Estimation of Distribution Algorithms (EDAs) based method to solve the PSP problem on HP model. Firstly, a composite fitness function containing the information of folding structure core formation is introduced to replace the traditional fitness function of RP model. It can help to select more optimum individuals for probabilistic model of EDAs algorithm. And a set of guided operators are used to increase the diversity of population and the likelihood of escaping from local optima. Secondly, an improved backtracking repairing algorithm is proposed to repair invalid individuals sampled by the probabilistic model of EDAs for the long sequence protein instances. A detection procedure of feasibility is added to avoid entering invalid closed areas when selecting directions for the residues. Thus, it can significant reduce the number of backtracking operation and the computational cost for long sequence protein. Experimental results demonstrate that the proposed method outperform the basic EDAs method. At the same time, it is very competitive with the other existing algorithms for the PSP problem on lattice HP models.

Glomerulus is the filtering unit of the kidney. In the computer aided diagnosis system designed for kidney disease, glomerulus extraction is an important step for analyzing kidney-tissue image. Against the disadvantages of traditional methods, this paper proposes a glomerulus extraction method using genetic algorithm for edge patching. Firstly, Canny edge detector is applied to get discontinuous edges of glomerulus. After labeling to remove the noises, genetic algorithm is used to search for optimal patching segments to join those edges together. Lastly, the edges and the patching segments with high fitness would be able to form the whole edge of the glomerulus. Experiments and comparisons indicate the proposed method can extract the glomerulus from kidney-tissue image both fast and accurately.

In a standard support vector machine (SVM), the training process has O(n(3)) time and O(n(2)) space complexities, where it is the size of training dataset. Thus, it is computationally infeasible for very large datasets. Reducing the size of training dataset is naturally considered to solve this problem. SVM classifiers depend on only support vectors (SVs) that lie close to the separation boundary. Therefore, we need to reserve the samples that are likely to be SVs. In this paper, we propose a method based on the edge detection technique to detect these samples. To preserve the entire distribution properties, we also use a clustering algorithm such as K-means to calculate the centroids of clusters. The samples selected by edge detector and the centroids of clusters are used to reconstruct the training dataset. The reconstructed training dataset with a smaller size makes the training process much faster, but without degrading the classification accuracies.

In this paper, an automatic segmentation method based on self-organizing feature map (SOFM) neural network (NN) is presented for color images. First, a binary tree clustering procedure is used to cluster the colors in an image. In each node of the tree, a SOFM NN is used as a classifier which is fed by image color values. The output neurons of the SOFM NN define the color classes for each node. In our method, the number of color classes for each node is two. For each node of the tree, Hotelling transform based splitting condition is used to define if the current color classes should be split. To speed up the entire algorithm, a nearest neighbor interpolation is used to get the small training set for SOFM NN. Once the colors in an image are clustered, it is easy to segment a target by analyzing the color feature in an image. The method is independent of the color scheme, so it is applicable to any type of color images. Our experimental results show the validity of the proposed method.

Protein structure prediction (PSP) is one of the most important problems in computational biology And it also is a very difficult optimization task, especially for long sequence instances. This paper proposes a novel clustering based niching EDA for HP model folding problem. The EDA individuals are clustered by the affinity propagation clustering method before submitting them to niching clearing. A cluster can be seen. as a niche in clearing procedure. The niche clearing radius can be adaptively determined by clustering. And an approach based on Boltzmann scheme is proposed to determine the niche capacity according to the adaptive clearing radius and niche fitness. Experimental results demonstrate that the proposed method outperforms the basic EDAs method. At the same time, it is very competitive with other existing algorithms for the PSP problem on lattice HP models.

A two-step identification method for nonlinear polynomial model using Evolutionary Algorithm (EA) Is proposed in this paper, and the method has the ability to select a parsimonious structure from a very large pool of model terms. In a nonlinear polynomial model, the number of candidate monomial terms increases drastically as the order of polynomial model increases, and it is impossible to obtain the accurate model structure directly even with state-of-art algorithms. The proposed method firstly carries out a pre-screening process to select a reasonable number of important monomial terms based on the importance index. In the next step, EA is applied to determine a set of significant terms to be included in the polynomial model. In this way, the whole identification algorithm is implemented very efficiently. Numerical simulations are carried out to demonstrate the effectiveness of the proposed identification method.

When a linear model is used for controlling nonlinear systems solely, it can't satisfy accuracy requirement. Whereas, although a neural network can deal with the accuracy problem, it may lead to instability. In this paper, an adaptive controller is proposed for nonlinear dynamical systems based on linear model and quasi-ARX neural network model. A switching algorithm is designed between the linear and nonlinear models. Theory analysis and simulations are given to show the effectiveness of the proposed method both on stability and accuracy.

Feature selection is a process to select a subset of original features. It can improve the efficiency and accuracy by removing redundant and irrelevant terms. Feature selection is commonly used in machine learning, and has been wildly applied in many fields, we propose a new feature selection method. This is an integrative hybrid method. It first uses Affinity Propagation and SVM sensitivity analysis to generate feature subset, and then use forward selection and backward elimination method to optimize the feature subset based on feature ranking. Besides, we apply this feature selection method to solve a new problem, Human resource selection. The data is acquired by questionnaire survey. The simulation results show that the proposed feature selection method is effective, it not only reduced human resource features but also increased the classification performance.

Pattern theorem in financial time-series is one of the most important technical analysis methods in financial prediction. Recent researches have achieved big progresses in identifying and recognizing time-series patterns. And most of the recent works on time-series deal with this task by using static approaches and mainly focus on the recognition accuracy, but considering that recognition of patterns in financial time-series, especially for stock time-series, are always time-consuming rather than pattern recognition in other fields, a dynamic recognition approach is more preferable so that investment on stock pattern become executable. In this paper we propose a dynamic approach for extracting and recognizing the patterns in stock-series. In our approach a slide window with flexible length is defined for extracting feature vertexes in stock-series, and in addition, a dynamic perceptual important point (PIP) locating method is proposed based on the PIP locating method for avoiding the computation expense problem and an artificial neural network (ANN) approach is involved for pattern recognition and window length identification.

Universal Learning Networks (ULNs) provide a generalized framework for many kinds of structures in neural networks with supervised learning. Multi-Branch Neural Networks (MBNNs) which use the Framework of ULNs have already been shown to have better representation ability in feedforward neural networks (FNNs). The multi-branch structure of MBNNs can he easily extended to recurrent neural networks (RNNs) because the characteristics of ULNs include the connection of multiple branches with arbitrary time delays. In this paper, therefore. RNNs with multi-branch structure are proposed and are shown to have better representation ability than conventional RNNs. RNNs can represent dynamical systems and are useful for time series prediction. The performance evaluation of RNNs with multi-branch structure was carried out using a benchmark of time series prediction. Simulation results showed that RNNs with multi-branch structure could obtain better performance than conventional RNNs, and also showed that they could improve the representation ability even if they are smaller-sized networks (C) 2009 Wiley Periodicals, Inc. Electron Comm Jpn 91(9): 37-44, 2008; Published Online in Wiley InterScience (www.interscience.wiley.com). DOI 10.1002/ecj.10157

A method of class association rule mining from incomplete databases is proposed using Genetic Network Programming (GNP). GNP is one of the evolutionary optimization techniques, which uses the directed graph structure. An incomplete database includes missing data in some tuples, however, the proposed method can extract important rules using these tuples, and users can define the conditions of important rules flexibly. Generally, it is not easy for Aprior-like methods to extract important rules from incomplete database, so we have estimated the performances of the rule extraction and classification of the proposed method using incomplete data set. The results showed that the accuracy of classification of the proposed method is favorable even if some tuples include missing data.

An efficient algorithm for building a classifier is proposed based on an important association rule mining using genetic network programming (GNP). The proposed method measures the significance of the association via the chi-squared test. Users can define the conditions of important association rules for building a classifier flexibly. The definition can include not only the minimum threshold chi-squared value, but also the number of attributes in the association rules. Therefore, all the extracted important rules can be used for classification directly. GNP is one of the evolutionary optimization techniques, which uses the directed graph structure as genes. Instead of generating a large number of candidate rules, Our method can obtain a sufficient number of important association rules for classification. In addition, our method suits association rule mining from dense databases such as medical datasets, where many frequently occurring items are found in each tuple. In this paper, we describe an algorithm for classification using important association rules extracted by GNP with acquisition mechanisms and present some experimental results of medical datasets. (C) 2008 Wiley Periodicals, Inc. Electron Comm Jpn, 91(2): 46-54, 2008; Published online in Wiley InterScience (www.interscience.wiley.com). DOI 10. 1002/ecj.10022

According to Hebb's cell assembly theory, the brain has the capability of function localization. On the other hand, it is suggested that in the brain there are three different learning paradigms: supervised, unsupervised, and reinforcement learning, which are related deeply to the three parts of brain: cerebellum, cerebral cortex, and basal ganglia, respectively. Inspired by the above knowledge of the brain in this paper we present a brainlike learning system consisting of three parts: supervised learning (SL) part, unsupervised learning (UL) part, and reinforcement learning (RL) part. The SL part is a main part learning inputoutput mapping
the UL part is a competitive network dividing input space into subspaces and realizes the capability of function localization by controlling firing strength of neurons in the SL part based on input patterns
the RL part is a reinforcement learning scheme, which optimizes system performance by adjusting the parameters in the UL part. Numerical simulations have been carried out and the simulation results confirm the effectiveness of the proposed brainlike learning system. © 2007 Wiley Periodicals, Inc.

Neural networks (NNs) can solve only simple problems if the network size is too small, but increasing the network size is costly in terms of memory space and calculation time. Thus, we have studied how to construct a network structure with high performance and low cost in space and time. One solution is a multibranch structure. Conventional NNs use the single-branch structure for connections, while the multibranch structure has multiple branches between nodes. In this paper, a new method which enables the multibranch NNs to have the localized property is proposed. It is well known that RBF networks have the localized property, which makes it possible to approximate functions faster than sigmoidal NNs. By using the multibranch structure having the localized property of RBF networks, NNs can obtain superior performance while maintaining lower costs in space and time. Simulation results of function approximations and a classification problem are presented to illustrate the effectiveness of multibranch NNs. (c) 2007 Wiley Periodicals, Inc.

The presence of similar patterns in regulatory sequences may aid users in identifying co-regulated genes or inferring regulatory modules. By modelling pattern occurrences in regulatory regions with Poisson statistics, this paper presents a log likelihood ratio statistics-based distance measure to calculate pair-wise similarities between regulatory sequences. We employed it within three clustering algorithms: hierarchical clustering, Self-Organising Map, and a self-adaptive neural network. The results indicate that, in comparison to traditional clustering algorithms, the incorporation of the log likelihood ratio statistics-based distance into the learning process may offer considerable improvements in the process of regulatory sequence-based classification of genes. © 2008 Inderscience Enterprises Ltd.

This paper presents an approach to the multiple sequence alignment (MSA) problem by applying genetic algorithms with a reserve selection mechanism. MSA is one of the most fundamental operations in bioinformatics, which plays an important part in predicting the structure, function and evolution of biological sequences. In order to solve the MSA problem efficiently, genetic algorithms (GAs) were applied. As the number and length of sequences increase, however, GAs are usually suffered from premature convergence where they are easily trapped into local optima. In this paper, we employ the reserve selection that is a new selection scheme to avoid premature convergence in GAs. Empirical studies demonstrate that genetic algorithms with reserve selection (GARS) could bring about a rise in the quality of multiple sequence alignment when compared with standard GAs.

Deceptive problems are a class of challenging problems for conventional genetic algorithms (GAs), which usually mislead the search to some local optima rather than the global optimum. This paper presents an improved genetic algorithm with reserve selection to solve deceptive problems. The concept "potential" of individuals is introduced as a new criterion for selecting individuals for reproduction, where some individuals with low fitness are also let survive only if they have high potentials. An operator called adaptation is further employed to release the potentials for approaching the global optimum. Case studies are done in two deceptive problems, demonstrating the effectiveness of the proposed algorithm.

An efficient algorithm for mining important association rule from multi-relational database using distributed mining ideas. Most existing data mining approaches look for rules in a single data table. However, most databases are multi-relational. In this paper, we present a novel distributed data-mining method to mine important rules in multiple tables (relations) and combine the method with genetic algorithm to enhance the mining efficiency. Genetic algorithm is in charge of finding antecedent rules and aggregate of transaction set that produces the corresponding rule from the chief attributes. Apriori and statistic method is in charge of mining consequent rules from the rest relational attributes of other tables according to the corresponding transaction set producing the antecedent rule in a distributed way. Our method has several advantages over most exiting data mining approaches. First, it can process multi-relational database efficiently. Second, rules produced have finer pattern. Finally, we adopt a new concept of extended association rules that contain more import and underlying information.

This paper presents a novel support vector regression (SVR) network for financial time series prediction. The SVR network consists of two layers of SVR: transformation layer and prediction layer. The SVRs in the transformation layer forms a modular network; but distinguished with conventional modular networks, the partition of the SVR modular network is based on the output domain that has much smaller dimension. Then the transformed outputs from the transformation layer are used as the inputs for the SVR in prediction layer. The whole SVR network gives an online prediction of financial time series. Simulation results on the prediction of currency exchange rate between US dollar and Japanese Yen show the feasibility and the effectiveness of the proposed method.

One of the central problems of functional genomics is gene classification. Microarray data are currently a major source of information about the functionality of genes. Various mathematical techniques, such as neural networks (NNs), self-organizing map (SOM) and several statistical methods, have been applied to classify the data in attempts to extract the underlying knowledge. As for conventional classification, the problem mainly addressed so far has been how to classify the multi-label gene data and how to deal with the imbalance problem. In this paper, we proposed an improved support vector machine (SVM) classifier with soft decision boundary. This boundary is a classification boundary based on belief degrees of data. The boundary can reflect the distribution of data, especially in the mutual part between classes and the excursion caused by the data imbalance.

propose an efficient data mining system for making quick response to users and providing a friendly interface. When data tuples have higher relationship, it could contain long frequent itemsets. If apriori algorithm mines all frequent itemsets in those tuples, its candidate itemsets will become very huge and it has to scan database huge times. Meanwhile, the number of rules mined by the apriori algorithm is huge. Our method avoids mining rules through huge candidate itemsets, just mines maximal frequent itemsets and scans the database for the frequent itemsets users are interested in. First, use GA to mine the maximal frequent itemsets and show them to users. Second, let users pick up one to deduce the association rules. Final, scan the database for the real support and confidence and show them to users. So, our method can not only save many times scanning the database and make quick response to users, but provide a friendly interface that let users select his interesting rules to mine.

Glomerulus extraction is an important step for automatic analysis of the kidney diseases in the computer-aided diagnosis system. A method based on searching the best fitting curve is proposed based on the characteristics of the renal biopsy images in microscope. This method can solve the problem of the large defect of the enhanced boundary, which lead to unsuccessful extraction. Firstly, a parametric equation is constructed based on the cubic spline interpolation function to draw the closed fitting curve. Secondly, the different scale binary images can be obtained by adjusting the parameters of the LOG filter. Finally, after labeling to remove the noises and thinning, a genetic algorithm is used to search the best fitting curve for glomerulus boundary. Experimental results indicate the correctness and effectiveness of this method.

Image segmentation, which is the first essential and fundamental issue in the image analysis and pattern recognition, is a classical difficult problem in image processing. In the computer-aided diagnosis system of the renal biopsy images in microscope, the correct segmentation of glomerulus is an important step for automatic analysis. Complex characteristics of renal biopsy images lead to the difficulty in boundary features description. A kind of feature operator based on the definition of the cavum boundary is proposed in this paper. According to this operator, a nonlinear thresholding surface can be constructed by neural network, and the appropriate surface can be selected to enhance the cavum boundary by the fault tolerance analysis. After denoising, the segmentation results can be obtained. Experimental results indicate that this method can enhance the boundary and suppress noises at the same time; it can obtain good segmented results and has a fine adaptability to various sample images.

Image segmentation plays an important role in image analysis and computer vision system. Among all segmentation techniques, the automatic thresholding methods are widely used because of their advantages of simple implement and time saving. Otsu method is one of thresholding methods and frequently used in various fields. Two-dimensional (2D) Otsu method behaves well in segmenting images of low signal-to-noise ratio than one-dimensional (1D). But it gives satisfactory results only when the numbers of pixels in each class are close to each other. Otherwise, it gives the improper results. In this paper, 2D histogram projection is used to correct the Otsu threshold. The 1D histograms are acquired by 2D histogram projection in x and y axes and a fast algorithm for searching the extrema of the projected histogram is proposed based on the wavelet transform in this paper. Experimental results show that the proposed method performs better than the traditional Otsu method for our renal biopsy samples. © 2008 IEEE.

To analyze the nuclei distributed in the glomeruli of the renal biopsy images automatically, the problem of missing or wrong extraction of them caused by the complex characteristics of the renal biopsy images must be solved. This paper introduces a dynamic thresholding method using eigenvalue feedback strategy based on multi-channel information. Firstly, the nonlinear thresholding surface adjusted by area eigenvalue feedback strategy is used to R-channel images of RGB color space and C-channel images of CMYK color space respectively to obtain the binary images. Through fusing the two images, the undetermined regions, which consist of the genuine and spurious nucleus, can be marked. And the reality degree is calculated to distinguish which one is genuine. Secondly, an area proportionality eigenvalue is considered in feedback strategy to divide the nuclei clusters into single ones. Then the nuclei extraction can be realized. The experimental results demonstrate the effectiveness of this method. © 2008 IEEE.

The discrete particle swarm optimization (DPSO) is a kind of particle swarm optimization (PSO) algorithm to find optimal solutions for discrete problems. This paper proposes an improved DPSO based on cooperative swarms, which partition the search space into lower dimensional subspaces. The k-means split scheme and regular split scheme are applied to split the solution vector into swarms. Then the swarms optimize the different components of the solution vector cooperatively. Some strategies are further used to improve the accuracy and convergence. Application of the proposed cooperative swarms based DPSO (CDPSO) on the traveling salesman problem (TSP) shows a significant improvement over conventional DPSOs. © 2008 IEEE.

Universal Learning Networks (ULNs) provide a generalized framework to many kinds of structures of neural networks with supervised learning. Multi-Branch Neural Networks (MBNNs) which use the framework of ULNs have been already shown that they have better representation ability in feedforward neural networks (FNNs). Multi-Branch structure of MBNNs can be easily extended to recurrent neural networks (RNNs) because the characteristics of ULNs include the connection of multiple branches with arbitrary time delays. In this paper, therefore, RNNs with Multi-Branch structure are proposed and they show that their representation ability is better than conventional RNNs. RNNs can represent dynamical systems and are useful for time series prediction. The performance evaluation of RNNs with Multi-Branch structure was carried out using a benchmark of time series prediction. Simulation results showed that RNNs with Multi-Branch structure could obtain better performance than conventional RNNs, and also showed that they could improve the representation ability even if they are smaller sized networks.

This paper proposes a graph-based evolutionary algorithm called Genetic Network Programming (GNP). Our goal is to develop GNP, which can deal with dynamic environments efficiently and effectively, based on the distinguished expression ability of the graph (network) structure. The characteristics of GNP are as follows. 1) GNP programs are composed of a number of nodes which execute simple judgment/processing, and these nodes are connected by directed links to each other. 2) The graph structure enables GNP to re-use nodes, thus the structure can be very compact. 3) The node transition of GNP is executed according to its node connections without any terminal nodes, thus the pst history of the node transition affects the current node to be used and this characteristic works as an implicit memory function. These structural characteristics are useful for dealing with dynamic environments. Furthermore, we propose an extended algorithm, "GNP with Reinforcement Learning (GNP-RL)" which combines evolution and reinforcement learning in order to create effective graph structures and obtain better results in dynamic environments. In this paper, we applied GNP to the problem of determining agents' behavior to evaluate its effectiveness. Tileworld was used as the simulation environment. The results show some advantages for GNP over conventional methods. © 2007 by the Massachusetts Institute of Technology.

Elevator Group Supervisory Control System (EGSCS) is a very large scale stochastic dynamic optimization problem. Due to its vast state space, significant uncertainty and numerous resource constraints such as finite car capacities and registered hall/car calls, it is hard to manage EGSCS using conventional control methods. Recently, many solutions for EGSCS using Artificial Intelligence (AI) technologies have been reported. Genetic Network Programming (GNP), which is proposed as a new evolutionary computation method several years ago, is also proved to be efficient when applied to EGSCS problem. In this paper, we propose an extended algorithm for EGSCS by introducing Reinforcement Learning (RL) into GNP framework, and an improvement of the EGSCS' performances is expected since the efficiency of GNP with RL has been clarified in some other studies like tile-world problem. Simulation tests using traffic flows in a typical office building have been made, and the results show an actual improvement of the EGSCS' performances comparing to the algorithms using original GNP and conventional control methods. Furthermore, as a further study, an importance weight optimization algorithm is employed based on GNP with RL and its efficiency is also verified with the better performances.

Genetic Network Programming (GNP) is an evolutionary computation which represents its solutions using graph structures. Since GNP can create quite compact programs and has an implicit memory function, it has been clarified that GNP works well especially in dynamic environments. In addition, a study on creating trading rules on stock markets using GNP with Importance Index (GNP-IMX) has been done. IMX is a new element which is a criterion for decision making. In this paper, we combined GNP-IMX with Actor-Critic (GNP-IMX&AC) and create trading rules on stock markets. Evolution-based methods evolve their programs after enough period of time because they must calculate fitness values, however reinforcement learning can change programs during the period, therefore the trading rules can be created efficiently. In the simulation, the proposed method is trained using the stock prices of 10 brands in 2002 and 2003. Then the generalization ability is tested using the stock prices in 2004. The simulation results show that the proposed method can obtain larger profits than GNP-IMX without AC and Buy&Hold.

The Elevator Group Supervisory Control Systems (EGSCS) are the control systems that systematically manage three or more elevators in order to efficiently transport the passengers in buildings. Double-deck elevators, where two elevators are connected with each other, serve passengers at two consecutive floors simultaneously. Double-deck Elevator systems (DDES) become more complex in their behavior than conventional single-deck elevator systems (SDES). Recently, Artificial Intelligence (AI) technology has been used in such complex systems. Genetic Network Programming (GNP), a graph-based evolutionary method, has been applied to EGSCS and its advantages are shown in some papers. GNP can obtain the strategy of a new hall call assignment to the optimal elevator when it performs crossover and mutation operations to judgment nodes and processing nodes. Meanwhile, Destination Floor Guidance System (DFGS) is installed in DDES, so that passengers can also input their destinations at elevator halls. In this paper, we have applied GNP to DDES and compared DFGS with normal systems. The waiting time and traveling time of DFGS are all improved because of getting more information from DFGS. The simulations showed the effectiveness of the double-deck elevators with DFGS in different building traffics.

Stability is one of the most important subjects in control systems. As for the stability of nonlinear dynamical systems, Lyapunov's direct method and linearized stability analysis method have been widely used. But, it is generally recognized that finding an appropriate Lyapunov function is fairly difficult especially for the nonlinear dynamical systems, and also it is not so easy for the linearized stability analysis to find the locally asymptotically stable region. Therefore, it is crucial and highly motivated to develop a new stability analysis method, which is easy to use and can easily study the locally asymptotically stable region at least approximately, if not exactly. On the other hand, as for the calculation of the higher order derivative, Universal Learning Networks ( ULNs) are equipped with a systematic mechanism that calculates their first and second order derivatives exactly.
So, in this paper, an approximate stability analysis method based on h approximation is proposed in order to overcome the above problems and its application to a nonlinear dynamical control system is discussed. The proposed method studies the stability of the original trajectory by investigating whether the perturbed trajectory can approach the original trajectory or not. The above investigation is carried out approximately by using the higher order derivatives of ULNs.
In summarizing the proposed method, firstly, the absolute values of the first order derivatives of any nodes of the trajectory with respect to any initial disturbances are calculated by using ULNs. If they approach zero at time infinity, then the trajectory is locally asymptotically stable. This is an alternative linearized stability analysis method for nonlinear trajectories without calculating Jacobians directly. In the method, the stability analysis of time-varying systems with multi-branches having any sample delays is possible, because the systems are modeled by ULNs. Secondly, the locally asymptotically stable region, where asymptotical stability is secured approximately, is obtained by finding the area where the first order terms of Taylor expansion are dominant compared to the second order terms with h approximation assuming that the higher order terms more than the third order are negligibly small in the area.
Simulations of an inverted pendulum balancing system are carried out. From the results of the simulations, it is clarified that the stability of the inverted pendulum control system is easily analyzed by the proposed method in terms of studying the locally asymptotically stable region. (c) 2005 Elsevier B. V. All rights reserved.

A new evolutionary method named 'Genetic Network Programming, GNP' has been proposed. GNP represents its solutions as directed graph structures which have some useful features inherently. For example, GNP has the implicit memory function which memorizes the past action sequences of agents, and GNP can re-use nodes repeatedly in the network flow, so very compact graph structures can be made. In this paper, the stock trading model using GNP with Candle Chart is proposed and its effectiveness is comfirmed by trading simulations.

Several methods of extracting association rules have been reported. A new evolutionary computation method named Genetic Network Programming (GNP) has also been developed recently and its efectiveness is shown for small datasets. However, it has not been tested for large datasets, particularly in datasets with a large number of attributes. The aim of this paper is to extract association rules from large and dense datasets using GNP considering a real world database with a huge number of attributes. We propose a new method where a large database is divided into many small datasets, then each GNP deals with one dataset having attributes with appropiate size, which was selected randomly from a large dataset and generated genetically. These GNPs are processed in parallel. We then propose some new genetic operations to improve the number of rules extracted and their quality as well. The proposed method improves remarkably on simulations. Fig. 1 shows the architecture of the proposed method. We use the CLIENT/SERVER model. CLIENT side carries out preprocessing of large database, assignment of files to each server, rule checking, and genetic operations on files. SERVER side carries out processing of each file using conventional GNP based mining method independently. The features and advantages of the proposed method are the following: Rule extraction is done in parallel. Each file generates its local pool of the rules. Files or datasets are treated as individuals in order to do new genetic operations over them and improve the rule extraction. Extracted rules are stored in a global pool. The rules are verified to avoid redundancy among them and it is assured that only new rules are stored.

According to Hebb's Cell assembly theory, the brain has the capability of function localization. On the other hand, it is suggested that the brain has three different learning paradigms: supervised, unsupervised and reinforcement learning. Inspired by the above knowledge of brain, we present a hierarchical learning system consisting of three parts: supervised learning (SL) part, unsupervised learning (UL) part and reinforcement learning (RL) part. The SL part is a main part learning input-output mapping; the UL part realizes the function localization of learning system by controlling firing strength of neurons in SL part based on input patterns; the RL part optimizes system performance by adjusting parameters in UL part. Simulation results confirm the effectiveness of the proposed hierarchical learning system.

This, palter investigates how genetic algorithms (GAs) can be improved to solve large-scale and complex problems more efficiently. First of all we review premature convergence, one of, the challenges confronted with when applying GAs to real-world problems. Next, some of the methods now available to prevent, premature convergence and their intrinsic defects are discussed. A qualitative analysis is then done oil the cause of premature convergence that is the loss of building blocks hosted in less-fit, individuals during the course of evolution. Thus, we propose a new improver - GAS with Reserve Selection (GARS), where a reserved area is set up to save, potential building, blocks and a selection mechanism based on individual uniqueness is employed to activate the potentials. Finally, case studies are done in a few standard problems well known in the literature, where the experimental results demonstrate the effectiveness and robustness of GARS in suppressing premature convergence, and also an enhancement is found in global optimization capacity.

This paper provides a deep insight into the performance of genetic algorithms with reserve selection (GARS), and investigates how parameters can be regulated to solve optimization problems more efficiently. First of all, we briefly present GARS, an improved genetic algorithm with a reserve selection mechanism which helps to avoid premature convergence. The comparable results to state-of-the-art techniques such as fitness scaling and sharing demonstrate both the effectiveness and the robustness of GARS in global optimization. Next, two strategies named Static RS and Dynamic RS are proposed for tuning the parameter reserve size to optimize the performance of GARS. Empirical studies conducted in several cases indicate that the optimal reserve size is problem dependent.

The Elevator Group Supervisory Control Systems (EGSCS) are the control systems that systematically manage three or more elevators in order to efficiently transport the passengers in buildings. Double-deck elevators, where two cages are connected with each other, are expected to be the next, generation elevator systems. Meanwhile, Destination Floor Guidance Systems (DFGS) are also expected in Double-Deck Elevator Systems (DDES). With these, the passengers could be served at two consecutive floors and could input, their destinations at elevator halls instead of conventional systems without DFGS. Such systems become more, complex than the traditional systems and require new control methods Genetic Network Programming (GNP), a graph-based evolutionary method, has been applied to EGSCS and its advantages are shown in some previous papers. GNP can obtain the strategy of a new hall call assignment to the optimal elevator because it, performs crossover and mutation operations to judgment nodes and processing nodes. In studies so far, the passenger's arrival has been assumed to take Exponential distribution for many years. In this paper, we have applied Erlang distribution and Binomial distribution in order to study how the passenger's arrival distribution affects EGSCS. We have found that the passenger's arrival distribution has great influence on EGSCS. It has been also clarified that GNP makes good performances under different conditions.

Particle swarm optimization (PSO) is a kind of evolutionary algorithm to find optimal (or near optimal) solutions for numerical and qualitative problems. Recently, a new variation on the traditional PSO algorithm, called cooperative particle swarm optimization (CPSO), has been proposed, employing cooperative behavior to significantly improve the performance of the original algorithm. However, a standard CPSO is focused only on continuous problems. In this paper, we present a new approach based on the CPSO to solve combination optimization problems by introducing dynamic splitting schemes. Reverse operation and simulated annealing techniques are further used to prevent the algorithm from being trapped in local minima. Finally, Traveling salesman problem (TSP) is applied to show the effectiveness of the proposed PSO. © 2007 SICE.

Our brain has three different learning paradigms: supervised, unsupervised and reinforcement learning. And it is suggested that those learning paradigms relate deeply to the cerebellum, cerebral cortex and basal ganglia in the brain, respectively. Inspired by these knowledge of brain, we present a brain-like learning system with those three different learning algorithms. The proposed system consists of three parts: the supervised learning (SL) part, the unsupervised learning (UL) part and the reinforcement learning (RL) part. The SL part, corresponding to the cerebellum of brain, learns an input-output mapping by supervised learning. The UL part, corresponding to the cerebral cortex of brain, is a competitive learning network, and divides an input space to subspaces by unsupervised learning. The RL part, corresponding to the basal ganglia of brain, optimizes the model performance by reinforcement learning. Numerical simulations show that the proposed brain-like learning system optimizes its performance automatically and has superior performance to an ordinary neural network.

これまで，多くの進化型計算手法がエージェントの行動系列の作成のために開発されてきたが，Genetic Network Programming（GNP）もその1 つである．本論文では，脳の機能局在をGNPに導入し，人工脳を構成論的に構築することを目指す．脳の機能局在とは，脳の異なる部分が特定の機能に対応していることをいう．GNP は有効グラフを基本構造としており，機能局在を実現するのに適している．本論文では，特に環境に応じて機能の切替えを行うGNP の獲得を重要度指標付きGNP を用いて行った．Many methods of generating behavior sequences of agents by evolution have been reported. A new evolutionary computation method named Genetic Network Programming (GNP) has been also developed recently along with these trends. The aim of this paper is to build an artificial model to realize functional localization of GNP considering the fact that the functional localization of the brain is realized in such a way that a different part of the brain corresponds to a different function. In this paper, it is especially stated that the switching function for functional localization can be realized using GNP with Importance Index (GNP IMX).

An efficient algorithm for building a classifier is proposed based on an important association rule mining using Genetic Network Programming (GNP). The proposed method measures the significance of the association via the chi-squared test. Users can define the conditions of important association rules for building a classifier flexibly. The definition can include not only the minimum threshold chi-squared value, but also the number of attributes in the association rules. Therefore, all the extracted important rules can be used for classification directly. GNP is one of the evolutionary optimization techniques, which uses the directed graph structure as genes. Instead of generating a large number of candidate rules, our method can obtain a sufficient number of important association rules for classification. In addition, our method suits association rule mining from dense databases such as medical datasets, where many frequently occurring items are found in each tuple. In this paper, we describe an algorithm for classification using important association rules extracted by GNP with acquisition mechanisms and present some experimental results of medical datasets.

The way of propagating and control of stochastic signals through Universal Learning Networks (ULNs) and its applications are proposed. ULNs have been already developed to form a superset of neural networks and have been applied as a universal framework for modeling and control of non-linear large-scale complex systems.
However, the ULNs cannot deal with stochastic variables. Deterministic signals can be propagated through a ULN, but the ULN does not provide any stochastic characteristics of the signals propagating through it.
The proposed method named Probabilistic Universal Learning Networks (PrULNs) can process stochastic variables and can train network parameters so that the signals behave with the pre-specified stochastic properties.
As examples of applications of the proposed method, control and identification of non-linear dynamic systems with noises are studied, and it is shown that the method are useful for dealing with the control and identification of the non-linear stochastic systems contaminated with noises. (c) 2005 Elsevier Ltd. All rights reserved.

Recently, Genetic Network Programming (GNP) has been proposed as one of the evolutionary algorithms. It represents its solutions as directed graph structures and the distinguished abilities have been shown. However, when we apply GNP to complex problems like the real world one, GNP must have robustness against the changes of environments and evolve quickly. Therefore, we introduced Automatically Generated Macro Nodes (AGMs) to GNP (GNP with AGMs). Actually GNP with AGMs has shown higher performances than the conventional GNP in terms of the fitness and the speed of evolution. In this paper, a new mechanism, AGMs with variable size, is introduced to GNP. Conventional AGMs have the fixed number of nodes and they evolve using only genetic operations, while a new method allows AGM to add nodes by necessity and delete nodes which do not contribute to the evolution of the AGM. The proposed GNP with AGMs of variable size is expected to evolve effectively and efficiently when it is applied to agent systems and also expected to make better behavior sequences of agents more easily than the conventional GNP algorithm. In the simulations, the proposed and conventional methods are applied to a tileworld problem and they are compared with each other. From the results, GNP with AGMs of variable size shows better fitness than GNP with AGMs of fixed size and the conventional GNP when adapting ten different environments.

Multiagent Systems with Symbiotic Learning and Evolution (Masbiole) has been proposed and studied, which is a new methodology of Multiagent Systems (MAS) based on symbiosis in the ecosystem. Masbiole employs a method of symbiotic learning and evolution where agents can learn or evolve according to their symbiotic relations toward others, i.e., considering the benefits/losses of both itself and an opponent. As a result, Masbiole can escape from Nash Equilibria and obtain better performances than conventional MAS where agents consider only their own benefits. This paper focuses on the evolutionary model of Masbiole, and its characteristics are examined especially with an emphasis on the behaviors of agents obtained by symbiotic evolution. In the simulations, two ideas suitable for the effective analysis of such behaviors are introduced; "Match Type Tile-world (MTT)" and "Genetic Network Programming (GNP)". MTT is a virtual model where tile-world is improved so that agents can behave considering their symbiotic relations. GNP is a newly developed evolutionary computation which has the directed graph type gene structure and enables to analyze the decision making mechanism of agents easily. Simulation results show that Masbiole can obtain various kinds of behaviors and better performances than conventional MAS in MTT by evolution.

Inspired by Hebb's cell assembly theory about how the brain worked, we have developed a function localization neural network (FLNN). The main part of a FLNN is structurally the same as an ordinary feedforward neural network, but it is considered to consist of several overlapping modules, which are switched according to input patterns. A FLNN constructed in this way has been shown to have better representation ability than an ordinary neural network. However, BP training algorithm for such FLNN is very easy to get stuck at a local minimum. In this paper, we mainly discuss the methods for improving BP training of the FLNN by utilizing the structural property of the network. Two methods are proposed. Numerical simulations are used to show the effectiveness of the improved BP training methods. © 2006 IEEE.

This paper proposes an improved model for the application of support vector machine (SVM) to achieve the real-world data classification. Being different from traditional SVM classifiers, the new model takes the thought about fuzzy theory into account. And a fuzzy decision-making function is also built to replace the sign function in the prediction stage of classification process. In the prediction part, the method proposed uses the decision value as the independent variable of fuzzy decision-making function to classify test data set into different classes, but not only the sign of which. This flexible design of decision-making model more approaches to the properties of real-world conditions in which interaction and noise influence exist around the boundary between different clusters. So many misclassifled cases can be modified when these sets are considered as fuzzy ones. In addition, a boundary offset is also introduced to modify the excursion produced by the imbalance of real-world dataset. Then an improved and more robust performance will be presented by using this adjustable fuzzy decision-making SVM model in simulations. © 2006 IEEE.

An efficient algorithm for important class association rule mining using Genetic Network Programming (GNP) is proposed. GNP is one of the evolutionary optimization techniques, which uses directed graph structures as genes. Instead of generating a large number of candidate rules, the method can obtain a sufficient number of important association rules for classification. The proposed method measures the significance of the association via the chi-squared test. Therefore, all the extracted important rules can be used for classification directly. In addition, the method suits class association rule mining from dense databases, where many frequently occurring items are found in each tuple. Users can define conditions of extracting important class association rules. In this paper, we describe an algorithm for class association rule mining with chi-squared test using GNP and present a classifier using these extracted rules. © 2006 IEEE.

A new graph-based evolutionary algorithm called Genetic Network Programming (GNP) has been proposed. The solutions of GNP are represented as graph structures, which can improve the expression ability and performance. In addition, GNP with Reinforcement Learning (GNP-RL) has been proposed to search for solutions efficiently. GNP-RL can use current information and change its programs during task execution, i.e., online learning. Thus, it has an advantage over evolution-based algorithms in case much information can be obtained during task execution. GNP-RL has a special stateaction space and it contributes to reducing the size of the Q-table and learning efficiently. The proposed method is applied to the controller of Khepera simulator and its performance is evaluated. © 2006 IEEE.

A new graph-based evolutionary algorithm named "Genetic Network Programming, GNP" has been already proposed. GNP represents its solutions as graph structures, which can improve the expression ability and performance. In addition, GNP with Reinforcement Learning (GNP-RL) was proposed a few years ago. Since GNP-RL can do reinforcement learning during task execution in addition to evolution after task execution, it can search for solutions efficiently. In this paper, GNP with Actor-Critic (GNP-AC) which is a new type of GNP-RL is proposed. Originally, GNP deals with discrete information, but GNP-AC aims to deal with continuous information. The proposed method is applied to the controller of the Khepera simulator and its performance is evaluated. © 2006 IEEE.

In this paper, we propose RasID-GA (an abbreviation of Adaptive Random Search with Intensification and Diversification combined with Genetic Algorithm) which improves the ability of diversification searching with Simplex Crossover (SPX). SPX generates the offspring based on uniform probability distribution and uses the M + 1 number of parent vectors, where M is the dimension of the vector. The RasIDGA with Simplex Crossover is compared with parallel RasIDs and GA with Simplex Crossover using 23 different objective functions having no local minima, a small number of local minima and a large number of local minima. © 2006 IEEE.

Elevator Group Supervisory Control System (EGSCS) is a very large scale stochastic dynamic optimization problem. Due to its vast state space, significant uncertainty, and numerous resource constraints such as finite car capacities and registered hall/car calls, it is hard to manage EGSCS using conventional control methods. Recently, many solutions for EGSCS using Artificial Intelligence (AI) technologies have been reported. Genetic Network Programming (GNP), which is proposed as a new evolutionary computation method several years ago, is also proved to be efficient when applied to EGSCS problem. In this paper, we propose an extended algorithm for EGSCS by introducing Reinforcement Learning (RL) into GNP framework, and expect to make an improvement of the EGSCS' performances since the efficiency of GNP with RL has been clarified in some other studies like tile-world problem. Simulation tests using traffic flows in a typical office building have been made, and the results show an actual improvement of the EGSCS' performances comparing to the algorithms using original GNP and conventional control methods. © 2006 IEEE.

A method of association rule mining using Genetic Network Programming (GNP) is proposed to improve the performance of association rule extraction from dense database. Rule extraction is done without identifying frequent itemsets used in Apriori-like methods. Association rules are represented as the connections of nodes in GNP. The proposed mechanisms calculate measurements of association rules directly from a database using GNP, and measure the significance of the association via the chi-squared test. The proposed system evolves itself by an evolutionary method and obtains candidates of association rules by genetic operations. Extracted association rules are stored in a pool all together through generations and reflected in genetic operators as acquired information. In this paper, we describe an algorithm capable of finding important association rules using GNP with sophisticated rule acquisition mechanisms and present some experimental results.

This paper proposes an improved model for the application of support vector machine (SVM) to achieve the real-world data classification. Being different from traditional SVM classifiers, the new model takes the thought about fuzzy theory into account. And a fuzzy decision-making function is also built to replace the sign function in the prediction stage of classification process. In the prediction part, the method proposed uses the decision value as the independent variable of fuzzy decision-making function to classify test data set into different classes, but not only the sign of which. This flexible design of decision-making model more approaches to the properties of real-world conditions in which interaction and noise influence exist around the boundary between different clusters. So many misclassified cases can be modified when these sets are considered as fuzzy ones. In addition, a boundary offset is also introduced to modify the excursion produced by the imbalance of real-world dataset. Then an improved and more robust performance will be presented by using this adjustable fuzzy decision-making SVM model in simulations.

Inspired by Hebb's cell assembly theory about how the brain worked, we have developed a function localization neural network (FLNN). The main part of a FLNN is structurally the same as an ordinary feedforward neural network, but it is considered to consist of several overlapping modules, which are switched according to input patterns. A FLNN constructed in this way has been shown to have better representation ability than an ordinary neural network. However, BP training algorithm for such FLNN is very easy to get stuck at a local minimum. In this paper, we mainly discuss the methods for improving BP training of the FLNN by utilizing the structural property of the network. Two methods are proposed. Numerical simulations are used to show the effectiveness of the improved BP training methods.

A new graph-based evolutionary algorithm called "Genetic Network Programming, GNP" has been proposed. The solutions of GNP are represented as graph structures, which can improve the expression ability and performance. In addition, GNP with Reinforcement Learning (GNP-RL) has been proposed to search for solutions efficiently. GNP-RL can use current information (state and reward) and change its programs during task execution. Thus, it has an advantage over evolution-based algorithms in case much information can be obtained during task execution. The GNP we proposed in the previous research deals with discrete information, but in this paper, we extend the conventional GNP-RL which can deal with numerical information. The proposed method is applied to the controller of Khepera simulator and its performance is evaluated.

An improved support vector machine (SVM) classifier model for classifying the real-world lopsided data is proposed. The most obvious differences between the model proposed and conventional SVM classifiers are the designs of decision-making functions and the introduction of an offset parameter. With considering about the vagueness of the real-world data sets, a fuzzy decision-making function is designed to take the place of the traditional sign function in the prediction part of SVM classifier. Because of the existence of the interaction and noises influence around the boundary between different clusters, this flexible design of decision-making model which is more similar to the real-world situations can present better performances. In addition, in this paper we mainly discuss an offset parameter introduced to modify the boundary excursion caused by the imbalance of the real-world datasets. Because noises in the real-world can also influence the separation boundary, a weighted harmonic mean (WHM) method is used to modify the offset parameter. Due to these improvements, more robust performances are presented in our simulations.

ニューラルネットワークはネットワークサイズが小さければ, 単純な問題しか解くことが出来ない.一方, サイズを大きくすると, 特に, 誤差逆伝播法を利用する場合, 学習に要する計算量やメモリ消費量が増大する.このような計算コストを削減し, 学習能力・汎化能力を向上させるための方法の1つがマルチブランチ構造である.従来のニューラルネットワークではノード間接続にシングルブランチを用いているが, マルチブランチ構造では, 任意の数のブランチでノード間を接続する.本論文では, ブランチ制御の機能局在の性質を利用して, マルチブランチ構造のニューラルネットワークの学習能力・汎化能力を向上させている.ブランチ制御では入力情報に依存してブランチを伝播する信号を調節し, ある入力情報に対してはあるブランチの信号を通過させ, 別のあるブランチでは信号を遮断する機能局在が可能である.ブランチ制御付きのマルチブランチニューラルネットワークを関数近似とTwo Spirals分類問題に適用し, 学習能力・汎化能力が向上することを検証している.

Genetic Network Programming (GNP) has been proposed as a new method of evolutionary computations and its basic characteristics are studied. GNP is constructed using graph structures whose gene consists of directed graphs, so it is possible to search solutions effectively especially for the dynamic real world problems due to the implicit memory function of its structure. Also GNP can easily implement a priori knowledge into its structure as functional nodes. There has been no example where GNP is applied to real world systems until now. In this paper, Elevator Group Supervisory Control System (EGSCS) which is a typical real world system is studied using GNP. From the simulations, the availability of GNP to EGSCS is confirmed and it is clarified that the proposed method can show better performances than other conventional methods.

Neural networks (NNs) can solve only a simple problem if the network size is too small, on the other hand, if the network size increases, it costs a lot in terms of memory space and calculation time. So, we have studied how to construct the network structure with high performances and low costs in space and time. A solution is a multi-branch structure. Conventional NNs use the single-branch for the connections, while the multi-branch structure has multi-branches between nodes. In this paper, a new method which enables the multi-branch NNs to have localized property is proposed. It is well known that RBF networks have localized property that makes it possible to approximate functions faster than sigmoidal NNs. By using the multi-branch structure having localized property of RBF networks, NNs could obtain high performances keeping the lower costs in space and time. Simulation results of function approximations and a classification problem illustrated the effectiveness of multi-branch NNs.

Various stock prices predicting and sell-buy strategy models have been so far proposed. They are classified as the fundamental analysis using the achievements of the companies and the trend of business, etc., and the technical analysis which carries out the numerical analysis of the movement of stock prices. On the other hand, as one of the methods for data mining which finds out the regularity from a vast quantity of stock price data, Genetic Algorithm (GA) has been so far applied widely. As a concrete example, the optimal values of parameters of stock indices like various moving averages and rates of deviation, etc. is computed by GA, and there have been developed various methods for predicting stock prices and determinig sell-buy strategy based on it. However, it is hard to determine which is the most effective index by the conventional GA. Moreover, the most effective one depends on the brands. So in this paper, a stock price prediction and sell-buy strategy model which searches for the optimal combination of various indices in the technical analysis has been proposed using Genetic Network programming and its effectiveness is confirmed by simulations.

According to the knowledge of brain science, it is suggested that there exists cerebral functional localization, which means that a specific part of the cerebrum is activated depending on various kinds of information human receives. The aim of this paper is to build an artificial model to realize functional localization based on Genetic Network Programming (GNP), a new evolutionary computation method recently developed. GNP has a directed graph structure suitable for realizing functional localization. We studied the basic characteristics of the proposed system by making GNP work in a functionally localized way.

Recently, artificial neural networks have been utilized for financial market applications. We have so far shown that multi-branch neural networks (MBNNs) could have higher representation and generalization ability than conventional NNs. In this paper, a prediction system of a stock price using MBNNs is proposed. Using the stock prices in time series and other information, MBNNs can learn to predict the price of the next day. The result of our simulations shows that the proposed system has better accuracy than a system using conventional NNs. © Springer-Verlag Berlin Heidelberg 2005.

Genetic Network Programming (GNP) has been proposed as a new method of evolutionary computations. Recently, GNP is applied to Elevator Group Supervisory Control System (EGSCS), that is, a benchmark of real world applications and its effectiveness is clarified. The EGSCS using GNP in the previous studies can control the elevator system using the conventional node functions. However, they do not have enough flexibility and generality for some uncertain factors due to the various different conditions in elevator systems. In this paper, several new frameworks of GNP for EGSCS are proposed in order to overcome the above problem considering the ranking calculation of elevators and node function optimization based on Real-coded GA. In the simulations, it is clarified that the proposed method can obtain better performances than the conventional methods. © 2005 SICE.

Recently, artificial neural networks have been utilized for financial market applications. We have so far shown that multi-branch neural networks (MBNNs) could have higher representation and generalization ability. In this paper, a prediction system of a stock price using MBNNs is proposed. The result of our simulations shows that the proposed system has better accuracy than a system using conventional NNs. © 2005 SICE.

"Multi Agent Systems with Symbiotic Learning and Evolution (Masbiole)" is an evolutionary computation modeled after the symbiotic relations in the ecosystem. In this paper, the simulations of "Robber and Police Game" that is assumed a basic model of Social Science are carried out to validate its performance, where two teams named robber and police compete each other. © 2005 SICE.

A method of association rule mining using Genetic Network Programming with sophisticated rule acquisition mechanisms is proposed to improve the performance of rule extraction. The system evolves itself by an evolutionary method and measures the significance of the association via the chi-squared test. Extracted rules are stored in a pool through generations and reflected in genetic operators as acquired information. © 2005 SICE.

A new graph-based evolutionary algorithm named "Genetic Network Programming, GNP" was proposed. GNP represents its solutions as graph structures, which can improve the expression ability and performance. And then, GNP with Reinforcement Learning (GNP with RL) has been proposed in order to search for solutions efficiently. GNP with RL can use the current information (state and reward) and change its programs during task execution, so it has an advantage over the evolution based algorithms in case much information can be obtained during task execution. In this paper, GNP with Actor-Critic (GNP-AC) which is a new type of GNP with RL is proposed. Originally, GNP deals with discrete information (ex. right, left, etc.), but GNP with AC aims to deal with continuous information (ex. the sensor value is "32"). The proposed method is applied to the controller of the Khepera simulator and its performance is evaluated. © 2005 SICE.

Since Genetic Network Programming (GNP) has been proposed as a new method of evolutionary computation, many studies have been done to its applications which cover not only virtual world problems but also real world systems like Elevator Group Supervisory Control System (EGSCS). In this paper, Reinforcement Learning is introduced to study its applicability and availability in the EGSCS based on the GNP structure with hall call assignment part embedded. © 2005 SICE.

A novel optimization method named RasID-GA (an abbreviation of Adaptive Random Search with Intensification and Diversification combined with Genetic Algorithm) is proposed in order to enhance the searching ability of conventional RasID, which is a kind of Random Search with Intensification and Diversification. RasID-GA is compared with conventional RasID and GA using 23 different objective functions, and it turns out that RasID-GA performs well compared with other methods. © 2005 IEEE.

Genetic Network Programming (GNP) whose gene consists of directed graphs has been proposed as a new method of evolutionary computations, and it is recently applied to the Elevator Group Supervisory Control System (EGSCS), a real world problem, to confirm its effectiveness. In the previous study, although the flow of traffic in the elevator system is known and fixed, it is changed dynamically with time in real elevator systems. Therefore, the EGSCS with an adaptive control should be studied considering such changes for practical applications. In this paper, the GNP with functional localization is applied to the EGSCS to construct such an adaptive system. In the proposed method, the switching GNP can switch the functionally localized GNPs (assigning GNPs) fitted to several kinds of traffic by detecting the change of the flow of traffic. From the simulations, the adaptability and effectiveness of the proposed method are clarified using the traffic data of a day in an office building. © 2005 IEEE.

Since Genetic Network Programming (GNP) has been proposed as a new method of evolutionary computation, many studies have been done on its applications which cover not only virtual world problems but also real world systems like Elevator Group Supervisory Control System (EGSCS) which is a very large scale stochastic dynamic optimization problem. From those researches, most of the significant features of GNP have been verified comparing to Genetic Algorithm (GA) and Genetic Programming (GP). Especially, the improvement of the performances on EGSCS using GNP showed an interesting and promising prospect in this field. On the other hand, some studies based on GNP with Reinforcement Learning (RL) revealed a better performance over conventional GNP on some problems such as tile-world models. As a basic study, Reinforcement Learning is introduced in this paper expecting to enhance EGSCS controller using GNP. © 2005 IEEE.

According to Hebb's cell assembly theory, the brain has the capability of function localization. On the other hand, it is suggested that the brain has three different learning paradigms: supervised, unsupervised and reinforcement learning. Inspired by the above knowledge of brain, we present a self-organizing function localization neural network (FLNN), that contains supervised, unsupervised and reinforcement learning paradigms. In this paper, we concentrate our discussion mainly on applying a simplified reinforcement learning called evaluative feedback to optimization of the self-organizing FLNN. Numerical simulations show that the self-organizing FLNN has superior performance to an ordinary artificial neural network (ANN).

A new evolutionary algorithm named "Genetic Network Programming, GNP" has been proposed. GNP represents its solutions as directed graphs, which realizes better expression ability than GA and GP which use string and tree structures, respectively. The aim of developing GNP is to deal with dynamic environments efficiently by using the distinguished expression ability and the inherently equipped functions of the network structure. However, since GNP is based on evolution, the programs cannot be changed until one generation ends. In this paper, we propose the extended algorithm, "GNP with Evolution and Learning" which combines evolution and reinforcement learning in order to adapt to dynamic environments quickly. The tileworld is used as a simulation environment and the results show some advantages of the proposed method.

Since the first neuron model was proposed, a lot of Neural Networks have been devised and been put into lots of practical uses. It is also true in the field of associative memory. Although so many useful memory models have been devised, there are still some problems, such as the limitation of storage capacity or too small attractor size to be stored.<br>In this paper, to solve the above problems, a novel associative memory is proposed. Its unique features are, (1) the memory network is obtained by training network parameters, (2) the size of the attractor of each stored memory can be controlled, and (3) some redundant nodes are introduced into the memory network in order to increase the storage capacity.<br>It is clarified from simulations that the proposed method can improve the memory functions, and can be applicable to the mutual associative memory easily.

A new evolutionary algorithm named "Genetic Network Programming, GNP" has been proposed. GNP represents its solutions as directed graphs, which realizes better expression ability than GA and GP which use string and tree structures, respectively. The aim of developing GNP is to deal with dynamic environments efficiently by using the distinguished expression ability and the inherently equipped functions of the network structure. However, since GNP is based on evolution, the programs cannot be changed until one generation ends. In this paper, we propose the extended algorithm, "GNP with Evolution and Learning" which combines evolution and reinforcement learning in order to adapt to dynamic environments quickly. The tileworld is used as a simulation environment and the results show some advantages of the proposed method.

Genetic Network Programming (GNP) extended from other evolutionary computations such as Genetic Algorithm (GA) and Genetic Programming (GP) has network structures as gene. Previously, the program size of conventional GNP was fixed and GNP programs have not introduced the concept of sub-routines, although GA and GP paid attention to sub-routines. In this paper, a new method where GNP with Automatically Generated Macro Nodes (GNP with AGMs) composed of a number of nodes is proposed for improving the performance of GNP. These AGMs also have network structures and are evolved like main GNP. In addition to that, AGMs have multiple inputs and outputs that have not been introduced in the past. In the simulations, comparisons between GNP program only and GNP with AGMs are carried out using the tile world. Simulation results shows that the proposed method brings better results compared with traditional GNP. And it is clarified from simulations that the node transition rules obtained by AGMs show the generalized rules able to deal with unknown environments.

A new graph-based evolutionary algorithm named "Genetic Network Programming, GNP" has been proposed. GNP represents its solutions as graph structures which have distinguished expression ability. In this paper, we propose GNP with Reinforcement Learning. Evolutionary algorithm of GNP makes a very compact graph structure and Reinforcement Learning of GNP improves search speed for solutions.

A new stock price prediction model is proposed based on Genetic Network Programming (GNP), i.e., an evolutionary computation recently developed. In the proposed prediction model, GNP is applied to searching for an optimal combination of two or more appropriate stock price indices, which is different from a conventional GA or GP based stock price prediction model, where GA or GP is usually used as an optimization technique to search for an optimal value of parameters in the stock price index. In this paper, a combination of several indices is shown to be more effective than a single index, because the most effective index usually differs from one brand to another. A series of simulation studies are carried out to confirm the effectiveness of the proposed new model.

In this paper, a new robust control method and its application to a photovoltaic (PV) supplied, separately excited DC motor loaded with a constant torque is discussed. The robust controller is designed against the load torque changes by using the first and second ordered derivatives of the universal learning networks (ULNs). These derivatives are calculated using the forward propagation algorithm, which is considered as an extended version of real time recurrent learning (RTRL). In this application, two ULN&apos;s are used: The first is the ULN identifier trained offline to emulate the dynamic performance of the DC motor system. The second is the ULN controller, which is trained online not only to make the motor speed follow a selected reference signal, but also to make the overall system operate at the maximum power point of the PV source. To investigate the effectiveness of the proposed robust control method, the simulation is carried out at four different values of the robustness coefficient in two different stages: The training stage, in which the simulation is done for a constant load torque. And the control stage, in which the controller performance is obtained when the load torque is changed. The simulation results showed that the robustness of the control system is improved although the motor load torque at the control stage is different from that at the training stage. (C) 2004 Elsevier Ltd. All rights reserved.

This paper presents a self-organizing function localization neural network (FLNN) inspired by Hebb's cell assembly theory about how the brain worked. The proposed self-organizing FLNN consists of two parts: main part and control part. The main part is an ordinary 3-layered feedforward neural network, but each hidden neuron contains a signal from the control part, controlling its firing strength. The control part consists of a SOM network whose outputs are associated with the hidden neurons of the main part. Trained with an unsupervised learning, SOM control part extracts structural features of input-output spaces and controls the firing strength of hidden neurons in the main part. Such self-organizing FLNN realizes capabilities of function localization and learning. Numerical simulations show that the self-organizing FLNN has superior performance than an ordinary neural network.

Quasi-ARX neural networks (NN) are NN based nonlinear models that not only have linear structures similar to linear ARX models, but also have useful interpretation in part of their parameters. However when applying an ordinary backpropagation (BP) for the training, it has potential risk that the BP algorithm is stuck at a local minimum, which results in a poorly trained model. In this paper, a homotopy continuation method is introduced to improve the convergence performance of BP training. The idea is to start the BP training with the criterion function for linear ARX model, which is gradually deformed first into one for quasi-ARX NN model with linear node functions, and then into the actual one for quasi-ARX NN with sigmoid node functions. By building the deformation into a usual recursive procedure for BP training of quasi-ARX NN model with adaptable node functions so that the proposed homotopy based BP algorithm is able to achieve improved convergence performance without much increase in the computation load. Numerical simulation results show that the proposed homotopy based BP has better performance than an ordinary BP.

In this paper, a functions localized network with branch gates (FLN-bg) is studied, which consists of a basic network and a branch gate network. The branch gate network is used to determine which intermediate nodes of the basic network should be connected to the output node with a gate coefficient ranging from 0 to 1. This determination will adjust the outputs of the intermediate nodes of the basic network depending on the values of the inputs of the network in order to realize a functions localized network. FLN-bg is applied to function approximation problems and a two-spiral problem. The simulation results show that FLN-bg exhibits better performance than conventional neural networks with comparable complexity. (C) 2003 Elsevier Ltd. All rights reserved.

This paper presents a constructive neural network with sigmoidal units and multiplication units, which can uniformly approximate any continuous function on a compact set in multi-dimensional input space. This network provides a more efficient and regular architecture compared to existing higher-order feedforward networks while maintaining their fast learning property. Proposed network provides a natural mechanism for incremental network growth. Simulation results on function approximation problem are given to highlight the capability of the proposed network. In particular, self-organizing process with RasID learning algorithm developed for the network is shown to yield smooth generation and steady learning.

This paper describes the application of Universal Learning Networks (ULNs) to the speed control of a separately excited dc motor drives fed from Photovoltaic (PV) generators via dc-dc buck-boost converters. In this application, two ULNs are used: the first is the Universal Learning Network Identifier (ULNI) that used to emulate the dynamic performance of the motor system. The second is the Universal Learning Network Controller (ULNC) that used to control the converter duty ratio so that the motor speed can follow an arbitrary reference signal. In addition to that, the overall system can operate at the Maximum Power Point (MPP) of the PV source. The free parameters of both networks are updated online by the forward propagation scheme, which is considered as an extended version of the Rel Time Recurrent Learning (RTRL). The simulation results showed good performance for the controller and the identifier networks. Promising results are also observed when the identifier network is trained in an environment contaminated with noise.

Recently, various attempts relevant to Multi Agent Systems (MAS) which is one of the most promising systems based on Distributed Artificial Intelligence have been studied to control large and complicated systems efficiently. In these trends of MAS, Multi Agent Systems with Symbiotic Learning and Evolution named Masbiole has been proposed. In Masbiole, symbiotic phenomena among creatures are considered in the process of learning and evolution of MAS. So we can expect more flexible and sophisticated solutions than conventional MAS. In this paper, we apply Masbiole to Iterative Prisoner's Dilemma Games (IPD Games) using Genetic Network Programming (GNP) which is a newly developed evolutionary computation method for constituting agents. Some characteristics of Masbiole using GNP in IPD Games are clarified.

Neural networks have attracted much interest in system identification and control communities because they can learn any nonlinear mapping. However, from a user's point of view, when neural networks are used as models for controller design, they do not have structures of easy use. This paper introduces a new neural network based prediction model for control of nonlinear systems. Distinctive features of the new model to conventional neural-network based ones are that it has not only meaningful interpretation on part of its parameters but also is linear for the input variables. The former feature makes parameter estimation easier and the latter allows us to derive a nonlinear controller directly from the identified prediction model. The modeling and the parameter estimation are described in detail. The usefulness of the new model is demonstrated by using numerical simulations.

This paper describes the use of Universal Learning Networks (ULNs) in the identification and control of a separately excited de motor loaded with a centrifugal pump and fed from Photovoltaic (PV) generator via dc-dc buck-boost converter. The Universal Learning Network Identifier (ULNI) is trained offline using the forward propagation algorithm to emulate the dynamic behavior of the de motor system. Then this identifier is used, instead of the motor system, for the online training of the Universal Learning Network Controller (ULNC). As a result, the motor speed can follow all arbitrarily selected reference signal. Furthermore, the overall system call operate at the Maximum Power Point (MPP) of the PV generator. which is the optimal operating point. The simulation results showed a good performance for the identifier and the controller as well.

Multi-branch neural networks have been proposed already in order to realize compact networks. It uses some branches between nodes, and this can improve the learning and generalization ability of the networks. In this paper, Branch Control is proposed on the multi-branch neural networks to further enhance the learning and generalization ability of the networks. Branch Control is to adjust the values of the signals on the branches depending on the network inputs using an additional branch control network. It has been clarified from simulation results of a function approximation problem that multi-branch neural networks with Branch Control could be improved more than that without Branch Control.

Recently, systems are becoming more complex and larger than ever, so numerous attempts have been made to introduce biological features into artificial systems, because many biological systems in the nature exist as one of the most complex systems.<br>Multi agent system with symbiotic learning and evolution have been recently proposed. It is named Masbiole. In this paper, Masbiole is reviewed and the method for evolving multi agent systems is proposed. From simulations on a multi objective knapsack problem, it has been clarified that Masbiole has better performance than that of conventional multi objective genetic algorithms.

A new evolutionary model with the network structure named Genetic Network Programming (GNP) has been proposed recently. GNP, that is, an expansion of GA and GP, represents solutions as a network structure and evolves it by using “offline learning (selection, mutation, crossover)”. GNP can memorize the past action sequences in the network flow, so it can deal with Partially Observable Markov Decision Process (POMDP) well. In this paper, in order to improve the ability of GNP, Q learning (an off-policy TD control algorithm) that is one of the famous online methods is introduced for online learning of GNP. Q learning is suitable for GNP because (1) in reinforcement learning, the rewards an agent will get in the future can be estimated, (2) TD control doesn't need much memory and can learn quickly, and (3) off-policy is suitable in order to search for an optimal solution independently of the policy. Finally, in the simulations, online learning of GNP is applied to a player for “Prisoner's dilemma game“ and its ability for online adaptation is confirmed. © 2003, The Institute of Electrical Engineers of Japan. All rights reserved.

A variety of neuron models combine the neural inputs through their summation and sigmoidal functions. Such structure of neural networks leads to shortcomings such as a large number of neurons in hidden layers and huge training data required. We introduce a kind of multiplication neuron which multiplies their inputs instead of summing to overcome the above problems. A hybrid universal learning network constructed by the combination of multiplication units arid summation units is proposed and trained for several well known benchmark problems. Different combinations of the above two are tried. It is clarified that multiplication is an essential computational element in many cases and the combination of the multiplication units with summation units in different layers in the networks improved the performance of the network. © 2003, The Institute of Electrical Engineers of Japan. All rights reserved.

Genetic algorithms are often well suited for optimization problems because of their parallel searching and evolutionary ability. Crossover and mutation are believed to be the main exploration operators. In this paper, we focus on how crossover and mutation work in binary-coded genetic algorithm and investigate their effects on bit's frequency of population. According to the analysis of equilibrium of crossover, we can see the bit-based simulated crossover (BSC) is strong crossover method. Furthermore, to increase robustness of binary-coded genetic algorithm, multi-generation inheritance evolutionary strategy(MGIS) was proposed. Simulation results demonstrate the effectiveness of the proposed method. © 2003, The Institute of Electrical Engineers of Japan. All rights reserved.

In this paper, RasID is enhanced and evaluated systematically. RasID is an abbreviation of Random Search with Intensification and Diversification. RasID can search for a global minimum based on a probabilitiy density function, which can be modified adaptively using information based on success and failure of the past searching. As a result, RasID performs intensified search and diversified search iteratively to find a global minimum.<br>The improvements of RasID are, (1) the probability density function is modified so that each variable can have its own unique density function, and (2) multiple candidates for solutions are created in the search to enhance the efficiency of the searching.<br>Modified RasID is compared systematically with typical optimization methods such as Evolutionary. Programming and Fast Evolutionary Programming introduced recently using 23 different complibated functions. From simulation results, it has been clarified that the performance of Modified RasID is comparable to the ones of EP and FEP in spite of using one individual.

In this paper, we propose a new auto correlation associative memory using universal learning networks (ULNs). The main purpose of this paper is to realize associative memory by training the network. Although so many useful models have been devised, there are some problems related to associative memory, such as the limitation of storage capacity or too small attractors of stored memories. To solve these problems, we obtain memory network by training network parameters not by calculating them in the conventional methods. Furthermore, we introduce "don't care nodes" into the networks just to enlarge network size and give more flexibility. We could verify that this method improves the memory capacity by computer simulations.

This paper proposes a stability analysis method based on the higher order derivatives of ULNs. In the proposed method, the following are proposed. Firstly, if the absolute values of the first order derivatives of any coordinates of the original trajectory with respect to any initial disturbances approach zero at time infinity, then the trajectory is locally asymptotically stable. Secondly, the (n, ε) locally asymptotically stable region, where asymptotical stability is secured approximately, is obtained by neglecting the higher order derivatives until nth order with e approximation.

This paper presents an adaptive neural network controller (ANNC) that is used to control the speed of a separately excited dc motor deriving a centrifugal pump load and fed from photovoltaic (PV) generator through dc-dc buck-boost converter. The controller is also used to track the maximum power point (MPP) of the PV generator by controlling the converter duty ratio. Such kind of controllers must have two objective functions to perform these two tasks, but in this research the objective function related to the MPP is converted to a constrained for the second objective function by making some approximation in the system equations. An adaptive neural network identifier (ANNI), which emulates the dynamic behavior of the motor system, plays an important role in computing the system Jacobian and hence updating the weights and biases of the ANNC. The weights and biases of both networks are updated on line using BP algorithm with adaptive learning rate. The computation of the adaptive learning rate is based on the value of the speed error through an empirical formula to get faster response with less oscillation and minimum overshoot. The transient response of the motor speed, current and voltage for a step change in the reference speed and the insolation are presented.

In the search for even better parsimonious neural network modeling, this paper describes a novel approach which attempts to exploit redundancy found in the conventional sigmoidal networks. A hybrid universal learning network constructed by the combination of proposed multiplication units with summation units is trained for several classification problems. It is clarified that the multiplication units in different layers in the network improve the performance of the network.

Symbiotic relation among agents is regarded as one of the most basic relations in the complex systems. In this paper, a method for constructing the required symbiotic relations among the agents is proposed, where the agent is made up of a layered neural network and its parameters are trained in order to realize the required symbiotic relations. From simulations of the ecosystems, whose agent corresponds to the species, it has been clarified that the proposed method can give an ecosystem model with more flexible and more powerful representation abilities than the conventional Lotka-Volterra model.

A new evolutionary computation method named Genetic Network Programming (GNP) was proposed recently. In this paper, an online learning method for GNP is proposed. This method uses Q learning to improve its state transition rules so that it can make GNP adapt to the dynamic environments efficiently.

This paper introduces a new method for applying multilayer perceptron (MLP) network to control of nonlinear systems. The MLP network is not used directly as a nonlinear controller, but used indirectly via an ARX-like macro-model. The ARX-like model incorporating MLP network is constructed in such a way that it has similar linear properties to a linear ARX model. The nonlinear controller is then designed in the same way as designing a linear controller based on a linear ARX model. Numerical simulations are carried to demonstrate the effectiveness of the new method.

This paper proposes a class of quasi-ARMAX models for non-linear systems. Similar to ordinary non-linear ARMAX models, the quasi-ARMAX models are flexible black-box models, but they have various linearity properties similar to those of linear ARMAX models. A modelling scheme is introduced to construct models consisting of two parts: a macro-part and a kernel-part. By using Taylor expansion and other mathematical transformation techniques, it is first constructed as a class of quasi-ARMAX interfaces (macro-parts) that have various linearity properties but contain some complicated coefficients. MIMO neurofuzzy models (kernel-parts) are then introduced to represent the complicated coefficients. It is shown that the proposed quasi-ARMAX models have both good approximation ability and some easy-to-use properties. The proposed models have been successfully applied to prediction, fault detection and adaptive control of non-linear systems.

This paper presents a homotopy approach to improving PEM identification of ARMAX model. PEM estimates of ARMAX model parameters are determined as the global minimum of criterion function, which is however not always unimodal because of the MA noise model part. An optimization-based PEM identification algorithm has a potential risk to be stuck at a local minimum that results in a poorly identified model. A homotopy continuation method is introduced to solve this problem. The idea is to start the estimation with the criterion function for PEM identification of the ARX model, which is gradually deformed into the actual one for PEM identification of the ARMAX model as the algorithm iterates. By building the deformation into the usual recursive procedure for the ARMAX identification and introducing a scheme to control the solution continuously staying in the global minima of the deformed criterion functions, the homotopy-based PEM identification algorithm is implemented in such a way that it has very good convergence performance, with only little increase in computation load compared to the usual PEM algorithm. (C) 2001 Elsevier Science Ltd. All rights reserved.

Recently, many people are involved in the research of complex systems which are made of a large number of elements interacting with each other.<br>In this paper, a new modeling method of mutual interaction of the complicated systems is proposed by using Spatial Distribution Universal Learning Networks (SdULNs) and Fuzzy inference algorithms.<br>In addition, the proposed model is applied to the modeling of ecosystems which are described as the Lotka-Volterra equation.<br>From simulations where apriori information on ecosystems is given, it has been cleared that the proposed method can present more flexible emergent dynamics than the conventional Lotka-Volterra equation.

A new control method of nonlinear dynamic systems is proposed based on the impulse responses of universal learning networks (ULNs). ULNs form a superset of neural networks. They consist of a number of interconnected nodes where the nodes may have any continuously differentiable nonlinear functions in them and each pair of nodes can be connected by multiple branches with arbitrary time delays. A generalized learning algorithm is derived for the ULNs, in which both the first order derivatives (gradients) and the higher order derivatives are incorporated. One of the distinguished features of the proposed control method is that the impulse response of the systems is considered as an extended part of the criterion function and it can be calculated by using the higher order derivatives of ULNs. By using the impulse response as the criterion function, nonlinear dynamics with not only quick response but also quick damping and small steady state error can be more easily obtained than the conventional nonlinear control systems with quadratic form criterion functions of state and control variables.

This paper studies how the generalization ability of models of dynamic systems can be improved by taking advantages of the second order derivatives of the outputs of networks with respect to the external inputs. The proposed method can be regarded as a direct implementation of the well-known regularization technique using the higher order derivatives of the Universal Learning Networks (ULNs). ULNs consist of a number of inter-connected nodes where the nodes may have any continuously differentiable nonlinear functions in them and each pair of nodes can be connected by multiple branches with arbitrary time delays. A generalized learning algorithm has been derived for the ULNs, in which both the first order derivatives (gradients) and the higher order derivatives are incorporated. First, the method for computing the second order derivatives of ULNs is discussed. Then a new method for implementing the regularization term is presented. Finally simulation studies on identification of a nonlinear dynamic system with noises are carried out to demonstrate the effectiveness of the proposed method. Simulation results show that the proposed method can improve the generalization ability of neural networks significantly especially in terms that (1) the robust network can be obtained even when the branches of trained ULNs are destructed, and (2) the obtained performance does not depend on the initial parameter values.

The implementation of weight initialization is directly related to the convergence of learning algorithms. In this paper we made a case study on the famous Mackey-Glass time series problem in order to try to find some relations between weight initialization of neural networks and pruning algorithms. The pruning algorithm used in simulations is Laplace regularizer method, that is, the backpropagation algorithm with Laplace regularizer added to the criterion function. Simulation results show that different kinds of initialization weight matrices display almost the same generalization ability when using the pruning algorithm, at least for the Mackey-Glass time series.

Recently, many methods of evolutionary computation such as Genetic Algorithm(GA) and Genetic Programming(GP) have been developed as a basic tool for modeling and optimizing the complex systems. Generally speaking, GA has the genome of string structure, while the genome in GP is the tree structure. Therefore, GP is suitable to construct the complicated programs, which can be applied to many real world problems. But, GP might be sometimes difficult to search for a solution because of its bloat. In this paper, a new evolutionary method named Genetic Network Programming(GNP), whose genome is network structure is proposed to overcome the low searching efficiency of GP and is applied to the problem on the evolution of behaviors of ants in order to study the effectiveness of GNP. In addition, the comparison of the performances between GNP and GP is carried out in simulations on ants behaviors.

In this paper, a stability analysis method based on the higher order derivatives of ULNs is proposed. In the proposed method, the following are proposed. Firstly, if the absolute values of the first order derivatives of any nodes with respect to any initial disturbances approach zero at time infinity, then the trajectory is locally asymptotically stable. Secondly, the locally asymptotically stable region, where asymptotical stability is secured approximately is obtained by comparing the first order derivatives and higher order derivatives.

In this paper, Universal Learning Networks with Branch Control of Relative Strength (ULNs with BR) is studied, which consists of basic networks and branch control networks. The branch control network can be used to determine which intermediate nodes of the basic network should be connected to the output node with a coefficient of relative strength ranging from zero to one. This determination will adjust the outputs of the intermediate nodes of the basic network. Therefore, by using ULNs with BC, locally functions distributed networks can be realized depending on the values of the inputs of the network. ULNs with BR is applied to a two-spirals problem. The simulation results show that ULNs with BC exhibits better performance than the conventional networks with comparable complexity.

A new designing method of a robust neural network controller against system environment changes using Universal Learning Network (ULN) is considered in this paper. With the introduced method, the worst values of system parameters can be searched as well as the optimization of controller parameters through a dual learning algorithm, which includes maximization and minimization procedures. Therefore, the robust controller can be obtained by minimizing the criterion function regarding the worst values of system parameters. Simulation results demonstrate that the system performance has been improved compared with the conventional method by using the proposed method.

Universal Learning Networks(ULNs) which are super set of supervised learning networks have been already proposed. They consist of a number of inter-connected nodes where the nodes may have any continuously differentiable nonlinear functions in them. Most of the functions used are sigmoidal functions. Disadvantages of exiting ULNs mainly lie in that long training time, a large number of nodes in hidden layers, and so on. In this paper, special ULNs with multiplication neurons(M neuron) are proposed, which have M neurons in the hidden layer and normal neurons with sigmoidal functions in the output layer. The computational power of networks models with multiplication neurons is compared with that of ULNs with existing neurons. In particular it is proved that ULNs with multiplication neurons are, with regard to the number of neurons that are needed, computationally more powerful than ULNs with normal sigmodial functions.

This paper discusses the problem of applying sigmoidal neural network to prediction and control of nonlinear dynamical systems. Instead of directly using neural networks as nonlinear models, we first develop a shield based on application specific knowledge, and then embed sigmoidal neural network model in the shield. An embedded sigmoidal neural network model obtained in this way not only has a structure favorable for certain applications such as controller design, but also has useful interpretation on part of model parameters. Corresponding to the meaningful part and the meaningless part of model parameters, a hierarchical training algorithm consisting of two learning loops is introduced to train the model, which has good performance on solving local minimum problems. The usefulness of the proposed prediction model is demonstrated by applying it to prediction and control of a simulated nonlinear system.

This paper discusses the problem of applying sigmoidal neural networks to identification of nonlinear dynamical systems. When using sigmoidal neural networks directly as nonlinear models, one often meets problems such as model parameters lack of physical meaning, sensitivity to noise in model training. In this paper, we introduce an embedded sigmoidal neural network model, in which the neural network is not used directly as a model, but is embedded in a shield such that part of the model parameters become meaningful. Corresponding to the meaningful part and the meaningless part of model parameters, a hierarchical training algorithm consisting of two learning loops is then introduced to train the model. Simulation results show that such a dual loop learning algorithm can solve the noise sensitivity and local minimum problems to some extent.

This paper studies how the generalization ability of models of dynamical systems can be improved by taking advantage of the second order derivatives of the outputs with respect to the external inputs. The proposed method can be regarded as a direct implementation of the well-known regularization technique using the higher order derivatives of the Universal Learning Networks (ULNs). ULNs consist of a number of interconnected nodes where the nodes may have any continuously differentiable nonlinear functions in them and each pair of nodes can be connected by multiple branches with arbitrary time delays. A generalized learning algorithm has been derived for the ULNs, in which both the first order derivatives (gradients) and the higher order derivatives are incorporated. First, the method for computing the second order derivatives of ULNs is discussed. Then, a new method for implementing the regularization term is presented. Finally, simulation studies on identification of a nonlinear dynamical system with noises are carried out to demonstrate the effectiveness of the proposed method. Simulation results show that the proposed method can improve the generalization ability of neural networks significantly, especially in terms that (1) the robust network can be obtained even when the branches of trained ULNs are destructed, and (2) the obtained performance does not depend on the initial parameter values. © 2001 Elsevier Science Ltd. All rights reserved.

Generalization ability of neural networks is the most important criterion to determine whether one algorithm is powerful or not. Many new algorithms have been devised to enhance the generalization ability of neural networks[1][2]. In this paper a new algorithm using the Gram-Schmidt orthogonalization algorithm [3] to the outputs of nodes in the hidden layers is proposed with the aim to reduce the interference among the nodes in the hidden layers, which is much more efficient than the regularizers methods. Simulation results confirm the above assertion.

本論文では，確率信号の処理能力を備えた確率一般化学習ネットワーク（Probabilistic Universal Learning Networks: PrULNs）を提案する．確率一般化学習ネットワークは一般化学習ネットワーク（Universal Learning Networks: ULNs）の拡張である．一般化学習ネットワークは非線形大規模複雑系のモデリングや制御系設計のための一般的な枠組みを提供しており，多くのニューラルネットワークのクラスをカバーしている． ULNs のために一般化された学習アルゴリズムが開発されているが，ほとんどの学習ネットワークがこのアルゴリズムによって学習できる．しかしながら，ULNsは確率信号が処理できない．ここで提案する PrULNs は信号の統計的特性量を計算する機構を備えており，また，ネットワークによって処理された信号が所望の確率特性を持つようにパラメータの学習をすることもできる． PrULNs は次のような問題を解決するのに役立つ：（1）学習ネットワークの汎化能力の向上；（2）従来の確率的制御より進んだ制御；（3）カオスのような複雑系の設計．Probabilistic Universal Learning Networks (PrULNs) are proposed which are learning networks with capability of dealing with stochastic signals. PrULNs are extension of Universal Learning Networks (ULNs). ULNs form a superset of neural networks and were proposed to provide a universal framework for modeling and control of nonlinear large-scale complex systems. A generalized learning algorithm has been devised for ULNs which can also be used in a unified manner for almost all kinds of learning networks. However, the ULNs can not deal with stochastic variables. Specific value of a stochastic signal can be propagated through a ULN, but the ULN does not provide any stochastic characteristic of the signals propagating through it. The PrULNs proposed here are equipped with machinery to calculate stochastic properties of signals and to train network parameters so that the signals behave with the pre-specified stochastic properties. The PrULNs will contribute to the solution of the following problems: (1) improving the generalization capability of the learning networks, (2) more sophisticated stochastic control than the conventional stochastic control, (3) designing problem for the complex systems such as chaotic systems.

Universal learning networks (ULN's) and robust control system design are discussed, ULN's provide a generalized framework to model and control complex systems. They consist of a number of inter-connected nodes where the nodes may have any continuously differentiable nonlinear functions in them and each pair of nodes can be connected by multiple branches with arbitrary time delays. Therefore, physical systems which can be described by differential or difference equations and also their controllers can be modeled in a unified way. So, ULN's constitute a superset of neural networks or fuzzy neural networks. In order to optimize the systems, a generalized learning algorithm is derived for the ULN's, in which both the first order derivatives (gradients) and the higher order derivatives are incorporated. The derivatives are calculated by using forward or backward propagation schemes. These algorithms for calculating the derivatives are extended versions of back propagation through time (BPTT) and real time recurrent learning (RTRL) by Williams in the sense that generalized nonlinear functions and higher order derivatives are dealt with. As an application of ULN's, the higher order derivative, one of the distinguished features of ULN's, is applied to realizing a robust control system in this paper, In addition, it is shown that the higher order derivatives are effective tools to realize sophisticated control of nonlinear systems. Other features of ULN's such as multiple branches with arbitrary time delays and using a priori information will be discussed in other papers.

A new chaos control method is proposed which is useful for taking advantage of chaos and avoiding it, The proposed method is based on the following facts: 1) chaotic phenomena can be generated and eliminated by controlling maximum Lyapunov exponent of systems and 2) maximum Lyapunov exponent can be formulated and calculated by using higher order derivatives of Universal Learning Networks (ULN's). ULN's consist of a number of inter-connected nodes where the nodes may have any continuously differentiable nonlinear functions in them and each pair of nodes can be connected by multiple branches with arbitrary time delays. A, generalized learning algorithm has been derived for the ULN's, in which both the first-order derivatives (gradients) and the higher order derivatives are incorporated. In simulations, parameters of ULN's with bounded node outputs are adjusted for maximum Lyapunov component to approach the target value. And, it has been shown that a fully connected ULN with three sigmoidal function nodes is able to generate and eliminate chaotic behaviors by adjusting the parameters.

How to determine the appropriate or optimal activation function in the neural networks for a specific learning samples remains open. In this paper the cascade-correlation algorithm which is an efficient constructive algorithm is used after implementation of some kinds of clustering algorithms to produce a modular network structure as a surrogate of activation node functions in the radial basis function (RBF) networks. In this way great improvement on the convergence rate of training algorithms and better approximation are achieved. Simulations with the two-spiral data sets proved the above assertion.

Recently many studies have been made on automatic design of the complex systems by using the evolutionary optimization techniques such as Genetic Algorithms (GA), Evolution Strategy (ES), Evolutionary Programming (EP) and Genetic Programming (GP). It is generally recognized that these techniques are very useful for optimizing fairly complex systems such as generation of intelligent behavior sequences of robots. In this paper, a new method named Genetic network Programming (GNP) is proposed in order to acquire these behavior sequences efficiently. GNP is composed of plural nodes for agents to execute simple judgment/processing and they are connected with each other to form a network structure. Agents behave according to the contents of the nodes and their connections in GNP. In order to obtain better structure, GNP changes itself by using evolutionary optimization techniques.

Using backpropagation algorithm (BP) to train neural networks is a widely adopted practice in both theory and practical applications. But its distributed weight representation, that is the weight matrix of final network after training by using BP are usually not sparsified, and prohibits its use in the rule discovery of inherent functional relations between the input and output data, so in this aspect some kinds of structure optimization are needed to improve its poor performance. In this paper with this in mind a new method to prune neural networks is proposed based on some statistical quantities of neural networks. Comparing with the other known pruning methods such as structural learning with forgetting (SLF) and RPROP algorithm, the proposed method can attain comparable or even better results over these methods without evident increase of the computational load. Detailed simulations using the Iris data sets exhibit our above assertion.

A Min Max robust control method is proposed for nonlinear systems based on the use of the higher order derivatives calculation of Universal Learning Networks (ULNs). An extended criterion function containing sensitivity terms is considered for controller design and the criterion function is evaluated at several specific operating points corresponding to certain system parameters. The ULNs learning is then performed in such a way that, at each step, it minimizes the worst criterion function among several operating points. It is found that the proposed control method is less time-consuming in the ULNs learning and a obtained controller has better performance than the conventional methods.

In this paper, a new Genetic Symbiosis Algorithm (GSA) is proposed based on the symbiotic concept found widely in ecosystems. Since in the conventional Genetic Algorithms (GA) reproduction is done using only the fitness function of each individual, there are some problems such as premature convergence to an undesirable solution at a very early stage of generation. In addition in some GA applications, it is sometimes required to maintain diversified solutions and to find out many locally optimal solutions. GSA is proposed to solve these problems by considering mutual symbiotic relations between Individuals. From simulations on optimizing a nonlinear function, it has been clarified that GSA can find more flexible solutions that can meet a variety of user's requests than the conventional methods.

In this paper, Universal Learning Networks with Branch Control (BrcULNs) are proposed, which consist of basic networks and branch control networks. The branch control network can be used to determine which branches of the basic network should be connected or disconnected. This determination depends on the inputs or the network flows of the basic network. Therefore, by using the BrcULNs, locally functions distributed networks can be realized depending on the values of the inputs of the network or the information of the network flows. The proposed network is applied to some function approximation problems. The simulation results show that the BrcULNs exhibit better performance than the conventional networks with comparable complexity.

Universal Learning Networks (ULNs) are proposed and their application to chaos control is discussed. ULNs provide a generalized framework to model and control complex systems. They consist of a number of inter-connected nodes where the nodes may have any continuously differentiable nonlinear functions in them and each pair of nodes can be connected by multiple branches with arbitrary time delays. Therefore, physical systems, which can be described by differential or difference equations and also their controllers, can be modeled in a unified way, and so ULNs may form a super set of neural networks and fuzzy neural networks. In order to optimize the ULNs, a generalized learning algorithm is derived, in which both the first order derivatives (gradients) and the higher order derivatives are incorporated. The derivatives are calculated by using forward or backward propagation schemes. These algorithms for calculating the derivatives are extended versions of Back Propagation Through Time (BPTT) and Real Time Recurrent Learning (RTRL) of Williams in the sense that generalized node functions, generalized network connections with multi-branch of arbitrary time delays, generalized criterion functions and higher order derivatives can be deal with. As an application of ULNs, a chaos control method using maximum Lyapunov exponent of ULNs is proposed. Maximum Lyapunov exponent of ULNs can be formulated by using higher order derivatives of ULNs, and the parameters of ULNs can be adjusted so that the maximum Lyapunov exponent approaches the target value. From the simulation results, it has been shown that a fully connected ULN with three nodes is able to display chaotic behaviors. © 2000 Elsevier Science Ltd.

This paper presents a case study for the overlapped multi-neural-network (OMNN). An overlapped multi-neural-network, structurally, is the same as an ordinary feedforward neural network, but it is considered as one consisting of several subnets. Ail subnets have the same input-output units, but some different hidden units. Input-output spaces are partitioned into several parts, each of which corresponds to one subnet of OMNN. Numerical simulations show that such an OR INN has superior performance in that It has better presentation ability than an ordinary neural network and better generalization ability than a non-overlapped multi-neural-network.

Recently many researches on group robot systems have been studied, where a number of robots behave in a group like birds' or ants. It is generally known that each robot has a limited intellectual power, but the robots can behave more intellectually in a group because they can interact each other. One of the most famous researches in these fields is Boids which is the artificial model of the birds behavior in the computer software. And there have been reported the multi-agent robot systems which can do many kinds of tasks efficiently by training the rules between environments and actions using reinforced learning. This paper also proposes a multi-agent system where a criterion function is defined regarding the behavior of the multi-agent system and parameters of mutual interaction of the agents are trained in order to optimize the above criterion function. From simulations, it has been shown that emergent behaviors of the agents can be developed by appropriately adjusting the parameters.

In this paper, we present a new learning method using prior information for three-layer neural networks. Usually when neural networks are used for identification of systems, all of their weights are trained independently, without considering their inter-relation of weights values. Thus the training results are not usually good. The reason for this is that each parameter has its influence on others during the learning. To overcome this problem, first, we give exact mathematical equation that describes the relation between weight values given a set of data conveying prior information. Then we present a new learning method that trains the part of the weights and calculates the others by using these exact mathematical equations. This method often keeps a priori given mathematical structure exactly during the learning, in other words, training is done so that the network follows predetermined trajectory. Numerical computer simulation results are provided to support the present approaches.

This paper deals with the problems of robust fault detection using Kullback discrimination information (KDI) in presence of nonlinear undermodeling. The systems to be diagnosed are assumed to contain certain unknown nonlinear elements. The fault detection is performed by applying the KDI to a linear ARMAX model with model uncertainty, in which error due to nonlinear undermodeling is described using a group of fuzzy models with adjustable parameters. The estimate of modeling error is considered in the KDI analysis and thresholding decision for robustness realization. The effectiveness of the proposed robust fault detection scheme is examined through numerical simulations.

Nonlinear elements such as friction, dead zone, backlash are mixed in most mechatronics systems, and these are factors that make control accuracy of systems decrease and cause an oscillation. It is difficult to deal with the problems of modeling and control of such systems using common sigmoidal neural networks because there exist nonsmooth nonlinearities in the systems, and there is no easy way to incorporate knowledge and experiences accumulated from past study. In this paper, a design method for nonlinear mechatronics control systems is proposed, in which both of the control object and its controller are represented by using a Universal Learning Network, and the network parameters are trained using a random search algorithm called RasID. In this approach, knowledge and experience about models and controllers are easily incorporated into the network including the nonlinear elements and their compensation elements expressed by non-differentiable functions. Some simulations of a nonlinear crane control system with dead-zone characteristic were carried out. The effectiveness of the proposed design method is illustrated via simulations.

Universal Learning Network (ULN) which is a super-set of supervised learning networks has been already proposed. Parameters in ULN are trained in order to optimize a criterion function as conventional neural networks, and after training they are used as constant parameters. In this paper, a new method to alter the parameters depending on the network flows is presented to enhance representation abilities of networks. In the proposed method, there exists two kinds of networks, the first one is a basic network which includes varying parameters and the other one is a network which calculates the optimal varying parameters depending on the network flows of the basic network. It is also proposed in this paper that any type of networks such as fuzzy inference networks, radial basis function networks and neural networks can be used for the basic and parameter calculation networks. From simulations where parameters in a neural network are altered by a fuzzy inference networks, it is shown that the networks with the same number of varying parameters have higher representation abilities than the conventional networks.

In this paper, a method for identifying nonlinear dynamic systems with noise is proposed by using Probabilistic Universal Learning Networks (PrULNs). PrULNs are extensions of Universal Learning Networks (ULNs). ULNs form a superset of neural networks and were proposed to provide a universal framework for modeling and control of nonlinear large-scale complex systems. But the ULN does not provide any stochastic characteristics of the signals propagating through it. The PrULNs are equipped with machinery to calculate stochastic properties of signals and to train network parameters so that the signals behave with the pre-specified stochastic properties. On the other hand, it is generally recognized that there exists an overfitting problem when identification of nonlinear dynamic systems with noise is done by neural networks. In this paper, it is shown from simulation results of identification of a nonlinear robot dynamics that PrULNs are useful for avoiding the overfitting.

A new model and learning method for controlling multi-agent systems that are composed of plural agents are presented. Since it is not possible to use the gradient method, the RasID learning method is employed. Simulation results show that the above model and method are effective for controlling the dynamics of the multi-agent systems because desired dynamics can be easily obtained by training fuzzy parameters.

Nth order asymptotic orbital stability analysis method has been proposed, which is a method to determine whether nonlinear system is stable or not with large fluctuations of the system states. In this paper, we discuss the stability of robust control of a nonlinear crane system using this method. The robust control system which is studied in this paper is more stable than ordinary control system even with the large disturbances. Nth order asymptotic orbital stability analysis is described by using the higher order derivatives of Universal Learning Networks (ULNs), and ULNs are tools for modeling, managing and controlling large scale complicated systems such as economic, social and living systems as well as industrial plants. In this paper, robust control system is constructed by ULNs too, and the controller is best tuned through learning. Finally, simulations of 1st order orbital change of a nonlinear crane system are carried out. From results of simulations, it is shown that the robust control method have better performance and robustness than commonly used method.

According to recent knowledge of brain science, it is suggested that there exists functions distribution, which means that specific parts exist in the brain for realizing specific functions. This paper introduces a new brain-like model called Learning Petri Network (LPN) that has the capability of functions distribution and learning, The idea is to use Petri net to realize the functions distribution and to incorporate the learning and representing ability of neural network into the Petri net. The obtained LPN can be used in the same way as a neural network to model and control dynamic systems, while it is distinctive to a neural network in that it has the capability of functions distribution, An application of the LPN to nonlinear crane control systems is discussed. It is shown via numerical simulations that the proposed LPN controller has superior performance to the commonly-used neural network one.

Probabilistic Universal Learning Networks (PrULNs) are proposed, which are learning networks with a capability of dealing with stochastic signals. PrULNs are extensions of Universal Learning Networks (ULNs). ULNs form a superset of neural networks and were proposed to provide a universal framework for modeling and control of nonlinear large-scale complex systems. A generalized learning algorithm has been devised for ULNs which can also be used in a unified manner for almost all kinds of learning networks. However, the ULNs can not deal with stochastic variables. Specific value of a stochastic signal can be propagated through a ULN, but the ULN does not provide any stochastic characteristics of the signals propagating through it. The PrULNs proposed here are equipped with machinery to calculate stochastic properties of signals and to train network parameters so that the signals behave with the pre-specified stochastic properties. The PrULNs will contribute to the solution of the following problems: (1) improving the generalization capability of the learning networks, (2) more sophisticated stochastic control than the conventional stochastic control, (3) designing problems for the complex systems such as chaotic systems. In this paper, PrULN is proposed and is applied to a nonlinear control system with noise.

A new control method of nonlinear dynamic systems is proposed based on impulse responses of Universal Learning Networks (ULNs). ULNs form a superset of neural networks. They consist of a number of inter-connected nodes where the nodes may have any continuously differentiable nonlinear functions in them and each pair of nodes can be connected by multiple branches with arbitrary (positive, zero, or even negative) time delays. A generalized learning algorithm is derived for the ULNs, in which both the first order derivatives (gradients) and the higher order derivatives are incorporated. The derivatives are calculated by using forward or backward propagation schemes. The algorithm can also be used in a unified manner for almost all kinds of learning networks. In this paper, not only the controlled object but also its controller are described by the ULNs and the controller is constructed by using the higher order derivatives of ULNs. The main feature of the proposed control method is to use impulse response defined by the higher order derivatives as a criterion function of the network. By using the impulse response, nonlinear dynamics with not only quick response but also quick damping can be more easily obtained than the conventional nonlinear control systems with quadratic form criterion functions of control variables.

In thermal power plants, it is an important theme to improve the control performance of main steam pressure and temperature etc. during load up/down. This paper focuses on temperature control that is the most difficult problem due to the non-linearity and long dead times of power plants. Model Reference Adaptive Control (MRAC) is applicable to the feed-forward control of power plants, but there are some problems. The most serious problem is that persistently exciting (PE) condition is not satisfied, and so it is difficult to estimate plant parameters using the well-known recursive least squares method. It is proposed in this paper that Jacobians of the neural networks (NN) are applied to identify the above mentioned plant parameters and control law is obtained by two methods, that is, one is the method to use the Jacobians of the NN plant model which is obtained by off line forward model learning, the other is the method to utilize the Hessian of the cost function. This method is evaluated by a detailed simulator that represents accurately the dynamics of power plants, and usefulness and effectiveness of the proposed method is proved.

Recently, a number of studies have been done to investigate complicated systems such as economical, social, ecological and living systems. And, it is well known that complicated systems can be analyzed by the concept of symbiosis which is made up of competition, exploitation and coexistence and so on. The purpose of this paper is to propose a new modeling method for complicated systems by using the concept of symbiosis. A useful modeling method is presented in order to improve the ability of representing the symbiosis phenomena using the technique of fuzzy inference.

In this paper, it is studied how the generalization ability of modeling of the dynamic systems can be improved by taking advantages of the second order derivatives of the criterion function with respect to the external inputs. The proposed method is based on the regularization theory proposed by Poggio, but a main distinctive point in this paper is that extension to dynamic systems from static systems has been taken into account and actual second order derivatives of the Universal Learning Network have been used to train the parameters of the networks. The second order derivatives term of the criterion function may minimize the deviation caused by the external input changes. Simulation results show that the method is useful for improving the generalization ability of identifying nonlinear dynamic systems using neural networks.

Recently Universal Learning Networks (ULNs) have been proposed, which are a super set of any kinds of supervised learning networks. One of the important features of the ULNs is that the ULNs have a systematic algorithm for calculating higher order derivatives of the criterion function with respect to parameters. And a robust control method and chaos control method using the second order derivatives of the ULNs have been proposed. In this paper, a new control design method of the nonlinear systems is proposed, which is an extension of the above robust control method in terms of considering not only the stability but also the quick response of the systems.

Chaotic behaviors are characterized mainly by Lyapunov numbers of a dynamic system. In this paper, a new method is proposed, which can control the maximum Lyapunov number of dynamic system that can be represented by Universal Learning Networks (ULNs). The maximum Lyapunov number of a dynamic system can be formulated by using higher order derivatives of ULNs and parameters of ULNs can be adjusted for the maximum Lyapunov number to approach to the target value by the combined gradient and random search method. Based on simulation results, a fully connected ULN with three nodes is possible to display chaotic behaviors.

Control systems using neural networks have been used recently in many fields, but some problems remain unsolved. One of the problems which should be overcome is to enhance the robustness of the neural network control systems. In this paper, a new robust control method is proposed, which is based on the second order derivatives of Universal Learning Network and fuzzy criterion function.

In this paper, a new random search method Ras Val for Neural Networks (NN) learning is proposed. RasVal (Random Search with Variable Search Length) is a kind of random search and it can find a global minimum instead of a local minimum using the capability of intensified and diversified searches. The main different point of RasVal from commonly used Random Search Methods (RSM) is that the shape of the probability density function for random searching can be adjusted based on the information of success or failure of the search. First, RasVal is described and after that, performance between RasVal, Back Propagation Method (BP) and Back Propagation Method with momentum (Mom. BP) are compared. The performance is evaluated by the simulations which include both static and dynamic Neural Networks (NN) learning problems. In the simulations, NN is trained to realize nonlinear functions and to control a nonlinear crane system by using RasVal, BP and Mom. BP. Simulation results show that Ras Val is superior or nearly equal to BP and Mom. BP because of the ability of intensification and diversification of the search.

This research is concerned with fault detection of nonlinear systems using Kullback discrimination information (KDI) as an index. A hybrid quasi-ARMAX model is proposed, which combines a linear ARMAX model and a multi-ARX-model based on interpolation. In the case where the faults occur on the ARMAX model part, a KDI-based "robust" fault detection is performed, in which multi-ARX-model part is treated as error due to nonlinear undermodeling. Ill other cases, the model is transformed into several local ARMAX models and fault detection is performed by using the KDI to discriminate each identified local model. In this paper, we mainly concentrate our discussion on the latter cases. Copyright (C) 1998 IFAC.

This paper presents a modeling scheme for nonlinear black-bet systems based on Universal Leaning Networks (ULN). The ULN, a superset of all kinds of neural networks, consists of two kinds of elements: nodes and branches corresponding to equations and their relations in traditional description of dynamic systems. Following the idea of ULN, a nonlinear black-bet, system is first represented by a set of related unknown equations, and then treated as the ULN with nodes and branches. Each unknown node function in the ULN is re-parameterized by using an adaptive fuzzy model. One of distinctive features of the black-bet model constructed in this way is that it can incorporate prier knowledge obtained from input-output data into its modeling and thus its parameters to be trained have explicit meanings useful for estimation and application.

This paper presents an adaptive control scheme for nonlinear black-bet systems based on the we of Universal Learning Networks (ULN). A ULN nonlinear controller is constructed in a similar way to linear stochastic control theory. In the obtained ULN controller, some node functions are known, while others are unknown. Each unknown node function is reparameterized using an adaptive fuzzy model. A robust adaptive algorithm is developed to adjust the unknown parameters in the controller. The effectiveness of the proposed control scheme is examined via numerical simulations.

This paper presents a novel random searching scheme called RasID for neural networks training. The idea is to introduce a sophisticated probability density function (PDF) for generating search vector. The PDF provides two parameters for realizing intensified search in the area where it is likely to find good solutions locally or diversified search in order to escape from a local minimum based on the success-failure of the past search. Gradient information is wed to improve the search performance. The proposed scheme Is applied to layered neural networks training and Is benchmarked against other deterministic and nondeterministic methods.

In this paper, we present a new control method firstly for nonlinear systems using Universal Learning Network(ULN) with radial basis function(RBF). ULN can model and control the large scale complicated systems such as industrial plants, economic, social and life phenomena. The basic idea of ULN is that large scale complicated control systems can be modeled by the network which consists of nonlinearly operated nodes and branches which may have arbitrary time delays including zero or minus ones. Second, a new learning algorithm is applied to the design of the optimal network controller of a nonlinear control system. The optimization method is named RasVal, which is a kind of random searching, and it can search for a global minimum systematically and effectively in a single framework which is not a combination of different methods. The searching for a global minimum is carried out based on the probability density functions of searching, which can be modified using information on success or failure of the past searching in order to execute intensified and diversified searching. Simulation studies were carried out in the following four cases to compare the learning speed and performance: (1). comparing Radial Basis Function(RBF) with Sigmoid Function(SF) based on the gradient method. (2). comparing RBF with SF based on RasVal. (3). comparing Ras Val with the gradient method for the RBF controller. (4). comparing RasVal with the gradient method for the SF controller. By applying RasVal and the gradient method to a nonlinear crane control system, it has been proved that the simulation results of ULN with RBF based on the gradient method are superior in performance to those of neural networks, and it has also been shown that the RBF control has better performance for the generalization capability than the neural network control based on the gradient method. On the other hand, it has been shown that the neural network control based on RasVal has better performance than the RBF control. At the same time, it has been shown that the RasVal is superior in performance to the commonly used back propagation learning algorithm whether the RBF controller or the NN controller is used. On the other hand, the generalization capability of a Radial Basis Function controller using RasVal was studied and it is shown that a new method is effective to overcome the over-fitting problem in nonlinear control systems.

This paper presents a hybrid quasi-ARMAX modeling and identification scheme for nonlinear systems. It is shown that a general nonlinear ARMAX system can be represented by using a constrained time-variant ARMAX model, whose coefficients are nonlinear functions of input-output variables of system. A hybrid quasi-ARMAX model are then obtained by embedding a group of fuzzy models in the coefficients of the constrained time-variant ARMAX model to describe the time-variant coefficients. One of the distinctive features of the hybrid quasi-ARMAX model is that its parameters to be estimated have useful explicit meanings which can be taken as advantages for setting initial values of parameter estimation and for improving the generalization ability of the model.

This paper proposes a hybrid quasi-ARMAX modeling and identification scheme for nonlinear systems. The idea is to incorporate a group of certain nonlinear nonparametric models (NNMs) into a linear ARMAX structure. Particular effort is made to find a, better compromise to the trade-off between the model flexibility and the simplicity for estimation by using knowledge information efficiently. As the result, we obtain a model equipped with a linear ARMAX structure, flexibility and simplicity The effectiveness and usefulness of the proposed hybrid model are examined by applying it to identification and control of nonlinear systems.