The recursive and hierarchical structure of full rooted trees is applicable to statistical models in various fields, such as data compression, image processing, and machine learning. In most of these cases, the full rooted tree is not a random variable; as such, model selection to avoid overfitting is problematic. One method to solve this problem is to assume a prior distribution on the full rooted trees. This enables the optimal model selection based on Bayes decision theory. For example, by assigning a low prior probability to a complex model, the maximum a posteriori estimator prevents the selection of the complex one. Furthermore, we can average all the models weighted by their posteriors. In this paper, we propose a probability distribution on a set of full rooted trees. Its parametric representation is suitable for calculating the properties of our distribution using recursive functions, such as the mode, expectation, and posterior distribution. Although such distributions have been proposed in previous studies, they are only applicable to specific applications. Therefore, we extract their mathematically essential components and derive new generalized methods to calculate the expectation, posterior distribution, etc.

<title>Abstract</title>Prediction based on a single linear regression model is one of the most common way in various field of studies. It enables us to understand the structure of data, but might not be suitable to express the data whose structure is complex. To express the structure of data more accurately, we make assumption that the data can be divided in clusters, and has a linear regression model in each cluster. In this case, we can assume that each explanatory variable has their own role; explaining the assignment to the clusters, explaining the regression to the target variable, or being both of them. Introducing probabilistic structure to the data generating process, we derive the optimal prediction under Bayes criterion and the algorithm which calculates it sub-optimally with variational inference method. One of the advantages of our algorithm is that it automatically weights the probabilities of being each number of clusters in the process of the algorithm, therefore it solves the concern about selection of the number of clusters. Some experiments are performed on both synthetic and real data to demonstrate the above advantages and to discover some behaviors and tendencies of the algorithm.

This paper deals with a prediction problem of a new targeting variable corresponding to a new explanatory variable given a training dataset. To predict the targeting variable, we consider a model tree, which is used to represent a conditional probabilistic structure of a targeting variable given an explanatory variable, and discuss statistical optimality for prediction based on the Bayes decision theory. The optimal prediction based on the Bayes decision theory is given by weighting all the model trees in the model tree candidate set, where the model tree candidate set is a set of model trees in which the true model tree is assumed to be included. Because the number of all the model trees in the model tree candidate set increases exponentially according to the maximum depth of model trees, the computational complexity of weighting them increases exponentially according to the maximum depth of model trees. To solve this issue, we introduce a notion of meta-tree and propose an algorithm called MTRF (Meta-Tree Random Forest) by using multiple meta-trees. Theoretical and experimental analyses of the MTRF show the superiority of the MTRF to previous decision tree-based algorithms.

We focus on smooth locally decodable codes (SLDC) and Private Information Retrieval (PIR). Recently, a relationship between SLDC and PIR are studied using information theoretical notations. In this paper, we clarify a relationship between SLDCs and PIR using set theoretical notations mainly.

We focus on smooth locally decodable codes (SLDC) and Private Information Retrieval (PIR). Recently, a relationship between SLDC and PIR are studied using information theoretical notations. In this paper, we clarify a relationship between SLDCs and PIR using set theoretical notations mainly.

We focus on smooth locally decodable codes (SLDC) and Private Information Retrieval (PIR). Recently, a relationship between SLDC and PIR are studied using information theoretical notations. In this paper, we clarify a relationship between SLDCs and PIR using set theoretical notations mainly.

A locally recoverable code (LRC code) is a code such that each codeword symbol can be recovered from other codeword symbols. In this research, we propose one possible generalization of previous LRC codes and we derive a bound on the rate of the proposed code.

A new type of spatially coupled low density parity check (SCLDPC) code is proposed. This code has two benefits. (1) This code requires less number of iterations to correct the erasures occurring through the binary erasure channel in the waterfall region than that of the usual SCLDPC code. (2) This code has lower error floor than that of the usual SCLDPC code. Proposed code is constructed as a coupled chain of the underlying LDPC codes whose code lengths exponentially increase as the position where the codes exist is close to the middle of the chain. We call our code spatially "Mt. Fuji" coupled LDPC (SFCLDPC) code because the shape of the graph representing the code lengths of underlying LDPC codes at each position looks like Mt. Fuji. By this structure, when the proposed SFCLDPC code and the original SCLDPC code are constructed with the same code rate and the same code length, L (the number of the underlying LDPC codes) of the proposed SFCLDPC code becomes smaller and M (the code lengths of the underlying LDPC codes) of the proposed SFCLDPC code becomes larger than those of the SCLDPC code. These properties of L and M enables the above reduction of the number of iterations and the bit error rate in the error floor region, which are confirmed by the density evolution and computer simulations.

This letter treats the problem of lossless fixed-to-variable length source coding in moderate deviation regime. We investigate the behavior of the overflow probability of the Bayes code. Our result clarifies that the behavior of the overflow probability of the Bayes code is similar to that of the optimal non-universal code for i.i.d. sources.

This paper investigates the problem of variable-length lossy source coding. We deal with the case where both the excess distortion probability and the overflow probability of codeword length are less than or equal to positive constants. The infimum of the thresholds on the overflow probability is characterized by a smooth max entropy-based quantity. Both non-asymptotic and asymptotic cases are analyzed. To show the achievability results, we do not utilize the random coding argument but give an explicit code construction.

In this research we apply Markov decision processes to asset management with sensor network. We propose a new asset management method which minimizes total cost with reference to a Bayes criterion.

In this research we apply Markov decision processes to recommender system with transitions of user classes. We propose a new recommender method which maximizes total reward with reference to a Bayes criterion.

In this research we apply Markov decision processes to asset management with sensor network. We propose a new asset management method which minimizes total cost with reference to a Bayes criterion.

In this research we apply Markov decision processes to recommender system with transitions of user classes. We propose a new recommender method which maximizes total reward with reference to a Bayes criterion.

We treat lossless fixed-to-variable length source coding under general sources for finite block length setting. We evaluate the threshold of the overflow probability for prefix and non-prefix codes in terms of the smooth max-entropy. We clarify the difference of the thresholds between prefix and non-prefix codes for finite block length. Further, we discuss our results under the asymptotic block length setting.

This letter deals with the Slepian-Wolf coding problem for general sources. The second-order achievable rate region is derived using quantity which is related to the smooth max-entropy and the conditional smooth max-entropy. Moreover, we show the relationship of the functions which characterize the second-order achievable rate region in our study and previous study.

In this study, we develop a new algorithm for decoding binary linear codes for symbol-pair read channels. The symbol-pair read channel was recently introduced by Cassuto and Blaum to model channels with higher write resolutions than read resolutions. The proposed decoding algorithm is based on linear programming (LP). For LDPC codes, the proposed algorithm runs in time polynomial in the codeword length. It is proved that the proposed LP decoder has the maximum-likelihood (ML) certificate property, i.e., the output of the decoder is guaranteed to be the ML codeword when it is integral. We also introduce the fractional pair distance d(fp) of the code, which is a lower bound on the minimum pair distance. It is proved that the proposed LP decoder corrects up to. inverted left perpendiculard(fp)/2inverted right perpendicular - 1 errors.

We treat lossless fixed-to-variable length source coding under general sources for finite block length setting. We evaluate the threshold of the overflow probability for prefix and non-prefix codes in terms of the smooth max-entropy. We clarify the difference of the thresholds between prefix and non-prefix codes for finite block length. Further, we discuss our results under the asymptotic block length setting.

Copyright © 2016 The Institute of Electronics, Information and Communication Engineers.This letter deals with the Slepian-Wolf coding problem for general sources. The second-order achievable rate region is derived using quantity which is related to the smooth max-entropy and the conditional smooth max-entropy. Moreover, we show the relationship of the functions which characterize the second-order achievable rate region in our study and previous study.

Copyright © 2016 The Institute of Electronics, Information and Communication Engineers.In this study, we develop a new algorithm for decoding binary linear codes for symbol-pair read channels. The symbol-pair read channel was recently introduced by Cassuto and Blaum to model channels with higher write resolutions than read resolutions. The proposed decoding algorithm is based on linear programming (LP). For LDPC codes, the proposed algorithm runs in time polynomial in the codeword length. It is proved that the proposed LP decoder has the maximum-likelihood (ML) certificate property, i.e., the output of the decoder is guaranteed to be the ML codeword when it is integral. We also introduce the fractional pair distance df p of the code, which is a lower bound on the minimum pair distance. It is proved that the proposed LP decoder corrects up to df p =2 - 1 errors.

In this paper, we develop a new decoding algorithm of binary linear codes for symbol-pair read channel. The Symbol-pair read channel has recently been introduced by Cassuto and Blaum to model channel whose write resolution is higher than read resolution. The proposed decoding algorithm is based on the linear programming (LP). It is proved that the proposed LP decoder has the maximum-likelihood (ML) certificate property, i.e., the output of the decoder is guaranteed to be the ML codeword when it is integral. We also introduce the fractional pair distance d(fp) of the code which is a lower bound on the minimum pair distance. It is proved that the proposed LP decoder corrects up to [d(fp)/2] - 1 errors.

There are many research on hybrid selective-repeat ARQ method. In many previous research purposes have been to maximize a throughput on the number of messages delivered to the user. Important theoretical limits were studied under the condition that a decoding error was very small and neglected.
In this research we propose the optimal type-I hybrid selective-repeat ARQ with finite receiver buffer. The proposed method maximizes an expected total reward regarding the number of messages delivered to the user correctly based on statistical decision theory.

A new theory known as compressed sensing considers the problem to acquire and recover a sparse signal from its linear measurements. In this paper, we propose a new support recovery algorithm from noisy measurements based on the linear programming (LP). LP is widely used to estimate sparse signals, however, we focus on the problem to recover the support of sparse signals rather than the problem to estimate sparse signals themselves. First, we derive an integer linear programming (ILP) formulation for the support recovery problem. Then we obtain the LP based support recovery algorithm by relaxing the ILP. The proposed LP based recovery algorithm has an attracting property that the output of the algorithm is guaranteed to be the maximum a posteiori (MAP) estimate when it is integer valued. We compare the performance of the proposed algorithm to a state-of-the-art algorithm named sparse matching pursuit (SMP) via numerical simulations.

Recently, studies of linear network coding have attracted attention. This paper considers a probabilistic error model for the deterministic linear network coding. In the previous studies of error-correcting codes, Kaneko et al. proposed a decoding algorithms for probabilistic error model. This is a maximum likelihood decoding algorithm that uses the optimization method called branch and bound method. This paper constructs a new model of deterministic linear network coding and proposes a maximum likelihood decoding algorithm that uses the branch and bound method.

Cooperative regenerating codes are a class of codes that enable a data collector to reconstruct the original data by connecting to a subset of storage nodes, and also can repair multiple failed nodes by downloading data from the surviving nodes and exchanging data among the new nodes. In cooperative regenerating codes, there exists a tradeoff between the storage size of each node and repair-bandwidth. In this study, we propose new classes of cooperative regenerating codes based on correlation of information stored on nodes. We also consider relationships between each class of cooperative regenerating codes and storage size and repair-bandwidth.

Linear Unequal Error Protection (UEP) codes were proposed by Masnick et al. and largely developed by Dunning et al. and Gils. On the other hand, random linear network coding is recently studied by many researchers. Kotter et al. proposed a new coding technique named subspace codes and applied it to random linear network coding. Moreover. Silva et al. proposed constructions of subspace codes using rank metric codes. In this paper, we propose subspace UEP codes and apply them to the random linear network coding. Then, we propose rank metric UEP codes and constructions of the subspace UEP codes using the rank metric UEP codes.

As a distributed storage system with regenerating function of failed nodes, regenerating codes have been studied recently. In a [n, k, d]-regenerating codes framework, a message can be reconstructed from any subsets of k nodes out of n nodes and any failed nodes can be repaired from any subsets of d nodes out of (n -1) nodes. The conditions of the reconstruction and regeneration can be generalized. In this paper, we propose the regenerating codes with generalized conditions and derive their optimal constructing algorithm under the multiple assignment map construction framework. The optimal codes of the framework are obtained by solving integer programming problems in a similar way of generalized secret sharing scheme's construction based on a multiple assignment map.

This paper considers universal lossless variable-length source coding problem and investigates the Bayes code from viewpoints of the distribution of its codeword lengths. First, we show that the codeword lengths of the Bayes code satisfy the asymptotic normality. This study can be seen as the investigation on the asymptotic shape of the distribution of codeword lengths. Second, we show that the codeword lengths of the Bayes code satisfy the law of the iterated logarithm. This study can be seen as the investigation on the asymptotic end points of the distribution of codeword lengths. Moreover, the overflow probability, which represents the bottom of the distribution of codeword lengths, is studied for the Bayes code. We derive upper and lower bounds of the infimum of a threshold on the overflow probability under the condition that the overflow probability does not exceed epsilon is an element of(0, 1). We also analyze the necessary and sufficient condition on a threshold for the overflow probability of the Bayes code to approach zero asymptotically.

This paper considers universal lossless variablelength source coding problem and deals with one of the fundamental limits and pointwise asymptotics of the Bayes code for stationary ergodic finite order Markov sources. As investigation of the fundamental limits, we show upper and lower bounds of the minimum rate such that the probability which exceeds it is less than epsilon is an element of (0, 1). Furthermore, we prove that the codeword length of the Bayes code satisfies the asymptotic normality (pointwise root n asymptotics) and the law of the iterated logarithm (pointwise root n log log n asymptotics), where n represents length of a source sequence and "log" is the natural logarithm.

This paper considers universal lossless variable-length source coding problem and investigates the Bayes code from viewpoints of the distribution of its codeword lengths. First, we show that the codeword lengths of the Bayes code satisfy the asymptotic normality. This study can be seen as the investigation on the asymptotic shape of the distribution of codeword lengths. Second, we show that the codeword lengths of the Bayes code satisfy the law of the iterated logarithm. This study can be seen as the investigation on the asymptotic end points of the distribution of codeword lengths. Moreover, the overflow probability, which represents the bottom of the distribution of codeword lengths, is studied for the Bayes code. We derive upper and lower bounds of the infimum of a threshold on the overflow probability under the condition that the overflow probability does not exceed ε∈(0,1). We also analyze the necessary and sufficient condition on a threshold for the overflow probability of the Bayes code to approach zero asymptotically.

In this paper, sequential prediction is studied. The typical assumptions about the probabilistic model in sequential prediction are following two cases. One is the case that a certain probabilistic model is given and the parameters are unknown. The other is the case that not a certain probabilistic model but a class of probabilistic models is given and the parameters are unknown. If there exist some parameters and some models such that the distributions that are identified by them equal the source distribution, an assumed model or a class of models can represent the source distribution. This case is called that specifiable condition is satisfied. In this study, the decision based on the Bayesian principle is made for a class of probabilistic models (not for a certain probabilistic model). The case that specifiable condition is not satisfied is studied. Then, the asymptotic behaviors of the cumulative logarithmic loss for individual sequence in the sense of almost sure convergence and the expected loss, i.e. redundancy are analyzed and the constant terms of the asymptotic equations are identified.

AdaBoost is an algorithm used for pattern recognition. This algorithm successively makes the model which minimizes an error for sample data. However, to consider prediction, there is no guarantee that such model is optimum. Regarding this issue, there are many studies about regularization method and variable selection method. Forward selection method is one of the methods of variable selection method and Stagewise is one of the methods of forward selection method. Each iteration, Stagewise selects one variable. Stagewise updates parameters slightly increase and not minimizes an error for sample data. It is known that Stagewise improves the rate of a prediction error. In this research, we apply the idea of Stagewise for AdaBoost and consider improvement the rate of a prediction error.

In this paper, we consider a completion problem of multiple related matrices. Matrix completion problem is the problem to estimate unobserved elements of the matrix from observed elements. It has many applications such as collaborative filtering, computer vision, biology, and so on. In cases where we can obtain some related matrices, we can expect that their simultaneous completion has better performance than completing each matrix independently. Collective matrix factorization is a powerful approach to jointly factorize multiple matrices. However, existing completion algorithms for the collective matrix factorization have some drawbacks. One is that most existing algorithms are based on non-convex formulations of the problem. Another is that only a few existing algorithms consider the strength of the relation among matrices and it results in worse performance when some matrices are actually not related. In this paper, we formulate the multiple matrix completion problem as the convex optimization problem. Moreover, it considers the strength of the relation among matrices. We also develop an optimization algorithm which solves the proposed problem efficiently based on the alternating direction method of multipliers (ADMM). We verify the effectiveness of our approach through numerical experiments on both synthetic data and real data set: MovieLens.

The objective of this research is to evaluate the epsilon-minimum overflow threshold of the Bayes codes for a Markov source. In the lossless variable-length source coding problem, typical criteria are the mean codeword length and the overflow probability. The overflow probability is the probability with which a codeword length per source symbol exceeds a threshold and the epsilon-minimum overflow threshold is defined. In the non-universal setting, the Shannon code is optimal under the mean codeword length and the epsilon-minimum overflow threshold for the Shannon code is derived for an i.i.d. source. On the other hand, in the universal setting, the Bayes code is one of universal codes which minimize the mean codeword length under the Bayes criterion. However, few studies have been done on the overflow probability for the Bayes codes. In this paper, we assume a stationary ergodic finite order Markov source and derive the upper and lower bounds of the epsilon-minimum overflow threshold of the Bayes codes.

In this paper, we study a privacy preserving linear regression analysis. We propose a new protocol of a distributed calculation method that calculates a least squares estimator, in the case that two parties have different types of explanatory variables. We show the security of privacy in the proposed protocol. Because the protocol have iterative calculations, we evaluate the number of iterations via numerical experiments. Finally, we show an extended protocol that is a distributed calculation method for k parties.

In previous research a role-playing game(RPG) is represented with Markov decision processes(MDP). But active learning method for RPG has not been studied yet. In this research we propose an active learning method which maximizes an expected total reward with respect to a Bayes criterion under the condition that the true parameter of MDP is unknown. We recognize the effectiveness of our proposed method by some simulations.

In this paper, we develop an iterative multiuser joint decoding of code-division multiple-access (CDMA) signals based on a distributed optimization algorithm. For the joint decoding problem, decoding algorithm based on the turbo principle is widely used. The algorithm consists of soft-input soft-output (SISO) channel decoder and SISO multiuser detector and it can be derived as an application of the sum-product algorithm. On the other hand, in the research area of error correcting codes, the decoding algorithm based on convex optimization has been attracting a great deal of attention. Decoding algorithm based on linear programming (LP) has decoding error rate which is comparable with sum-product algorithm with stronger theoretical guarantees. We formulate the joint decoding problem of CDMA signals as an convex optimization problem and we present a relax form of the problem. Moreover, we propose a distributed algorithm which efficiently solves the relaxed optimization problem. The proposed algorithm is based on alternating direction method of multipliers (ADMM). We also see the performance of the proposed decoder through numerical simulations. © 2013 IEEE.

Predictive word is an input technology showing candidate words which a system predict by user partial input. We treat predictive methods using an N-gram model. The model is generally produced by analyzing train data. The data is more sparse in proportion to an N-gram order, because of enormous combinations of words in the sequences. An issue of producing the model is how to combine a lower order model into a higher order one. Many researchers proposed models composed of weighed each-order one, such as a mixture distribution or an interpolation created by discount parameters considering about extremely lower frequent sequence. But these methods have no theoretical guarantee about prediction errors. In this paper, we treat the issue as a statistical problem that the model order is unknown, and discuss prediction errors from a point of view about Bayesian decision theory. We present that an optimal prediction method with reference to the Bayes criterion for minimizing the errors. Experimental results using Japanese documents show that our method performs good predictive words.

Experiments usually aim to study how changes in various factors affect the response variable of interest. Since the response model used most often at present in experimental design is expressed through the effect of each factor, it is straightforward to ascertain how each factor affects the response variable. However, since the response model contains redundant parameters, in the analysis of variance we must calculate the degrees of freedom defined by the number of independent parameters. In this letter, we propose the idea of calculating the degrees of freedom over the model based on an orthonormal system for the first time. In this way, we can easily obtain the number of independent parameters associated with any component, which reduces the risk of mistakes in the calculation of the number of independent parameters and facilitates the implementation of estimation procedures. Copyright © 2013 The Institute of Electronics, Information and Communication Engineers.

The document classification problem has been investigated by various techniques, such as a vector space model, a support vector machine, a random forest, and so on. On the other hand, J.Ziv et al. have proposed a document classification method using Ziv-Lempel algorithm to compress the data. Furthermore, the Context-Tree Weighting (CTW) algorithm has been proposed as an outstanding data compression, and for the document classification using the CTW algorithm experimental results have been reported. In this paper, we assume that each document with same category arises from Markov model with same parameters for the document classification. Then we propose an analysis method to estimate a classification error probability for the document with the finite length.

We propose new algorithms for fault diagnosis problem based on the dual decomposition method and the augmented Lagrangian method. Our algorithms are convergent and those outputs are same as that of Linear Programming (LP) based fault diagnosis algorithm. The proposed algorithms have smaller computational complexity than ordinary LP solver. Experimental results show the practical potentials of the proposed algorithms.

Lossless variable-length source coding with unequal cost function is considered for general sources. In this problem, the codeword cost instead of codeword length is important. The infimum of average codeword cost has already been determined for general sources. We consider the overflow probability of codeword cost and determine the infimum of achievable overflow threshold. Our analysis is on the basis of information-spectrum methods and hence valid through the general source.

We analyze the asymptotic properties of the cumulative logarithmic loss in the decision problem based on the Bayesian principle and explicitly identify the constant terms of the asymptotic equations as in the case of previous studies by Clarke and Barron and Gotoh et al. We assume that the set of models is given that identify a class of parameterized distributions, it has a nested structure and the source distribution is not contained in all the families of parameterized distributions that are identified by each model. The cumulative logarithmic loss is the sum of the logarithmic loss functions for each time decision-, e. g., the redundancy in the universal noiseless source coding.

A stream cipher is an important class of encryption algorithms. Its safety depends on the structure of the pseudo-random number generator used. There are various types of pseudo-random number generators in existence, and attack algorithms used on them have been studied individually. In this paper, we express the problem of attacks on a general stream cipher as a probabilistic inference problem, and formulate the optimal key estimation. We also propose a unified framework of attack algorithms that can be applied to a wide variety of stream ciphers. The optimal key estimation, however, has computational complexity. To reduce the complexity, an approximation algorithm based on a probabilistic inference is proposed. We also describe some attack algorithms used on practical pseudo-random number generators. Finally, the proposed algorithm is evaluated by through a computer simulation.

In this paper, we propose a general markov decision process model for the recommendation system. Furthermore, by using historical data, we derive the optimal recommendation lists from the proposed model based on bayesian decision theory. In the recommendation research area, there were little studies that considered both the purchased items and the past recommended items within a given period. In these circumstances, markov decision process based recommend method that can take these two things into account has been proposed. Our method also uses both things as with the previous method. Here, the unique thing about this paper is not only that we generalize the existing model, but also that we formulate the process to get the recommendation lists as the statistical decision problem. As a result, we can obtain the most suitable recommendation lists with respect to the purpose of the recommendation for a wide variety of recommendation scene. By using artificial data, we show the experimental results that our method can obtain more rewards than the conventional method gets.

It has recently been shown that the model in experimental design can be expressed in terms of an orthonormal system. In this case, the model is expressed by using Fourier coefficients instead of the effect of each factor. As there is an abundance of software for calculating the Fourier transform, such a system allows for a straightforward implementation of the procedures for estimating the Fourier coefficients by using Fourier transform. However, Fourier coefficients themselves do not provide a direct representation of the effect of each factor, and the relation between the Fourier coefficients and the effect of each factor has not yet been clarified. In this paper, we present theorems of the relation between the Fourier coefficients and the effect of each factor. By using these theorems, the effect of each factor can be easily obtained from the computed Fourier coefficients. Therefore, with the aid of an orthonormal system, it is possible to easily implement the estimation procedures as well as to understand how each factor affects the response variable in the model.

Source coding theorem reveals the minimum achievable code length under the condition that the error probability is smaller than or equal to some small constant. In the single user communication system, the source coding theorem was proved for general sources. The class of general source is quite large and it is important result since the result can be applied for a wide class of sources. On the other hand there are several studies to evaluate the achievable code length more precisely for the restricted class of sources by using the restriction. In the multi-user communication system, although the source coding theorem was proved for general correlated sources, there is no study to evaluate the achievable code length more precisely. In this study, we consider the stationary memoryless correlated sources and show the coding theorem for Slepian-Wolf type problem more precisely than the previous result.

The overflow probability is one of criteria that evaluate the performance of fixed-to-variable length (FV) codes. In the single source coding problem, there were many researches on the overflow probability. Recently, the source coding problem for correlated sources, such as Slepian-Wolf coding problem or source coding problem with side information, is one of main topics in information theory. In this paper, we consider the source coding problem with side information. In particular, we consider the FV code in the case that the encoder and the decoder can see side information. In this case, several codes were proposed and their mean code lengths were analyzed. However, there was no research about the overflow probability. We shall show two lemmas about the overflow probability. Then we obtain the condition that there exists a FV code under the condition that the overflow probability is smaller than or equal to some constant.

In this paper, we develop linear-programming (LP) decoding for multiple-access channels with binary linear codes. For single-user channels, LP decoding has attracted much attention in recent years as a good approximation to maximum-likelihood (ML) decoding. We demonstrate how the ML decoding problem for multiple-access channels with binary linear codes can be formulated as an LP problem. This is not straightforward, because the objective function of the problem is generally a non-linear function of the codeword symbols. We introduce auxiliary variables such that the objective function is a linear function of these variables. The ML decoding problem then reduces to the LP problem. As in the case for single-user channels, we formulate the relaxed LP problem to reduce the complexity for practical implementation, and as a result propose a decoder that has the desirable property known as the ML certificate property (i.e., if the decoder outputs an integer solution, the solution is guaranteed to be the ML codeword). Although the computational complexity of the proposed algorithm is exponential in the number of users, we can reduce this complexity for Gaussian multiple-access channels. Furthermore, we compare the performance of the proposed decoder with a decoder based on the sum-product algorithm.

Allocation methods for Cartesian product files on multiple disks using linear error-correcting codes are discussed. In this paper, we propose an allocation method using unequal error protection (UEP) codes. Codewords of an UEP code have some special bits which are protected against a greater number of errors than the other bits. We firstly assume a model that "*", which means "don't care", appears with different probability in each attribute of queries. In this case, the average access time can be calculated using the split distance distribution. Finally, we illustrate the average access time of the methods using UEP codes.

F.P.Preparata et al. have proposed a fault diagnosis model to find all faulty units in the multicomputer system by using outcomes which each unit tests some other units. In this paper, for probabilistic diagnosis models, we show an efficient diagnosis algorithm to obtain a posteriori probability that each of units is faulty given the test outcomes. Furthermore, we propose a method to analyze the diagnostic error probability of this algorithm.

Experiments usually aim to study how changes in various factors affect the response variable of interest. Since the response model used most often at present in experimental design is expressed through the effect of each factor, it is straightforward to ascertain how each factor affects the response variable. However, since the response model contains redundant parameters, we must calculate the degrees of freedom defined by the number of independent parameters in the analysis of variance. In this paper, we show that through a description of experimental design based on an orthonormal system, the response model can be expressed using only independent parameters. Hence, we do not have to calculate the degrees of freedom defined by the number of independent parameters.

We discuss disk allocation methods for Cartesian product files by introducing error correcting codes, and have clarified the performance of the methods by system evaluation models developed by using rate distortion theory. Let us assume q(n) Cartesian product files with n attributes and q actual values in each attribute, and store q(n) files into G(&lt;= q(n)) disks. For a partial match access request, we represent new disk allocation methods which able to access the disks in parallel as much as possible, where the partial match access request includes an indefinite case (don't care: "*") in some attributes and the * requires to access the files with corresponding to the attribute for the all actual attribute values. In this paper, we propose to apply unequal error protection codes to the case where the probabilities of occurrence of the * in the attributes for a partial match access request are not the same. We show the disk allocation methods have desirable properties as n becomes large.

In this chapter, we propose that the model of experimental design be expressed as an orthonormal system, and show that the model contains no redundant parameters. Then, the model is expressed by using Fourier coefficients instead of the effect of each factor. As there is an abundance of software for calculating the Fourier transform, such a system allows for a straightforward implementation of the procedures for estimating the Fourier coefficients by using Fourier transform. In addition, the effect of each factor can be easily obtained from the Fourier coefficients. Therefore, it is possible to implement easily the estimation procedures as well as to understand how each factor affects the response variable in a model based on an orthonormal system. Moreover, the analysis of variance can also be performed in a model based on an orthonormal system. Hence, it is clear that two main procedures in the experimental design, that is, the estimation of the effects and the analysis of variance can be executed in a description of experimental design on the basis of an orthonormal system.

In this paper, we develop linear-programming (LP) decoding for multiple-access channels with binary linear codes. For single-user channels, LP decoding has attracted much attention in recent years as a good approximation to maximum-likelihood (ML) decoding. We demonstrate how the ML decoding problem for multiple-access channels with binary linear codes can be formulated as an LP problem. This is not straightforward, because the objective function of the problem is generally a nonlinear function of the codeword symbols. We introduce auxiliary variables such that the objective function is a linear function of these variables. The ML decoding problem then reduces to the LP problem. As in the case for single-user channels, we formulate the relaxed LP problem to reduce the complexity for practical implementation, and as a result propose a decoder that has the desirable property known as the ML certificate property (i.e., if the decoder outputs an integer solution, the solution is guaranteed to be the ML codeword). Although the computational complexity of the proposed algorithm is exponential in the number of users, we can reduce this complexity for Gaussian multiple-access channels. Furthermore, we compare the performance of the proposed decoder with a decoder based on the sum-product algorithm. Copyright © 2011 The Institute of Electronics, Information and Communication Engineers.

Document classification is one of important topics in the field of NLP (Natural Language Processing). In the previous research a document classification method has been proposed which minimizes an error rate with reference to a Bayes criterion. But when the number of documents in training data is small, the accuracy of the previous method is low. So in this research we use estimating data in order to estimate prior distributions. When the training data is small the accuracy using estimating data is higher than the accuracy of the previous method. But when the training data is big the accuracy using estimating data is lower than the accuracy of the previous method. So in this research we also propose another technique whose accuracy is higher than the accuracy of the previous method when the training data is small, and is almost the same as the accuracy of the previous method when the training data is big. © 2011 The Institute of Electrical Engineers of Japan.

Maximum likelihood (ML) decoding of linear block codes can be considered as an integer linear programming (ILP) Since it is an NP hard problem in general there are many researches about the algorithms to approximately solve the problem One of the most popular algorithms is linear programming (LP) decoding proposed by Feldman et al LP decoding is based on the LP relaxation which Is a method to approximately solve the ILP corresponding to the ML decoding problem Advanced algorithms for solving ILP (approximately or exactly) include cutting plane method and branch and bound method As applications of these methods adaptive LP decoding and branch and bound decoding have been proposed by Taghavi et al and Yang et al respectively Another method for solving ILP is the branch and cut method which is a hybrid of cutting plane and branch and bound methods The branch and cut method is widely used to solve ILP however it is unobvious that the method works well for the ML decoding problem In this paper we show that the branch and cut method is certainly effective for the ML decoding problem Furthermore the branch and cut method consists of some technical components and the performance of the algorithm depends on the selection of these components It is important to consider how to select the technical components in the branch and cut method We see the differences caused by the selection of those technical components and consider which scheme is most effective for the ML decoding problem through numerical simulations

We propose that the model in experimental design be ex pressed in terms of an orthonormal system Then we can easily estimate the effects using Fourier transforms We also provide the theorems with respect to the sum of squares needed in analysis of variance Using these theorems It is clear that we can execute the analysis of variance in this model

In digital signal processing, the sampling theorem states that any real valued function f can be reconstructed from a sequence of values of f that are discretely sampled with a frequency at least twice as high as the maximum frequency of the spectrum of f. This theorem can also be applied to functions over finite domain. Then, the range of frequencies of f can be expressed in more detail by using a bounded set instead of the maximum frequency. A function whose range of frequencies is confined to a bounded set is referred to as bandlimited function. And a sampling theorem for bandlimited functions over Boolean domain has been obtained. Here, it is important to obtain a sampling theorem for bandlimited functions not only over Boolean domain (GF(2)(n) domain) but also over GF(q)(n) domain, where q is a prime power and GF(q) is Galois field of order q. For example, in experimental designs, although the model can be expressed as a linear combination of the Fourier basis functions and the levels of each factor can be represented by GF(q), the number of levels often take a value greater than two. However, the sampling theorem for bandlimited functions over GF(q)(n) domain has not been obtained. On the other hand, the sampling points are closely related to the codewords of a linear code. However, the relation between the parity check matrix of a linear code and any distinct error vectors has not been obtained, although it is necessary for understanding the meaning of the sampling theorem for bandlimited functions. En this paper, we generalize the sampling theorem for bandlimited functions over Boolean domain to a sampling theorem for bandlimited functions over GF(q)(n) domain. We also present a theorem for the relation between the parity check matrix of a linear code and any distinct error vectors. Lastly, we clarify the relation between the sampling theorem for functions over GF(q)(n) domain and linear codes.

In this paper, we consider density evolution analyses of low-density parity-check (LDPC) codes for a finite-state Markov channel (FSMC). Since operations in the subgraph corresponding to the estimation process do not satisfy symmetry conditions, all densities involved in each value of the channel state should be kept track. In [6], to avoid the complexity to compute plural pdfs, only one pdf involved in the channel state is computed exploiting marginalization operations. We suppose that this approach is not accurate enough to track the estimation process of the joint estimation-decoding. In this paper, we present a procedure of the density evolution analysis of LDPC codes for an FSMC of which all densities involved in each value of the channel state are kept track. © 2010 Taylor &amp
Francis Group, LLC.

In coding theory, it is important to calculate an upper bound for the size of codes given the length and minimum distance. The Linear Programming (LP) bound is known as a good upper bound for the size of codes. On the other hand, Unequal Error Protection (UEP) codes have been studied in coding theory. In UEP codes, a codeword has special bits which are protected against a greater number of errors than other bits. In this paper, we propose a LP bound for UEP codes. Firstly, we generalize the distance distribution (or weight distribution) of codes. Under the generalization, we lead to the LP bound for UEP codes. And we show a numerical example of the LP bound for UEP codes. Lastly, we compare the proposed bound with a modified Hamming bound.

We consider the source coding problem with side information. Especially, we consider the FV code in the case that the encoder and the decoder can see side information. We obtain the condition that there exists a FV code under the condition that the overflow probability is smaller than or equal to some constant.

Lossless fixed-to-variable(FV) length codes are considered. The overflow probability is one of criteria that evaluate the performance of FV code. In the single source coding problem, there were many researches on the overflow probability. Recently, the source coding problem for correlated sources, such as Slepian-Wolf coding problem or source coding problem with side information, is one of main topics in information theory. In this paper, we consider the source coding problem with side information. Especially, we consider the FV code in the case that the encoder and the decoder can see side information. In this case, several codes were proposed and their mean code lengths were analyzed. However, there was no research about the overflow probability. We shall show two lemmas about the overflow probability. Then we obtain the condition that there exists a FV code under the condition that the overflow probability is smaller than or equal to some constant.

In digital signal processing, the sampling theorem states that any real valued function f can be reconstructed from a sequence of values of f that are discretely sampled with a frequency at least twice as high as the maximum frequency of the spectrum of f . This theorem can also be applied to functions over finite domain. Then, the range of frequencies of f can be expressed in more detail by using a bounded set instead of the maximum frequency. A function whose range of frequencies is confined to a bounded set is referred to as bandlimited function. And a sampling theorem for bandlimited functions over Boolean domain has been obtained. Here, it is important to obtain a sampling theorem for bandlimited functions not only over Boolean domain (GF(2) n domain) but also over GF(q)n domain, where q is a prime power and GF(q) is Galois field of order q. For example, in experimental designs, although the model can be expressed as a linear combination of the Fourier basis functions and the levels of each factor can be represented by GF(q), the number of levels often take a value greater than two. However, the sampling theorem for bandlimited functions over GF(q)n domain has not been obtained. On the other hand, the sampling points are closely related to the codewords of a linear code. However, the relation between the parity check matrix of a linear code and any distinct error vectors has not been obtained, although it is necessary for understanding the meaning of the sampling theorem for bandlimited functions. In this paper, we generalize the sampling theorem for bandlimited functions over Boolean domain to a sampling theorem for bandlimited functions over GF(q)n domain. We also present a theorem for the relation between the parity check matrix of a linear code and any distinct error vectors. Lastly, we clarify the relation between the sampling theorem for functions over GF(q)n domain and linear codes. © 2010 The Institute of Electronics, Information and Communication Engineers.

In the field of experimental design, it is important to construct orthogonal designs. In this paper, we propose a new algorithm to construct orthogonal design. This algorithm uses Ukita’s algorithm, which is essentially based on projective geometries, and uses orthogonal designs constructed by error-correcting codes. We show some numerical examples of the proposed algorithm, and show that the proposed algorithm can construct good orthogonal designs with low complexity even if there are high order effects. © 2010 Taylor &amp
Francis Group, LLC.

The universal lossless source coding problem is one of the most important problem in communication systems. The aim of source coding is to compress data to reduce costs in digital communication. Traditional universal source coding schemes are usually designed for stationary sources. Recently, some universal codes for nonstationary sources have been proposed. Independent piecewise identically distributed (i.p.i.d.) sources are simple nonstationary sources that parameter changes discontinuously. In this paper, we assume new i.p.i.d. sources class, and we prove that Bayes codes minimize the mean redundancy when parameter transition pattern is known and parameter is unknown. © 2010 Taylor &amp
Francis Group, LLC.

The joint source-channel coding problem is considered. The joint source-channel coding theorem reveals the existence of a code for the pair of the source and the channel under the condition that the error probability is smaller than or equal to epsilon asymptotically. The separation theorem guarantees that we can achieve the optimal coding performance by using the two-stage coding. In the case that epsilon = 0, Han showed the joint source-channel coding theorem and the separation theorem for general sources and channels. Furthermore the epsilon-coding theorem (0 &lt;= epsilon &lt; 1) in the similar setting was studied. However, the separation theorem was not revealed since it is difficult in general. So we consider the separation theorem in this setting.

This paper describes an extension of XEX* mode, which is a method to convert a block cipher into a tagged tweakable block cipher, a notion introduced by Rogaway in 2004 as an extension of the tweakable block cipher by Liskov et al. Our extension attaches an additional encryption function to the original XEX*, which has some limitation but is slightly faster than the encryption implemented by XEX*. We prove our scheme's security in a general form, where the offset function, a key component of our construction, is not restricted to the one used by XEX*. We also provide some applications of our result, in particular to OCB 2.0, an authenticated encryption based on XEX*.

Code construction for digital fingerprinting, which is a copyright protection technique for multimedia, is considered. Digital fingerprinting should deter collusion attacks, where several fingerprinted copies of the same content are mixed to disturb their fingerprints. In this paper, we consider the averaging attack, which is known to be effective for multimedia fingerprinting with the spread spectrum technique. We propose new methods tor constructing fingerprinting codes to increase the coding rate of conventional fingerprinting codes, while they guarantee to identify the same number of colluders. Due to the new fingerprinting codes, the system can deal with a larger number of users to supply digital contents.

The context tree models are widely used in a lot of research fields. Patricia[7] like trees are applied to the context trees that are expanded according to the increase of the length of a source sequence in the previous researches of non-predictive source coding and model selection. The space complexity of the Patricia like context trees are O(t) where t is the length of a source sequence. On the other hand, the predictive Bayes source coding algorithm cannot use a Patricia like context tree, because it is difficult to hold and update the posterior probability parameters on a Patricia like tree. So the space complexity of the expanded trees in the predictive Bayes coding algorithm is O(t(2)). In this paper, we propose an efficient predictive Bayes coding algorithm using a new representation of the posterior probability parameters and the compact context tree holding the parameters whose space complexity is O(t).

The context tree models are widely used in a lot of research fields. Patricia[7] like trees are applied to the context trees that are expanded according to the increase of the length of a source sequence in the previous researches of non-predictive source coding and model selection. The space complexity of the Patricia like context trees are O(t) where t is the length of a source sequence. On the other hand, the predictive Bayes source coding algorithm cannot use a Patricia like context tree, because it is difficult to hold and update the posterior probability parameters on a Patricia like tree. So the space complexity of the expanded trees in the predictive Bayes coding algorithm is O(t(2)). In this paper, we propose an efficient predictive Bayes coding algorithm using a new representation of the posterior probability parameters and the compact context tree holding the parameters whose space complexity is O(t).

In this paper, we generalize the sampling theorem for bandlimited functions over the Boolean domain to a sampling theorem for bandlimited functions over a GF(q)(n) domain. We also present a theorem for the relation between the parity check matrix of a linear code and any distinct error vectors. Lastly, we clarify the relation between the sampling theorem for functions over a GF(q)(n) domain and linear codes.

In this paper. we analyze the robustness for low-density parity-check (LDPC) codes over the Gilbert-Elliott (GE) channel. For this purpose we propose a density evolution method for the case where LDPC decoder uses the mismatched parameters for the GE channel. Using this method, we derive the region of tuples of true parameters and mismatched decoding parameters for the GE channel. where the decoding error probability approaches asymptotically to zero.

A new ensemble of low-density parity-check (LDPC) codes for correcting a solid burst erasure is proposed. This ensemble is an instance of a combined matrix ensemble obtained by concatenating some LDPC matrices. We derive a new bound on the critical minimum span ratio of stopping sets for the proposed code ensemble by modifying the bound for ordinary code ensemble. By calculating this bound, we show that the critical minimum span ratio of stopping sets for the proposed code ensemble is better than that of the conventional one with keeping the same critical exponent of stopping ratio for both ensemble. Furthermore from experimental results, we show that the average minimum span of stopping sets for a solid burst erasure of the proposed codes is larger than that of the conventional ones.

This paper introduces two channel models of correlated parallel channels. Then we analyze structure of error correcting codes over these correlated parallel channels. We derive necessary and sufficient conditions for these codes and some code construction is presented. We also show some upper and lower bounds on the coding rate of the error correcting codes for correlated parallel channels. The introduced channel models are related to burst error channels and the codes analyzed in this paper can be used as asymmetric interleaving codes for burst error channels.

In this paper, we investigate the design of low-density parity-check (LDPC) codes for the Gilbert-Elliott (GE) channel. Recently, Eckford et al. proposed a design method of irregular LDPC codes using approximate density-evolution (DE) for Markov channels [7]. In the design method proposed by Eckford et al., the probability density function (PDF) of the messages from variable nodes to check nodes is approximated by the Gaussian distribution. In this paper, we first show the method to obtain the accurate PDF of the messages from variable nodes to check nodes by utilizing two DE steps for the Gaussian distribution. We call this method the iterative density approximation (IDA). Using this method, we can design the good LDPC codes. Next, we propose an efficient design method of irregular LDPC codes by using Beta approximation to the PDF of the channel state probability for the GE channel. Consequently, we show that the complexity to calculate PDFs of the channel messages is considerably reduced though the rates of LDPC codes obtained by using the proposed approximation are almost the same as that of the IDA method.

In this paper, we consider a, ramp scheme for Key Pre-distribution System (KPS). In the ramp scheme, it can be regarded ass one kind of generalization of KPS, and the required resources can be reduced at the cost of a security degradation which depends on the size of users. We define a ramp scheme for KPS, show lower bound on the amount of user's information needed to generate a common key, and design a protocol that realize a ramp scheme for KPS.

In this study, we consider a joint source-channel coding problem. The joint source-channel coding theorem reveals the existence of a code for the pair of the source V and the channel W under the condition that the probability of error is smaller than or equal to E asymptotically. We show the separation theorem for general sources and general channels.

This paper presents MACs that combine a block cipher and its component such as a reduced-round version. Our MACs are faster than the standard MAC modes such as CBC-MAC, and provably secure if the block cipher is pseudorandom and its component is a permutation with a small differential probability. Such a MAC scheme was recently proposed by one of authors, and we provide improvements about security and treading-off between speed and amount of preprocessing.

Reliability-based hybrid ARQ (RB-HARQ) is a kind of incremental-redundancy ARQ recently introduced. In the RB-HARQ, the receiver returns both NAK signal and set of unreliable bit indices if the received sequence is not accepted. Since each unreliable bit index is determined by the bitwise posterior probability, better approximation of that probability becomes crucial as the number of retransmissions increases. Assuming the systematic code for the initial transmission, the proposed RB-HARQ scheme has the following two features: (a) the sender retransmits newly encoded and interleaved parity bits corresponding to the unreliable information bits; (b) the retransmitted parity bits as well as the initial received sequence are put all together to perform the message passing decoding i.e. the suboptimal MAP decoding. Finally, simulation results are shown to evaluate the above two features.

Digital fingerprinting, a copyright protection technique for digital contents, is considered. Digital fingerprinting should deter collusion attacks, where several fingerprinted copies of the same content are mixed to disturb their fingerprints. In this paper, we consider the averaging attack, which has effect for multimedia fingerprinting. We propose new collusion-secure fingerprinting codes based on finite geometries (FGs) which increase the rate of conventional collusion-secure codes, while they guarantee to identify the same number of colluders. Due to the new FG-based fingerprinting codes, the system can deal with a larger number of users to distribute a digital content.

Word segmentation is the most fundamental and important process for Japanese or Chinese language processing. Because there is no separation between words in these languages, we firstly have to separate the sequence into words. On this problem, it is known that the approach by probabilistic language model is highly efficient, and this is shown practically. On the other hand, recently, a word-valued source has been proposed as a new class of source model for the source coding problem. This model can be supposed to reflect more of the probability structure of natural languages. We may regard Japanese sentence or Chinese sentence as the sequence emitting from a non-prefix-free WVS. In this paper as the first phase of applying WVS to natural language processing, we formulate a word segmentation problem for the sequence from non-prefix-free WVS. Then, we examine the performance of word segmentation for the models by numerical computations.

Morphological analysis is one of important topics in the field of NLP(Natural Language Processing). In many previous research a HMM(Hidden Markov Model) with unknown parameters has been used as a language model. In this research we also use the HMM as the language model. And we assume that sate transitions in the HMM are dominated by a second order Markov chain. At first we propose two types of morphological analysis methods which minimize the error rate with reference to a Bayes criterion. But the computational complexity of the proposed Bayes optimal morphological analysis methods are exponential order. So we also propose approximate methods.

We study a tweakable blockcipher for arbitrarily long message (also called a tweakable enciphering scheme) that consists of a universal hash function and an expansion, a keyed function with short input and long output. Such schemes, called HCTR and HCH, have been recently proposed. They used (a variant of) the counter mode of a blockcipher for the expansion. We provide a security proof of a structure that underlies HCTR and HCH. We prove that the expansion can be instantiated with any function secure against Known-plaintext attacks (KPAs), which is called a weak pseudorandom function (WPRF). As an application of our proof, we provide efficient blockcipher-based schemes comparable to HCH and HCTR. For the double-block-length case, our result is an interesting extension of previous attempts to build a double-block-length cryptographic permutation using WPRF.

The CTW(Context Tree Weighting) algorithm is an efficient universal coding algorithm on context tree models. The CTW algorithm has been interpreted as the non-predictive Bayes coding algorithm assuming a special prior distribution over context tree models. An efficient recursive calculation method using a gathering context tree in the CTW algorithm is well known. Although there exist efficient recursive algorithms for the Bayes codes assuming a special class of prior distributions, the basic property of the prior distribution class has been scarcely investigated. In this paper we show the exact definition of a prior distribution class on context tree models that has the similar property to the class of conjugate priors. We show the posterior distribution is also included in the same distribution class as the prior distribution class. So we can also construct an efficient algorithm of predictive Bayes codes on context tree models by using the prior distribution class. Lastly the asymptotic mean code length of the codes is investigated.

In this paper, transmission schemes with noiseless and high capacity feedback channel is considered. We propose two types of transmission schemes using LDPC codes and clarify the density evolution analysis method for these proposed schemes. We investigate the performance of the proposed schemes by the density evolution analysis and computer simulations. The result shows some interesting characteristics for schemes with high capacity feedback channel.

In this paper, transmission schemes with noiseless and high capacity feedback channel is considered. We propose two types of transmission schemes using LDPC codes and clarify the density evolution analysis method for these proposed schemes. We investigate the performance of the proposed schemes by the density evolution analysis and computer simulations. The result shows some interesting characteristics for schemes with high capacity feedback channel.

In this paper, we present an iterative algorithm for computing the secrecy capacity of broadcast channel with confidential message (BCC) in the situation that the main channel is less noisy than the eavesdropper's channel. The global convergence of the algorithm is proved, and an expression for its convergence rate is derived.

In this paper, we generalize the achievability of variable-length coding from two viewpoints. One is the definition of an overflow probability, and the other is the definition of an achievability. We define the overflow probability as the probability of codeword length, not per symbol, is larger than eta(n) and we introduce the epsilon-achievability of variable-length codes that implies an existence of a code for the source under the condition that the overflow probability is smaller than or equal to epsilon.
Then we show that the epsilon-achievabitity of variable-length codes is essentially equivalent to the epsilon-achievability of fixed-length codes for general sources. Moreover we show the condition of epsilon-achievability for some restricted sources given epsilon.

In this letter, we generalize the achievability of variable-length coding from two viewpoints. One is the definition of an overflow probability, and the other is the definition of an achievability. We define the overflow probability as the probability of codeword length, not per symbol, is larger than ηn and we introduce the e-achievability of variable-length codes that implies an existence of a code for the source under the condition that the overflow probability is smaller than or equal to ∈. Then we show that the e-achievability of variable-length codes is essentially equivalent to the e-achievability of fixed-length codes for general sources. Moreover by using above results, we show the condition of e-achievability for some restricted sources given ∈. Copyright © 2007 The Institute of Electronics, Information and Communication Engineers.

Recently, a word-valued source has been proposed as a new class of information source models. A word-valued source is regarded as a source with a probability distribution over a word set. Although a word-valued source is a nonstationary source in general, it has been proved that an entropy rate of the source exists and the Asymptotic Equipartition Property (AEP) holds when the word set of the source is prefix-free. However, when the word set is not prefix-free (non-prefix-free), only an upper bound on the entropy density rate for an i.i.d. word-valued source has been derived so far. In this paper, we newly derive a lower bound on the entropy density rate for an i.i.d. word-valued source with a finite non-prefix-free word set. Then some numerical examples are given in order to investigate the behavior of the bounds.

We propose a new fixed-rate error correction system with a feedback channel. In our system, the receiver transmits a list of positions of unreliable information bits based on the log a-posteriori probability ratios by outputs of a soft-output decoder to the transmitter. This method is just like that of the reliability-based hybrid ARQ scheme. To dynamically select an appropriate interleaving function with feedback information is a key feature of our system. By computer simulations, we show that the performance of a system with a feedback channel is improved by dynamically selecting an appropriate interleaving function.

Reliability Based Hybrid ARQ (RB-HARQ) is an ARQ scheme using the modified feedback. In the RB-HARQ, the receiver returns both NAK signal and unreliable bit indices if the received sequence is not acknowledged. Since the unreliable bit index is determined by the bitwise posterior probability, better approximation of that probability is crucial. Assuming the systematic codes, the proposed RB-HARQ scheme has two features for this approximation: (1) The sender retransmits newly encoded parity bits corresponding to the unreliable information bits; (2) The retransmitted parity bits as well as the initial received sequence are put all together to perform the message passing decoding.

Orthogonal Arrays (OAs) have been playing important roles in the field of experimental design. It has been known that OAs are closely related to error-correcting codes. Therefore, many OAs can be constructed from error-correcting codes. But these OAs are suitable for only cases that equal interaction effects can be assumed, for example, all two-factor interaction effects. Since these cases are rare in experimental design, we cannot say that OAs from error-correcting codes are practical. In this paper. we define OAs with unequal strength. In terms of our terminology. OAs from error-correcting codes are OAs with equal strength. We show that OAs with unequal strength are closer to practical OAs than OAs with equal strength. And we clarify the relation between OAs with unequal strength and unequal error-correcting codes. Finally, we propose some construction methods of OAs with unequal strength from unequal error-correcting codes.

We propose transformation of a parity-check matrix of any low-density parity-check code. A code with transformed parity-check matrix is an equivalent of a code with the original parity-check matrix. For the binary erasure channel, performance of a message-passing algorithm with a transformed parity-check matrix is better than that with the original matrix.

If we can use a high capacity feedback channel, the receiver may transmit the indices of unreliable bits in a received sequence instead of ACK/NAK signals. We consider the error correction schemes using the indices of unreliable bits. The suitable parity symbols are incrementally transmitted using the feedback information. The bitwise reliability can be calculated by message passing algorithms on the variable graph constructed by adding the new nodes and arcs corresponding to the received parity bits to a former graph at each stage.

The reliability-based heuristic search methods for maximum likelihood decoding (MLD) generate test error patterns (or, equivalently, candidate codewords) according to their heuristic values. Test error patterns are stored in lists and its space complexity is crucially large for MLD of long block codes. Based on the decoding algorithms both of Battail and Fang and of its generalized version suggested by Valembois and Fossorier, we propose a new method for reducing the space complexity of the heuristic search methods for MLD including the well-known decoding algorithm of Han et al. If the heuristic function satisfies a certain condition, the proposed method guarantees to reduce the space complexity of both the Battail-Fang and Han et al. decoding algorithms. Simulation results show the high efficiency of the proposed method.

The Reliability Based Hybrid ARQ (RBH-ARQ) is one of hybrid ARQ schemes. In the RBH-ARQ, the modified feedback is composed of both ACK/NAK signal and unreliable bit indices which is evaluted from bitwise posterior probabilities. Using the modified feedback information, a sender encodes and retransmits the codeword. In this paper, we propose several error correction procedures with the RBH-ARQ schemes based on a systematic convolutional code. These have differences in their use of received retransmitted informations. To evalute and compare the peformances of these algorithms, we consider use of received retransmitted informations and a constitution of the encoder in the RBH-ARQ schemes.

Matrix clustering is defined as a method to obtain submatrices with specified area and density ( the ratio of 1 element about submatrix ) for the given sparse binary matrix.This method has been proposed as a data mining technique for Customer Relationship Management(CRM) and makes it possible to cluster the related items and customers.In this paper, we extend the apriori algorithm and propose a new matrix clustering algorithm which obtains all submatrices satisfying some specific condition.As a result, it is possible to analyze the purchase information of the customers in detail.

The problem of universal codes with side information is investigated from Bayes criterion. We propose side information context tree models which are an extension of context tree models to sources with side information. Assuming a special class of the prior distributions for side information context tree models, we propose an efficient algorithm of Bayes code for the models. The asymptotic code length of the Bayes codes with side information is also investigated.

The best-known algorithm for the decoding of low-density parity-check (LDPC) codes is the sum-product algorithm (SPA). The SPA is a message-passing algorithm on a graphical model called a factor graph (FG). The performance of the SPA depends on a structure of loops in a FG. Pearl showed that loops in a graphical model could be erased by the clustering method. This method clusters plural nodes into a single node. In this paper, we show several examples about a decoding on a FG to which the clustering method is applied. And we propose an efficient decoding algorithm for it.

The probabilistic reasoning given distribution evidence, virtual evidence, indirect evidence or likelihood have been investigated in previous research.In this paper, we classify the reasoning into two types and define each type of the reasoning by mathematical formulas. We clarify the difference between two types by the definition. From the definition, we show that the first type of reasoning is solved by the minimization of Kullback-Leibler(K-L) information under marginal constraints and the second type is calculated by the ordinary probabilistic reasoning methods such as Belief Propagation(BP). We also show that the Iterative Scaling Procedure(ISP) is applied to the first type reasoning. Moreover, we propose an efficient propagation algorithm, which are based on ISP, for the reasoning on Junction trees. Both the space and the time complexities of the proposed algorithm are lower than that of the previous research

Nishiara and Morita defined an i.i.d. word-valued source which is defined as a pair of an i.i.d. source with a countable alphabet and a function which transforms each symbol into a word over finite alphabet. They showed the asymptotic equipartition property (AEP) of the i.i.d. word-valued source and discussed the relation with source coding algorithm based on a string parsing approach. However, their model is restricted in the i.i.d. case and any universal code for a class of word-valued sources isn't discussed. In this paper, we generalize the i.i.d. word-valued source to the ergodic word-valued source which is defined by an ergodic source with a countable alphabet and a function from each symbol to a word. We show existence of entropy rate of the ergodic word-valued source and its formula. Moreover, we show the recurrence time theorem for the ergodic word-valued source with a finite alphabet. This result clarifies that Ziv-Lempel code (ZL77 code) is universal for the ergodic word-valued source.

In this paper, we discuss the difference in code lengths between the code based on the minimum description length (MDL) principle (the MDL code) and the Bayes code under the condition that the same prior distribution is assumed for both codes. It is proved that the code length of the Bayes code is smaller than that of the MDL code by o(1) or O(1) for the discrete model class and by O(1) for the parametric model class. Because we can assume the same prior for the Bayes code as for the code based on the MDL principle, it is possible to construct the Bayes code with equal or smaller code length than the code based on the MDL principle. From the viewpoint of mean code length per symbol unit (compression rate), the Bayes code is asymptotically indistinguishable from the MDL two-stage codes.

Ill many practical transmission systems, transmission error rate of the channel is time-varying. In previous works for controlling the rime-varying channel, the state of the channel is estimated and is uniquely determined. Then, the optimum value is chosen as a control parameter for the determined state. However, there is a discrepancy between the actual channel state and estimated state. The optimum control parameter determined for the estimated state does not necessarily provide the best performance presented by some object function in error control such as throughput efficiency. In th is study, the expected value of throughput is calculated by using the posterior probability for each state and an algorithm in which the control parameter is selected to maximize the expected performance is proposed. From the viewpoint of the statistical decision theory, it is shown that this selection is the optimum in maximizing the throughput under the Bayes criterion. Using simulation, the proposed method is evaluated and its effectiveness is shown. (C) 2001 Scripta Technica

In this paper, the generalized proof are shown for the coding theorem of [1]. Consequently, the further discussion is obtained.

In this paper, we show maximal rates in the case that random number generators generate a random sequence with an arbitrary prescribed distribution from a random sequence with an arbitrary given distribution.

In this paper, we show the probability of length overflow of several codes by using the variance and the asymptotic normality of the codelength.

We analyse the extended stochastic complexity (ESC) which has been proposed by K. Yamanishi. The ESC can be applied to learning algorithms for on-line prediction and batch-learning settings. Yamanishi derived the upper bound of ESC satisfying uniformly for all data sequences and that of the asymptotic expectation of ESC. However, Yamanishi concentrates mainly on the worst case performance and the lower bound has not been derived. In this paper, we show some interesting properties of ESC which are similar to Bayesian statistics: the Bayes rule and the asymptotic normality. We then derive the asymptotic formula of ESC in the meaning of almost sure and mean convergence within an error of o(1) using these properties.

For the two-dimensional BCH code, several decoding methods exceeding the BCH bound and correcting the errors that cannot be corrected by the conventional limited distance decoding method have been proposed. This article proposes an algorithm that allows reduction of the computational volume in a decoding method exceeding the BCH bound by solving the equation for unknown variables beforehand and limiting the range of the error location. Further, this decoding method exceeding the BCH bound is applied to soft-decision decoding methods that use limited-distance decoding multiple times, and especially to Chase decoding, the decoding of Tanaka et al., and that of Kaneko et al. It is shown that the amount of computation and the decoding error rate are improved.

We shall generalize B.S. Clarke and A.R. Barron's analysis of the Bayes method for the FSMX sources. The FSMX source considered here is specified by the set of all states and its parameter value. At first, we show the asymptotic codelengths of individual sequences of the Bayes codes for the FSMX sources. Secondly, we show the asymptotic expected codelengths. The Bayesian posterior density and the maximum likelihood estimator satisfy asymptotic normality for the finite ergodic Markov source, and this is the key of our analysis.

We shall generalize B.S. Clarke and A.R. Barron's analysis of the Bayes method for the FSMX sources. The FSMX source considered here is specified by the set of all states and its parameter value. At first, we show the asymptotic codelengths of individual sequences of the Bayes codes for the FSMX sources. Secondly, we show the asymptotic expected codelengths. The Bayesian posterior density and the maximum likelihood estimator satisfy asymptotic normality for the finite ergodic Markov source, and this is the key of our analysis.

This paper presents a new architecture for multiple-input signature analyzers. The proposed signature analyzer with H delta inputs is designed by parallelizing a GLFSR(delta, m), where delta is the number of input signals and m is the number of stages in the feedback shift register. The GLFSR, developed by Pradhan and Gupta, is a general framework for representing LFSR-based signature analyzers. The parallelization technique described in this paper can be applied to any kind of GLFSR signature analyzer, e.g., SISRs, MISRs, multiple MISRs and MLFSRs. It is shown that a proposed signature analyzer with H delta inputs requires less complex hardware than either single GLFSR(H delta, m)s or a parallel construction of the H original GLFSR(delta, m)s. It is also shown that the proposed signature analyzer, while requiring simpler hardware, has comparable aliasing probability with analyzers using conventional GLFSRs for some CUT error models of the same test response length and test lime. The proposed technique would be practical for testing CUTs with a large number of output sequences, since the test circuit occupies a smaller area on the LSI chip than the conventional multiple-input signature analyzers of comparable aliasing probability.

This paper presents a new architecture for multiple-input signature analyzers. The proposed signature analyzer with H delta inputs is designed by parallelizing a GLFSR(delta, m), where delta is the number of input signals and m is the number of stages in the feedback shift register. The GLFSR, developed by Pradhan and Gupta, is a general framework for representing LFSR-based signature analyzers. The parallelization technique described in this paper can be applied to any kind of GLFSR signature analyzer, e.g., SISRs, MISRs, multiple MISRs and MLFSRs. It is shown that a proposed signature analyzer with H delta inputs requires less complex hardware than either single GLFSR(H delta, m)s or a parallel construction of the H original GLFSR(delta, m)s. It is also shown that the proposed signature analyzer, while requiring simpler hardware, has comparable aliasing probability with analyzers using conventional GLFSRs for some CUT error models of the same test response length and test lime. The proposed technique would be practical for testing CUTs with a large number of output sequences, since the test circuit occupies a smaller area on the LSI chip than the conventional multiple-input signature analyzers of comparable aliasing probability.

In this paper, we shall derive the upper bounds on error rates of the statistical model selection using the information criteria, P(e)*. The similar bounds were derived by J.Suzuki [2]. We shall generalize the results for the general model class.

In this paper, we shall derive the upper bounds on error rates of the statistical model selection using the information criteria, P(e)*. The similar bounds were derived by J.Suzuki [2]. We shall generalize the results for the general model class.

In this paper, new algorithms are proposed based on statistical decision theory in the field of Markov decision processes under the condition that a tradition probability matrix is unknown. In previous researches on RL(reinforcement learning), learning is based on only the estimation of an unknown transition probability matrix and the maximum reward is not received in a finite period, though their purpose is to maximize a reward. In our algorithms it is possible to maximize the reward within a finite period with respect to Bayes criterion. Moreover, we propose some techniques to reduce the computational complexity of our algorithm from exponential order to polynomial order.

In this paper, new algorithms are proposed based on statistical decision theory in the field of Markov decision processes under the condition that a tradition probability matrix is unknown. In previous researches on RL(reinforcement learning), learning is based on only the estimation of an unknown transition probability matrix and the maximum reward is not received in a finite period, though their purpose is to maximize a reward. In our algorithms it is possible to maximize the reward within a finite period with respect to Bayes criterion. Moreover, we propose some techniques to reduce the computational complexity of our algorithm from exponential order to polynomial order.

Bayesian theory is effective in statistics, lossless source coding, machine learning, etc. It is often, however, computationally expensive since the calculation of posterior probabilities and of mixture distributions is not tractable. In this paper, we propose a new method for approximately calculating mixture distributions in a discrete hypothesis class.

This paper presents a near architecture for multiple-input signature analyzers. The proposed signature analyzer with H delta inputs is designed by parallelizing a GLFSR(delta,m), where delta is the number of input signals and m is the number of stages in the feedback shift register. The GLFSR, developed by Pradhan and Gupta, is a general framework for rep resenting LFSR-based signature analyzers. The parallelization technique described in this paper can be applied to any kind of GLFSR signature analyzer, e.g., SISRs: MISRs, multiple MISRs and MLFSRs. It is show that a proposed signature analyzer with HS inputs requires less complex hardware than either single GLFSR(H delta,m)s or parallel construction H original GLFSR(delta,m)s. It is also shown that the proposed parallelization technique can be applied to a test pattern generator in BIST, since the GLFSR is also used to generate patterns for a CUT. The proposed technique would be practical for testing CUTs with a large number of input and output sequences, since the test circuit occupies a smaller area on the LSI chip than conventional test circuits.

Bayesian theory is effective in statistics, lossless source coding, machine learning, etc. It is often, however, computationally expensive since the calculation of posterior probabilities and of mixture distributions is not tractable. In this paper, we propose a new method for approximately calculating mixture distributions in a discrete hypothesis class.

In this paper, we consider the problem to predict the class of an unknown sample after learning from queries, We propose to evaluate a learning algorithm by a loss function for the prediction under a constraint, In this paper, the error probability for the prediction and the number of queries is defined as the loss function and the constraint, respectively, Then our objective is to minimize the error probability, the error probability is determined by what presentation order for instances to query and how to predict, Since the optimal prediction has been shown in previous researches, we only have to select the optimal presentation order for instances to query, We propose a lower bound used in the branch-and-bound algorithm to select the optimal presentation order for instances, Lastly, we show the efficiency of the algorithm using the derived lower bound by numerical computation.

In this paper, we obtain an accurate asymptotic formula of the Bayes optimal codelength in the case that the sources are not always i.i.d. sources by deriving the asymptotic codelength of sufficient statistic codes. © 1997 IEEE.

Recently, the high-speed data transmission techniques that have been developed for communication systems have in turn necessitated the implementation of high-speed error correction circuits. Parallel processing has been found to be an effective method of speeding up operations, since the maximum achievable clock frequency is generally bounded by the physical constraints of the circuit. This paper presents a parallel encoder and decoder architecture which can be applied to both binary and nonbinary cyclic codes. The architecture allows H symbols to be processed in parallel, where H is an arbitrary integer, although its hardware complexity is not proportional to the number of parallel symbols H. As an example, we investigate hardware complexity for a Reed-Solomon code and a binary BCH code. It is shown that both the hardware complexity and the delay for a parallel circuit is much less than that with the parallel operation of H conventional circuits. Although the only problem with this parallel architecture is that the encoder's critical path length increases with H, the proposed architecture is more efficient than a setup using H conventional circuits for high data rate applications. It is also suggested that a parallel Reed-Solomon encoder and decoder, which can keep up with optical transmission rates, i.e., several giga bits/sec, could be implemented on one LSI chip using current CMOS technology.

We discuss the method for representing uncertain knowledge information and also on a reasoning system that guarantees a given reliability based on information theory and probability theory. In this paper, we propose a new reasoning system which consists of inductive and deductive inferences, by using a knowledge representation method with a range of certainty factors. In inductive inference, a new atom is computed from the observed data or facts, together with the upperbound and lowerbound between which its probability must fall. Also in deductive inference, which includes negation, conjunction, disjunction, chain and modus ponens, a new atom is deduced with a range of certainty factors computed from the results of inductive inference. Properties and behaviors of the system are clarified. It can be concluded that the new reasoning system is effective of conventional expert systems.

We show that the received sequences, which are rejected and not discarded on ARQ schemes, make the next decision with repeat request powerful by a random coding arguments. We first give the ordinary ARQ of the basic algorithm with some criterion, and define the ARQ with cumulating the rejected sequences, which naturally extended from the ordinary one. Then we study the error exponent of decoding the second received sequences under the condition that the first is rejected. We assume the block coded ARQ with noiseless feedback. © 1994 IEEE.

The context tree weighting (CTW) algorithm [Willems et al., 1993] has high compressibility for universal coding with respect to FSMX sources. The present authors propose an algorithm by reinterpreting the CTW algorithm from the viewpoint of Bayes coding. This algorithm can be applied to a wide class of prior distribution for finite alphabet FSMX sources. The algorithm is regarded as both a generalized version of the CTW procedure and a practical algorithm using a context tree of the adaptive Bayes coding which has been studied in Mataushima et al. (1991). Moreover, the proposed algorithm is free from underflow which frequently occurs in the CTW procedure. © 1994 IEEE.

Considering inductive inference and deductive inference as not individual processes but a serial process of information processing, the serial inference procedure is studied from two purposes: one is the compression of observed facts, and the other is the prediction of new wffs. A new scheme of the serial inference process is proposed by using the correspondence to source coding and prediction problem. The optimal inference procedures for the two purposes are shown in the proposed scheme.

The problem of distortionless encoding when the parameters of the probabilistic model of a source are unknown is considered from a statistical decision theory point of view. A class of predictive and nonpredictive codes is proposed that are optimal within this framework. Specifically, it is shown that the codeword length of the proposed predictive code coincides with that of the proposed nonpredictive code for any source sequence. A bound for the redundancy for universal coding is given in terms of the supremum of the Bayes risk. If this supremum exists, then there exists a minimax code whose mean code length approaches it in the proposed class of codes, and the minimax code is given by the Bayes solution relative to the prior distribution of the source parameters that maximizes the Bayes risk.

In designing systems with a human‐computer interface with a minimum number of interactions, there are two issues to consider: the determination of a proper sequence of questions by the user, and proper termination by the computer system, based on previous instructions. In short, these issues are those of feature selection ordering and a stopping rule for pattern recognition processes. Conventional treatments of these problems have been investigated from the viewpoint of an average error rate. When the number of patterns is large, however, and the number of instructions to be terminated is large, from the point of view of user interface efficiency, and the average error rate is not an effective indicator at the intermediate stage. In this paper, a new method is proposed for determining feature selection ordering and a stopping rule which focuses on the remaining patterns at each stage, and which maximizes the value of mutual information between the user's responses and the required pattern. Important properties associated with this scheme have been demonstrated while evaluating its performance via computer simulation. One is that the average information gain at each stage decreases monotonically, and another is that this scheme produces the minimum error rate. Copyright © 1990 Wiley Periodicals, Inc., A Wiley Company