For massive multiple-input multiple-output (MIMO) systems, pilot contamination reduces the data transmission capacity owing to the inter-cell interference of non-orthogonal pilots reusage. To develop efficient mobile communication, it is necessary to mitigate pilot contamination. To address this problem, we propose an annealing-based pilot allocation method using Digital Annealer to provide solution for Ising machine. The proposed method is a max k-cut-based approach, where the graph represents the potential strength of pilot contamination among users in other cells. By using this proposed method, users who have strong relationship with pilot contamination will be assigned different pilots. Experiment results show that the proposed method can realize optimal pilot allocation and mitigate pilot contamination. Compared with conventional methods, the proposal achieves the best performance which can increase the minimum achievable rate and show higher SINR, especially when the numbers of users and cells are large.

With the dramatically increasing video streaming in the total network traffic, it is critical to develop effective algorithms to ensure the quality of content delivery service. Adaptive bitrate (ABR) control is the most essential technique which determines the proper bitrate to be chosen based on network conditions, thus realize high-quality video streaming. In this paper, a novel ABR strategy is proposed based on Ising machine by using the quadratic unconstrained binary optimization (QUBO) method and Digital Annealer (DA) for the first time. The proposed method is evaluated by simulation with the real-world measured throughput and compared with other state-of-the-art methods. Experiment results show that the proposed QUBO-based method can outperform the existing methods, which demonstrating the superior of the proposed QUBO-based method.

Recent advancements of virtualization technologies for parallel processing involve scheduling containerized tasks in a workflow. Since a container can include multiple tasks, it can be reused or shared among applications. If every task in a workflow uses its dedicated container without sharing among any tasks, each container image must be downloaded for each task. As a result, many computational resources are required to process and the communication latency related to container image downloading can become a bottleneck for the makespan. In task scheduling algorithms for workflows, this characteristic produces a new challenging issue that how effectively shares containers among tasks to avoid redundant container image download processes and redundant task allocations. One of the fundamental problems is that no policy has been established for simultaneously satisfying effective container sharing, maintaining the degree of task parallelism, and effective computational resource utilization. In this paper, we propose a clustering-based containerized task scheduling algorithm for clouds, namely, shareable functional task clustering for utilizing virtualized resources (SF-CUV). The objective of SF-CUV is to minimize the makespan with less computational resources and containers than other algorithms by clustering tasks and sharing each container among tasks. SF-CUV consists of two phases: (i)task clustering and pre-virtual CPU (vCPU) allocation phase to derive an accurate scheduling priority, and (ii)task ordering and actual task reallocation phase. Experimental results obtained via simulation and in a real environment show that SF-CUV can utilize both vCPUs and containers with a shorter makespan compared with other approaches.

These days, attention is thought to be an efficient way to recognize an image. Vision Transformer (ViT) uses a Transformer for images and has very high performance in image recognition. ViT has fewer parameters than Big Transfer (BiT) and Noisy Student. Therefore, we consider that Self-Attention-based networks are slimmer than convolution-based networks. We use a ViT as a Discriminator in a Generative Adversarial Network (GAN) to get the same performance with a smaller model. We name it ViT-GAN. Besides, we find parameter sharing is very useful to make parameter-efficient ViT. However, the performances of ViT heavily depend on the number of data samples. Therefore, we propose a new method of Data Augmentation. Our Data Augmentation, in which the strength of Data Augmentation varies adaptively, helps ViT for faster convergence and better performance. With our Data Augmentation, we show ViT-based discriminator can achieve almost the same FID but the number of the parameters of the discriminator is 35% fewer than the original discriminator.

With the live video streaming becomes more and more common in daily life such as live meeting and live video call, it is an urgent task to ensure high-quality and low-delay live video streaming service. High user quality of experience (QoE) should be ensured to satisfy the requirement of user, for which latency is one of the important factors. In this paper, a high-QoE live streaming method is proposed with reinforcement learning. Experiments are conducted to evaluate the proposed method. Results demonstrate that the proposal shows the best performance with highest QoE compared with conventional methods in three network conditions. In Ferry case, the QoE is almost twice of the QoE of other methods.

Learned image compression (LIC) has achieved superior coding performance than traditional image compression standards such as HEVC intra in terms of both PSNR and MS-SSIM. However, most LIC frameworks are based on floating-point arithmetic which has two potential problems. First is that using traditional 32-bit floating-point will consume huge memory and computational cost. Second is that the decoding might fail because of the floating-point error coming from different encoding/decoding platforms. To solve the above two problems. 1) We linearly quantize the weight in the main path to 8-bit fixed-point arithmetic, and propose a fine tuning scheme to reduce the coding loss caused by the quantization. Analysis transform and synthesis transform are fine tuned layer by layer. 2) We exploit look-up-table (LUT) for the cumulative distribution function (CDF) to avoid the floating-point error. When the latent node follows non-zero mean Gaussian distribution, to share the CDF LUT for different mean values, we restrict the range of latent node to be within a certain range around mean. As a result, 8-bit weight quantization can achieve negligible coding gain loss compared with 32-bit floating-point anchor. In addition, proposed CDF LUT can ensure the correct coding at various CPU and GPU hardware platforms.

Dynamic adaptive streaming over HTTP (DASH) has been widely adopted in modern video streaming services. In DASH, the core technique is adaptive bitrate (ABR) control which can adjust the requested video bitrate level according to the network conditions to tradeoff between video quality and rebuffering risk. It is a challenge for the ABR methods in the scenarios when multiple DASH streaming users compete over the network bottleneck. This paper proposes a client-side ABR control method, flexible relaxation assisted by buffer (FRAB), to achieve fair, stable and efficient video streaming among different users. The idea of FRAB is to relax the change of the video quality based on current buffer level, which can enhance the stability of video streaming. Meanwhile, by flexibly adjusting the relaxation, the efficiency and fairness among all users are improved. FRAB is evaluated in real experiments under three different network conditions and compared with conventional multi-user ABR algorithms. Results indicate FRAB has the best performance in fairness, which reduces the unfairness by a maximum of 69.5% under real-world measured network condition. It also improves the efficiency by 71.3% comparing with PANDA, and enhances the stability by 73.3% comparing with TFDASH. The experiment results demonstrated that the proposed method has superior performances in multi-user DASH video streaming.

Recently, aging of the population and medical cost inflation are emerging as social issues to be solved. To address this problem, estimations of bioinformation based on Internet of Things (IoT) and machine learning get more attention to researchers. In this paper, in order to estimate R-R Interval (RRI) without using specialized and professional wearable devices, we propose a deep learning based RRI estimation method by using mainly smartphone sensors. For dataset, we collect ECG (for label), 3-axis acceleration, pressure, illuminance, GPS and temperature (for training data) under different exercise types (walking and running) by using a smartphone and smart clothing called hitoe. To construct a regression model, we adopt a dual stage attention-based RNN model. From the evaluation results, we confirm that the proposed method can estimate RRI and LF/HF with acceptable accuracy.

In order to simplify deployment and management of IoT services, Network Function Virtualization (NFV) and Service Function Chaining (SFC) are promising solutions, and much researchers have conducted these topics. To enhance the reliability of former research efforts, in this paper, we propose an orchestration framework for IoT-centric SFC by using Docker and Kubernetes. The framework enables an automatic IoT service deployment by satisfying service requirements and computing and network resource constraints. In such deployment, we apply a Virtual Network Function (VNF)/Service Function (SF) placement problem to achieve efficient utilization of the resources. We set an objective function as minimizing both numbers of SF instances and communications and build a mathematical model based on Integer Linear Programming (ILP). To validate it, we implement a model for the framework and evaluate the performances by carrying out a numerical evaluation and a real experiment. From the evaluation results, we confirm that the proposed approach can reduce the number of SF placements and the number of communications among SF instances.

Thanks to breakthrough and evolution of deep learning in computer vision areas, adaptation of deep learning into communication systems are getting lots of attention to researchers. Recently, a channel estimation method by using a deep learning-based image super-resolution (SR) technique, namely ChannelNet, has been proposed. Inspired by this research, in this paper, we propose a deep SR based channel estimation method by applying more accurate deep learning-based SR network architecture, EDSR. In order to enhance intelligibility and reliability of deep SR based channel estimation methods, we evaluate the performance of several deep SR based channel estimation methods (SRCNN, ChannelNet and EDSR) by carrying out practical 5G simulations. From the evaluations, the results conclude that the deep SR based channel estimation methods can potentially improve accuracy of channel estimation and reduce BER characteristics.

Adaptive Loop Filter (ALF) is a new technique proposed by the latest video coding standard Versatile Video Coding (VVC). To the best of our knowledge, this paper is the first implementation to design a hardware architecture of ALF in VVC decoder. The implementation reduces 60% of the memory access, saves approximately 50% of the hardware resources including adders and multipliers cost, increases the throughput and makes the hardware configurable for luma and chroma components. The synthesis result demonstrates that the architecture achieves a throughput of 4k@120fps and a maximum frequency of 367MHz under the TSMC 65nm process.

Conventional object detection methods are in the pixel domain and require full decoding with high computational complexity. In this paper, we propose a fast object detection method in the intra compressed domain of High Efficiency Video Coding (HEVC), which significantly accelerates the object detection process that uses compressed video images. Considering the characteristics of various coding features, we select 3 types of data for object detection, including partitioning depths, prediction modes, and residuals. To achieve a more discriminative representation of the residuals, we design an iterative restoration algorithm that can generate the details of the original image and reduce the noise in the residuals. Extensive evaluations on multiple HEVC test sequences and large-scale object detection dataset BDD100K confirm the effectiveness of our method. With a slight reduction in detection accuracy, our compressed domain detection system runs 1.8 times faster than the pixel domain.

Dynamic adaptive streaming over NDN typically relies on past information of network conditions and streaming quality. In this paper, we address the cold start problem associated with reinforcement learning based NDN adaptive streaming where a new consumer often found choosing bitrate arbitrarily due to the lack of experience. The idea is to construct a shared Q-Table which is continuously updated by previous consumers. Based on this Q-Table, a new consumer is expected to start choosing the segment bitrate more proactively. Simulations through ns-3 show that the proposed approach could help the consumers to find an optimal action from the beginning of the session.

Learned image compression (LIC) has illustrated good ability for reconstruction quality driven tasks (e.g. PSNR, MS-SSIM) and machine vision tasks such as image understanding. However, most LIC frameworks are based on pixel domain, which requires the decoding process. In this paper, we develop a learned compressed domain framework for machine vision tasks. 1) By sending the compressed latent representation directly to the task network, the decoding computation can be eliminated to reduce the complexity. 2) By sorting the latent channels by entropy, only selective channels will be transmitted to the task network, which can reduce the bitrate. As a result, compared with the traditional pixel domain methods, we can reduce about 1/3 multiply-add operations (MACs) and 1/5 inference time while keeping the same accuracy. Moreover, proposed channel selection can contribute to at most 6.8% bitrate saving.

Recently, requirement of city monitoring and maintenance using ICT techniques increases with the help of transportation system. In addition, the spread of COVID-19 has increased the demand for managing pedestrian traffic volume. To contribute to these trends, in this paper, we propose a new pedestrian radar map system in order to estimate pedestrian density on streets and sidewalks. Our system uses e-bikes to collect 360-degree images and visualize pedestrian positions as a radar map. In evaluations, we confirm the accuracies of the radar maps and pedestrian density by using KITTI dataset and by carrying out a field experiment.

Convolutional neural networks (CNNs) have achieved great successes on many computer vision tasks, such as image recognition, video processing, and target detection. In recent years, many hardware designs have been devoted to accelerating CNN inference. In order to further speed up CNN inference and reduce data waste, this work proposed a reconfigurable sliced systolic array: 1) Depending on the number of network nodes in each layer, the slice mode could be dynamically configured to achieve high throughput and resource utilization. 2) To take full advantage of convolution reuse and weight reuse, this work designed a tile-column sliding (TCS) processing dataflow. 3) A four-stage for loop algorithm was employed, which divides the CNN calculation into several parts based on the input nodes and output nodes. The entire CNN inference is carried out using integer-only arithmetic originated from TensorLite. Experimental results prove that these strategies lead to significant improvement in inference performance and energy efficiency.

As the demand for high-resolution videos grows, the next generation video coding standard Versatile Video Coding introduces many new proposals, including Adaptive Multiple Transforms (AMT), to improve coding efficiency. This paper presents a reconfigurable transform core for the VVC standard where the implementation of 1D DST-VII and DCT-VIII for all transform sizes are enabled. To offer a very low circuit complexity, a simple approximation strategy with a little coding performance loss is proposed. An 8×8 Processing Element (PE) array is employed as the core computational unit, where each PE can be configured dynamically based on the transform type. In addition, the transforms of larger sizes can be realized in the finite PE units with the Partitioned Matrix Multiplication (PMM) scheme. The experimental and synthesis results show that this design can save at least 29.1% area compared with other works in literature with the negligible degradation of video quality and a slight increase in the bit rate.

With the increasing video demand in daily network traffic, it is an urgent task to develop effective algorithms to facilitate high-quality content delivery service. Recently, numerous adaptive streaming algorithms have been proposed to improve the user perceived experience. However, these algorithms were mainly developed from the perspective of single user. There is not yet systematical evaluation and comparison of the bitrate adaptation methods for multi-user video streaming. Besides, the Quality of Experience (QoE) metrics were not unified. In this work, we propose a new mininet-based testbed framework which is able to conduct real-time video streaming emulation in various multi-user scenarios. Seven state-of-the-art adaptation methods are incorporated into the testbed. Meanwhile, ITU-T P.1203 model, the world's first standard for measuring QoE of HTTP adaptive streaming, is implemented to calculate the mean opinion scores of different methods. Using the developed testbed, the performance of current adaptation methods in multi-user network is analyzed and compared. A variety of experiments are carried out by changing the user number and network conditions, in which the QoE of different users are investigated. It is found that current algorithms perform inconsistently in various network scenarios. In the excessive user and limited bandwidth cases, machine learning and scheduling techniques show superiority in providing high and equal QoE for all users. While in the high-delay case, the buffer-based approaches show robust performance. Overall, the findings of this work give an insight for designing and choosing adaptive streaming strategies in different multi-user network conditions.

Intra prediction is an essential component in the image coding. This paper gives an intra prediction framework completely based on neural network modes (NM). Each NM can be regarded as a regression from the neighboring reference blocks to the current coding block. (1) For variable block size, we utilize different network structures. For small blocks 4×4 and 8×8, fully connected networks are used, while for large blocks 16×16 and 32×32, convolutional neural networks are exploited. (2) For each prediction mode, we develop a specific pre-trained network to boost the regression accuracy. When integrating into HEVC test model, we can save 3.55%, 3.03% and 3.27% BD-rate for Y, U, V components compared with the anchor. As far as we know, this is the first work to explore a fully NM based framework for intra prediction, and we reach a better coding gain with a lower complexity compared with the previous work.

This paper enhances the intra prediction by using multiple neural network modes (NM). Each NM serves as an end-To-end mapping from the neighboring reference blocks to the current coding block. For the provided NMs, we present two schemes (appending and substitution) to integrate the NMs with the traditional modes (TM) defined in high efficiency video coding (HEVC). For the appending scheme, each NM is corresponding to a certain range of TMs. The categorization of TMs is based on the expected prediction errors. After determining the relevant TMs for each NM, we present a probability-Aware mode signaling scheme. The NMs with higher probabilities to be the best mode are signaled with fewer bits. For the substitution scheme, we propose to replace the highest and lowest probable TMs. New most probable mode (MPM) generation method is also employed when substituting the lowest probable TMs. Experimental results demonstrate that using multiple NMs will improve the coding efficiency apparently compared with the single NM. Specifically, proposed appending scheme with seven NMs can save 2.6%, 3.8%, and 3.1% BD-rate for Y, U, and V components compared with using single NM in the state-of-The-Art works.

Recent researches in video frame interpolation show great progress. In this paper, we propose a novel video compression method which incorporates deep learning based frame interpolation into HEVC which is the current video compression standard. Experimental results show that our approach can outperform HEVC in some scenarios.

Recently, an efficient and automatic infrastructure maintenance service is mandatory. To address this demand, in this paper, we introduce a segmentation-based road damage detection method by using U-Net. To train the model, we collect 4K images by using a smartphone mounted on a bicycle and build our own road damage dataset. In addition, to improve detection accuracy, we apply focal loss and image patch for loss function and input image, respectively. From the evaluation, the result confirms that the method demonstrates to extract road damages with acceptable accuracy.

In this paper, we propose a deep learning-based distance estimation method from a single-shot image. In the proposal, we model the estimation as a regression problem, and estimate the distance between a pedestrian and a camera by using three main features; size of bounding box, image blur and image features. By using KITTI dataset, we evaluate the accuracy of the proposed model.

Recently, in order to build barrier-free smart cities, Japanese government is actively promoting to construct open data regarding roads and buildings, including existence of bumps, road slope and corridor width. To contribute this activity, in this paper, we introduce a corridor width estimation method by using an RGB-D camera. In this method, we assume that the RGB-D camera is mounted on the electric wheelchairs for a mobile sensing platform. The corridor width estimation is mainly operated by three steps: planes extraction, side walls extraction and distance calculation. We evaluate the accuracy of this method at eight corridors in Waseda University.

To contribute to publishing pedestrian networks as open data, this paper introduces a curb height estimation method by using 2DLiDAR mounted on e-Bike. We develop a prototype system by using e-Bike, 2DLiDAR and other sensors, and collect actual curb data around Waseda University. From accuracy evaluations, the results confirm that the prototype can estimate road level differences and positions with high accuracy.

Under complex network conditions, adaptive video streaming requires additional state information for optimal quality selection. In this paper, we present the applicability of reinforcement learning techniques on NDN adaptive streaming. Both buffer-based and throughput-based adaptation are studied and observed their characteristics. The Q-learning algorithm is used to learn state-action values. Based on a greedy policy, the simulation results demonstrate that RL agents tend to choose the best possible bitrate which consequently reduces the quality fluctuation in adaptive streaming.

Learned image compression (LIC) has reached the traditional hand-crafted methods such as JPEG2000 and BPG in terms of the coding gain. However, the large model size of the network prohibits the usage of LIC on resource-limited embedded systems. This paper presents a LIC with 8-bit fixed-point weights. First, we quantize the weights in groups and propose a non-linear memory-free codebook. Second, we explore the optimal grouping and quantization scheme. Finally, we develop a novel weight clipping fine tuning scheme. Experimental results illustrate that the coding loss caused by the quantization is small, while around 75% model size can be reduced compared with the 32-bit floating-point anchor. As far as we know, this is the first work to explore and evaluate the LIC fully with fixed-point weights, and our proposed quantized LIC is able to outperform BPG in terms of MS-SSIM.

Recently learned image compression has achieved many great progresses, such as representative hyperprior and its variants based on convolutional neural networks (CNNs). However, CNNs are not fit for scalable coding and multiple models need to be trained separately to achieve variable rates. In this paper, we incorporate differentiable quantization and accurate entropy models into recurrent neural networks (RNNs) architectures to achieve a scalable learned image compression. First, we present an RNN architecture with quantization and entropy coding. To realize the scalable coding, we allocate the bits to multiple layers, by adjusting the layer-wise lambda values in Lagrangian multiplier-based rate-distortion optimization function. Second, we add an RNN-based hyperprior to improve the accuracy of entropy models for multiple-layer residual representations. Experimental results demonstrate that our performance can be comparable with recent CNN-based hyperprior methods on Kodak dataset. Besides, our method is a scalable and flexible coding approach, to achieve multiple rates using one single model, which is very appealing.

For the next-generation video coding standard Versatile Video Coding (VVC), several new contributions have been proposed to improve the coding efficiency, especially in the transformation operations. This paper proposes a unified 32× 32 block-based transform architecture for the VVC standard that enables 2D Discrete Sine Transform-VII (DST-VII) and Discrete Cosine Transform-VIII (DCT-VIII) of all sizes. It mainly gives three contributions: 1) The N-Dimensional Reduced Adder Graph (RAG-n) algorithm is adopted to design the minimal adder-oriented computational units. 2) The storage of the asymmetric transform units can be realized in the dual-port SRAM-based transpose memory. 3) The pipelined 2D transformations of mixed block sizes are achieved with the throughput rate of 32 samples per cycle. The synthesis results indicate that this architecture can reduce area by up to 73.1% compared with other state-of-the-art works. Moreover, power saving ranging from 4.9% to 9.9% can be achieved. Regarding the transpose memory, at least 21.9% of the area can be saved by using SRAM.

With the continuous development of mobile devices, it becomes possible for people to demand higher definition videos. To alleviate the pressure of deploying the video codec in mobile multimedia, a learning-based low complexity in-loop filter is proposed in this paper. Depthwise separable convolution is combined with batch normalization to construct this model. To enhance its performance, the knowledge from a pre-trained teacher model is transferred to it. However, the over-smoothing problem in the inter frames caused by double enhancing effect remains. To solve this, a Wiener-based filtering algorithm that tries to restore the distortion from the learned residuals is designed and introduces an adequate filtering effect. The experimental results show that our proposed methods achieve considerable BD-rate reduction than HEVC anchor. Compared with the previous state-of-the-art work VR-CNN, our model achieves 1.65% extra BD-rate reduction, 79.1% decrease in FLOPs, 25% decrease in encoding complexity, and 70% decoding complexity decrease.

In this paper, a coding framework VIP-ICT-Codec is introduced. Our method is based on the VTM (Versatile Video Coding Test Model). First, we propose a color space conversion from RGB to YUV domain by using a PCA-like operation. A method for the PCA mean calculation is proposed to de-correlate the residual components of YUV channels. Besides, the correlation of UV components is compensated considering that they share the same coding tree in VVC. We also learn a residual mapping to alleviate the over-filtered and under-filtered problem of specific images. Finally, we regard the rate control as an unconstraint Lagrangian problem to reach the target bpp. The results show that we achieve 32.625dB at the validation phase.

In this paper, we provide a detailed description on our submitted method Kattolab to Workshop and Challenge on Learned Image Compression (CLIC) 2020. Our method mainly incorporates discretized Gaussian Mixture Likeli-hoods to previous state-of-the-art learned compression algorithms. Besides, we also describes the acceleration strategies and bit optimization with the low-rate constraint. Experimental results have demonstrated that our approach Kattolab achieves 0.9761 in terms of MS-SSIM at the rate constraint of 0.15 bpp during the validation phase.

Lossless image compression is an important task in the field of multimedia communication. Traditional image codecs typically support lossless mode, such as WebP, JPEG2000, FLIF. Recently, deep learning based approaches have started to show the potential at this point. HyperPrior is an effective technique proposed for lossy image compression. This paper generalizes the hyperprior from lossy model to lossless compression, and proposes a L2-norm term into the loss function to speed up training procedure. Besides, this paper also investigated different parameterized models for latent codes, and propose to use Gaussian mixture likelihoods to achieve adaptive and flexible context models. Experimental results validate our method can outperform existing deep learning based lossless compression, and outperform the JPEG2000 and WebP for JPG images.

Image compression has been an important research topic for many decades. Recently, deep learning has achieved great success in many computer vision tasks, and its use in image compression has gradually been increasing. In this paper, we present an energy compaction-based image compression architecture using a convolutional autoencoder (CAE) to achieve high coding efficiency. Our main contributions include three aspects: 1) we propose a CAE architecture for image compression by decomposing it into several down(up)sampling operations; 2) for our CAE architecture, we offer a mathematical analysis on the energy compaction property and we are the first work to propose a normalized coding gain metric in neural networks, which can act as a measurement of compression capability; 3) based on the coding gain metric, we propose an energy compaction-based bit allocation method, which adds a regularizer to the loss function during the training stage to help the CAE maximize the coding gain and achieve high compression efficiency. The experimental results demonstrate our proposed method outperforms BPG (HEVC-intra), in terms of the MS-SSIM quality metric. Additionally, we achieve better performance in comparison with existing bit allocation methods, and provide higher coding efficiency compared with state-of-the-art learning compression methods at high bit rates.

Recently, a fifth-generation cellular system (5G) is widely expected to provide plenty of wireless network resources (i.e., broadband capacity). In this paper, to validate 5G system performances, such as physical-layer and TCP-layer throughputs, we carry out a field trial at an actual indoor train station, named Haneda International Airport Terminal Station. In the field trial, we deploy the prototype 5G system (Central Unit, Distribution Unit, Radio Unit and 5G UE (tablet)) on the train station platform and evaluate mobile 5G downlink throughputs. Through the actual measurements, the results confirm that the prototype 5G system can achieve mobile broadband capacity (more than 1 Gbps) even when the UE is located anywhere at the indoor train station platform.

The amount of video data is so large that efficient video compression is required for the storage and transmission. Intra prediction is one of important components in video compression. In this paper, we examine various Convolutional Neural Network (CNN) structures to estimate optimal intra prediction mode as Most Probable Modes (MPMs). Moreover, we investigate several combinations of the MPMs obtained by the CNN and MPMs derived from High Efficiency Video Coding Test Model (HM). From these experimental results, we find that using 6 MPMs from both CNN and HM with moderate number of channels or kernel size is preferred.

Different from the traditional quaternary tree (QT) structure utilized in the previous generation video coding standard H.265/HEVC, a brand new partition structure named quadtree with nested multi-type tree (QTMT) is applied in the latest codec H.266/VVC. The introduction of QTMT brings in superior encoding performance at the cost of great time-consuming. Therefore, a fast intra partition algorithm based on variance and Sobel operator is proposed in this paper. The proposed method settles the novel asymmetrical partition issue in VVC by well balancing the reduction of computational complexity and the loss of encoding quality. To be more concrete, we first terminate further splitting of a coding unit (CU) when the texture of it is judged as smooth. Then, we use Sobel operator to extract gradient features to decide whether to split this CU by QT, thus terminating further MT partitions. Finally, a completely novel method to choose only one partition from five QTMT partitions is applied. Obviously, homogeneous area tends to use a larger CU as a whole to do prediction while CUs with complicated texture are prone to be divided into small sub-CUs and these sub-CUs usually have different textures from each other. We calculate the variance of variance of each sub-CU to decide which partition will distinguish the sub-textures best. Our method is embedded into the latest VVC official reference software VTM-7.0. Comparing to anchor VTM-7.0, our method saves the encoding time by 49.27% on average at the cost of only 1.63% BDBR increase. As a traditional scheme based on variance and gradient to decrease the computational complexity in VVC intra coding, our method outperforms other relative existing state-of-the-art methods, including traditional machine learning and convolution neural network methods.

Image compression is a fundamental research field and many well-known compression standards have been developed for many decades. Recently, learned compression methods exhibit a fast development trend with promising results. However, there is still a performance gap between learned compression algorithms and reigning compression standards, especially in terms of widely used PSNR metric. In this paper, we explore the remaining redundancy of recent learned compression algorithms. We have found accurate entropy models for rate estimation largely affect the optimization of network parameters and thus affect the rate-distortion performance. Therefore, in this paper, we propose to use discretized Gaussian Mixture Likelihoods to parameterize the distributions of latent codes, which can achieve a more accurate and flexible entropy model. Besides, we take advantage of recent attention modules and incorporate them into network architecture to enhance the performance. Experimental results demonstrate our proposed method achieves a state-of-the-art performance compared to existing learned compression methods on both Kodak and high-resolution datasets. To our knowledge our approach is the first work to achieve comparable performance with latest compression standard Versatile Video Coding (VVC) regarding PSNR. More importantly, our approach generates more visually pleasant results when optimized by MS-SSIM.

Surveillance cameras are commonly used for security purpose. However, most of them are used for verification after incidents happen. In this paper, we propose a method for proactive video surveillance system using human pose estimation. Our method estimates anomaly scores of human actions using the video descriptors (human pose and bounding box) extracted by pose estimation and tracking methods. We estimate human poses, apply PCA, estimate GMM parameters, and finally calculate anomaly scores based on the GMM. We evaluate our method and compare with higher-level recognition-based method. Experimental results demonstrate effectiveness of human pose for video anomaly detection.

Recently, deep learning techniques have contributed to solving a multitude of computer vision tasks. In this paper, we propose a deep-learning approach for video alignment, which involves finding the best correspondences between two overlapping videos. We formulate the video alignment task as a variant of the well-known machine comprehension (MC) task in natural language processing. While MC answers a question about a given paragraph, our technique determines the most relevant frame sequence in the context video to the query video. This is done by representing the individual frames of the two videos by highly discriminative and compact descriptors. Next, the descriptors are fed into a multi-stage network that is able, with the help of the bidirectional attention flow mechanism, to represent the context video at various granularity levels besides estimating the query-aware context part. The proposed model was trained on 10k video-pairs collected from 'YouTube'. The obtained results show that our model outperforms all known state of the art techniques by a considerable margin, confirming its efficacy.

Quadtree with nested multi-Type tree (QTMT) partition structure in Versatile Video Coding (VVC) contributes to superior encoding performance compared to the basic quad-Tree (QT) structure in High Efficiency Video Coding (HEVC). However, the improvement of performance leads to an un-Avoidable increase of computational complexity. To achieve a balance between coding efficiency and compression quality, we propose a fast intra partition algorithm based on variance and gradient to solve the rectangular partition problem in VVC. First, further splitting of smooth areas is terminated. Then, QT partition is chosen depending on the gradient features extracted by Sobel operator. Finally, one partition from five possible QTMT partitions is directly chosen by computing the variance of variance of sub-CUs. The theoretical basis of our method is that a homogeneous area tends to be predicted with a larger coding unit (CU), and sub-parts of a split CU are prone to have different textures from each other. To our knowledge, this is the first attempt to apply traditional method to accelerating the rectangular partition problem in VVC intra prediction. Experimental results show that the proposed method can save averagely 53.17% encoding time with only 1.62% BDBR increase and 0.09dB BDPSNR loss compared to anchor VTM4.0.

Recently, the demand for high quality 360-degree video delivery is increasing, however, 360-degree videos require extremely high bitrate and large network capacity. Therefore, an efficient (i.e., higher quality and lower traffic) 360-degree video delivery is mandatory. To address this fact, this paper introduces and evaluates a tile-based 360-degree video delivery system that equips viewport movement prediction and rate adaptation.

In this paper, a dual learning-based method in intra coding is introduced for PCS Grand Challenge. This method is mainly composed of two parts: intra prediction and reconstruction filtering. They use different network structures, the neural network-based intra prediction uses the full-connected network to predict the block while the neural network-based reconstruction filtering utilizes the convolutional networks. Different with the previous filtering works, we use a network with more powerful feature extraction capabilities in our reconstruction filtering network. And the filtering unit is the block-level so as to achieve a more accurate filtering compensation. To our best knowledge, among all the learning-based methods, this is the first attempt to combine two different networks in one application, and we achieve the state-of-the-art performance for AI configuration on the HEVC Test sequences. The experimental result shows that our method leads to significant BD-rate saving for provided 8 sequences compared to HM-16.20 baseline (average 10.24% and 3.57% bitrate reductions for all-intra and random-access coding, respectively). For HEVC test sequences, our model also achieved a 9.70% BD-rate saving compared to HM-16.20 baseline for allintra configuration.

In this paper, we present a road infrastructure monitoring system using e-bikes to support smart community. Smartphone and IoT (Internet of Things) devices are attached to e-bikes, and electric power is supplied from the batteries of e-bikes. Image processing techniques based on deep learning are applied and run on IoT devices or cloud backbone. A prototype system is implemented, which performs from image capturing to web browsing of the processed images.

Adaptive streaming on Named Data Networking appears to be a challenging task since the provider is unknown to the consumer. In this paper, we present an approach to efficiently adjust the streaming bit-rate by deploying IoT devices. Each NDN node is designed to send IoT devices the information of available bandwidth. The IoT devices then assist the consumers if the current network can deliver 4K video segments. The experiments using NDNx and NDN-JS show that the consumer in the proposed approach could select representation bit-rate more accurately than the traditional rate adaptation method.

Recently, Ultra HDTV (UHDTV) services become popular over satellite and on the internet. On the contrary, there are tremendously huge volume of High Definition Television (HDTV) and Standard Definition Television (SDTV) contents stored in broadcasting companies and storage devices. In this paper, we propose a color space conversion (also known as gamut mapping) method from BT. 709 (used for current HDTV broadcast) to BT. 2020 (used for UHDTV broadcast), which estimates and restores lost color information. It learns an end-to-end conversion method from BT. 709 image to BT. 2020 image with restoring lost color information using Convolutional Neural Network (CNN). By experiments, we confirm that our method can achieve 2.31dB gain against the conventional method on average.

Intelligent sensing is an important application field of video coding. For the next generation video coding standard Versatile Video Coding (VVC), several new contributions have been proposed to improve the coding efficiency, especially in the transformation operations. This paper proposes a unified transform architecture for VVC standard that enables 1D Discrete Sine Transform-VII (DST-VII) and Discrete Cosine Transform-VIII (DCT-VIII) of all sizes. In order to minimize the number of adders, the N-Dimensional Reduced Adder Graph (RAG-n) algorithm is adopted to design the logical computation of the VVC transforms. In addition, a pipeline operation is employed to achieve a high throughput rate of 32 samples per cycle. The experimental and synthesis results show that our proposal can save 16.96% area and 9.89% power consumption than the direct implementation by the multipliers. Moreover, this design can reduce the normalized area (NA) by at least 57.53% compared with other advanced works.

Abstract-Virtualized service and network functions are deployed on virtual machines (VMs) to realize essential processing to realize service function chaining (SFC). Issues on SFC is SF allocation to a VM and to minimize the response time and number of function instances. In this paper, we propose an SF clustering-based scheduling algorithm, called 'SF-clustering for utilizing virtual CPUs' (SFCUV), to solve the SF allocation and SF selection problems simultaneously. Experimental results show that SF-CUV can utilize vCPUs to minimize the response time.

To develop and interoperate smart city applications efficiently, smart city IoT platforms require efficient handling of various types of sensor devices, networking and computing resources, and different domain applications. To address this fact, the authors of this paper proposed a research project named 'Fed4IoT', which is an acronym for federation of IoT and cloud infrastructures, to provide scalable and interoperable smart city applications. The project primarily proposes two key technologies: IoT device virtualization and context-information sharing. In this paper, we introduce the IoT device virtualization (one of key technology in Fed4IoT) that enables efficient utilization of networking and computing resources by applying a micro-service sharing and dynamic resource scaling. In the performance evaluations, we implement three micro services, such as object detection, garbage detection and road damage detection, as smart city applications and dynamically and flexibly allocate computing resources to three micro services by using Docker and Kubernetes. Through the evaluations we validate that the proposed IoT device virtualization achieves the efficient computing resource utilization and reduction of networking resources.

Recently, the demand for immersive VR (360-degree) video delivery is increasing, and an efficient (high quality and low traffic) 360-degree video streaming methodology is mandatory. To address this fact, in this paper, we introduce a tile-based 360-degree DASH streaming with viewport prediction. We evaluate the prediction accuracy of future viewport movement patterns by using clustering and image and audio information. In addition, we perform rate adaptation based on viewport prediction results. Through evaluations, we confirm the tile-based 360-degree DASH streaming achieves higher objective image quality and lower total video traffic volume.

Towards an intelligent video surveillance system (high throughput and high detection accuracy), in this paper, we quantitatively validate the network performance of 60GHz IEEE 802.11ad and the contribution of 4K resolution to image-based human pose detection. In the performance evaluation of 11ad, we evaluate TCP throughputs by changing the communication distances (up-to 6m). In the performance evaluation of human pose detection, we evaluate the detection accuracy of human pose detection by changing a distance between a camera and a subject (up-to 100 m) and input image resolutions.

To realize highly reliable video surveillance and provide ultrahigh-definition/immersive video streaming, it is planned to adopt the 5G cellular system using millimeter-wave (mmWave) as the wireless-network infrastructure. However, mmWave communication has a challenging issue: mmWave communication is extremely sensitive to obstacles, such as walls, pillars, and even human bodies, and this issue easily increases the packet loss rates and round trip time (RTT) (or disconnection from the base station) due to a no line of sight (NLOS) environment. Therefore, in this work, 5G throughput performances were evaluates in an indoor train station by considering the effect of an NLOS environment caused by blockage by human bodies. In addition, to improve the robustness of TCP transmission in a high-RTT and high-packet-loss environment (e.g., an NLOS environment), a state-of-the-art TCP, TCP-FSO, was used. In the evaluations, the MATLAB 5G library was used to simulate the 5G environment, and a Linux software-based network emulator, Traffic Control, was used to emulate the 5G network. From the evaluations, it the 5G mobile throughput characteristics were confirmed in three different crowded patterns (low, middle, and high density), and the TCP-FSO advantage against CUBIC-TCP was validated.

To recognize transportation modes without any additional sensor devices, we demonstrate that the transportation modes can be recognized from communication quality factors. In the demonstration, instead of using global positioning system (GPS) and accelerometer sensors, we collect mobile TCP throughputs, received-signal strength indicators (RSSIs), and cellular base-station IDs (Cell IDs) through in-line network measurement when the user enjoys mobile services, such as video streaming. In accuracy evaluations, we conduct two different field experiments to collect the data in six typical transportation modes (static, walking, riding a bicycle, riding a bus, riding a train and riding a subway), and then construct the classifiers by applying a support-vector machine (SVM), k-nearest neighbor (k-NN), random forest (RF), and convolutional neural network (CNN). Our results show that these transportation modes can be recognized with high accuracy by using communication quality factors as well as the use of accelerometer sensors.

New video coding standards bring high coding efficiency at the cost of large computational complexity. Especially, in 8K sequence, computation cost becomes much larger due to its super high spatial resolution. In this paper, we propose a method to improve 8K UHDTV coding performance of H.265/HEVC. Our proposed method pre-analyzes spatial and motion complexity, and selects an optimal GOP size or intra/inter prediction mode. Experimental results suggest that the proposed method can improve coding efficiency and reduce computational cost.

The popularity of high dynamic range (HDR) makes the inverse tone mapping become an important technique for HDR display. In this paper, we propose a convolutional neural network (CNN)-based inverse tone mapping method to generate a high-quality HDR image from one single standard dynamic range (SDR) image. First, we present a CNN design with a three- channel input, which considers both luminance and chrominance. Second, we propose to use overlapped inputs to remove the boundary artifacts, caused by zero paddings in CNN. Experimental results demonstrate the high quality of our generated HDR images compared to the ground truth.

Recently, with the spread of Internet of Things (IoT), video surveillance system is widely used to detect abnormal activity using cameras and sensors. In such video surveillance system, contact-free heart rate measurement is a highly required technology to predict the occurrence of abnormal activities and suspicious humans. In this paper, our main contributions can be summarized by two aspects. First, we compare the accuracy performance of heart rate measurement using six heart beat waveform acquisition methods and two heart rate calculation methods. Second, we discuss the influence of image resolutions of RGB cameras on accurate performance, to investigate the requirement of surveillance cameras in actual scenarios.

To develop and interoperate smart city applications efficiently, smart city IoT platforms require efficient handling of various types of sensor devices, networking and computing resources, and different domain applications. To address this fact, in this paper, we introduce an IoT device virtualization that enables efficient utilization of computing resources. The proposal applies a micro-service sharing and dynamic resource scaling. In the performance validations, we implement an early prototype using Docker, Kubernetes, and Apache Kafka. Through the preliminary experiment, we confirm that the proposal can improve the application processing time by appropriately sharing and scaling micro services.

In video streaming on the internet, effective encoding recipes (i.e. bitrate-resolution pairs) are a main obstacle to deliver high-quality video streams. We developed a method to generate an encoding recipe that considers subjective visual quality with one just-noticeable difference (JND) distance. However, this method requires excessive computation time, which is not directly applicable for live streaming. In this paper, in order to provide a light-weight method for live streaming, we developed three acceleration techniques: resolution extrapolation, VMAF skipping and sampled objective measure calculation. These techniques are heuristic, but greatly contribute to reducing computational cost. Experimental results demonstrate that the proposed method achieves a significant reduction in computation time without significant effects on rate-JND characteristics.

Recently, ultra high definition television (UHDTV) services have become popular using satellites and the Internet. However, there are expansive volumes of high definition television (HDTV) and standard definition television (SDTV) contents held by broadcasting companies and in storage devices. Herein we propose two color space conversion (also known as gamut mapping) methods from BT.709 (used for current HDTV broadcast) to BT.2020 (used for UHDTV broadcast) that restore or estimate lost color information. One of our methods anisotropically diffuses the BT.709 chromaticities with regard to the direction of the original chromaticities in the BT.2020 color space, generating chromaticities out of BT.709 gamut. The other learns an end-to-end conversion method from a BT.709 image to a BT.2020 image and restores lost color information using convolutional neural network (CNN). Using these methods along with BT.709 images, we obtain BT.2020 images with chromaticities from the BT.709 color gamut.

Dynamic adaptive video streaming over HTTP (DASH) is widely studied and has been adopted in modern video players to ensure user quality of experience (QoE). In DASH, adaptive bitrate control is a key part whose ultimate goal is to maximize video bitrate while minimizing rebuffering. Throughput prediction plays an important role in helping select the proper video bitrate dynamically. In this paper, we studied the influence of throughput prediction on adaptive video streaming. Because the real-world network is dynamic, different methods need to be tested with large-scale deployments and analyzed statistically. However, this is difficult in academic research. Therefore, we established a reproducible trace-based emulation environment, which enables us to compare different methods quantitatively under the artificially same condition, with limited experiments. The throughput prediction methods are implemented into DASH to evaluate the effect on QoE for video streaming. The results indicate that the prediction method using long short-term memory (LSTM) performs better than the other methods. However, throughput prediction alone is not enough to ensure high QoE. To further improve the QoE, we proposed the decision map method (DMM), where the buffer occupancy is also incorporated to make a selection. By using this decision map, the choice of bitrate can be smarter than that when only prediction information is used. The total QoE is further improved by 32.1% in the ferry trace, which shows the effectiveness of DMM in further improving the performance of throughput prediction in adaptive bitrate control.

In this paper, a novel technique is introduced to address the video alignment task which is one of the hot topics in computer vision. Specifically, we aim at finding the best possible correspondences between two overlapping videos without the restrictions imposed by previous techniques. The novelty of this work is that the video alignment problem is solved by drawing an analogy between it and the machine comprehension (MC) task in natural language processing (NLP). Simply, MC seeks to give the best answer to a question about a given paragraph. In our work, one of the two videos is considered as a query, while the other as a context. First, a pre-trained CNN is used to obtain high-level features from the frames of both the query and context videos. Then, the bidirectional attention flow mechanism; that has achieved considerable success in MC; is used to compute the query-context interactions in order to find the best mapping between the two input videos. The proposed model has been trained using 10k of collected video pairs from”YouTube”. The initial experimental results show that it is a promising solution for the video alignment task when compared to the state of the art techniques.

Named Data Networking (NDN) has been recently introduced as a new network architecture. The implementation of adaptive bit-rate video streaming on NDN is one of the most challenging tasks. The in-network caching feature of NDN could mislead a consumer to overestimate the end-to-end throughput for the next video segment. In this paper, we propose a strategy for more efficient rate adaptation by allowing Interest message adaptation in a hop-by-hop fashion. Moreover, the proactive content caching is designed to be utilized on the related router when a bottleneck on the transmission path was detected. The experiments using NDN-JS and DASH-JS demonstrate that our proposed solution provides better average stream bit-rate than the traditional rate adaptation method. Also, the link bandwidth consumption for video data transmission using the proposed scheme could be reduced efficiently.

We introduce a perceptual video quality driven video encoding solution for optimized adaptive streaming. By using multiple bitrate/resolution encoding like MPEG-DASH, video streaming services can deliver the best video stream to a client, under the conditions of the client's available bandwidth and viewing device capability. However, conventional fixed encoding recipes (i.e., resolution-bitrate pairs) suffer from many problems, such as improper resolution selection and stream redundancy. To avoid these problems, we propose a novel video coding method, which generates multiple representations with constant JustNoticeable Difference (JND) interval. For this purpose, we developed a JND scale estimator using Support Vector Regression (SVR), and designed a pre-encoder which outputs an encoding recipe with constant JND interval in an adaptive manner to input video.

Image compression has been investigated as a fundamental research topic for many decades. Recently, deep learning has achieved great success in many computer vision tasks, and is gradually being used in image compression. In this paper, we present a lossy image compression architecture, which utilizes the advantages of convolutional autoencoder (CAE) to achieve a high coding efficiency. First, we design a novel CAE architecture to replace the conventional transforms and train this CAE using a rate-distortion loss function. Second, to generate a more energy-compact representation, we utilize the principal components analysis (PCA) to rotate the feature maps produced by the CAE, and then apply the quantization and entropy coder to generate the codes. Experimental results demonstrate that our method outperforms traditional image coding algorithms, by achieving a 13.7% BD-rate decrement on the Kodak database images compared to JPEG2000. Besides, our method maintains a moderate complexity similar to JPEG2000.

Image quality assessment (IQA) is an inherent problem in the field of image processing. Recently, deep learning-based image quality assessment has attracted increased attention, owing to its high prediction accuracy. In this paper, we propose a fully-blind and fast image quality predictor (FFIQP) using convolutional neural networks including two strategies. First, we propose a distortion clustering strategy based on the distribution function of intermediate-layer results in the convolutional neural network (CNN) to make IQA fully blind. Second, by analyzing the relationship between image saliency information and CNN prediction error, we utilize a pre-saliency map to skip the non-salient patches for IQA acceleration. Experimental results verify that our method can achieve the high accuracy (0.978) with subjective quality scores, outperforming existing IQA methods. Moreover, the proposed method is highly computationally appealing, achieving flexible complexity performance by assigning different thresholds in the saliency map.

Because mobile users demand a high quality and energyfriendly video delivery service that efficiently uses wireless resources, we introduce an energy-efficient video delivery system by applying moving route navigation and playout buffer control based on the mobile throughput history data. The proposed system first determines the optimal travel route to achieve high-speed and energy-efficient communications. Then when a user enters a high throughput area, our system temporarily extends the video playout buffer size, and the user aggressively downloads video segments via a high-speed and energy-efficient wireless connection until the extended buffer is filled. After leaving this area, the user consumes video segments from the extended buffer in order to keep smooth video playback without wireless communications. We carry out computer simulations, laboratory and field experiments and confirm that the proposed system can achieve energy-efficient mobile video delivery.

Throughput prediction is one of the promising techniques to improve the quality of service (QoS) and quality of experience (QoE) of mobile applications. To address the problem of predicting future throughput distribution accurately during the whole session, which can exhibit large throughput fluctuations in different scenarios (especially scenarios of moving user), we propose a history-based throughput prediction method that utilizes time series analysis and machine learning techniques for mobile network communication. This method is called the Hybrid Prediction with the Autoregressive Model and Hidden Markov Model (HOAH). Different from existing methods, HOAH uses Support Vector Machine (SVM) to classify the throughput transition into two classes, and predicts the transmission control protocol (TCP) throughput by switching between the Autoregressive Model (AR Model) and the Gaussian Mixture Model-Hidden Markov Model (GMM-HMM). We conduct field experiments to evaluate the proposed method in seven different scenarios. The results show that HOAH can predict future throughput effectively and decreases the prediction error by a maximum of 55.95% compared with other methods.

Image compression has been investigated for many decades. Recently, deep<br />
learning approaches have achieved a great success in many computer vision<br />
tasks, and are gradually used in image compression. In this paper, we develop<br />
three overall compression architectures based on convolutional autoencoders<br />
(CAEs), generative adversarial networks (GANs) as well as super-resolution<br />
(SR), and present a comprehensive performance comparison. According to<br />
experimental results, CAEs achieve better coding efficiency than JPEG by<br />
extracting compact features. GANs show potential advantages on large<br />
compression ratio and high subjective quality reconstruction. Super-resolution<br />
achieves the best rate-distortion (RD) performance among them, which is<br />
comparable to BPG.

In recent years, the system using acoustic communication is increasing. However, because acoustic communication uses low frequency, transmission rate is lower than radio wave communication. In wireless communication, MIMO-OFDM is proposed for improvement quality and transmission rate. In this paper, we introduce a software-defined acoustic communication platform by using MATLAB and implement acoustic MIMO-OFDM transmission into the platform. Also, we evaluate BER characteristics in various experimental parameters in MATLAB simulation and real environment. Moreover, we evaluate image quality in actual acoustic image transmission by using the acoustic communication platform and we can successfully transmit the image via acoustic MIMO-OFDM.

Image compression has been investigated as a fundamental research topic for<br />
many decades. Recently, deep learning has achieved great success in many<br />
computer vision tasks, and is gradually being used in image compression. In<br />
this paper, we present a lossy image compression architecture, which utilizes<br />
the advantages of convolutional autoencoder (CAE) to achieve a high coding<br />
efficiency. First, we design a novel CAE architecture to replace the<br />
conventional transforms and train this CAE using a rate-distortion loss<br />
function. Second, to generate a more energy-compact representation, we utilize<br />
the principal components analysis (PCA) to rotate the feature maps produced by<br />
the CAE, and then apply the quantization and entropy coder to generate the<br />
codes. Experimental results demonstrate that our method outperforms traditional<br />
image coding algorithms, by achieving a 13.7% BD-rate decrement on the Kodak<br />
database images compared to JPEG2000. Besides, our method maintains a moderate<br />
complexity similar to JPEG2000.

This paper proposes a transmission control protocol (TCP) for long distance high-speed wireless communications, including free-space optical communications (FSOC). Extreme high frequency of wireless com- munications enables high-speed bit rate, but frequent signal error, including burst error, can be a quite severe problem for ordinary high-speed TCPs. To achieve 10 Gbps or higher data transfer throughput on FSOC, the proposed TCP (designated "TCP-FSO") has improved and new features including multi-layer congestion control, retransmission control with packet loss point estimation, delay-based ACK congestion control, and ACK retransmission control. We evaluated data transfer throughput of TCP-FSO and the other TCPs, by throughput model analysis and experiment on real implementa-Tion. Obtained results show that TCP-FSO achieves far higher data transfer throughput than other high-speed TCPs. For example, it achieved a thou- sand times higher throughput than the other high-speed TCPs in a real FSOC environment.

In this paper, we consider a color space conversion method from BT.709 used for current HDTV (High Definition Television) broadcast to BT.2020 that will be used for UHDTV (Ultra HDTV) broadcast, with lost color information restoration. Our method anisotropically diffuses BT.709 chromaticities considering the direction to the original chromaticities in BT.2020 color space. With this method and BT.709 images, we obtained BT.2020 images that had chromaticities out of BT.709 color gamut.

For 8K UHDTV video, which has high spatial resolution, CU size has large influence to encoding efficiency. In addition, a moving object with random motion has also a large impact on encoding efficiency of inter prediction. In this paper, we therefore propose a method to improve H.265/HEVC encoding efficiency by CU size expansion and inter/intra prediction mode selection. Moreover, we evaluated encoding efficiency and complexity by chancing max CU sizes and inter/intra prediction modes. From these evaluation results, the proposed method achieved high efficiency and low computational cost.

As mobile multimedia services have significantly evolved and diversified with the spread of smartphones and Internet of Things (IoT) devices, low-delay multimedia cloud computing is the need of the hour. To address this demand, in this study, we introduce an edge cloud system that equips a multimedia service function chaining capability. A prototype implementation of the proposed edge cloud system has three main features: 1) edge computing deployment by using OpenStack, 2) multimedia service slicing and chaining, and 3) efficient resource management in edge networks. Based on these features, the proposed system achieves lower multimedia processing delay compared to a conventional cloud computing platform. We deploy the proposed system in our laboratory and validate the system performance by using typical multimedia application, such as human detection in video surveillance.

To provide energy-efficient (i.e., longer lifetime of sensors) and network-friendly (i.e., reducing network traffic) field sensing, we propose an edge-centric field monitoring system which applies efficient sensors and camera control. The proposed system detects conditions in a monitoring area and controls sensing frequency (sampling rate) of sensors, and capture rate and encoding rate of surveillance cameras, according to the detected conditions. In addition, the system applies a Multi-access Edge Computing (MEC) platform to provide fast feedback control to the sensors and cameras. In performance evaluations, we assume that the monitoring target is landslide detection and create a miniature 'artificial landslide generation' environment in our laboratory. By using the environment, we evaluate the system performance, and evaluation results indicate that the proposed system can reduce network traffic and save energy consumption efficiently.

To reduce the backbone video traffic generated by video surveillance, we propose an intelligent video surveillance system that offers multi-modal sensor-based event detection and event-driven video rate adaptation. Our proposed system can detect pedestrian existence and movements in the monitoring area by using multi-modal sensors (camera, laser scanner and infrared distance sensor) and control surveillance video quality according to the detected events. We evaluate event detection accuracy and video traffic volume in the experiment scenarios where up to six pedestrians pass through and/or stop at the monitoring area. Evaluation results conclude that our system can significantly reduce video traffic while ensuring high-quality surveillance.

Recently, to provide a low-latency mobile computing platform, Mobile Edge Computing (MEC) is proposed. In this paper, we first summarized the feature capabilities of MEC, such as content distribution and caching, computational offloading and multimedia Internet of Things (IoT). Then, to understand recent research efforts of multimedia MEC, we briefly highlight the research efforts in terms of above three capabilities: how to achieve edge caching in video distribution, how to schedule computational offloading to the cloud and how the communication quality degradation affects to the user experience of multimedia IoT. Finally, we addressed the emerging research issues of multimedia MEC to improve reliability and robustness of multimedia MEC.

Throughput prediction is one of good solutions to improve quality of mobile applications (e.g., YouTube or Netflix) for video streaming delivery services in mobile networks. This is because such applications require monitoring the network performances to control content quality, thus guarantee quality of service (QoS) and quality of experience (QoE). In this paper, we propose a history-based TCP throughput prediction method incorporating communication quality features using SVR (Support Vector Regression). By taking history of communication quality features such as historical throughput and Received Signal Strength Indication (RSSI) into consideration, the throughput prediction error can be decreased. We conduct experiments with the proposed method and compare the prediction accuracy with a variety of methods in different scenarios of various moving modes of users. Results show that the proposed model could predict throughput effectively in various scenarios and decrease throughput prediction errors by a maximum of 26.47% compared with other methods.

The increasing demand of mobile applications has brought large amount of mobile traffic. To meet users' requirements for high-quality video delivery, it is an urgent task to provide fair-quality video delivery for various users and situations. In this paper, we evaluate QoS and QoE characteristics and validate QoE unfriendliness in heterogeneous DASH contents distributions to provide QoE-fair video delivery,. In the evaluations, we employ multiple contents with different resolutions, frame rates, and rate-distortion characteristics. By using heterogeneous DASH contents, we evaluate the effect of playout buffer size on QoS and QoE performances. Evaluation results show that smaller playout buffer size contributes to QoE improvement under network congestion. In addition, we confirm that suppression of playback stall is a particularly important factor to achieve QoE fairness compared to other QoS metrics such as initial delay and representation.

In recent years, various approaches have been investigated towards blind image quality assessment (IQA) with high accuracy and low complexity. In this paper we develop a pre-saliency map based blind IQA method, which takes advantage of saliency information in prior of quality prediction for performance enhancement by two steps. 1) We split the image into patches and design a convolution neural network (CNN) to predict the patch-wise quality score. Then we explore the relation between image saliency information and CNN prediction error to present a statistical analysis. 2) Based on the analysis, we propose a patch quality aggregation algorithm by removing non-salient patches which are likely to bring large prediction error and assigning large weights for salient patches. Experimental results validate that our method can achieve high accuracy (0.978) with subjective quality scores, which outperforms existing IQA methods. Meanwhile, the proposed method can reduce 52.7% computational time than the IQA without pre-saliency map.

In this paper, we performed human moving pattern recognition using communication quality: cellular download throughputs, Received Signal Strength Indicators (RSSIs) and cellular base station IDs. We apply three machine learning algorithms, such as Support Vector Machine (SVM), K-Nearest Neighbor (KNN) and Random Forest (RF) and evaluate recognition accuracy of human moving patterns. Results conclude that the communication quality can recognize moving patterns with high accuracy.

In this paper, to provide high-quality and high-visibility surveillance, we introduce video surveillance system equipped image enhancement system. Because quality of enhanced image heavily depends on encoding characteristic, we evaluate the subjective and objective image quality of image enhancement under various video encoding rates by using night and day sequences.

In this paper, we evaluate QoS (Quality of Service) and QoE (Quality of Experience) characteristics of 2K and 4K video distribution using MPEG-DASH (Dynamic Adaptive Streaming over HTTP). Evaluation results conclude that smaller playout buffer size contributes to QoE improvement.

Because mobile users demand high quality and energy-friendly video delivery service that efficiently uses wireless resources, we introduce a green video delivery system using moving route navigation and playout buffer control based on the mobile throughput history data. Our system first determines the optimal travel route to realize high-speed and energy-efficient communications. Then when a user enters a high throughput area, our system temporarily extends the video playout buffer size, and if allowed, the user remains in the high throughput area and aggressively downloads video segments until filling the extended buffer via a high-speed and energyefficient wireless connection. After leaving this area, the user consumes video segments from the extended buffer in order to keep smooth video playback without wireless communications. Computer simulations along with laboratory and field experiments on the system performance confirm that our system can achieve green mobile video delivery.

In this study, we propose a broadcasting and communication integrated platform that can retransmit 8K broadcasting contents from a TV receiving set to mobile devices. We then evaluate the QoS characteristics and the qualities of 8K contents in this platform. Evaluation results conclude that 8K broadcasting contents can be retransmitted to mobile devices smoothly and their qualities are nearly as high as those of assumed 8K broadcasting service in Japan.

Nowadays, researches on accident prevention using train-mounted cameras had been progressing. Our proposed method considers temporal continuity between frames by using motion vectors in addition to conventional thresholding on similarity values obtained by a human detection method using HOG features. Experiments show the effectiveness of our method as compared with a previous method using only HOG features.

In order to analyze system latency of edge computing for multimedia data processing, we propose a model of system latency of two different distributed processing scenarios: pipeline and parallel processing. We validate the model by using actual cloud services and human detection processing.

Urban surveillance systems are being applied in a rapid pace with mature but inefficient solutions. The inefficiency is revealed with two aspects, too concentrated bandwidth and processing requirement. To solve this problem, we proposed a content oriented surveillance system based on Information-Centric Network. However, we can't simply replace TCP/IP streaming structure with named contents streaming structure because it can't improve the surveillance system's efficiency enough. In this paper, we took the ICN network's profits even further with the named contents. Instead of streaming live video to the central data center and processing multiple data stream in the same time, we have designed the nodes to process the captured raw data and produce objective contents for the central data center. With the extremely size difference in raw data and actual valued contents from it, we could apply the method in investigating area people traffic conditions and even in disaster and anti-terrorism scenarios. There was a field experiment performed to evaluate tourists' densities and dressing habits during winter season of March. The experiment expressed the benefits of our system and compared our method with traditional surveillance systems in saving network bandwidth and functionalities.

異なる視点から撮影した画像をもとに3次元情報を得ることは，コンピュータビジョンの重要なテーマの一つである．これまで多くのステレオ法が研究され，二眼・三眼・マルチベースラインの各ステレオや移動カメラのシステムへ適用されてきた．一般的に奥行き推定精度を上げるには，カメラ間距離を大きくするべきである．しかし距離を大きくすることは対応点探索を困難にする．現実には撮影対象となる物体は完全散乱反射特性ではないなど視点依存の輝度変動があり，視点間距離が大きくなるほど影響が大きい．この論文では，視点依存の輝度変動に頑健なマルチベースラインステレオ法を提案する．提案法では近傍カメラ間比較を総和する新しいコスト関数を導入する．最初に完全散乱仮定の下で，同一のベースライン長をもつ二眼ステレオと同等な奥行き推定精度であることを理論的に証明する．次に視点依存の輝度変動をもつ実際の画像セットを用いることにより，二眼ステレオだけでなく従来のマルチベースラインステレオと同等あるいは優れた精度をもつことを実験的に示す．

In recent years, video sequence with a depth map can be taken by a camera with RGB-D (depth) sensor like Kinect. On the other hand, quality degradation at occluded regions is inevitable when conventional 2-D video coding is applied. In this paper, we propose usage of an occluded region detection supported by the depth map for improvement of 2-D image coding quality. Experiments show that further improvement of picture quality at the whole of frame and occluded regions is addressed for H.265/HEVC compressed streams.

Image super-resolution quality is degraded by the noise component of an image. However the noise should not be eliminated because of its importance in high-definition impression for digital cinema. We therefore propose an image super-resolution method using the synthesis of the super-resolution signal and noise components respectively after dividing an original image into signal and noise components. The signal component's resolution is enhanced using registration between the signal component and its multi-scale Wavelet components with resolution-enhanced parameter optimization.

Applications and malware affecting them are dramatically changing. It isn't certain whether the currently used features can classify normal traffic or malware traffic correctly. In this paper, we evaluated the features used in previous studies while taking into account secular changes to classify normal traffic into the normal category and anomalous traffic into the anomalous category correctly. A secular change in this study is a difference in a feature between the date the training data were caputred and the date the test data were captured in the same circumstance. The evaluation is based on the Euclidean distance between the normal codebook or anomalous codebook made by vector quantization and the test data. We report on what causes these secular changes and which features with little or no secular change are effective for malware detection. © 2013 Springer-Verlag Berlin Heidelberg.

Application-layer multicast has been considered as a promising method for streaming live media to users in P2P networks. However, the inefficient bandwidth utilization and the vulnerability to peer dynamics remain important issues. In this paper, we introduce a new approach to constructing an efficient overlay leveraging the strong features of tree and mesh methods. A core overlay is built in a fashion of multiple sub-trees, which allows each peer to receive data from other peers located in any different sub-trees. We propose a set of auxiliary connections, which connect between two nodes that do not have permanent parent/child relationship, to achieve a push-based data transmission from multiple parents without redundancy. Simulations through ns2 demonstrate that our scheme provides a good streaming quality with low end-to-end delay. Also, the proposed auxiliary links is beneficial in improving the resiliency of the overlay and utilizing the bandwidth of peers.

The use of mobile terminals has been expanding dramatically in recent years as they evolve from a means of dispatching and gathering information to a highly functional tool that supports personal lifestyles and behavior. A mobile terminal is likely to store various kinds of personal information such as a calendar and contact information as well as key data to carry out online transactions. Losing one's mobile terminal therefore creates the possibility that one's personal information may fall into the wrong hands and be used for malicious purposes.
We therefore propose a method of personal authentication using sensor data in a mobile terminal. First, we applied the LPC cepstrum to this authentication and checked for validity. We also evaluated the effectiveness of gait authentication using several frames.

We propose a novel image super-resolution method from digital cinema to 8K ultra high-definition television using registration of wavelet multi-scale components with affine transformation. The proposed method features that an original image is divided into signal and noise components by the wavelet soft-shrinkage with detection of white noise level. The signal component enhances resolution by registration between a signal component and its wavelet multi-scale components with affine transformation and parameters optimization. The affine transformation enhances super-resolution image quality because it increases registration candidates. The noise component enhances resolution with power control considering cinema noise representation. Super-resolution image outputs by synthesis of super-resolved signal and noise components. Experiments show that the proposed method has objectively better PSNR measurement and subjectively better appearance in comparison with conventional super-resolution methods. © 2013 IEEE.

In this paper, an image noise in the digital cinema is analyzed at first. Based on this analysis, we propose the image super-resolution method from digital cinema to ultrahigh definition television. The proposed method separates the digital cinema into signal and noise components by the spatio-temporal Wavelet analysis, super-resolves these components respectively and synthesize them.

In this paper, we propose a system which automatically generates slideshows for music, by utilizing images retrieved from photo sharing web sites, based on query words extracted from song lyrics. The proposed system consists of two major steps: (1) query extraction from song lyrics, (2) image selection from web image search results. Moreover, in order to improve the display duration of each image in the slideshow, we adjust image transition timing by analyzing the duration of each lyric line in the input song. We have conducted subjective evaluation experiments, which prove that the proposal can generate impressive music slideshows for any input song. © 2010 International Society for Music Information Retrieval.

圧縮動画像に対して原画像を利用しないNo Reference型PSNR/SSIM推定手法を提案する。提案手法では、原画像を参照せずに符号化ストリームから推定可能な統計量を用いたSSIMの近似式を導出する。推定する統計量は量子化誤差分散と信号分散の減少量であり、前者は既存のPSNR推定に使用され、後者はSSIM推定特有のものである。また実画像を用いた実験として、SSIM推定の基礎的な性能評価と、H.264/AVC圧縮ストリームに対するNR型PSNR/SSIM推定の評価を行った。前者の結果として、フレーム単位の推定を行うとSSIM推定値が真値から外れる点、後者の結果として、イントラフレームにおいては良好な推定が行えるが、インターフレームでは改善の余地が見られる点を示した。

本稿では量子化制約条件を考慮した複数フレームの合成によりPSNRを改善する手法を提案する。複数フレームの合成の際には同一領域を合成する必要があり、動き検出の性能が全体の性能の向上に繋がると考えられる。そこで動き検出において、いくつかの改善手法を示す。また、反復処理における問題点についても述べた後、改善手法として量子化の制約条件を外れた場合に行うDCT係数の補正処理を制限する手法について説明し、それぞれシミュレーション実験によりその有用性を示す。

This research presents a formal user evaluation of a typical visualization method for content-based music information retrieval (MIR) systems, and also proposes a novel interface to improve MIR usability. Numerous interfaces to visualize content-based MIR systems have been proposed, but reports on user evaluations of such proposed GUIs are scarce. This research aims to evaluate the effectiveness of a typical 2-D visualization method for content-based MIR systems, by conducting comparative user evaluations against the traditional list-based format to present MIR results to the user. Based on the observations of the experimental results, we next propose a 3-D visualization system, which features a function to specify sub-regions of the feature space based on genre classification results, and a function which allows users to select features that are assigned to the axes of the 3-D space. Evaluation of this GUI conclude that the functions of the 3-D system can significantly improve both the efficiency and usability of MIR systems. © 2009 International Society for Music Information Retrieval.

In this paper, we introduce a new mesh-based approach for data delivery which is organized over multiple tree-shaped core routes. Given that both tree and mesh approaches have their own strong points, we simply combine them together. We evaluate the proposal through ns-2 simulator. The simulation results demonstrate that our approach can provide higher average received quality and has acceptable data delivery delay when compared with a single tree method. We also show that, over a static overlay, the push-based data delivery on mesh can provide the received quality close to pull-based data delivery method with less latency. As well, it has lower control overhead than the pull-based method when the peer number is large.

Recently it has been shown that scalable (layered) image is appropriate for the Internet because of its better flexibility and functionality. At the same time, the pre-fetching technology also proves to be an efficient method to improve the performance of image-delivery with the low delay and adjustable quality. However, how to deploy the pre-fetching to deliver scalable contents has not been mentioned. Therefore, this paper presents an analysis of the pre-fetching algorithm to deliver layered image contents over the Internet. Based on the analysis of web request probability and the relations between different layers in the scalable imaging delivering system, a new pre-fetching algorithm is proposed to drastically improve cache performance. This proposal is a request frequency-based approach, in which a progressive image format can be used.

In order to deliver multimedia contents to the end-users with low delay and acceptable quality, the pre-fetching technologies have been shown the efficiency if we can predict user&apos;s future requests and prefetch the data needed previously. However, because both the network topology and Web access are changed dynamically, whether the pre-fetching can get the stable performance under dynamic environment becomes an important issue. Therefore, this paper talks about the robustness of the pre-fetching algorithm, which has been proposed by us to deliver layered image contents over the Internet. Based on the web access distribution, the request pattern in the scalable imaging delivering system is analyzed in this paper. Furthermore, we check the robustness of our proposal by changing related parameters. Simulation results show that Our proposal is stable against variable conditions even if access patterns are dynamically changed.

The total amount of information to process or analyze is jumping sharply with the quick spread of computers and networks. Our project, «Highly scalable monitoring architecture for information explosion», develops a monitoring system allows observing systems, merging the system logs, and discovering intelligence to share. More concretely, the project builds the total system to maintain, optimize, and protect autonomically. This paper reports the outcomes of the project after first-half of the development period.The rest of the paper is organized as follows. Section 2 describes the concept and details of the monitoring system on a single node, and Section 3 addresses the aggregation of the collected information in distributed environments. Section 4 and Section 5 introduce applications of the monitoring systems. Section 6 summarizes the project and mentions future plans. © 2009 IEEE.

In this paper, we introduce a new mesh-based approach for data delivery which is organized over multiple tree-shaped core routes. Given that both tree and mesh approaches for multicast overlay construction have their own strong points, we simply combine them. During session, the mesh connections are formed dynamically to facilitate the data delivery on tree-shaped core routes which may experience bandwidth bottlenecks. Unlike existing mesh-based approaches, our proposal employs push-based data delivery along mesh connection. A well-organized mesh construction scheme is defined to avoid data redundancies. We evaluated the proposal through ns-2 simulator. The simulation results demonstrate that our approach can provide higher average received quality and has acceptable control overhead when compared to a single tree method. We also show that, over a static overlay, the push-based data delivery on mesh can provide the received quality close to supposed pull-based data delivery method with less latency. Moreover, it has lower control overhead than the pull-based method when the peer number is large.

パノラマ画像とは広範囲なシーンが撮影された視野の広い画像のことであるが、このパノラマ画像を取得する手法として、高精度にキャリブレーションされたマルチカメラシステムを利用する方法、複数の画像を繋ぎ合わせる方法、曲面ミラーや広角レンズを用いる方式、など様々な手法が研究されている。しかし、これらの方法はそれぞれ問題点もある。マルチカメラシステムを利用する場合は、カメラ間での高い精度でのキャリブレーションが必要になり、システムが複雑になってしまう。複数の画像を繋ぎ合わせる場合は、通常のカメラを使用できるが、画像間である程度重なりとなる領域が必要になることや、撮影時の日照条件や露光の変化による画像間の輝度、色彩などの違いによって、画像間の繋ぎ目に違和感が生じてしまうという問題がある。一方、曲面ミラーや広角レンズを用いる場合は、手軽に直接パノラマ画像を取得できるが、視線方向での解像度の不均一性や、全体的な解像度の低さが問題となる。そこで本研究では、双曲面ミラーを利用して取得した全方位画像と、通常画像をハイブリッド合成することで、高解像度で、輝度、色彩が補正されたパノラマ画像の生成を行う。また、同時に全方位画像と通常画像の対応付け手法に関する検討も行う。

自動採譜処理において複数音高推定は長年研究されているテーマであるが、現在のところ実用的なシステムはほとんど出来ていない。計算機による自動採譜が難しい理由は、単に計算量だけの問題でなく、計算機の性能向上によって解決されるものではない。そこで、採譜作業を全て計算機に委ねずにユーザと協調して採譜を行う採譜支援システムを提案する。本稿では、ユーザが特定の和音中の最高音を入力することで、その基本周波数以上の周波数帯域をカットする方式を採る。また、コード進行の音楽知識を用いて高精度の推定を実現する。さらにMIDI音源と実音源に対して実際にシステムを構築し、その有効性を確認した。

本稿ではMotion JPEGにおいて、動き検出を用いたフレーム間での合成に加え、量子化制約条件を考慮した符号化ひずみ低減方法を提案する。従来手法である動き検出を用いた画像の合成と量子化制約条件を利用したDCT係数の補正について述べた後、提案手法について説明する。また、シミュレーションによる実験を行い、提案手法の性能を従来手法と比較することによって評価する。

本稿では,静止画において画素シフトした複数の画像を合成することで画質が改善されることを応用し,動画像に拡張をする.イントラ符号化された動画像では近傍フレームを参照することで画質の改善が期待できる.また,複数の符号化方式を混在させることにより動き補償による画質改善の効果が得られない領域であっても画質を改善させることができる.

HDTVの普及により,映像を高解像・大画面で視聴する機会が増えている.画像の拡大手法としてBicubic法が広く用いられるが,画像にボケが発生する.本稿ではIM-GPDCTと呼ばれる超解像技術を圧縮動画像へ適用する方法を提案した.まず,IM-GPDCTにおける拡張領域パディング方法が高解像度化に与える影響について検討した.次に,拡大法により周波数分布が異なるという特性をIM-GPDCTのアルゴリズムの中で利用することで性能改善を行った.また従来のIM-GPDCTで発生するリンギングの発生を抑制した.さらにIM-GPDCTを圧縮動画へ適用し,複数フレームと動き検出を用いた画質改善を行った.

Content Delivery Overlays improves end-user performance by replicating Web contents on a group of geographically distributed sites interconnected over the Internet. However, with the development whereby overlay systems can manage dynamically changing files, an important issue to be resolved is consistency management, which means the cached replicas on different sites must be updated if the originals change. In this paper, based on the analytical formulation of object freshness, web access distribution and network topology, we derive a novel algorithm as follows: (1) For a given content which has been changed on its original server, only a limited number of its replicas instead of all replicas are updated. (2) After a replica has been selected for update, the latest version will be sent from an algorithm-decided site instead of from its original server. Simulation results verify that the proposed algorithm provides much better consistency management than conventional methods with the reduced the old hit ratio and network traffic.

The purpose of this study is to maintain efficient backup routes for reconstructing overlay trees quickly. In most conventional methods, after a node leaves the trees, its child nodes start searching for the new parents. In this reactive approach, it takes a lot of time to find a new parent. In this paper, we propose a proactive approach to finding a next parent as the backup route node over the overlay tree before the current parent leaves. A proactive approach allows a node to find its new parent node immediately and switch to the backup route node smoothly. In our proposal, the structure of the overlay tree using a redundant degree can decide a backup route node without so much overhead. Simulations demonstrate our proactive approach can recover from node departures 2 times faster than reactive approaches, and can construct overlay trees with lower overheads than another proactive method. Additionally we carried out experiments over actual networks and their results support the effectiveness of our approach. We confirmed that our proposal achieved better streaming quality than conventional approaches.

This paper proposes a feature space modification method for feature extraction of music, which is effective for the development of a content-based music information retrieval (MIR) system based on user preferences. The proposed method conducts clustering of all songs in the music collection, and utilizes the resulting cluster IDs as training data for feature space modification. and is capable to automatically generate a feature space which is suitable to the content of any music collection. Experiment results prove that the proposed method improves accuracy of user preference based MIR.

Content Distribution Overlays improves end-user performance by replicating Web contents on a group of geographically distributed sites interconnected over the Internet. However, with the development whereby overlay systems can manage dynamically [2] changing files, an important issue to be resolved is consistency management, which means the cached replicas on different sites must be updated if the originals change. In this paper, based on the analytical formulation of object freshness time, web access distribution and network topology, we derive a novel algorithm as follows: (1) For a given content which has been changed at its original server, only a limited number of its replicas instead of all replicas are updated. (2) After a replica has been selected for update, the latest version will be sent from an algorithm-decided site instead of from its original server. Simulation results verify that the proposed algorithm provides much better consistency management than conventional methods with the reduced update overhead and network traffic.

This paper proposes a feature space modification method for feature extraction of music, which is effective for the development of a content-based music information retrieval (MIR) system based on user preferences. The proposed method conducts clustering of all songs in the music collection, and utilizes the resulting cluster IDs as training data for feature space modification, and is capable to automatically generate a feature space which is suitable to the content of any music collection. Experiment results prove that the proposed method improves accuracy of user preference based MIR.

The purpose of this study is to maintain efficient backup routes for restoring overlay trees. In most conventional methods, after a node leaves the trees, its children start searching for a new parent. In this reactive approach, it takes a lot of time to find a new parent. In this paper, we propose a proactive approach to find a new parent over the overlay trees before the current parent leaves. A proactive approach can find respective new parents immediately and switch to the backup route smoothly. In our proposal, the structure of the overlay tree using a redundant degree enables to decide a new parent without so much overhead information. Simulations demonstrate our proactive approach can recover from node departures 2 times faster than reactive approaches, and can construct overlay trees with lower overheads than another proactive method. Additionally we carried out experiments over actual networks and their results support the effectiveness of our approach. We confirmed that our proposal achieved better streaming quality than conventional approaches.

This paper proposes a novel on-demand multipath routing protocol for a mobile ad hoe network. By applying a newly developed route update procedure with combined metrics of delay, hop count and disjointness, each intermediate node deliberately selects multipath candidates while contributing to suppression of unnecessary routing packets. Extension of RREQ/RREP packets with a source route list is also incorporated, not only to alleviate limitation of the hop-count based approaches but rather to provide more efficient multiple routes. Computer simulations using ns-2 simulator are performed with comparison to conventional methods and effectiveness of the proposed methods is quantitatively validated.

CDN (Content Delivery Networks) improves end-user performance by replicating web contents on a group of geographically distributed servers. However, repeatedly keeping the entire replica of the original objects into many content servers consumes too much server resource. This problem becomes more serious for the large-sized objects such as streaming media, e.g. high quality video. In this paper, we therefore propose an efficient replication method for layered video streams in CDN, which can reduce user response delays and storage costs simultaneously. Based on an analytical formulation of the cooperative replication of layers and segments of each video stream, we derive a replication algorithm which solves next three problems quantitatively. (1) How many servers should be selected to replicate a given video stream? (2) For a single video stream, how many layers and segments should be stored in a given server? (3) After selecting a group of servers for each video stream, how do we allocate the replication priority (i.e. order) to each server? Simulation results verify that the proposed algorithm efficiently resolves the above problems and provides much better performance than conventional methods.

CDN (Content Delivery Networks) improves end-user performance by replicating web contents on a group of geographically distributed servers. However, repeatedly keeping the entire replica of the original objects into many content servers consumes too much server resource. This problem becomes more serious for the large-sized objects such as streaming media, e.g. high quality video. In this paper, we therefore propose an efficient replication method for layered video streams in CDN, which can reduce user response delays and storage costs simultaneously. Based on an analytical formulation of the cooperative replication of layers and segments of each video stream, we derive a replication algorithm which solves next three problems quantitatively. (1) How many servers should be selected to replicate a given video stream? (2) For a single video stream, how many layers and segments should be stored in a given server? (3) After selecting a group of servers for each video stream, how do we allocate the replication priority (i.e. order) to each server? Simulation results verify that the proposed algorithm efficiently resolves the above problems and provides much better performance than conventional methods.

TCP is a popular internet protocol for reliable end-to-end data delivery, but it cannot be directly applied to wireless networks in which packet loss may be induced by higher BER or handover than congestion. TCP assumes that such packet loss is caused by network congestion and initiates congestion control procedures. In this paper, we present a novel protocol using Explicit Handover Notification to improve TCP performance over wireless links. Additionally, we execute computer simulations using network simulator and compare with other various protocols.

This paper presents a method of gender classification by integrating facial, hairstyle, and clothing images. Initially, input images are separated into facial, hairstyle and clothing regions, and independently learned PCAs and GMMs based on thousands of sample images are applied to each region. The classification results are then integrated into a single score using some known priors based on the Bayes rule. Experimental results showed that our integration strategy significantly reduced error rate in gender classification compared with the conventional facial only approach.

In this paper, we propose a motion vector search algorithm that employs the Sum of Absolute Difference (SAD) minimization in DCT domain as a distortion measure. In addition, block based motion estimation is carried out with multiple modes in this frequency domain, and a mode is selected according to the Lagrangian cost function [1, 2, 3]. Four modes are designed. The first two modes are a mode with 1 motion vector (INTER1V) and a mode with 4 motion vectors (INTER4V) per macroblock that are similar to the H.263 [4]. The last two modes use 4 motion vectors per macroblock (INTER1V+) or per block (INTER4V+), where the 8×8 DCT coefficients are firstly separated into 4 groups and a motion vector is independently assigned to each group. The average SNR (Signal to Noise Ratio) gains between 0.3 and 1.0dB are obtained for tested sequences when compared with TMN. Our proposal method provides true motion vectors for low frequency and minimizes distortion for higher frequency. Therefore our proposed method also reduces subjective visual degradation.

CDN (Content Delivery Networks) improves end-user performance by replicating web contents on a group of geographically distributed content servers. Replication Algorithm plays an important role in helping users to retrieve Web objects from the content servers. If a user can directly get the requested objects from the content server, he need not to contact the remote origin server and the user delay can be reduced. However, current replica strategies in CDN are to simply and repeatedly keep the complete replica of the original object on many content servers. This method has some disadvantages, including too much consumed server space and a waste of the storage cost. It is more serious for replicating some large-sized objects such as streaming media, which are being distributed over the Internet more and more. In this paper, we discuss a replication strategy for scalable video streaming in CDN to reduce user response and storage cost. Based on theoretical analysis, assuming layered video coding, we propose a novel replication algorithm which deals with following three problems. (1) How many content servers should be selected to replicate a given video content? (2) For a single video content, how many layers should be kept in a given content server? (3) After selecting a group of content servers for each video content, how do we decide the replication priority for each content server? Simulation results show that the proposed algorithm can efficiently resolve the above problems, and provide much better performance than the conventional methods.

With the advance of high-speed network technologies, availability and popularity of streaming media contents over the Internet has grown rapidly in recent years. Because of their distinct statistical properties and user viewing patterns, traditional delivery and caching schemes for normal web objects such as HTML files or images can not be efficiently applied to streaming media such as audio and video. In this paper, we therefore propose an integrated caching scheme for streaming media with segment-based caching and hierarchically distributed proxies. Firstly, each stream is divided into segments and their caching algorithms are considered to determine how to distribute the segments into different level proxies efficiently. Secondly, by introducing two kinds of segment priorities, segment replacing algorithms are proposed to determine which stream and which segments should be replaced when the cache is full. Finally, a Web-friendly caching scheme is proposed to integrate the streaming caching with the conventional caching of normal web objects. Performance of the proposed algorithms is verified by carrying out simulations.

The efficient distribution of stored information has become a major concern in the Internet. Since the web workload characteristics show that more than 60% of network traffic is caused by image documents, how to efficiently distribute image documents from servers to end clients is an important issue. Proxy cache is an efficient solution to reduce network traffic. And it has been shown that an image caching method (Graceful Caching) based on hierarchical coding format performs better than conventional caching schemes in recent years. However, as the capacity of the cache is limited, how to efficiently allocate the cache memory to achieve a minimum expected delay time is still a problem to be resolved. This paper presents an integrated caching algorithm to deal with the above problem for image databases, web browsers, proxies and other similar applications in the Internet. By analyzing the web request distribution of the Graceful Caching, both replacing and pre-fetching algorithms are proposed. We also show that our proposal can be carried out based on information readily available in the proxy server; it flexibly adapts its parameters to the hit rates and access pattern of users&#039; requesting documents in the Graceful Caching. Finally we verify the performance of this algorithm by simulations.

The emergence of the Internet as a pervasive communication medium has led to the rise of many applications of streaming media. However, because of their distinct statistical properties and user viewing patterns, traditional delivery and caching schemes for web objects such as HTML files or images can not be efficiently applied to streaming media such as audio and video. In this paper, we therefore propose an integrated caching scheme for streaming media with segment-based caching and with hierarchically distributed proxies. Firstly, each stream is divided into segments and their caching algorithms are considered to determine how to distribute the segments into different level proxies efficiently. Then, by introducing two kinds of segment priorities, segments replacing algorithms are proposed to determine which stream and which segments should be replaced when the cache is full. Finally, a web friendly caching scheme is proposed to integrate the streaming caching with the conventional caching of normal web objects. Performance of the proposed algorithms is verified by carrying out simulations.

有線・無線の音声・動画のストリーミングを考える場合、有線での伝送方式をそのまま無線に伝送させるのは効率が悪い。そこで求められるのが有線・無線ゲートウェイである。有線伝送では輻輳におけるバケット廃棄対策に重きをおき、無線伝送ではランダム誤り対策に重きをおき、それぞれを有線・無線ゲートウェイにおいて変換することにより効率を上げることが可能である。本稿ではその有線・無線ゲートウェイに音声ストリームであるPCM及びMP3(MPEG I Audio Layer III)を階層化させたものをUEPO (Unequal Error Protection)を用いて伝送させ、それらの可能性に関して検討した。

The efficient distribution of stored information has become a major concern in the Internet. Since the web workload characteristics shows that more than 60% of network traffic is caused by image, documents, how to efficiently distribute image documents from servers to end clients is an important issue. Proxy cache is an efficient solution to reduce network traffic. An it has been shown that an image caching method (Graceful Caching) based on hierarchical coding format showed better performance than conventional. caching schemes in recent years. However, as the capacity of the cache is limited, how to efficiently allocate the cache memory to achieve a minimum expected delay time is still a problem to be resolved.
This paper presents an integrated caching algorithm to deal with the above problem in the Internet. By analyzing the web request distribution of Graceful Caching, both replacement and pre-fetching algorithms are proposed. We also show that our proposal can be carried out based on information readily available in the proxy server and it flexibly adapts its parameters to the hit rates and access pattern of users' requesting documents in the Graceful Caching. We finally verify the performance of this algorithm by simulations.

This paper presents a cache scheme for image databases, web browsers, proxies and other similar applications in Internet The current cache system employs a hard strategy: either the image Is stored in a cache or not even its data is quite big. We have proposed a caching scheme featuring human memory mechanism (image database system featuring graceful oblivion), which is called Graceful Caching. The previous work shows that Graceful Caching reduces user response time by assigning a variable amount of memory to each image. Several recent studies suggest that prefetching, techniques could be employed to further improve the cache performance, by anticipating and prefetching future client requests. Some approaches based on request frequency have been proposed to the conventional caching (hard system) and got better results. In this paper, we try to apply a request frequency based approach. to Graceful Caching system, in which a progressive image format is used. Based on the analysis of web request probability in Graceful Caching, we go on to propose a new prefetching algorithm that drastically improves cache performance of system. We verify the performance of this algorithm by simulations.

A new wavelet coding method is presented to exploit both intra/inter-band correlations. While the traditional zerotree approach can exploit correlation between subbands, a wavelet decomposition has properties more suitable for the runlength coding method, which can exploit spatial correlation. The proposed method classifies wavelet coefficients into groups according to their parents' magnitude. Each group is chosen in order of the magnitude and scanned to keep spatial correlation. It outperforms the zerotree-based coding approach especially at low bit rate.

This paper describes novel methods to improve coding efficiency of the wavelet transform from the viewpoint of scanning techniques. Firstly, statistical properties of the wavelet transform are examined using real image sequences. They indicate energy distribution or correlations of wavelet transformed coefficients. Taking them into account, efficient scanning methods are investigated for two different approaches. The first approach treats transform coefficients, located in spatially neighboring position, in 8 × 8 blocks. This is called block based approach, and its optimum scanning path is examined similar to the zigzag scan for DCT (discrete cosine transform). Furthermore, a new scanning algorithm utilizing correlation between transform coefficients is proposed. The second approach carries out scanning over the entire subimage created by the wavelet transform. This is called subband based approach, in which scanning is applied to each subband. A new scanning algorithm is presented utilizing a block-wise non-linear filter which removes isolated non-zero coefficients. Simulations are carried out, and their results show the effectiveness of the above approaches especially for low bit-rate applications.

This paper presents novel algorithms exploiting multiple camera inputs and segmentation techniques, which can be applied to image fusion, disparity detection and object extraction. Differently focused images, stereo pairs and both of them are used for fusion, disparity detection and object extraction, respectively. Firstly, image fusion is done by segmentation of each image and determination of focused regions per segment. An efficient decision criterion is developed taking a method of auto-focus into consideration. Secondly, disparity detection is executed by recursively applying segmentation and disparity detection per segment. A new clustering criterion is proposed in order to achieve good segmentation and high compression ratio of disparity maps simultaneously. Finally, object extraction is carried out by utilizing both the fusion result and the disparity map. Experiments are carried out, and they show us effectiveness of the proposed algorithms.

This paper presents a novel framework for structure recovery utilizing a scaled orthographic view and perspective views simultaneously. Perspective views lead to precise recovery based on the triangulation principle; however, many parameters such as camera positions, camera poses and focal length, must be measured beforehand. Thus, a scaled orthographic view is introduced as a subsidiary system to achieve them automatically. Camera parameters ate calculated implicitly owing to the scaled orthographic properties, and then the structure recovery is done by the orthogonality of camera coordinate systems and the triangulation principle. The experimental results provide sufficient accuracy of structure recovery.

This paper presents a novel framework which proves the superiority of overlapped motion compensation. Window design problem is revised by introducing a statistical model of motion estimation process. The result clarifies relationship between the optimum window and image characteristics in an explicit formula and quantifies prediction error reduction achieved by overlapped motion compensation. Experimental results using real image sequences support the proposed theory and demonstrate its superiority. Overlapping in warping prediction is also considered and its effectiveness is shown.

This paper presents mathematical frameworks on temporal predictive processing in the MPEG video compression standard. Firstly, a coding gain is derived based on traditional prediction theories. The optimum ordering of three different picture types (I,P,B-pictures) is clarified according to image source characteristics. Secondly, a novel framework on the target bit assignment is presented with some experimental backgrounds. The solution consists of simple formulae, but brings drastic SNR gains to the conventional TM5 algorithm.

This paper describes theories on overlapped motion compensation and their applications to a wavelet codec aimed for very low bit-rate environment below 64 kb/s. The theories are concerned with the evaluation of prediction efficiency improved by overlapped motion compensation and also its smoothing effect on the discontinuities at block boundaries encountered when motion vectors do not coincide among neighboring blocks. They contribute to determine the optimum window shape for overlapped motion compensation in a developed wavelet codec, which suffers from reduction of coding efficiency when there are such discontinuities in a signal to be transformed. Regarding the wavelet codec, a new scanning method of transform coefficients and alternate use of normal and spatially-reversed basis functions for cascaded wavelet transform are introduced. Finally, the implementation of the codec on a Video Image Signal Processor (VISP) is discussed. © 1994 IEEE

This paper presents a filter design algorithm in order to realize high efficient image codec system based on subband decomposition, and also describes unified frameworks to interpret various image coding techniques. First, various image coding techniques based on the linear transform are dealt with in an unified manner, where both the matrix representation in time domain and the multirate filter bank concept are introduced. Reconsideration about the DPCM is stressed here because the subband coding and the transform coding have been already formulated in some common frameworks. Neither the filter&#039;s frequency response nor its orthogonality are taken into consideration because they don&#039;t necessarily play an important role in our formulation, particularly in case of the DPCM and the SSKF. Secondly, statistical optimization of short tap and linear phase PR (perfect reconstruction) filter banks is considered. The UCG (unified coding gain) that we proposed at the VSIP &#039;91 as a performance measure of energy compaction properties of the multirate filter bank is expanded to multiple layer cases, and several new examples are presented. The short tap structure leads to low computational complexity, and the linear phase property contributes not only to remove phase distortion but also to solve the so-called border problem. Simulation results are also shown, and the validity of our approach is confirmed.

In this paper, two analytical methods for evaluating the coding efficiency of subband coding are proposed, and optimization of filter coefficients of the perfect reconstruction FIR filter banks is considered, based on a new performance measure called unified coding gain. First, matrix representation of the subband coding in the time domain is considered, and conventional subband filter banks are classified into orthogonal ones such as the QMF and nonorthogonal ones such as the SSKF. For the orthogonal filter banks, the coding gain shown by Jayant and Noll is introduced, and their theoretical performance evaluation is carried out. However, this first method cannot be applied to nonorthogonal filter banks any longer because the coding gain is defined on the assumption of filter orthogonality. Therefore, an optimum bit allocation problem for subband coding is considered, and the unified coding gain, which can be applied to arbitrary subband filter banks, is derived as a new performance measure to take the place of the coding gain. This second method enables us to estimate the coding efficiency of arbitrary transform techniques as well as the subband approaches, and its result suggests that the SSKF(5 × 3) outperforms the QMF as long as the number of subbands is not too large, even though its filter length is much shorter. This result encourages us to find filter coefficients that maximize the unified coding gain according to filter length. In addition, new perfect reconstruction FIR filter banks which have not only low computational complexity but also good energy compaction properties are presented. © 1991.

One of the merits with the introduction of variable bit-rate coding to moving pictures is the constancy of picture quality. However, considering the fact that conventional SNR cannot reflect the result of subjective assessment sufficiently, other quality measures should be considered to design the variable bit-rate coders. In this paper, weighted SNR (WSNR), which is calculated by weighting coding errors with the spatial frequency response of human vision, is introduced as a quality measure in place of SNR, and the variable bit-rate coder based on this measure is considered. Then, subjective assessment is carried out and the effectiveness of our approach is confirmed. © 1991.

In this paper, two analytical methods to evaluate coding performance of subband coding are proposed, and optimization of its filter coefficients from the viewpoint of energy compaction property is considered. The first method is based on matrix representation of subband coding in time domain, where the coding gain given by Jayant and Noll is introduced as a performance measure for filter banks with orthogonal property. The second method is based on the optimum bit allocation problem for subband coding (multirate filter bank), where the unified coding gain is derived as a new performance measure which can be applied to arbitrary transform techniques. We then try to find filter coefficients which maximize the unified coding gain according to input characteristics. This approach leads to optimization of filter coefficients from the viewpoint of energy compaction property.

As a promising means of realizing broadband ISDN, the ATM (asynchronous transfer mode), which is based on fast packet switching, is considered interesting. In video communication by ATM, the constancy and improvement of picture quality by variable rate coding is expected, while the quality deterioration by cell loss is a serious problem. This paper proposes a video coding method introducing hierarchical coding to cope with the cell loss problem. The proposed hierarchical coding extracts the low‐frequency components of the image by iterating the hierarchical processing based on the spatial filter. Then it is possible to suppress the effect of the quality deterioration caused by cell loss, as long as the low‐frequency components are retained. First, the configuration of the proposed system is discussed. Simulations are executed to observe the effect of the cell loss and the coding characteristics, demonstrating the effectiveness of the proposed method in coping with the cell loss. Then discussions are made on the optimal number of hierarchical processings to cope with the cell loss and the protection level for hierarchical information. Furthermore, it is shown that a new type of video communication can be realized by introducing hierarchical coding. Copyright © 1990 Wiley Periodicals, Inc., A Wiley Company