Learned Image Compression (LIC), which uses neural networks to compress images, has experienced significant growth in recent years. The hyperprior-module-based LIC model has achieved higher performance than classical codecs. However, the LIC models are too heavy (in calculation and parameter amounts) to apply to edge devices. To solve this problem, some former papers focus on structural pruning for LIC models. However, they either cause noticeable performance decrement or neglect the appropriate pruning threshold for each LIC model. These problems keep their pruning results sub-optimal. This paper proposes a Pruning Threshold Searching on the hyperprior module for different-quality LIC models. Our method removes most parameters and calculations while keeping the performance the same as the models before pruning. We removed at least 49.8% of parameters and 28.5% of calculations for the Channel-Wise-Context-Model-based models and 29.1% of parameters for the Cheng-2020 models.

Transmission latency presents a significant challenge when operating remote equipment, such as a robotic arm. To address this, we developed a platform and conducted experiments to reduce transmission latency to near-zero levels. These experiments employed Long Short-Term Memory (LSTM) to anticipate future motion trends, leveraging both the controller's movement variables and Electromyography (EMG) data from the operator's arm muscles. Our findings indicate the potential to decrease transmission latency by approximately 500ms. Additionally, our research confirms a direct correlation between prediction accuracy and the brevity of prediction time, suggesting that shorter prediction times yield more accurate results when using EMG. In the context of video transmission for a remotely located robotic arm, we applied video prediction techniques using the Predictive Coding Network (PredNet) to counter network latency. Our results suggest that these predictive methods can effectively compensate for a latency period of 300ms, thereby highlighting their potential for reducing transmission latency in remote robotic operations.

Lossless image compression is an important research field in image compression. Recently, learning-based lossless image compression methods achieved impressive performance compared with traditional lossless methods, such as WebP, JPEG2000, and FLIF. The aim of the lossless image compression algorithms is to use shorter codelength to represent images. To encode an image with fewer bytes, eliminating the redundancies among the pixels in the image is highly important. Hence, in this paper, we explore the idea of combining an autoregressive model for the raw images based on the end-to-end lossless architecture proposed to enhance the performance. Furthermore, inspired by the successful achievements of Channel-conditioning models, we propose a Multivariant Mixture distribution Channel-conditioning model (MMCC) in our network architecture to boost performance. The experimental results show that our approach outperforms most classical lossless compression methods and existing learning-based lossless methods.

Recent state-of-the-art Learned Image Compression methods feature spatial context models, achieving great rate-distortion improvements over hyperprior methods. However, the autoregressive context model requires serial decoding, limiting run-time performance. The Checkerboard context model allows parallel decoding at a cost of reduced RD performance. We present a series of multistage spatial context models allowing both fast decoding and better RD performance. We split the latent space into square patches and decode serially within each patch while different patches are decoded in parallel. The proposed method features a comparable decoding speed to Checkerboard while reaching the RD performance of Autoregressive and even also outperforming Autoregressive. Inside each patch, the decoding order must be carefully decided as a bad order negatively impacts performance; therefore, we also propose a decoding order optimization algorithm.

Learned image compression (LIC) methods have exhibited promising progress and superior rate-distortion performance compared with classical image compression standards. Most existing LIC methods are Convolutional Neural Networks-based (CNN-based) or Transformer-based, which have different advantages. Exploiting both advantages is a point worth exploring, which has two challenges: 1) how to effectively fuse the two methods? 2) how to achieve higher performance with a suitable complexity? In this paper, we propose an efficient parallel Transformer-CNN Mixture (TCM) block with a controllable complexity to incorporate the local modeling ability of CNN and the non-local modeling ability of transformers to improve the overall architecture of image compression models. Besides, inspired by the recent progress of entropy estimation models and attention modules, we propose a channel-wise entropy model with parameter-efficient swin-transformer-based attention (SWAtten) modules by using channel squeezing. Experimental results demonstrate our proposed method achieves state-of-the-art rate-distortion performances on three different resolution datasets (i.e., Kodak, Tecnick, CLIC Professional Validation) compared to existing LIC methods. The code is at https://github.com/jmliu206/LIC-TCM.

Recently, thanks to advancing computing power and evolving image and sensor processing, SLAM (Simultaneous Localization and Mapping) is collecting much attention in broad areas. Inspired by the former research efforts on SLAM, we develop a real-time LiDAR-based indoor localization system in this paper. The system estimates the user's self-position by using LiDAR SLAM and Visual-Inertial Odometry (VIO) and visualizes the user's time-series positions on the building floor map (global map) by adopting a 2D-to-2D localization method. For the system validation, we designed and implemented the prototype handheld system and carried out the validation tests on one floor of our university building. The validation tests confirmed that the system achieves real-time indoor localization.

High frame rate videos are recently used for training a variety of image processing tasks such as image/video deblurring and frame interpolation. Yet, the number of publicly available datasets with high frame rate videos are still limited. In this paper, we propose the iPhone 240fps dataset, a dataset which consists of real world 240fps videos. We conduct experiments with deblurring and frame interpolation models to show the effectiveness of this dataset. We also use this dataset to conduct experiments with a state-of-the-art video super-resolution model. We take advantage of the flexibility to change the frame rate of the dataset and test which fps is best suited for training the model. We believe this dataset can make contributions to future works in image and video processing. The dataset is publicly available on our cloud storage††https://waseda.box.com/s/4ihxczdiythmeml38ndgsq4l1q90vnpp:

Spectral Normalization is one of the best methods for stabilizing the training of Generative Adversarial Network. Spectral Normalization limits the gradient of discriminator between the distribution between real data and fake data. However, even with this normalization, GAN's training sometimes fails. In this paper, we reveal that more severe restriction is sometimes needed depending on the training dataset, then we propose a novel stabilizer which offers an adaptive normalization method, called ABCAS. Our method decides discriminator's Lipschitz constant adaptively, by checking the distance of distributions of real and fake data. Our method improves the stability of the training of Generative Adversarial Network and achieved better Fréchet Inception Distance score of generated images. We also investigated suitable spectral norm for three datasets. We show the result as an ablation study.

While the rollout of the fifth-generation mobile network (5G) is underway across the globe with the intention to deliver 4K/8K UHD videos, Augmented Reality (AR), and Virtual Reality (VR) content to the mass amounts of users, the coverage and throughput are still one of the most significant issues, especially in the rural areas, where only 5G in the low-frequency band are being deployed. This called for a highperformance adaptive bitrate (ABR) algorithm that can maximize the user quality of experience given 5G network characteristics and data rate of UHD contents.Recently, many of the newly proposed ABR techniques were machine-learning based. Among that, Pensieve is one of the state-of-the-art techniques, which utilized reinforcement-learning to generate an ABR algorithm based on observation of past decision performance. By incorporating the context of the 5G network and UHD content, Pensieve has been optimized into Pensieve 5G. New QoE metrics that more accurately represent the QoE of UHD video streaming on the different types of devices were proposed and used to evaluate Pensieve 5G against other ABR techniques including the original Pensieve. The results from the simulation based on the real 5G Standalone (SA) network throughput shows that Pensieve 5G outperforms both conventional algorithms and Pensieve with the average QoE improvement of 8.8% and 14.2%, respectively. Additionally, Pensieve 5G also performed well on the commercial 5G NR-NR Dual Connectivity (NR-DC) Network, despite the training being done solely using the data from the 5G Standalone (SA) network.

Due to the recent demand for huge Uplink through-put on Mobile networks driven by the rapid development of social media platforms, UHD 4K/8K video, and VR/AR contents, Uplink MIMO (UL-MIMO) has now been deployed on commercial 5G networks with reasonable availability of supported User Equipment (UE) for consumers. By utilizing up to 2 Tx antenna ports, UL-MIMO-capable UE promised to achieve up to two times the uplink throughput in ideal conditions, while providing improved uplink performance over UE with 1Tx in challenging conditions.In Japan, SoftBank, one of the carriers, introduced 5G Standalone (SA) services for the Fixed Wireless Access (FWA) application back in October 2021. Mobile services were commenced in May 2022, which provide UL-MIMO for supported UE on C-Band or Band n77 (3.7 GHz). In this paper, the uplink performance of UL-MIMO-capable UE will be compared against the conventional UL-1Tx UE on trains, which is the most popular method of transportation for the Japanese. The results show that UL-MIMO-capable UE delivers an average of 19.8% better throughput on moving trains with up to 33.5% in the more favorable signal conditions. A moderate relationship between downlink 5G NR SS-RSRP and uplink throughput also has been observed.

For next-generation logistics systems using autonomous vehicles and drones, spoofing of the GNSS location data induces serious problems. Although signal-based anti-spoofing has been studied, it is difficult to apply to current commercial GNSS modules in many cases. We investigate possibilities to detect spoofing of GNSS location data using multiple sensing devices and a decision tree classifier. Multiple features using the GNSS, beacons, and the IMU are defined and create a model to detect spoofing. Experimental results using learning-based classifier indicates the higher performances and generalization capability. The results also show that distance from beacons is useful to detect GNSS spoofing and indicate prospects of installation for the future drone highways.

In this paper, we introduce an information-centric service mesh for autonomous in-network computing. The information-centric service mesh offers service-centric networking, not connectivity-based networking provided by the current container orchestration, like Kubernetes (K8s). We implement the information-centric service mesh by adopting an ambassador container style and using Cefore and Cefpyco which is the CCNx software implementation developed by NICT in Japan. For the proof-of-concept test, we demonstrate the end-to-end delay of more complex service chain models (up to 17 nodes) on our K8s cluster.

5G Standalone (SA) is the goal of the 5G evolution, which aims to provide higher throughput and lower latency than the existing LTE network. One of the main applications of 5G is the real-time distribution of Ultra High-Definition (UHD) content with a resolution of 4K or 8K. In Q2/2021, Advanced Info Service (AIS), the biggest operator in Thailand, launched 5G SA, providing both 5G SA/NSA service nationwide in addition to the existing LTE network. While many parts of the world are still in process of rolling out the first phase of 5G in Non-Standalone (NSA) mode, 5G SA in Thailand already covers more than 76% of the population. In this paper, UHD video will be a real-time live streaming via MPEG-DASH over different mobile network technologies with minimal buffer size to provide the lowest latency. Then, performance such as the number of dropped segments, MAC throughput, and latency are evaluated in various situations such as stationary, moving in the urban area, moving at high speed, and also an ideal condition with maximum SINR. It has been found that 5G SA can deliver more than 95% of the UHD video segment successfully within the required time window in all situations, while 5G NSA produced mixed results depending on the condition of the LTE network. The result also reveals that the LTE network failed to deliver more than 20 % of the video segment within the deadline, which shows that 5G SA is absolutely necessary for low-latency UHD video streaming and 5G NSA may not be good enough for such task as it relies on the legacy control signal.

Demands for high immersive communications using 3D content are increasing. In order to contribute to development of this immersive communications, this paper reports a prototype implementation of real-time point cloud streaming system and shows live-action demonstrations of live point cloud streaming using HoloLens. From the demonstration, we confirm that the prototype system could have a potential capability of realization of live streaming without any point cloud playback interruptions.

The latest Versatile Video Coding (VVC) standard incorporates a series of effective and complex new intra coding tools, which obtains superior coding efficiency than the High Efficiency Video Coding (HEVC). However, this makes the intra coding more complicated and time-consuming. A fast algorithm for VVC is proposed from two aspects of model selection and early terminating to reduce coding complexity in this paper. The relationship between bins with HOG and intra modes is created for the mode selection, decreasing the planar modes for SATD and RDO. Moreover, we analyze the maximum bins to determine the final modes, and we use the modes of left and upper blocks as a reference for the current CU, which can early terminate RDO. The proposed algorithm is implemented on VVC test model, and the experimental results show that it can achieve 36.61% time savings with only 0.94% BDBR increases averagely, which outperforms other relative existing state-of-the-art methods.

There is a growing number of images that are analyzed by machines rather than just humans. Recently, most machine vision tasks are based on decoded images which require an image compression (encoding/decoding) framework. However, using the decoded images in the pixel-domain has two drawbacks: 1) the complexity is high for the decoder part, 2) the accuracy (e.g., mIoU, mean absolute error, and average precision) of machine vision tasks will be degraded since decoded images only aim to optimize the human perceived quality (e.g., PSNR) so that information required for machine vision tasks will be lost during the decoding process. In this paper, we improve the machine vision tasks in the compressed domain. 1) A gate module is utilized to effectively select some compressed-domain features. 2) Knowledge distillation is introduced to improve the accuracy. 3) A training strategy is explored to support multiple tasks including the image compression. The experimental results show that we can achieve better rate-accuracy/distortion and lower complexity compared with the state-of-the-art pixel-domain work that can take both machine and human vision tasks.

Learned image compression (LIC) has reached a comparable coding gain with traditional hand-crafted methods such as VVC intra. However, the large network complexity prohibits the usage of LIC on resource-limited embedded systems. Network quantization is an efficient way to reduce the network burden. This paper presents a quantized LIC (QLIC) by channel splitting. First, we explore that the influence of quantization error to the reconstruction error is different for various channels. Second, we split the channels whose quantization has larger influence to the reconstruction error. After the splitting, the dynamic range of channels is reduced so that the quantization error can be reduced. Finally, we prune several channels to keep the number of overall channels as origin. By using the proposal, in the case of 8-bit quantization for weight and activation of both main and hyper path, we can reduce the BD-rate by 0.61%-4.74% compared with the previous QLIC. Besides, we can reach better coding gain compared with the state-of-the-art network quantization method when quantizing MS-SSIM models. Moreover, our proposal can be combined with other network quantization methods to further improve the coding gain. The moderate coding loss caused by the quantization validates the feasibility of the hardware implementation for QLIC in the future.

Recently, learned image compression (LIC) has shown a superior ability in the compression ratio as well as the quality of the reconstructed image. By adopting the framework of variational autoencoder, LIC [1] can outperform the intra prediction of the latest traditional coding standard VVC. To accelerate the coding speed, most LIC frameworks are operated on GPU with the floating-point arithmetic. However, the mismatch of floating-point calculation results on various hardware platforms will cause the decoding error if encoding and decoding are performed on different platforms. Therefore, LIC with a fixed-point arithmetic [2-3] is highly required. This paper gives an FPGA design for a LIC with 8-bit fixed-point quantization. Different from existing FPGA accelerators [4-6], we propose a fine-grained pipeline architecture to realize high DSP efficiency. Cascading DSP and the deconvolution with zero skipping are also developed to enhance the hardware performance.

Learned Image Compression (LIC) gradually became more and more famous in these years. The hyperprior-module-based LIC models have achieved remarkable rate-distortion performance. However, the memory cost of these LIC models is too large to actually apply them to various devices, especially to portable or edge devices. The parameter scale is directly linked with memory cost. In our research, we found the hyperprior module is not only highly over-parameterized, but also its latent representation contains redundant information. Therefore, we propose a novel pruning method named ERHP in this paper to efficiently reduce the memory cost of hyperprior module, while improving the network performance. The experiments show our method is effective, reducing at least 22.6% parameters in the whole model while achieving better rate-distortion performance.

Learned image compression allows achieving state-of-the-art accuracy and compression ratios, but their relatively slow runtime performance limits their usage. While previous attempts on optimizing learned image codecs focused more on the neural model and entropy coding, we present an alternative method to improving the runtime performance of various learned image compression models. We introduce multi-threaded pipelining and an optimized memory model to enable GPU and CPU workloads' asynchronous execution, fully taking advantage of computational resources. Our architecture alone already produces excellent performance without any change to the neural model itself. We also demonstrate that combining our architecture with previous tweaks to the neural models can further improve runtime performance. We show that our implementations excel in throughput and latency compared to the baseline and demonstrate the performance of our implementations by creating a real-time video streaming encoder-decoder sample application, with the encoder running on an embedded device.

Learned image compression (LIC) has shown its superior compression ability. Quantization is an inevitable stage to generate quantized latent for the entropy coding. To solve the non-differentiable problem of quantization in the training phase, many differentiable approximated quantization methods have been proposed. However, the derivative of quantized latent to non-quantized latent are set as one in most of the previous methods. As a result, the quantization error between non-quantized and quantized latent is not taken into consideration in the gradient descent. To address this issue, we exploit the gradient scaling method to scale the gradient of non-quantized latent in the back-propagation. The experimental results show that we can outperform the recent LIC quantization methods.

This demo paper gives a real-time learned image codec on FPGA. By using Xilinx VCU128, the proposed system reaches 720P@30fps codec, which is 7.76x faster than prior work.

Most machine vision tasks (e.g., semantic segmentation) are based on images encoded and decoded by image compression algorithms (e.g., JPEG). However, these decoded images in the pixel domain introduce distortion, and they are optimized for human perception, making the performance of machine vision tasks suboptimal. In this paper, we propose a method based on the compressed domain to improve segmentation tasks. i) A dynamic and a static channel selection method are proposed to reduce the redundancy of compressed representations that are obtained by encoding. ii) Two different transform modules are explored and analyzed to help the compressed representation be transformed as the features in the segmentation network. The experimental results show that we can save up to 15.8% bitrates compared with a state-of-the-art compressed domain-based work while saving up to about 83.6% bitrates and 44.8% inference time compared with the pixel domain-based method.

With the COVID19 pandemic, video streaming traffic is increasing rapidly. Especially, the live streaming traffic accounts for large amount due to the fact that many events have been switched to the online forms. Therefore, the demand to ensure a high-quality streaming experience is increasing urgently. Since the network condition is expected to fluctuate dynamically, the video streaming needs to be controlled adaptively according to the network condition to provide high quality of experience (QoE) for users. In this paper, a method was proposed to control the live video streaming using the actor-critic reinforcement learning (RL) technique. In this method, the historical video streaming logs such as throughput, buffer size, rebuffering time, latency are taken consideration as the states of RL, then the model is established to map the states to an action such as bitrate decision. In this study, the live streaming simulation is utilized to evaluate the method since the model needs training and the simulation can generate data much faster than real experiment. Experiments were conducted to evaluate the proposed method. Results demonstrate that the total QoE in Bus and Car scenarios show the best performance. The QoE of Tram case shows the lowest due to the low bandwidth.

Convolutional neural network (CNN) filters have achieved significant performance in video artifacts reduction. However, the high complexity of existing methods makes them difficult to be applied in actual usage. In this article, an efficient low-complexity CNN filter is proposed. We utilized depth separable convolution merged with the batch normalization as the backbone of our proposed CNN filter and presented a frame-level residual mapping (RM) to use one network to filter both intra- A nd intersamples. It is known that there will be an oversmoothing problem for the interframes if we directly use the filter trained with intrasamples. In this article, the proposed RM can effectively solve the oversmoothing problem. Besides, RM is flexible and can be combined with other learning-based filters. The experimental results show that our proposed method achieves a significant bjÃntegaard-delta(BD)-rate reduction than H.265/high efficiency video coding. The experiments show that the proposed network achieves about 1.2% BD-rate reduction and 79.1% decrease in FLOPs than VR-CNN. Our performance is better with less complexity than the previous work. The measurement on H.266/versatile video coding and ablation studies also ensure the effectiveness of the proposed method.

Fixed-point arithmetic is a technique for treating weights and intermediate values as integers in deep learning. Since deep learning models generally store each weight as a 32-bit floating-point value, storing by 8-bit integers can reduce the size of the model. In addition, memory usage can be reduced, and inference can be much faster by hardware acceleration when special hardware for int8 inference is provided. On the other hand, when inferences are carried out by fixed-point weights, accuracy of the model is reduced due to loss of dynamic range of the weights and intermediate layer values. For this reason, inference frameworks such as TensorRT and TensorFlow Lite, provide a function called "calibration"to suppress the deterioration of the accuracy caused by quantization by measuring the distribution of input data and numerical values in the intermediate layer when quantization is performed. In this paper, after quantizing a pre-trained model that performs super-resolution, speed and accuracy are measured using TensorRT. As a result, the trade-off between the runtime and the accuracy is confirmed. The effect of calibration is also confirmed.

Frame interpolation methods generate intermediate frames by taking consecutive frames as inputs. This enables the generation of high frame rate videos from low frame rate videos. Recently, many deep learning-based frame interpolation methods have been proposed. One way of frame interpolation is by using the bi-directional optical flow. In many cases, these methods use backward warping to warp the input images to the desired frame. However, forward warping can also be used to warp the input frames. In this paper, we propose a frame interpolation method that utilizes both forward warping and backward warping. Experimental results show that utilizing both warping methods can enhance the performance compared to only using backward warping.

Convolutional neural network (CNN)-based filters have achieved great success in video coding. However, in most previous works, individual models were needed for each quantization parameter (QP) band, which is impractical due to limited storage resources. To explore this, our work consists of two parts. First, we propose a frequency and spatial QP-adaptive mechanism (FSQAM), which can be directly applied to the (vanilla) convolution to help any CNN filter handle different quantization noise. From the frequency domain, a FQAM that introduces the quantization step (Qstep) into the convolution is proposed. When the quantization noise increases, the ability of the CNN filter to suppress noise improves. Moreover, SQAM is further designed to compensate for the FQAM from the spatial domain. Second, based on FSQAM, a QP-adaptive CNN filter called QA-Filter that can be used under a wide range of QP is proposed. By factorizing the mixed features to high-frequency and low-frequency parts with the pair of pooling and upsampling operations, the QA-Filter and FQAM can promote each other to obtain better performance. Compared to the H.266/VVC baseline, average 5.25% and 3.84% BD-rate reductions for luma are achieved by QA-Filter with default all-intra (AI) and random-access (RA) configurations, respectively. Additionally, an up to 9.16% BD-rate reduction is achieved on the luma of sequence BasketballDrill. Besides, FSQAM achieves measurably better BD-rate performance compared with the previous QP map method.

For massive multiple-input multiple-output (MIMO) systems, pilot contamination reduces the data transmission capacity owing to the inter-cell interference of non-orthogonal pilots reusage. To develop efficient mobile communication, it is necessary to mitigate pilot contamination. To address this problem, we propose an annealing-based pilot allocation method using Digital Annealer to provide solution for Ising machine. The proposed method is a max k-cut-based approach, where the graph represents the potential strength of pilot contamination among users in other cells. By using this proposed method, users who have strong relationship with pilot contamination will be assigned different pilots. Experiment results show that the proposed method can realize optimal pilot allocation and mitigate pilot contamination. Compared with conventional methods, the proposal achieves the best performance which can increase the minimum achievable rate and show higher SINR, especially when the numbers of users and cells are large.

End-to-end Learned image compression (LIC) has reached the traditional
hand-crafted methods such as BPG (HEVC intra) in terms of the coding gain.
However, the large network size prohibits the usage of LIC on resource-limited
embedded systems. This paper reduces the network complexity by quantizing both
weights and activations. 1) For the weight quantization, we study different
kinds of grouping and quantization scheme at first. A channel-wise non-linear
quantization scheme is determined based on the coding gain analysis. After
that, we propose a fine tuning scheme to clip the weights within a certain
range so that the quantization error can be reduced. 2) For the activation
quantization, we first propose multiple non-linear quantization codebooks with
different maximum dynamic ranges. By selecting an optimal one through a
multiplexer, the quantization range can be saturated to the greatest extent. In
addition, we also exploit the mean-removed quantization for the analysis
transform outputs in order to reduce the bit-width cost for the specific
channel with the large non-zero mean. By quantizing each weight and activation
element from 32-bit floating point to 8-bit fixed point, the memory cost for
both weight and activation can be reduced by 75% with negligible coding
performance loss. As a result, our quantized LIC can still outperform BPG in
terms of MS-SSIM. To our best knowledge, this is the first work to give a
complete analysis on the coding gain and the memory cost for a quantized LIC
network, which validates the feasibility of the hardware implementation.

With the dramatically increasing video streaming in the total network traffic, it is critical to develop effective algorithms to ensure the quality of content delivery service. Adaptive bitrate (ABR) control is the most essential technique which determines the proper bitrate to be chosen based on network conditions, thus realize high-quality video streaming. In this paper, a novel ABR strategy is proposed based on Ising machine by using the quadratic unconstrained binary optimization (QUBO) method and Digital Annealer (DA) for the first time. The proposed method is evaluated by simulation with the real-world measured throughput and compared with other state-of-the-art methods. Experiment results show that the proposed QUBO-based method can outperform the existing methods, which demonstrating the superior of the proposed QUBO-based method.

Recent advancements of virtualization technologies for parallel processing involve scheduling containerized tasks in a workflow. Since a container can include multiple tasks, it can be reused or shared among applications. If every task in a workflow uses its dedicated container without sharing among any tasks, each container image must be downloaded for each task. As a result, many computational resources are required to process and the communication latency related to container image downloading can become a bottleneck for the makespan. In task scheduling algorithms for workflows, this characteristic produces a new challenging issue that how effectively shares containers among tasks to avoid redundant container image download processes and redundant task allocations. One of the fundamental problems is that no policy has been established for simultaneously satisfying effective container sharing, maintaining the degree of task parallelism, and effective computational resource utilization. In this paper, we propose a clustering-based containerized task scheduling algorithm for clouds, namely, shareable functional task clustering for utilizing virtualized resources (SF-CUV). The objective of SF-CUV is to minimize the makespan with less computational resources and containers than other algorithms by clustering tasks and sharing each container among tasks. SF-CUV consists of two phases: (i)task clustering and pre-virtual CPU (vCPU) allocation phase to derive an accurate scheduling priority, and (ii)task ordering and actual task reallocation phase. Experimental results obtained via simulation and in a real environment show that SF-CUV can utilize both vCPUs and containers with a shorter makespan compared with other approaches.

With the dramatically increasing video streaming in the total network
traffic, it is critical to develop effective algorithms to promote the content
delivery service of high quality. Adaptive bitrate (ABR) control is the most
essential technique which determines the proper bitrate to be chosen based on
network conditions, thus realize high-quality video streaming. In this paper, a
novel ABR strategy is proposed based on Ising machine by using the quadratic
unconstrained binary optimization (QUBO) method and Digital Annealer (DA) for
the first time. The proposed method is evaluated by simulation with the
real-world measured throughput, and compared with other state-of-the-art
methods. Experiment results show that the proposed QUBO-based method can
outperform the existing methods, which demonstrating the superior of the
proposed QUBO-based method.

In this paper, we propose a learned video codec with a residual prediction
network (RP-Net) and a feature-aided loop filter (LF-Net). For the RP-Net, we
exploit the residual of previous multiple frames to further eliminate the
redundancy of the current frame residual. For the LF-Net, the features from
residual decoding network and the motion compensation network are used to aid
the reconstruction quality. To reduce the complexity, a light ResNet structure
is used as the backbone for both RP-Net and LF-Net. Experimental results
illustrate that we can save about 10% BD-rate compared with previous learned
video compression frameworks. Moreover, we can achieve faster coding speed due
to the ResNet backbone. This project is available at
https://github.com/chaoliu18/RPLVC.

These days, attention is thought to be an efficient way to recognize an image. Vision Transformer (ViT) uses a Transformer for images and has very high performance in image recognition. ViT has fewer parameters than Big Transfer (BiT) and Noisy Student. Therefore, we consider that Self-Attention-based networks are slimmer than convolution-based networks. We use a ViT as a Discriminator in a Generative Adversarial Network (GAN) to get the same performance with a smaller model. We name it ViT-GAN. Besides, we find parameter sharing is very useful to make parameter-efficient ViT. However, the performances of ViT heavily depend on the number of data samples. Therefore, we propose a new method of Data Augmentation. Our Data Augmentation, in which the strength of Data Augmentation varies adaptively, helps ViT for faster convergence and better performance. With our Data Augmentation, we show ViT-based discriminator can achieve almost the same FID but the number of the parameters of the discriminator is 35% fewer than the original discriminator.

With the live video streaming becomes more and more common in daily life such as live meeting and live video call, it is an urgent task to ensure high-quality and low-delay live video streaming service. High user quality of experience (QoE) should be ensured to satisfy the requirement of user, for which latency is one of the important factors. In this paper, a high-QoE live streaming method is proposed with reinforcement learning. Experiments are conducted to evaluate the proposed method. Results demonstrate that the proposal shows the best performance with highest QoE compared with conventional methods in three network conditions. In Ferry case, the QoE is almost twice of the QoE of other methods.

Learned image compression (LIC) has achieved superior coding performance than traditional image compression standards such as HEVC intra in terms of both PSNR and MS-SSIM. However, most LIC frameworks are based on floating-point arithmetic which has two potential problems. First is that using traditional 32-bit floating-point will consume huge memory and computational cost. Second is that the decoding might fail because of the floating-point error coming from different encoding/decoding platforms. To solve the above two problems. 1) We linearly quantize the weight in the main path to 8-bit fixed-point arithmetic, and propose a fine tuning scheme to reduce the coding loss caused by the quantization. Analysis transform and synthesis transform are fine tuned layer by layer. 2) We exploit look-up-table (LUT) for the cumulative distribution function (CDF) to avoid the floating-point error. When the latent node follows non-zero mean Gaussian distribution, to share the CDF LUT for different mean values, we restrict the range of latent node to be within a certain range around mean. As a result, 8-bit weight quantization can achieve negligible coding gain loss compared with 32-bit floating-point anchor. In addition, proposed CDF LUT can ensure the correct coding at various CPU and GPU hardware platforms.

Dynamic adaptive streaming over HTTP (DASH) has been widely adopted in modern video streaming services. In DASH, the core technique is adaptive bitrate (ABR) control which can adjust the requested video bitrate level according to the network conditions to tradeoff between video quality and rebuffering risk. It is a challenge for the ABR methods in the scenarios when multiple DASH streaming users compete over the network bottleneck. This paper proposes a client-side ABR control method, flexible relaxation assisted by buffer (FRAB), to achieve fair, stable and efficient video streaming among different users. The idea of FRAB is to relax the change of the video quality based on current buffer level, which can enhance the stability of video streaming. Meanwhile, by flexibly adjusting the relaxation, the efficiency and fairness among all users are improved. FRAB is evaluated in real experiments under three different network conditions and compared with conventional multi-user ABR algorithms. Results indicate FRAB has the best performance in fairness, which reduces the unfairness by a maximum of 69.5% under real-world measured network condition. It also improves the efficiency by 71.3% comparing with PANDA, and enhances the stability by 73.3% comparing with TFDASH. The experiment results demonstrated that the proposed method has superior performances in multi-user DASH video streaming.

Recently, aging of the population and medical cost inflation are emerging as social issues to be solved. To address this problem, estimations of bioinformation based on Internet of Things (IoT) and machine learning get more attention to researchers. In this paper, in order to estimate R-R Interval (RRI) without using specialized and professional wearable devices, we propose a deep learning based RRI estimation method by using mainly smartphone sensors. For dataset, we collect ECG (for label), 3-axis acceleration, pressure, illuminance, GPS and temperature (for training data) under different exercise types (walking and running) by using a smartphone and smart clothing called hitoe. To construct a regression model, we adopt a dual stage attention-based RNN model. From the evaluation results, we confirm that the proposed method can estimate RRI and LF/HF with acceptable accuracy.

In order to simplify deployment and management of IoT services, Network Function Virtualization (NFV) and Service Function Chaining (SFC) are promising solutions, and much researchers have conducted these topics. To enhance the reliability of former research efforts, in this paper, we propose an orchestration framework for IoT-centric SFC by using Docker and Kubernetes. The framework enables an automatic IoT service deployment by satisfying service requirements and computing and network resource constraints. In such deployment, we apply a Virtual Network Function (VNF)/Service Function (SF) placement problem to achieve efficient utilization of the resources. We set an objective function as minimizing both numbers of SF instances and communications and build a mathematical model based on Integer Linear Programming (ILP). To validate it, we implement a model for the framework and evaluate the performances by carrying out a numerical evaluation and a real experiment. From the evaluation results, we confirm that the proposed approach can reduce the number of SF placements and the number of communications among SF instances.

Thanks to breakthrough and evolution of deep learning in computer vision areas, adaptation of deep learning into communication systems are getting lots of attention to researchers. Recently, a channel estimation method by using a deep learning-based image super-resolution (SR) technique, namely ChannelNet, has been proposed. Inspired by this research, in this paper, we propose a deep SR based channel estimation method by applying more accurate deep learning-based SR network architecture, EDSR. In order to enhance intelligibility and reliability of deep SR based channel estimation methods, we evaluate the performance of several deep SR based channel estimation methods (SRCNN, ChannelNet and EDSR) by carrying out practical 5G simulations. From the evaluations, the results conclude that the deep SR based channel estimation methods can potentially improve accuracy of channel estimation and reduce BER characteristics.

Adaptive Loop Filter (ALF) is a new technique proposed by the latest video coding standard Versatile Video Coding (VVC). To the best of our knowledge, this paper is the first implementation to design a hardware architecture of ALF in VVC decoder. The implementation reduces 60% of the memory access, saves approximately 50% of the hardware resources including adders and multipliers cost, increases the throughput and makes the hardware configurable for luma and chroma components. The synthesis result demonstrates that the architecture achieves a throughput of 4k@120fps and a maximum frequency of 367MHz under the TSMC 65nm process.

Conventional object detection methods are in the pixel domain and require full decoding with high computational complexity. In this paper, we propose a fast object detection method in the intra compressed domain of High Efficiency Video Coding (HEVC), which significantly accelerates the object detection process that uses compressed video images. Considering the characteristics of various coding features, we select 3 types of data for object detection, including partitioning depths, prediction modes, and residuals. To achieve a more discriminative representation of the residuals, we design an iterative restoration algorithm that can generate the details of the original image and reduce the noise in the residuals. Extensive evaluations on multiple HEVC test sequences and large-scale object detection dataset BDD100K confirm the effectiveness of our method. With a slight reduction in detection accuracy, our compressed domain detection system runs 1.8 times faster than the pixel domain.

Dynamic adaptive streaming over NDN typically relies on past information of network conditions and streaming quality. In this paper, we address the cold start problem associated with reinforcement learning based NDN adaptive streaming where a new consumer often found choosing bitrate arbitrarily due to the lack of experience. The idea is to construct a shared Q-Table which is continuously updated by previous consumers. Based on this Q-Table, a new consumer is expected to start choosing the segment bitrate more proactively. Simulations through ns-3 show that the proposed approach could help the consumers to find an optimal action from the beginning of the session.

Learned image compression (LIC) has illustrated good ability for reconstruction quality driven tasks (e.g. PSNR, MS-SSIM) and machine vision tasks such as image understanding. However, most LIC frameworks are based on pixel domain, which requires the decoding process. In this paper, we develop a learned compressed domain framework for machine vision tasks. 1) By sending the compressed latent representation directly to the task network, the decoding computation can be eliminated to reduce the complexity. 2) By sorting the latent channels by entropy, only selective channels will be transmitted to the task network, which can reduce the bitrate. As a result, compared with the traditional pixel domain methods, we can reduce about 1/3 multiply-add operations (MACs) and 1/5 inference time while keeping the same accuracy. Moreover, proposed channel selection can contribute to at most 6.8% bitrate saving.

Recently, requirement of city monitoring and maintenance using ICT techniques increases with the help of transportation system. In addition, the spread of COVID-19 has increased the demand for managing pedestrian traffic volume. To contribute to these trends, in this paper, we propose a new pedestrian radar map system in order to estimate pedestrian density on streets and sidewalks. Our system uses e-bikes to collect 360-degree images and visualize pedestrian positions as a radar map. In evaluations, we confirm the accuracies of the radar maps and pedestrian density by using KITTI dataset and by carrying out a field experiment.

Convolutional neural networks (CNNs) have achieved great successes on many computer vision tasks, such as image recognition, video processing, and target detection. In recent years, many hardware designs have been devoted to accelerating CNN inference. In order to further speed up CNN inference and reduce data waste, this work proposed a reconfigurable sliced systolic array: 1) Depending on the number of network nodes in each layer, the slice mode could be dynamically configured to achieve high throughput and resource utilization. 2) To take full advantage of convolution reuse and weight reuse, this work designed a tile-column sliding (TCS) processing dataflow. 3) A four-stage for loop algorithm was employed, which divides the CNN calculation into several parts based on the input nodes and output nodes. The entire CNN inference is carried out using integer-only arithmetic originated from TensorLite. Experimental results prove that these strategies lead to significant improvement in inference performance and energy efficiency.

As the demand for high-resolution videos grows, the next generation video coding standard Versatile Video Coding introduces many new proposals, including Adaptive Multiple Transforms (AMT), to improve coding efficiency. This paper presents a reconfigurable transform core for the VVC standard where the implementation of 1D DST-VII and DCT-VIII for all transform sizes are enabled. To offer a very low circuit complexity, a simple approximation strategy with a little coding performance loss is proposed. An 8×8 Processing Element (PE) array is employed as the core computational unit, where each PE can be configured dynamically based on the transform type. In addition, the transforms of larger sizes can be realized in the finite PE units with the Partitioned Matrix Multiplication (PMM) scheme. The experimental and synthesis results show that this design can save at least 29.1% area compared with other works in literature with the negligible degradation of video quality and a slight increase in the bit rate.

With the increasing video demand in daily network traffic, it is an urgent task to develop effective algorithms to facilitate high-quality content delivery service. Recently, numerous adaptive streaming algorithms have been proposed to improve the user perceived experience. However, these algorithms were mainly developed from the perspective of single user. There is not yet systematical evaluation and comparison of the bitrate adaptation methods for multi-user video streaming. Besides, the Quality of Experience (QoE) metrics were not unified. In this work, we propose a new mininet-based testbed framework which is able to conduct real-time video streaming emulation in various multi-user scenarios. Seven state-of-the-art adaptation methods are incorporated into the testbed. Meanwhile, ITU-T P.1203 model, the world's first standard for measuring QoE of HTTP adaptive streaming, is implemented to calculate the mean opinion scores of different methods. Using the developed testbed, the performance of current adaptation methods in multi-user network is analyzed and compared. A variety of experiments are carried out by changing the user number and network conditions, in which the QoE of different users are investigated. It is found that current algorithms perform inconsistently in various network scenarios. In the excessive user and limited bandwidth cases, machine learning and scheduling techniques show superiority in providing high and equal QoE for all users. While in the high-delay case, the buffer-based approaches show robust performance. Overall, the findings of this work give an insight for designing and choosing adaptive streaming strategies in different multi-user network conditions.

Intra prediction is an essential component in the image coding. This paper gives an intra prediction framework completely based on neural network modes (NM). Each NM can be regarded as a regression from the neighboring reference blocks to the current coding block. (1) For variable block size, we utilize different network structures. For small blocks 4×4 and 8×8, fully connected networks are used, while for large blocks 16×16 and 32×32, convolutional neural networks are exploited. (2) For each prediction mode, we develop a specific pre-trained network to boost the regression accuracy. When integrating into HEVC test model, we can save 3.55%, 3.03% and 3.27% BD-rate for Y, U, V components compared with the anchor. As far as we know, this is the first work to explore a fully NM based framework for intra prediction, and we reach a better coding gain with a lower complexity compared with the previous work.

This paper enhances the intra prediction by using multiple neural network modes (NM). Each NM serves as an end-To-end mapping from the neighboring reference blocks to the current coding block. For the provided NMs, we present two schemes (appending and substitution) to integrate the NMs with the traditional modes (TM) defined in high efficiency video coding (HEVC). For the appending scheme, each NM is corresponding to a certain range of TMs. The categorization of TMs is based on the expected prediction errors. After determining the relevant TMs for each NM, we present a probability-Aware mode signaling scheme. The NMs with higher probabilities to be the best mode are signaled with fewer bits. For the substitution scheme, we propose to replace the highest and lowest probable TMs. New most probable mode (MPM) generation method is also employed when substituting the lowest probable TMs. Experimental results demonstrate that using multiple NMs will improve the coding efficiency apparently compared with the single NM. Specifically, proposed appending scheme with seven NMs can save 2.6%, 3.8%, and 3.1% BD-rate for Y, U, and V components compared with using single NM in the state-of-The-Art works.

Convolutional neural network (CNN)-based filters have achieved great success
in video coding. However, in most previous works, individual models are needed
for each quantization parameter (QP) band. This paper presents a generic method
to help an arbitrary CNN-filter handle different quantization noise. We model
the quantization noise problem and implement a feasible solution on CNN, which
introduces the quantization step (Qstep) into the convolution. When the
quantization noise increases, the ability of the CNN-filter to suppress noise
improves accordingly. This method can be used directly to replace the (vanilla)
convolution layer in any existing CNN-filters. By using only 25% of the
parameters, the proposed method achieves better performance than using multiple
models with VTM-6.3 anchor. Besides, an additional BD-rate reduction of 0.2% is
achieved by our proposed method for chroma components.

Recent researches in video frame interpolation show great progress. In this paper, we propose a novel video compression method which incorporates deep learning based frame interpolation into HEVC which is the current video compression standard. Experimental results show that our approach can outperform HEVC in some scenarios.

Recently, an efficient and automatic infrastructure maintenance service is mandatory. To address this demand, in this paper, we introduce a segmentation-based road damage detection method by using U-Net. To train the model, we collect 4K images by using a smartphone mounted on a bicycle and build our own road damage dataset. In addition, to improve detection accuracy, we apply focal loss and image patch for loss function and input image, respectively. From the evaluation, the result confirms that the method demonstrates to extract road damages with acceptable accuracy.

In this paper, we propose a deep learning-based distance estimation method from a single-shot image. In the proposal, we model the estimation as a regression problem, and estimate the distance between a pedestrian and a camera by using three main features; size of bounding box, image blur and image features. By using KITTI dataset, we evaluate the accuracy of the proposed model.

Recently, in order to build barrier-free smart cities, Japanese government is actively promoting to construct open data regarding roads and buildings, including existence of bumps, road slope and corridor width. To contribute this activity, in this paper, we introduce a corridor width estimation method by using an RGB-D camera. In this method, we assume that the RGB-D camera is mounted on the electric wheelchairs for a mobile sensing platform. The corridor width estimation is mainly operated by three steps: planes extraction, side walls extraction and distance calculation. We evaluate the accuracy of this method at eight corridors in Waseda University.

To contribute to publishing pedestrian networks as open data, this paper introduces a curb height estimation method by using 2DLiDAR mounted on e-Bike. We develop a prototype system by using e-Bike, 2DLiDAR and other sensors, and collect actual curb data around Waseda University. From accuracy evaluations, the results confirm that the prototype can estimate road level differences and positions with high accuracy.

Under complex network conditions, adaptive video streaming requires additional state information for optimal quality selection. In this paper, we present the applicability of reinforcement learning techniques on NDN adaptive streaming. Both buffer-based and throughput-based adaptation are studied and observed their characteristics. The Q-learning algorithm is used to learn state-action values. Based on a greedy policy, the simulation results demonstrate that RL agents tend to choose the best possible bitrate which consequently reduces the quality fluctuation in adaptive streaming.

Learned image compression (LIC) has reached the traditional hand-crafted methods such as JPEG2000 and BPG in terms of the coding gain. However, the large model size of the network prohibits the usage of LIC on resource-limited embedded systems. This paper presents a LIC with 8-bit fixed-point weights. First, we quantize the weights in groups and propose a non-linear memory-free codebook. Second, we explore the optimal grouping and quantization scheme. Finally, we develop a novel weight clipping fine tuning scheme. Experimental results illustrate that the coding loss caused by the quantization is small, while around 75% model size can be reduced compared with the 32-bit floating-point anchor. As far as we know, this is the first work to explore and evaluate the LIC fully with fixed-point weights, and our proposed quantized LIC is able to outperform BPG in terms of MS-SSIM.

Recently learned image compression has achieved many great progresses, such as representative hyperprior and its variants based on convolutional neural networks (CNNs). However, CNNs are not fit for scalable coding and multiple models need to be trained separately to achieve variable rates. In this paper, we incorporate differentiable quantization and accurate entropy models into recurrent neural networks (RNNs) architectures to achieve a scalable learned image compression. First, we present an RNN architecture with quantization and entropy coding. To realize the scalable coding, we allocate the bits to multiple layers, by adjusting the layer-wise lambda values in Lagrangian multiplier-based rate-distortion optimization function. Second, we add an RNN-based hyperprior to improve the accuracy of entropy models for multiple-layer residual representations. Experimental results demonstrate that our performance can be comparable with recent CNN-based hyperprior methods on Kodak dataset. Besides, our method is a scalable and flexible coding approach, to achieve multiple rates using one single model, which is very appealing.

Convolutional Neural Network (CNN)-based filters have achieved significant
performance in video artifacts reduction. However, the high complexity of
existing methods makes it difficult to be applied in real usage. In this paper,
a CNN-based low complexity filter is proposed. We utilize depth separable
convolution (DSC) merged with the batch normalization (BN) as the backbone of
our proposed CNN-based network. Besides, a weight initialization method is
proposed to enhance the training performance. To solve the well known over
smoothing problem for the inter frames, a frame-level residual mapping (RM) is
presented. We analyze some of the mainstream methods like frame-level and
block-level based filters quantitatively and build our CNN-based filter with
frame-level control to avoid the extra complexity and artificial boundaries
caused by block-level control. In addition, a novel module called RM is
designed to restore the distortion from the learned residuals. As a result, we
can effectively improve the generalization ability of the learning-based filter
and reach an adaptive filtering effect. Moreover, this module is flexible and
can be combined with other learning-based filters. The experimental results
show that our proposed method achieves significant BD-rate reduction than
H.265/HEVC. It achieves about 1.2% BD-rate reduction and 79.1% decrease in
FLOPs than VR-CNN. Finally, the measurement on H.266/VVC and ablation studies
are also conducted to ensure the effectiveness of the proposed method.

For the next-generation video coding standard Versatile Video Coding (VVC), several new contributions have been proposed to improve the coding efficiency, especially in the transformation operations. This paper proposes a unified 32× 32 block-based transform architecture for the VVC standard that enables 2D Discrete Sine Transform-VII (DST-VII) and Discrete Cosine Transform-VIII (DCT-VIII) of all sizes. It mainly gives three contributions: 1) The N-Dimensional Reduced Adder Graph (RAG-n) algorithm is adopted to design the minimal adder-oriented computational units. 2) The storage of the asymmetric transform units can be realized in the dual-port SRAM-based transpose memory. 3) The pipelined 2D transformations of mixed block sizes are achieved with the throughput rate of 32 samples per cycle. The synthesis results indicate that this architecture can reduce area by up to 73.1% compared with other state-of-the-art works. Moreover, power saving ranging from 4.9% to 9.9% can be achieved. Regarding the transpose memory, at least 21.9% of the area can be saved by using SRAM.

With the continuous development of mobile devices, it becomes possible for people to demand higher definition videos. To alleviate the pressure of deploying the video codec in mobile multimedia, a learning-based low complexity in-loop filter is proposed in this paper. Depthwise separable convolution is combined with batch normalization to construct this model. To enhance its performance, the knowledge from a pre-trained teacher model is transferred to it. However, the over-smoothing problem in the inter frames caused by double enhancing effect remains. To solve this, a Wiener-based filtering algorithm that tries to restore the distortion from the learned residuals is designed and introduces an adequate filtering effect. The experimental results show that our proposed methods achieve considerable BD-rate reduction than HEVC anchor. Compared with the previous state-of-the-art work VR-CNN, our model achieves 1.65% extra BD-rate reduction, 79.1% decrease in FLOPs, 25% decrease in encoding complexity, and 70% decoding complexity decrease.

In this paper, a coding framework VIP-ICT-Codec is introduced. Our method is based on the VTM (Versatile Video Coding Test Model). First, we propose a color space conversion from RGB to YUV domain by using a PCA-like operation. A method for the PCA mean calculation is proposed to de-correlate the residual components of YUV channels. Besides, the correlation of UV components is compensated considering that they share the same coding tree in VVC. We also learn a residual mapping to alleviate the over-filtered and under-filtered problem of specific images. Finally, we regard the rate control as an unconstraint Lagrangian problem to reach the target bpp. The results show that we achieve 32.625dB at the validation phase.

In this paper, we provide a detailed description on our submitted method Kattolab to Workshop and Challenge on Learned Image Compression (CLIC) 2020. Our method mainly incorporates discretized Gaussian Mixture Likeli-hoods to previous state-of-the-art learned compression algorithms. Besides, we also describes the acceleration strategies and bit optimization with the low-rate constraint. Experimental results have demonstrated that our approach Kattolab achieves 0.9761 in terms of MS-SSIM at the rate constraint of 0.15 bpp during the validation phase.

Lossless image compression is an important task in the field of multimedia communication. Traditional image codecs typically support lossless mode, such as WebP, JPEG2000, FLIF. Recently, deep learning based approaches have started to show the potential at this point. HyperPrior is an effective technique proposed for lossy image compression. This paper generalizes the hyperprior from lossy model to lossless compression, and proposes a L2-norm term into the loss function to speed up training procedure. Besides, this paper also investigated different parameterized models for latent codes, and propose to use Gaussian mixture likelihoods to achieve adaptive and flexible context models. Experimental results validate our method can outperform existing deep learning based lossless compression, and outperform the JPEG2000 and WebP for JPG images.

Image compression has been an important research topic for many decades. Recently, deep learning has achieved great success in many computer vision tasks, and its use in image compression has gradually been increasing. In this paper, we present an energy compaction-based image compression architecture using a convolutional autoencoder (CAE) to achieve high coding efficiency. Our main contributions include three aspects: 1) we propose a CAE architecture for image compression by decomposing it into several down(up)sampling operations; 2) for our CAE architecture, we offer a mathematical analysis on the energy compaction property and we are the first work to propose a normalized coding gain metric in neural networks, which can act as a measurement of compression capability; 3) based on the coding gain metric, we propose an energy compaction-based bit allocation method, which adds a regularizer to the loss function during the training stage to help the CAE maximize the coding gain and achieve high compression efficiency. The experimental results demonstrate our proposed method outperforms BPG (HEVC-intra), in terms of the MS-SSIM quality metric. Additionally, we achieve better performance in comparison with existing bit allocation methods, and provide higher coding efficiency compared with state-of-the-art learning compression methods at high bit rates.

Recently, a fifth-generation cellular system (5G) is widely expected to provide plenty of wireless network resources (i.e., broadband capacity). In this paper, to validate 5G system performances, such as physical-layer and TCP-layer throughputs, we carry out a field trial at an actual indoor train station, named Haneda International Airport Terminal Station. In the field trial, we deploy the prototype 5G system (Central Unit, Distribution Unit, Radio Unit and 5G UE (tablet)) on the train station platform and evaluate mobile 5G downlink throughputs. Through the actual measurements, the results confirm that the prototype 5G system can achieve mobile broadband capacity (more than 1 Gbps) even when the UE is located anywhere at the indoor train station platform.

The amount of video data is so large that efficient video compression is required for the storage and transmission. Intra prediction is one of important components in video compression. In this paper, we examine various Convolutional Neural Network (CNN) structures to estimate optimal intra prediction mode as Most Probable Modes (MPMs). Moreover, we investigate several combinations of the MPMs obtained by the CNN and MPMs derived from High Efficiency Video Coding Test Model (HM). From these experimental results, we find that using 6 MPMs from both CNN and HM with moderate number of channels or kernel size is preferred.

Different from the traditional quaternary tree (QT) structure utilized in the previous generation video coding standard H.265/HEVC, a brand new partition structure named quadtree with nested multi-type tree (QTMT) is applied in the latest codec H.266/VVC. The introduction of QTMT brings in superior encoding performance at the cost of great time-consuming. Therefore, a fast intra partition algorithm based on variance and Sobel operator is proposed in this paper. The proposed method settles the novel asymmetrical partition issue in VVC by well balancing the reduction of computational complexity and the loss of encoding quality. To be more concrete, we first terminate further splitting of a coding unit (CU) when the texture of it is judged as smooth. Then, we use Sobel operator to extract gradient features to decide whether to split this CU by QT, thus terminating further MT partitions. Finally, a completely novel method to choose only one partition from five QTMT partitions is applied. Obviously, homogeneous area tends to use a larger CU as a whole to do prediction while CUs with complicated texture are prone to be divided into small sub-CUs and these sub-CUs usually have different textures from each other. We calculate the variance of variance of each sub-CU to decide which partition will distinguish the sub-textures best. Our method is embedded into the latest VVC official reference software VTM-7.0. Comparing to anchor VTM-7.0, our method saves the encoding time by 49.27% on average at the cost of only 1.63% BDBR increase. As a traditional scheme based on variance and gradient to decrease the computational complexity in VVC intra coding, our method outperforms other relative existing state-of-the-art methods, including traditional machine learning and convolution neural network methods.

Image compression is a fundamental research field and many well-known compression standards have been developed for many decades. Recently, learned compression methods exhibit a fast development trend with promising results. However, there is still a performance gap between learned compression algorithms and reigning compression standards, especially in terms of widely used PSNR metric. In this paper, we explore the remaining redundancy of recent learned compression algorithms. We have found accurate entropy models for rate estimation largely affect the optimization of network parameters and thus affect the rate-distortion performance. Therefore, in this paper, we propose to use discretized Gaussian Mixture Likelihoods to parameterize the distributions of latent codes, which can achieve a more accurate and flexible entropy model. Besides, we take advantage of recent attention modules and incorporate them into network architecture to enhance the performance. Experimental results demonstrate our proposed method achieves a state-of-the-art performance compared to existing learned compression methods on both Kodak and high-resolution datasets. To our knowledge our approach is the first work to achieve comparable performance with latest compression standard Versatile Video Coding (VVC) regarding PSNR. More importantly, our approach generates more visually pleasant results when optimized by MS-SSIM.

Surveillance cameras are commonly used for security purpose. However, most of them are used for verification after incidents happen. In this paper, we propose a method for proactive video surveillance system using human pose estimation. Our method estimates anomaly scores of human actions using the video descriptors (human pose and bounding box) extracted by pose estimation and tracking methods. We estimate human poses, apply PCA, estimate GMM parameters, and finally calculate anomaly scores based on the GMM. We evaluate our method and compare with higher-level recognition-based method. Experimental results demonstrate effectiveness of human pose for video anomaly detection.

Recently, deep learning techniques have contributed to solving a multitude of computer vision tasks. In this paper, we propose a deep-learning approach for video alignment, which involves finding the best correspondences between two overlapping videos. We formulate the video alignment task as a variant of the well-known machine comprehension (MC) task in natural language processing. While MC answers a question about a given paragraph, our technique determines the most relevant frame sequence in the context video to the query video. This is done by representing the individual frames of the two videos by highly discriminative and compact descriptors. Next, the descriptors are fed into a multi-stage network that is able, with the help of the bidirectional attention flow mechanism, to represent the context video at various granularity levels besides estimating the query-aware context part. The proposed model was trained on 10k video-pairs collected from 'YouTube'. The obtained results show that our model outperforms all known state of the art techniques by a considerable margin, confirming its efficacy.

Quadtree with nested multi-Type tree (QTMT) partition structure in Versatile Video Coding (VVC) contributes to superior encoding performance compared to the basic quad-Tree (QT) structure in High Efficiency Video Coding (HEVC). However, the improvement of performance leads to an un-Avoidable increase of computational complexity. To achieve a balance between coding efficiency and compression quality, we propose a fast intra partition algorithm based on variance and gradient to solve the rectangular partition problem in VVC. First, further splitting of smooth areas is terminated. Then, QT partition is chosen depending on the gradient features extracted by Sobel operator. Finally, one partition from five possible QTMT partitions is directly chosen by computing the variance of variance of sub-CUs. The theoretical basis of our method is that a homogeneous area tends to be predicted with a larger coding unit (CU), and sub-parts of a split CU are prone to have different textures from each other. To our knowledge, this is the first attempt to apply traditional method to accelerating the rectangular partition problem in VVC intra prediction. Experimental results show that the proposed method can save averagely 53.17% encoding time with only 1.62% BDBR increase and 0.09dB BDPSNR loss compared to anchor VTM4.0.

Recently, the demand for high quality 360-degree video delivery is increasing, however, 360-degree videos require extremely high bitrate and large network capacity. Therefore, an efficient (i.e., higher quality and lower traffic) 360-degree video delivery is mandatory. To address this fact, this paper introduces and evaluates a tile-based 360-degree video delivery system that equips viewport movement prediction and rate adaptation.

In this paper, a dual learning-based method in intra coding is introduced for PCS Grand Challenge. This method is mainly composed of two parts: intra prediction and reconstruction filtering. They use different network structures, the neural network-based intra prediction uses the full-connected network to predict the block while the neural network-based reconstruction filtering utilizes the convolutional networks. Different with the previous filtering works, we use a network with more powerful feature extraction capabilities in our reconstruction filtering network. And the filtering unit is the block-level so as to achieve a more accurate filtering compensation. To our best knowledge, among all the learning-based methods, this is the first attempt to combine two different networks in one application, and we achieve the state-of-the-art performance for AI configuration on the HEVC Test sequences. The experimental result shows that our method leads to significant BD-rate saving for provided 8 sequences compared to HM-16.20 baseline (average 10.24% and 3.57% bitrate reductions for all-intra and random-access coding, respectively). For HEVC test sequences, our model also achieved a 9.70% BD-rate saving compared to HM-16.20 baseline for allintra configuration.

In this paper, we present a road infrastructure monitoring system using e-bikes to support smart community. Smartphone and IoT (Internet of Things) devices are attached to e-bikes, and electric power is supplied from the batteries of e-bikes. Image processing techniques based on deep learning are applied and run on IoT devices or cloud backbone. A prototype system is implemented, which performs from image capturing to web browsing of the processed images.

Adaptive streaming on Named Data Networking appears to be a challenging task since the provider is unknown to the consumer. In this paper, we present an approach to efficiently adjust the streaming bit-rate by deploying IoT devices. Each NDN node is designed to send IoT devices the information of available bandwidth. The IoT devices then assist the consumers if the current network can deliver 4K video segments. The experiments using NDNx and NDN-JS show that the consumer in the proposed approach could select representation bit-rate more accurately than the traditional rate adaptation method.

Recently, Ultra HDTV (UHDTV) services become popular over satellite and on the internet. On the contrary, there are tremendously huge volume of High Definition Television (HDTV) and Standard Definition Television (SDTV) contents stored in broadcasting companies and storage devices. In this paper, we propose a color space conversion (also known as gamut mapping) method from BT. 709 (used for current HDTV broadcast) to BT. 2020 (used for UHDTV broadcast), which estimates and restores lost color information. It learns an end-to-end conversion method from BT. 709 image to BT. 2020 image with restoring lost color information using Convolutional Neural Network (CNN). By experiments, we confirm that our method can achieve 2.31dB gain against the conventional method on average.

Intelligent sensing is an important application field of video coding. For the next generation video coding standard Versatile Video Coding (VVC), several new contributions have been proposed to improve the coding efficiency, especially in the transformation operations. This paper proposes a unified transform architecture for VVC standard that enables 1D Discrete Sine Transform-VII (DST-VII) and Discrete Cosine Transform-VIII (DCT-VIII) of all sizes. In order to minimize the number of adders, the N-Dimensional Reduced Adder Graph (RAG-n) algorithm is adopted to design the logical computation of the VVC transforms. In addition, a pipeline operation is employed to achieve a high throughput rate of 32 samples per cycle. The experimental and synthesis results show that our proposal can save 16.96% area and 9.89% power consumption than the direct implementation by the multipliers. Moreover, this design can reduce the normalized area (NA) by at least 57.53% compared with other advanced works.

Recently deep learning based image compression has made rapid advances with promising results based on objective quality metrics. However, a rigorous subjective quality evaluation on such compression schemes have rarely been reported. This paper aims at perceptual quality studies on learned compression. First, we build a general learned compression approach, and optimize the model. In total six compression algorithms are considered for this study. Then, we perform subjective quality tests in a controlled environment using high-resolution images. Results demonstrate learned compression optimized by MS-SSIM yields competitive results that approach the efficiency of state-of-the-art compression. The results obtained can provide a useful benchmark for future developments in learned image compression.

Abstract-Virtualized service and network functions are deployed on virtual machines (VMs) to realize essential processing to realize service function chaining (SFC). Issues on SFC is SF allocation to a VM and to minimize the response time and number of function instances. In this paper, we propose an SF clustering-based scheduling algorithm, called 'SF-clustering for utilizing virtual CPUs' (SFCUV), to solve the SF allocation and SF selection problems simultaneously. Experimental results show that SF-CUV can utilize vCPUs to minimize the response time.

In this paper, we provide a detailed description on our approach designed for<br />
CVPR 2019 Workshop and Challenge on Learned Image Compression (CLIC). Our<br />
approach mainly consists of two proposals, i.e. deep residual learning for<br />
image compression and sub-pixel convolution as up-sampling operations.<br />
Experimental results have indicated that our approaches, Kattolab, Kattolabv2<br />
and KattolabSSIM, achieve 0.972 in MS-SSIM at the rate constraint of 0.15bpp<br />
with moderate complexity during the validation phase.

To develop and interoperate smart city applications efficiently, smart city IoT platforms require efficient handling of various types of sensor devices, networking and computing resources, and different domain applications. To address this fact, the authors of this paper proposed a research project named 'Fed4IoT', which is an acronym for federation of IoT and cloud infrastructures, to provide scalable and interoperable smart city applications. The project primarily proposes two key technologies: IoT device virtualization and context-information sharing. In this paper, we introduce the IoT device virtualization (one of key technology in Fed4IoT) that enables efficient utilization of networking and computing resources by applying a micro-service sharing and dynamic resource scaling. In the performance evaluations, we implement three micro services, such as object detection, garbage detection and road damage detection, as smart city applications and dynamically and flexibly allocate computing resources to three micro services by using Docker and Kubernetes. Through the evaluations we validate that the proposed IoT device virtualization achieves the efficient computing resource utilization and reduction of networking resources.

Compression has been an important research topic for many decades, to produce a significant impact on data transmission and storage. Recent advances have shown a great potential of learning based image and video compression. Inspired from related works, in this paper, we present an image compression architecture using a convolutional autoencoder, and then generalize image compression to video compression, by adding an interpolation loop into both encoder and decoder sides. Our basic idea is to realize spatial-temporal energy compaction in learning image and video compression. Thereby, we propose to add a spatial energy compaction-based penalty into loss function, to achieve higher image compression performance. Furthermore, based on temporal energy distribution, we propose to select the number of frames in one interpolation loop, adapting to the motion characteristics of video contents. Experimental results demonstrate that our proposed image compression outperforms the latest image compression standard with MS-SSIM quality metric, and provides higher performance compared with state-of-the-art learning compression methods at high bit rates, which benefits from our spatial energy compaction approach. Meanwhile, our proposed video compression approach with temporal energy compaction can significantly outperform MPEG-4, and is competitive with commonly used H.264. Both our image and video compression can produce more visually pleasant results than traditional standards.

Recently, the demand for immersive VR (360-degree) video delivery is increasing, and an efficient (high quality and low traffic) 360-degree video streaming methodology is mandatory. To address this fact, in this paper, we introduce a tile-based 360-degree DASH streaming with viewport prediction. We evaluate the prediction accuracy of future viewport movement patterns by using clustering and image and audio information. In addition, we perform rate adaptation based on viewport prediction results. Through evaluations, we confirm the tile-based 360-degree DASH streaming achieves higher objective image quality and lower total video traffic volume.

Towards an intelligent video surveillance system (high throughput and high detection accuracy), in this paper, we quantitatively validate the network performance of 60GHz IEEE 802.11ad and the contribution of 4K resolution to image-based human pose detection. In the performance evaluation of 11ad, we evaluate TCP throughputs by changing the communication distances (up-to 6m). In the performance evaluation of human pose detection, we evaluate the detection accuracy of human pose detection by changing a distance between a camera and a subject (up-to 100 m) and input image resolutions.

To realize highly reliable video surveillance and provide ultrahigh-definition/immersive video streaming, it is planned to adopt the 5G cellular system using millimeter-wave (mmWave) as the wireless-network infrastructure. However, mmWave communication has a challenging issue: mmWave communication is extremely sensitive to obstacles, such as walls, pillars, and even human bodies, and this issue easily increases the packet loss rates and round trip time (RTT) (or disconnection from the base station) due to a no line of sight (NLOS) environment. Therefore, in this work, 5G throughput performances were evaluates in an indoor train station by considering the effect of an NLOS environment caused by blockage by human bodies. In addition, to improve the robustness of TCP transmission in a high-RTT and high-packet-loss environment (e.g., an NLOS environment), a state-of-the-art TCP, TCP-FSO, was used. In the evaluations, the MATLAB 5G library was used to simulate the 5G environment, and a Linux software-based network emulator, Traffic Control, was used to emulate the 5G network. From the evaluations, it the 5G mobile throughput characteristics were confirmed in three different crowded patterns (low, middle, and high density), and the TCP-FSO advantage against CUBIC-TCP was validated.

To recognize transportation modes without any additional sensor devices, we demonstrate that the transportation modes can be recognized from communication quality factors. In the demonstration, instead of using global positioning system (GPS) and accelerometer sensors, we collect mobile TCP throughputs, received-signal strength indicators (RSSIs), and cellular base-station IDs (Cell IDs) through in-line network measurement when the user enjoys mobile services, such as video streaming. In accuracy evaluations, we conduct two different field experiments to collect the data in six typical transportation modes (static, walking, riding a bicycle, riding a bus, riding a train and riding a subway), and then construct the classifiers by applying a support-vector machine (SVM), k-nearest neighbor (k-NN), random forest (RF), and convolutional neural network (CNN). Our results show that these transportation modes can be recognized with high accuracy by using communication quality factors as well as the use of accelerometer sensors.

New video coding standards bring high coding efficiency at the cost of large computational complexity. Especially, in 8K sequence, computation cost becomes much larger due to its super high spatial resolution. In this paper, we propose a method to improve 8K UHDTV coding performance of H.265/HEVC. Our proposed method pre-analyzes spatial and motion complexity, and selects an optimal GOP size or intra/inter prediction mode. Experimental results suggest that the proposed method can improve coding efficiency and reduce computational cost.

The popularity of high dynamic range (HDR) makes the inverse tone mapping become an important technique for HDR display. In this paper, we propose a convolutional neural network (CNN)-based inverse tone mapping method to generate a high-quality HDR image from one single standard dynamic range (SDR) image. First, we present a CNN design with a three- channel input, which considers both luminance and chrominance. Second, we propose to use overlapped inputs to remove the boundary artifacts, caused by zero paddings in CNN. Experimental results demonstrate the high quality of our generated HDR images compared to the ground truth.

Recently, with the spread of Internet of Things (IoT), video surveillance system is widely used to detect abnormal activity using cameras and sensors. In such video surveillance system, contact-free heart rate measurement is a highly required technology to predict the occurrence of abnormal activities and suspicious humans. In this paper, our main contributions can be summarized by two aspects. First, we compare the accuracy performance of heart rate measurement using six heart beat waveform acquisition methods and two heart rate calculation methods. Second, we discuss the influence of image resolutions of RGB cameras on accurate performance, to investigate the requirement of surveillance cameras in actual scenarios.

To develop and interoperate smart city applications efficiently, smart city IoT platforms require efficient handling of various types of sensor devices, networking and computing resources, and different domain applications. To address this fact, in this paper, we introduce an IoT device virtualization that enables efficient utilization of computing resources. The proposal applies a micro-service sharing and dynamic resource scaling. In the performance validations, we implement an early prototype using Docker, Kubernetes, and Apache Kafka. Through the preliminary experiment, we confirm that the proposal can improve the application processing time by appropriately sharing and scaling micro services.

In video streaming on the internet, effective encoding recipes (i.e. bitrate-resolution pairs) are a main obstacle to deliver high-quality video streams. We developed a method to generate an encoding recipe that considers subjective visual quality with one just-noticeable difference (JND) distance. However, this method requires excessive computation time, which is not directly applicable for live streaming. In this paper, in order to provide a light-weight method for live streaming, we developed three acceleration techniques: resolution extrapolation, VMAF skipping and sampled objective measure calculation. These techniques are heuristic, but greatly contribute to reducing computational cost. Experimental results demonstrate that the proposed method achieves a significant reduction in computation time without significant effects on rate-JND characteristics.

This paper introduces recent trends in video streaming and four methods proposed by the authors for video streaming. Video traffic dominates the Internet as seen in current trends, and new visual contents such as UHD and 360-degree movies are being delivered. MPEG-DASH has become popular for adaptive video streaming, and machine learning techniques are being introduced in several parts of video streaming. Along with these research trends, the authors also tried four methods: route navigation, throughput prediction, image quality assessment, and perceptual video streaming. These methods contribute to improving QoS/QoE performance and reducing power consumption and storage size.

Recently, ultra high definition television (UHDTV) services have become popular using satellites and the Internet. However, there are expansive volumes of high definition television (HDTV) and standard definition television (SDTV) contents held by broadcasting companies and in storage devices. Herein we propose two color space conversion (also known as gamut mapping) methods from BT.709 (used for current HDTV broadcast) to BT.2020 (used for UHDTV broadcast) that restore or estimate lost color information. One of our methods anisotropically diffuses the BT.709 chromaticities with regard to the direction of the original chromaticities in the BT.2020 color space, generating chromaticities out of BT.709 gamut. The other learns an end-to-end conversion method from a BT.709 image to a BT.2020 image and restores lost color information using convolutional neural network (CNN). Using these methods along with BT.709 images, we obtain BT.2020 images with chromaticities from the BT.709 color gamut.

Dynamic adaptive video streaming over HTTP (DASH) is widely studied and has been adopted in modern video players to ensure user quality of experience (QoE). In DASH, adaptive bitrate control is a key part whose ultimate goal is to maximize video bitrate while minimizing rebuffering. Throughput prediction plays an important role in helping select the proper video bitrate dynamically. In this paper, we studied the influence of throughput prediction on adaptive video streaming. Because the real-world network is dynamic, different methods need to be tested with large-scale deployments and analyzed statistically. However, this is difficult in academic research. Therefore, we established a reproducible trace-based emulation environment, which enables us to compare different methods quantitatively under the artificially same condition, with limited experiments. The throughput prediction methods are implemented into DASH to evaluate the effect on QoE for video streaming. The results indicate that the prediction method using long short-term memory (LSTM) performs better than the other methods. However, throughput prediction alone is not enough to ensure high QoE. To further improve the QoE, we proposed the decision map method (DMM), where the buffer occupancy is also incorporated to make a selection. By using this decision map, the choice of bitrate can be smarter than that when only prediction information is used. The total QoE is further improved by 32.1% in the ferry trace, which shows the effectiveness of DMM in further improving the performance of throughput prediction in adaptive bitrate control.

In this paper, a novel technique is introduced to address the video alignment task which is one of the hot topics in computer vision. Specifically, we aim at finding the best possible correspondences between two overlapping videos without the restrictions imposed by previous techniques. The novelty of this work is that the video alignment problem is solved by drawing an analogy between it and the machine comprehension (MC) task in natural language processing (NLP). Simply, MC seeks to give the best answer to a question about a given paragraph. In our work, one of the two videos is considered as a query, while the other as a context. First, a pre-trained CNN is used to obtain high-level features from the frames of both the query and context videos. Then, the bidirectional attention flow mechanism; that has achieved considerable success in MC; is used to compute the query-context interactions in order to find the best mapping between the two input videos. The proposed model has been trained using 10k of collected video pairs from”YouTube”. The initial experimental results show that it is a promising solution for the video alignment task when compared to the state of the art techniques.

To ensure good quality of experience for user when transmitting video content, throughput prediction can contribute to the selection of proper bitrate. In this paper, we propose a throughput prediction method with recurrent neural network (RNN) model. Experiments are conducted to evaluate the methods, and the results indicate that proposed method can decrease the prediction error by a maximum of 29.39% compared with traditional methods.

Image quality assessment (IQA) is an inherent research topic in image processing field for several decades. Recently, machine learning has achieved success in many multimedia tasks and can be applied in IQA. Especially, screen content images (SCIs) is greatly increasing in various applications, but the characteristics of SCIs makes it difficult to directly apply general IQA methods to predict qualities. In this paper, we propose a fast no-reference SCIs quality prediction method. First, we use the convolutional neural networks (CNNs) to predict the quality scores of each patch. Second, we present a SCIs-oriented quality aggregation algorithm for acceleration. Experimental results demonstrate that our method can achieve the high accuracy (0.957) with subjective quality scores, outperforming existing methods. Moreover, our method is computationally appealing, achieving flexible complexity performance by selecting different groups of patches.

The popularity of high dynamic range (HDR) makes the inverse tone mapping become an important technique for HDR display. In this paper, we propose a convolutional neural network (CNN) based inverse tone mapping method to generate a high-quality HDR image from one single standard dynamic range (SDR) image. First, we present a CNN design with a three-channel input, which considers both luminance and chrominance. Second, we propose to use overlapped inputs to remove the boundary artifacts, caused by zero padding in CNN. Experimental results demonstrate the high quality of our generated HDR images compared to the ground truth and conventional inverse tone mapping methods.

In order to recognize the transportation modes without any additional sensor devices, we propose a recognition method by using communication quality factors. In the proposed method, instead of Global Positioning System (GPS) and accelerometer sensors, we collect mobile TCP throughputs, Received Signal Strength Indicators (RSSIs), and cellular base station IDs (Cell IDs) through in-line network measurement when the user enjoys mobile services, such as video streaming service. In accuracy evaluations, we conduct two different field experiments to collect the data in five typical transportation modes (static, walking, riding a bicycle, a bus and a train,) and then construct the classifiers by applying Support Vector Machine (SVM), k-Nearest Neighbor (k-NN) and Random Forest (RF). Results conclude that these transportation modes can be recognized by using communication quality factors with high accuracy as well as the use of accelerometer sensors.

In this paper, to sustain a high-quality 24-hour video surveillance (i.e., high reliability) and reduce redundant video traffic volume (i.e., network friendliness), we propose an edge-centric video surveillance system that provides flexible adaptive control of the image enhancement process and video quality based on an event-driven adaptation. In the proposed system, the video bitrate is adaptively controlled according to the contrast of captured videos and conditions in a monitored area (e.g., 'normal', 'caution', and 'alert'). To confirm the system performance, we evaluate objective image quality, accuracy of human detection and video traffic volume generated by the proposed system. Evaluations conclude that the system can reduce the video traffic while sustaining high-quality visibility.

Named Data Networking (NDN) has been recently introduced as a new network architecture. The implementation of adaptive bit-rate video streaming on NDN is one of the most challenging tasks. The in-network caching feature of NDN could mislead a consumer to overestimate the end-to-end throughput for the next video segment. In this paper, we propose a strategy for more efficient rate adaptation by allowing Interest message adaptation in a hop-by-hop fashion. Moreover, the proactive content caching is designed to be utilized on the related router when a bottleneck on the transmission path was detected. The experiments using NDN-JS and DASH-JS demonstrate that our proposed solution provides better average stream bit-rate than the traditional rate adaptation method. Also, the link bandwidth consumption for video data transmission using the proposed scheme could be reduced efficiently.

We introduce a perceptual video quality driven video encoding solution for optimized adaptive streaming. By using multiple bitrate/resolution encoding like MPEG-DASH, video streaming services can deliver the best video stream to a client, under the conditions of the client's available bandwidth and viewing device capability. However, conventional fixed encoding recipes (i.e., resolution-bitrate pairs) suffer from many problems, such as improper resolution selection and stream redundancy. To avoid these problems, we propose a novel video coding method, which generates multiple representations with constant JustNoticeable Difference (JND) interval. For this purpose, we developed a JND scale estimator using Support Vector Regression (SVR), and designed a pre-encoder which outputs an encoding recipe with constant JND interval in an adaptive manner to input video.

Image compression has been investigated as a fundamental research topic for many decades. Recently, deep learning has achieved great success in many computer vision tasks, and is gradually being used in image compression. In this paper, we present a lossy image compression architecture, which utilizes the advantages of convolutional autoencoder (CAE) to achieve a high coding efficiency. First, we design a novel CAE architecture to replace the conventional transforms and train this CAE using a rate-distortion loss function. Second, to generate a more energy-compact representation, we utilize the principal components analysis (PCA) to rotate the feature maps produced by the CAE, and then apply the quantization and entropy coder to generate the codes. Experimental results demonstrate that our method outperforms traditional image coding algorithms, by achieving a 13.7% BD-rate decrement on the Kodak database images compared to JPEG2000. Besides, our method maintains a moderate complexity similar to JPEG2000.

Image quality assessment (IQA) is an inherent problem in the field of image processing. Recently, deep learning-based image quality assessment has attracted increased attention, owing to its high prediction accuracy. In this paper, we propose a fully-blind and fast image quality predictor (FFIQP) using convolutional neural networks including two strategies. First, we propose a distortion clustering strategy based on the distribution function of intermediate-layer results in the convolutional neural network (CNN) to make IQA fully blind. Second, by analyzing the relationship between image saliency information and CNN prediction error, we utilize a pre-saliency map to skip the non-salient patches for IQA acceleration. Experimental results verify that our method can achieve the high accuracy (0.978) with subjective quality scores, outperforming existing IQA methods. Moreover, the proposed method is highly computationally appealing, achieving flexible complexity performance by assigning different thresholds in the saliency map.

Adaptive rate video streaming on a novel Internet architecture, called Named Data Networking (NDN), appears to be a promising technology. However, given the source of content on NDN is unknown to the consumer, rate adaptation becomes inefficient. Moreover, the traffic at some nodes holding popular contents still can be congested. In this paper, we utilize the Internet of Things (IoT) device for assisting content delivery at an NDN router that has low bandwidth. The location of IoT devices installation is determined by a metric calculated from link bandwidth, delay, and utilization. We performed the experiments using DASH-NDN-JS on the setup network. The results demonstrate that our proposal could support NDN router to deliver video content to additional consumers efficiently. Thus, the network scalability can be improved with a reasonable cost.

Because mobile users demand a high quality and energyfriendly video delivery service that efficiently uses wireless resources, we introduce an energy-efficient video delivery system by applying moving route navigation and playout buffer control based on the mobile throughput history data. The proposed system first determines the optimal travel route to achieve high-speed and energy-efficient communications. Then when a user enters a high throughput area, our system temporarily extends the video playout buffer size, and the user aggressively downloads video segments via a high-speed and energy-efficient wireless connection until the extended buffer is filled. After leaving this area, the user consumes video segments from the extended buffer in order to keep smooth video playback without wireless communications. We carry out computer simulations, laboratory and field experiments and confirm that the proposed system can achieve energy-efficient mobile video delivery.

Throughput prediction is one of the promising techniques to improve the quality of service (QoS) and quality of experience (QoE) of mobile applications. To address the problem of predicting future throughput distribution accurately during the whole session, which can exhibit large throughput fluctuations in different scenarios (especially scenarios of moving user), we propose a history-based throughput prediction method that utilizes time series analysis and machine learning techniques for mobile network communication. This method is called the Hybrid Prediction with the Autoregressive Model and Hidden Markov Model (HOAH). Different from existing methods, HOAH uses Support Vector Machine (SVM) to classify the throughput transition into two classes, and predicts the transmission control protocol (TCP) throughput by switching between the Autoregressive Model (AR Model) and the Gaussian Mixture Model-Hidden Markov Model (GMM-HMM). We conduct field experiments to evaluate the proposed method in seven different scenarios. The results show that HOAH can predict future throughput effectively and decreases the prediction error by a maximum of 55.95% compared with other methods.

Image compression has been investigated for many decades. Recently, deep<br />
learning approaches have achieved a great success in many computer vision<br />
tasks, and are gradually used in image compression. In this paper, we develop<br />
three overall compression architectures based on convolutional autoencoders<br />
(CAEs), generative adversarial networks (GANs) as well as super-resolution<br />
(SR), and present a comprehensive performance comparison. According to<br />
experimental results, CAEs achieve better coding efficiency than JPEG by<br />
extracting compact features. GANs show potential advantages on large<br />
compression ratio and high subjective quality reconstruction. Super-resolution<br />
achieves the best rate-distortion (RD) performance among them, which is<br />
comparable to BPG.

In recent years, the system using acoustic communication is increasing. However, because acoustic communication uses low frequency, transmission rate is lower than radio wave communication. In wireless communication, MIMO-OFDM is proposed for improvement quality and transmission rate. In this paper, we introduce a software-defined acoustic communication platform by using MATLAB and implement acoustic MIMO-OFDM transmission into the platform. Also, we evaluate BER characteristics in various experimental parameters in MATLAB simulation and real environment. Moreover, we evaluate image quality in actual acoustic image transmission by using the acoustic communication platform and we can successfully transmit the image via acoustic MIMO-OFDM.

Image compression has been investigated as a fundamental research topic for<br />
many decades. Recently, deep learning has achieved great success in many<br />
computer vision tasks, and is gradually being used in image compression. In<br />
this paper, we present a lossy image compression architecture, which utilizes<br />
the advantages of convolutional autoencoder (CAE) to achieve a high coding<br />
efficiency. First, we design a novel CAE architecture to replace the<br />
conventional transforms and train this CAE using a rate-distortion loss<br />
function. Second, to generate a more energy-compact representation, we utilize<br />
the principal components analysis (PCA) to rotate the feature maps produced by<br />
the CAE, and then apply the quantization and entropy coder to generate the<br />
codes. Experimental results demonstrate that our method outperforms traditional<br />
image coding algorithms, by achieving a 13.7% BD-rate decrement on the Kodak<br />
database images compared to JPEG2000. Besides, our method maintains a moderate<br />
complexity similar to JPEG2000.

Named Data Networking (NDN) has been recently introduced as a new future network architecture. The end-to-end throughput estimation for adaptive bit-rate video streaming on NDN is one of the most challenging topics. Specifically, the end-to-end throughput estimation on NDN appears to be unreliable, since the provider of content is unknown to the consumer. Moreover, partial caching on NDN router's Content Store could temporarily lead to packet loss due to throughput overestimation. In this paper, we present an active Interest adaptation scheme which operates by proactively estimating the throughput in a hop-by-hop fashion. The consumer node is then assisted with the most recent available end-to-end bandwidth. Therefore, the video player can promptly adapt to the change of network condition. The implementation evaluation using NDN-JS and DASH-JS on the setup network demonstrate that our proposed solution provides better average stream bit-rate and consumes less network bandwidth than the traditional system.

This paper proposes a transmission control protocol (TCP) for long distance high-speed wireless communications, including free-space optical communications (FSOC). Extreme high frequency of wireless com- munications enables high-speed bit rate, but frequent signal error, including burst error, can be a quite severe problem for ordinary high-speed TCPs. To achieve 10 Gbps or higher data transfer throughput on FSOC, the proposed TCP (designated "TCP-FSO") has improved and new features including multi-layer congestion control, retransmission control with packet loss point estimation, delay-based ACK congestion control, and ACK retransmission control. We evaluated data transfer throughput of TCP-FSO and the other TCPs, by throughput model analysis and experiment on real implementa-Tion. Obtained results show that TCP-FSO achieves far higher data transfer throughput than other high-speed TCPs. For example, it achieved a thou- sand times higher throughput than the other high-speed TCPs in a real FSOC environment.

In this paper, we consider a color space conversion method from BT.709 used for current HDTV (High Definition Television) broadcast to BT.2020 that will be used for UHDTV (Ultra HDTV) broadcast, with lost color information restoration. Our method anisotropically diffuses BT.709 chromaticities considering the direction to the original chromaticities in BT.2020 color space. With this method and BT.709 images, we obtained BT.2020 images that had chromaticities out of BT.709 color gamut.

For 8K UHDTV video, which has high spatial resolution, CU size has large influence to encoding efficiency. In addition, a moving object with random motion has also a large impact on encoding efficiency of inter prediction. In this paper, we therefore propose a method to improve H.265/HEVC encoding efficiency by CU size expansion and inter/intra prediction mode selection. Moreover, we evaluated encoding efficiency and complexity by chancing max CU sizes and inter/intra prediction modes. From these evaluation results, the proposed method achieved high efficiency and low computational cost.

As mobile multimedia services have significantly evolved and diversified with the spread of smartphones and Internet of Things (IoT) devices, low-delay multimedia cloud computing is the need of the hour. To address this demand, in this study, we introduce an edge cloud system that equips a multimedia service function chaining capability. A prototype implementation of the proposed edge cloud system has three main features: 1) edge computing deployment by using OpenStack, 2) multimedia service slicing and chaining, and 3) efficient resource management in edge networks. Based on these features, the proposed system achieves lower multimedia processing delay compared to a conventional cloud computing platform. We deploy the proposed system in our laboratory and validate the system performance by using typical multimedia application, such as human detection in video surveillance.

To provide energy-efficient (i.e., longer lifetime of sensors) and network-friendly (i.e., reducing network traffic) field sensing, we propose an edge-centric field monitoring system which applies efficient sensors and camera control. The proposed system detects conditions in a monitoring area and controls sensing frequency (sampling rate) of sensors, and capture rate and encoding rate of surveillance cameras, according to the detected conditions. In addition, the system applies a Multi-access Edge Computing (MEC) platform to provide fast feedback control to the sensors and cameras. In performance evaluations, we assume that the monitoring target is landslide detection and create a miniature 'artificial landslide generation' environment in our laboratory. By using the environment, we evaluate the system performance, and evaluation results indicate that the proposed system can reduce network traffic and save energy consumption efficiently.

To reduce the backbone video traffic generated by video surveillance, we propose an intelligent video surveillance system that offers multi-modal sensor-based event detection and event-driven video rate adaptation. Our proposed system can detect pedestrian existence and movements in the monitoring area by using multi-modal sensors (camera, laser scanner and infrared distance sensor) and control surveillance video quality according to the detected events. We evaluate event detection accuracy and video traffic volume in the experiment scenarios where up to six pedestrians pass through and/or stop at the monitoring area. Evaluation results conclude that our system can significantly reduce video traffic while ensuring high-quality surveillance.

Throughput prediction is essential for ensuring high quality of service for video streaming transmissions. However, current methods are incapable of accurately predicting throughput in mobile networks, especially for moving user scenarios. Therefore, we propose a TCP throughput prediction method TRUST using machine learning for mobile networks. TRUST has two stages: user movement pattern identification and throughput prediction. In the prediction stage, the long short-term memory (LSTM) model is employed for TCP throughput prediction. TRUST takes all the communication quality factors, sensor data and scenario information into consideration. Field experiments are conducted to evaluate TRUST in various scenarios. The results indicate that TRUST can predict future throughput with higher accuracy than the conventional methods, which decreases the throughput prediction error by maximum 44% under the moving bus scenario.

Recently, to provide a low-latency mobile computing platform, Mobile Edge Computing (MEC) is proposed. In this paper, we first summarized the feature capabilities of MEC, such as content distribution and caching, computational offloading and multimedia Internet of Things (IoT). Then, to understand recent research efforts of multimedia MEC, we briefly highlight the research efforts in terms of above three capabilities: how to achieve edge caching in video distribution, how to schedule computational offloading to the cloud and how the communication quality degradation affects to the user experience of multimedia IoT. Finally, we addressed the emerging research issues of multimedia MEC to improve reliability and robustness of multimedia MEC.

Throughput prediction is one of good solutions to improve quality of mobile applications (e.g., YouTube or Netflix) for video streaming delivery services in mobile networks. This is because such applications require monitoring the network performances to control content quality, thus guarantee quality of service (QoS) and quality of experience (QoE). In this paper, we propose a history-based TCP throughput prediction method incorporating communication quality features using SVR (Support Vector Regression). By taking history of communication quality features such as historical throughput and Received Signal Strength Indication (RSSI) into consideration, the throughput prediction error can be decreased. We conduct experiments with the proposed method and compare the prediction accuracy with a variety of methods in different scenarios of various moving modes of users. Results show that the proposed model could predict throughput effectively in various scenarios and decrease throughput prediction errors by a maximum of 26.47% compared with other methods.

The increasing demand of mobile applications has brought large amount of mobile traffic. To meet users' requirements for high-quality video delivery, it is an urgent task to provide fair-quality video delivery for various users and situations. In this paper, we evaluate QoS and QoE characteristics and validate QoE unfriendliness in heterogeneous DASH contents distributions to provide QoE-fair video delivery,. In the evaluations, we employ multiple contents with different resolutions, frame rates, and rate-distortion characteristics. By using heterogeneous DASH contents, we evaluate the effect of playout buffer size on QoS and QoE performances. Evaluation results show that smaller playout buffer size contributes to QoE improvement under network congestion. In addition, we confirm that suppression of playback stall is a particularly important factor to achieve QoE fairness compared to other QoS metrics such as initial delay and representation.

In recent years, various approaches have been investigated towards blind image quality assessment (IQA) with high accuracy and low complexity. In this paper we develop a pre-saliency map based blind IQA method, which takes advantage of saliency information in prior of quality prediction for performance enhancement by two steps. 1) We split the image into patches and design a convolution neural network (CNN) to predict the patch-wise quality score. Then we explore the relation between image saliency information and CNN prediction error to present a statistical analysis. 2) Based on the analysis, we propose a patch quality aggregation algorithm by removing non-salient patches which are likely to bring large prediction error and assigning large weights for salient patches. Experimental results validate that our method can achieve high accuracy (0.978) with subjective quality scores, which outperforms existing IQA methods. Meanwhile, the proposed method can reduce 52.7% computational time than the IQA without pre-saliency map.

In this paper, we performed human moving pattern recognition using communication quality: cellular download throughputs, Received Signal Strength Indicators (RSSIs) and cellular base station IDs. We apply three machine learning algorithms, such as Support Vector Machine (SVM), K-Nearest Neighbor (KNN) and Random Forest (RF) and evaluate recognition accuracy of human moving patterns. Results conclude that the communication quality can recognize moving patterns with high accuracy.

In this paper, to provide high-quality and high-visibility surveillance, we introduce video surveillance system equipped image enhancement system. Because quality of enhanced image heavily depends on encoding characteristic, we evaluate the subjective and objective image quality of image enhancement under various video encoding rates by using night and day sequences.

In this paper, we evaluate QoS (Quality of Service) and QoE (Quality of Experience) characteristics of 2K and 4K video distribution using MPEG-DASH (Dynamic Adaptive Streaming over HTTP). Evaluation results conclude that smaller playout buffer size contributes to QoE improvement.

Because mobile users demand high quality and energy-friendly video delivery service that efficiently uses wireless resources, we introduce a green video delivery system using moving route navigation and playout buffer control based on the mobile throughput history data. Our system first determines the optimal travel route to realize high-speed and energy-efficient communications. Then when a user enters a high throughput area, our system temporarily extends the video playout buffer size, and if allowed, the user remains in the high throughput area and aggressively downloads video segments until filling the extended buffer via a high-speed and energyefficient wireless connection. After leaving this area, the user consumes video segments from the extended buffer in order to keep smooth video playback without wireless communications. Computer simulations along with laboratory and field experiments on the system performance confirm that our system can achieve green mobile video delivery.

In this study, we propose a broadcasting and communication integrated platform that can retransmit 8K broadcasting contents from a TV receiving set to mobile devices. We then evaluate the QoS characteristics and the qualities of 8K contents in this platform. Evaluation results conclude that 8K broadcasting contents can be retransmitted to mobile devices smoothly and their qualities are nearly as high as those of assumed 8K broadcasting service in Japan.

Nowadays, researches on accident prevention using train-mounted cameras had been progressing. Our proposed method considers temporal continuity between frames by using motion vectors in addition to conventional thresholding on similarity values obtained by a human detection method using HOG features. Experiments show the effectiveness of our method as compared with a previous method using only HOG features.

In order to analyze system latency of edge computing for multimedia data processing, we propose a model of system latency of two different distributed processing scenarios: pipeline and parallel processing. We validate the model by using actual cloud services and human detection processing.

Throughput prediction contributes a lot to adaptive bitrate control, adjusting the quality of video streaming accordingly to offer smooth media transmission and save energy at the same time. To solve the problem of throughput prediction for real time communication, this paper puts forward a new history-based throughput prediction method applying Hidden Markov Model in mobile networks. The main purpose of this method is to predict future throughput for real time communication in mobile network. Our novel approach utilizes Hidden Markov Model (HMM) with Gaussian Mixture Model (GMM) to deal with history time series of throughput and judge fluctuation factor with total variance when predicting future throughput. By conducting experiments with the new methodology, we compare the accuracy of the proposed method with three other conventional prediction methods. Results show our proposed method could identify data fluctuation effectively and predict future 100s throughput with high accuracy in various situations.

With increase of mobile wireless LAN systems, frequency contamination by the overlapping basic service set (OBSS) becomes a critical issue. In this paper, we focus on HCF controlled channel access (HCCA) to alleviate the OBSS problem. HCCA considers single BSS (SBSS) environment only and suppresses traffic of co-existing DCF (distributed coordination function) based WiFi access points (APs). We propose two methods that utilize capture effects to improve the coexistence capability with DCF networks, that we call DCF friendliness. The first method adjusts transmission timing in HCCA WLAN by applying inter-AP coordination. The second method enables simultaneous communication with surrounding DCF WLANs by changing the frame interval of HCCA to DIFS+1 SlotTime. Simulations show that both the proposed method can achieve higher throughput and better DCF friendliness.

In this paper, we propose a method to improve H.265/HEVC encoding performance for 8K UHDTV moving pictures by detecting amount or complexity of object motions. In 8K video, which has high spatial resolution, motion has a big influence on encoding efficiency. The proposed method estimates motion complexity by external process, and selects an optimal prediction mode and search ranges of motion vectors. By this method, we achieved highly efficient and low computation encoding.

In order to provide high-quality and highly reliable video delivery services for mobile users, especially train passengers, we propose a proactive content caching scheme that uses transportation systems. In our system, we place content servers with cache capability [e.g., content centric networking/named data networking (CCN/NDN)] in every train and station. Video segments encapsulated by MPEG-Dynamic Adaptive Streaming over HTTP (MPEG-DASH) are distributed and pre-cached by the station servers before the trains arrive at the stations. The trains receive content via high-speed wireless transport, such as wireless LANs or millimeter waves, when they stop at the stations. We developed prototype systems based on hypertext transfer protocol and CCN/NDN protocol, evaluate their performance through two field experiments that uses actual trains, and compare with traditional video streaming over cellular networks. Such evaluations indicate that our system can achieve high-quality video delivery without interruption for up to 50 users simultaneously.

Urban surveillance systems are being applied in a rapid pace with mature but inefficient solutions. The inefficiency is revealed with two aspects, too concentrated bandwidth and processing requirement. To solve this problem, we proposed a content oriented surveillance system based on Information-Centric Network. However, we can't simply replace TCP/IP streaming structure with named contents streaming structure because it can't improve the surveillance system's efficiency enough. In this paper, we took the ICN network's profits even further with the named contents. Instead of streaming live video to the central data center and processing multiple data stream in the same time, we have designed the nodes to process the captured raw data and produce objective contents for the central data center. With the extremely size difference in raw data and actual valued contents from it, we could apply the method in investigating area people traffic conditions and even in disaster and anti-terrorism scenarios. There was a field experiment performed to evaluate tourists' densities and dressing habits during winter season of March. The experiment expressed the benefits of our system and compared our method with traditional surveillance systems in saving network bandwidth and functionalities.

Information Centric Networking (ICN) is a new networking paradigm in which the network provides users with named content, instead of communication channels between hosts. However, many issues, such as naming, routing, resource control, and security, still need to be resolved before it can be realized practically. Further, the energy efficiency of ICNs has not been sufficiently considered. In this study, we evaluate the energy consumption of an ICN network by first measuring the power consumption of actual PCs/smartphones and constructing an energy consumption model. Then, by using the model and the simulator ndnSIM, we calculate the energy consumed by the network. We also evaluate the energy consumed for video delivery, with a view toward power-saving video delivery.

In order to tackle recent mobile traffic explosion, content offloading using Wifi access points from cellular network is deployed but the Wifi system itself is sometimes heavily congested. In this paper, we propose a content offloading system using transportation systems such as a train. In addition, we aim to incorporate CCN (Content Centric Networking) into our system to exploit its in-network caching capability. We compare three protocol designs, pure IP, IP+CCN hybrid and pure CCN, and show the efficiency of our system using a network simulator.

In recent years, researches of underwater sensor networks have continued to investigate environment and resources of the sea. Acoustic waves are used instead of radio waves for wireless communication in underwater. However, dedicated hardware is very expensive, experiments on the sea are very time-consuming, and huge water spaces are necessary to study the underwater acoustic communication. In this paper, we present a cheap and tractable software-defined acoustic communication platform running on PCs using MATLAB, and evaluate its characteristics in a variety of communication methods by changing modulation schemes, error correction codes, transmission power and frequency by using commercial speaker and microphone devices. Our current implementation achieves data rate of up to 4.5 Kbps.

A system is proposed that uses content-centric networking to deliver content to mobile users. It uses sensor data obtained from the device they are using to identify their means of transport. The content is then delivered in a manner that best matches that means. The means of transport is identified using a combination of features: average magnetic quantity, average value and standard deviation of acceleration, vehicle speed, and maximum power spectrum of acceleration.

An accurate (reliable) motion estimation (ME) algorithm using block matching for sequences including spatially periodic patterns is proposed and evaluated in comparison to conventional algorithms. Mismatch between accurate motion vectors (MVs) and MVs obtained by block matching (BM) which minimizes BM distortion is evaluated. BM distortion caused by MV sampling error (DVSE), which causes ME errors, is discovered. A new ME algorithm which uses the maximum value of DVSE (maxDVSE) is proposed. As for the proposed algorithm, maxDVSE is calculated for each block as a threshold of accurate vector candidates, and all candidate vectors for each block in a frame are stored. Accurate vectors are selected from the candidate vectors by finding so-called "anchor vectors" (namely, vectors at a unique point on spatially periodic patterns). The MVs are selected in a process starting from one block and propagated to neighboring blocks. Simulations are performed and the results show that average precision of the proposed ME algorithm is 98.9%, which is superior to that of other conventional algorithms.

Proactive content caching scheme utilizing transportation systems, especially on trains, was proposed in order to provide a robust content delivery with efficient wireless resource usage. This system requires content servers with NDN capability to be placed on every station and trains. The mechanism is to pre-cache the contents that users request to the station server before the train arrives, and the train server caches the content during its stoppage time. With this mechanism, users are able to have a continuous playback of videos while riding on trains. In this paper, we have proposed a browser-based implementation, called DASH-NDN-JS, for this proactive content caching scheme. We evaluate this scheme, and experiment with multiple users to see how it will affect the video quality each user will achieve and the bandwidth consumption between the connections. Our evaluations conclude that the increase of users lowers the video quality, but avoids congestion depending on what video content each user will want to request.

Block-based Motion Estimation (ME) has been widely used in various video coding standards to remove temporal redundancy. However, this ME has limitation that it can only compensate for a parallel translation. Various methods have been proposed for other motions such as zooming. In recent years, 8K UHDTV (7,680 x 4,320 pixels) has been developed. Since 8K has large motion by zooming that is difficult to be predicted by block matching, it is important to improve zoom motion estimation. In this paper, to handle zooming in 8K video sequences, we propose a method for improving the picture quality by global zoom estimation based on motion vector analysis extracted by block matching.

© Springer International Publishing Switzerland 2015. This paper presents a live version music identification system by modifying the conventional cover song identification system. The proposed system includes two stages: a live version identification phase and an audio scenedetection phase. We improve the accuracy of the system by weighting similarity scores in the live version identification phase and discriminating scenes by using RMS, pulse clarity and similarity scores. Results show that the proposed method performs better than the previous method. The final algorithm achieves 70% accuracy on average.

Providing robust video streaming along with efficient wireless resource usage is necessary for mobile users, especially on subway, and mobile carriers. To achieve this, we propose a highly-reliable buffer strategy based on long-term throughput prediction. Our approach has two elements which are called "long-term throughput prediction" and "guaranteed playout buffer filling mechanism." To avoid any video freeze due to network quality degradation, our approach calculates the optimal amount of playout buffer and schedules video download timing in a theoretical manner. We evaluate its performance via experiments in real environment. Evaluations conclude that our approach can provide highly-reliable video streaming and also achieve to reduce the average playout buffer size on the client.

In wireless networks, it is important to realize energy-efficient video delivery. To do this, we introduce energy-efficient video streaming over named data networking (NDN). In our proposed approach, we focus on two areas, namely, to improve the throughput performance by Interest aggregation, and to reduce the overhead energy using playout buffer-size control. We evaluate the power savings realized by our method using a hardware power-measurement tool. Our results show that Interest aggregation can realize an approximately 50% reduction in energy compared to conventional NDN implementation, and a large playout buffer size can reduce the energy by approximately 37% compared to the case of a small playout buffer size.

Content Centric Networking (CCN) has been emerged as a new networking paradigm. In CCN, adaptive video streaming is one of the most promising technologies. However, rate adaptation and outbound face selection are the primary concern since a node cannot reliably estimate the end-to-end throughput. In this paper, we propose a strategy for outbound face selection considering both response time and bit-rate of lost packets. These two aspects facilitate the node to choose which outbound face offers the best QoE. In a case where bit-rate of lost packets is greater than a half of transmitted bit-rate, the node will re-select outbound faces to avoid intermittent streaming. The experiments performed using CCNx demonstrate that our approach provides better average stream bit-rate than the earlier response time based method.

In this paper, we evaluate performances of 4K/2K video streaming using DASH-JS over 2.4/5GHz WiFi networks. We confirmed that fluctuation of reception quality was mainly caused by the minimum representation rate of MPEG-DASH contents and overlapped ON cycles for packet transmission.

Effective use of various wireless network interfaces is expected. In this paper, we evaluate Quality of Service characteristics of mobile 4K video using a radio communication quality map. We collect radio communication quality (e.g. throughput) by smartphones and visualize the quality in a heat map. Using this map, we select two routes that are expected to bring the highest and the lowest communication quality. We then evaluate QoS characteristic of 4K video streaming using MPEG-DASH on these two routes. The result shows that a good route user can achieve 1.5 times higher communication quality than a bad route user.

In this paper, we propose a method to control H.265/HEVC encoding for 8K UHDTV moving pictures by detecting amount or complexity of object motions. In 8K video, which has very high spatial resolution, motion has a big influence on encoding efficiency and processing time. The proposed method estimates motion features by external process which uses local feature points matching between two frames, selects an optimal prediction mode and determines search ranges of motion vectors. Experiments show we can detect motion complexity of 8K movies by using local feature matching between frames and we can select optimal configurations of encoding. By our method, we achieved highly efficient and low computation encoding.

Improving Quality of Service (QoS) in wireless networks is important and necessary for mobile users. We have previously proposed Comfort Route (CR) Navigation, which navigates users to their destinations using high QoS communication areas, such as Wi-Fi APs, rather than the geographical Shortest Route (SR). In this paper, we employ an analytical model to estimate the CR gain in a theoretical manner which assumes that available cellular and Wi-Fi throughputs are uniform within their coverage. The CR gain is computed by using basic parameters, including wireless network bandwidth and transmission time. To validate our model, we compare simulation results and real observation. These results conclude that the CR gain could estimate by using our analytical model.

To improve Quality of Service in wireless networks while mobile users travel, we introduce Comfort Route Navigation (CRN) which is a navigation system based on a user centric mobility management for next generation wireless networks. CRN provides an optimal route that satisfies user needs, such as obtaining maximum wireless resources. To achieve CRN, we construct an access point (AP) map of Shinjuku city. To reflect quality of APs in this map, we evaluate throughputs at seven public Wi-Fi spots. Based on these observations, we evaluate performance of our CRN via computer simulations and in a real environment. These evaluations conduct that a CRN user could obtain higher communication quality rather than a Shortest Route (SR) user. These evaluations also conclude that CRN gain depends heavily on the quality and location of the best broadband spot.

In recent years, video sequence with a depth map can be taken by a camera with RGB-D (depth) sensor like Kinect. On the other hand, quality degradation at occluded regions is inevitable when conventional 2-D video coding is applied. In this paper, we propose usage of an occluded region detection supported by the depth map for improvement of 2-D image coding quality. Experiments show that further improvement of picture quality at the whole of frame and occluded regions is addressed for H.265/HEVC compressed streams. © 2014 IEEE.

It is important for multimedia mobile users to obtain sufficient resources and a stable capacity of resources. Network resources such as available bandwidth for mobile users, however, fluctuate depending on the level of background traffic. The deployment of several access points enables mobile users to access more network resources on the way to their destination by taking the longcut route rather than the shortcut route. These users pay an extra cost to take the longcut route but obtain greater gains in throughput over the shortcut route. Previous research has shown that the gain-to-cost ratio can be more than double for the longcut route. No considerations, however, have been made for the case in which resource fluctuation is due to network congestion. This paper investigates the quality of service (QoS) for a longcut route with fewer resources (real resources) compared with that for one with the total resources (ideal resources) in the longcut route calculation. The results show that on average, users have gains of approximately double in real resources than their gains in ideal resources. This finding indicates that users can greatly improve their gains if they can accurately predict the resource fluctuation.

The peak signal-to-noise ratio (PSNR) used as an index of image quality usually requires original images, but this is difficult for consumer generated content such as videos on YouTube. Therefore, we developed two blind PSNR estimation methods without bit-stream analysis in which multiple support vector machines are prepared to learn differently encoded images in PSNR
using an entire frame and dividing the frame into two areas. We confirmed that higher estimation accuracy is possible for the latter method against that using the entire frame.

With the increasing demand for video streaming in mobile environment, we have been tackling a research on throughput prediction based on the measurement history data in order to improve service quality[1][2]. Since throughput fluctuation occurs sharply in mobile environment, useless packet loss or congestion occurs when available network bandwidth is insufficient compared to the bit rate of video contents. Hence, it is possible to improve the quality of service by deducing the available throughput and selecting appropriate bit rate for the video contents. In this paper, we propose a throughput prediction method that considers mobile locations of a train and their throughput history and show promising experimental results.

Extending battery life for smartphones while using wireless networks is important. In this paper, we propose Comfort Route for Energy Saving (CRFES), which helps users extend the battery life of their smartphones by navigating to their destinations via Quality of Service (QoS)-compliant energy-efficient spots such as Wi-Fi spots. To create CRFES, we construct maps of Wi-Fi spots using logging software we developed for Android phones. In addition to locations, we record actual throughput and energy consumption observed in cellular/Wi-Fi networks. So comfort route navigation is done by choosing a route which minimizes energy consumption instead of choosing the shortest path. Evaluations are carried out by computer simulation and one real city map. The results conclude that CRFES can save the battery life by approximately 50% by spending 1.6 times time cost to reach their destinations on average.

Traffic sign recognition contributes to the safety of drivers and people around the car. The system analyzes the road ahead images taken by on-vehicle camera, but blurred and distorted images make it difficult to recognize correct traffic sign. In this paper, we propose a method that deforms from ellipses to perfect circles and applies template matching with distorted sign templates.

Height data is very important not only as an indicator of growth in low birth weight infants, newborn, infants (infants below), but also for early detection of disease such as dwarfism. In this paper, we consider height estimation by image processing that targets lying infants for the purpose of reducing the burden of infants, doctors and parents. Furthermore, we compare results of using a single image and multiple images obtained by moving pictures, and show the latter contributes to improvement.

Feature analysis of audio compression is necessary to achieve high accuracy in musical content recognition and content-based music information retrieval (MIR). Bit rate differences are expected to adversely affect musical content analysis and content-based MIR results because the frequency response might be changed by the encoding. In this paper, we specifically examine its effect on the chroma vector, which is a commonly used feature vector for music signal processing. We analyze sound qualities extracted from encoded music files with different bit rates and compare them with the chroma features of original songs obtained using datasets for chord recognition. © 2014 Springer International Publishing.

Providing robust content delivery along with efficient wireless resource usage is important for next generation wireless networks. To achieve this, we propose a proactive content caching scheme utilizing transportation systems, especially trains. In our system, we place content servers with CCN capability to every train and station. Segments of video contents are pre-cached by the station servers before trains arrive at stations. Trains receive the contents via high-speed wireless transport while they stop at the stations. We develop a prototype system based on IP and CCN Hybrid protocols. We evaluate its performance by field experiment and compare with traditional CDN scenarios using cellular networks. Evaluations conclude that our system can achieve high-speed and high-reliable video delivery without freezing.

Acquisition of three-dimensional information through images captured from different viewpoints is one of important themes to be solved in computer vision. Lots of stereo algorithms have been studied, and they have been applied to two-camera, trinocular, multiple-baseline stereos and moving-camera systems. Generally, the larger distance between cameras should be required for better depth accuracy. However, this often makes the correspondence issue more difficult. Brightness variation depending on viewpoints can be considered as one of the reasons in real scenes, which may be caused by non-Lambertien reflectance property of an observed object. In this paper, the authors propose a novel multiple-baseline stereo algorithm that is robust against brightness variation depending on viewpoints. It introduces a new cost function summarizing neighbor-camera comparisons and the authors prove its theoretical accuracy under the condition in which Lambertien assumption is comparable to a two-camera stereo algorithm that has the same baseline length. The experimental results by using real image sets that include brightness variation depending on viewpoints indicate it can show comparable or better accuracies not only to a two-camera stereo but also to a multiple-baseline stereo.

In recent years, video sequence with a depth map can be taken by a camera with RGB-D (depth) sensor like Kinect. On the other hand, quality degradation at occluded regions is inevitable when conventional 2-D video coding is applied. In this paper, we propose usage of an occluded region detection supported by the depth map for improvement of 2-D image coding quality. Experiments show that further improvement of picture quality at the whole of frame and occluded regions is addressed for H.265/HEVC compressed streams.

Image super-resolution quality is degraded by the noise component of an image. However the noise should not be eliminated because of its importance in high-definition impression for digital cinema. We therefore propose an image super-resolution method using the synthesis of the super-resolution signal and noise components respectively after dividing an original image into signal and noise components. The signal component's resolution is enhanced using registration between the signal component and its multi-scale Wavelet components with resolution-enhanced parameter optimization.

Three-dimensional (3D) video service is expected to be introduced as a next-generation television service. Stereoscopic video is composed of two 2D video signals for the left and right views, and these 2D video signals are encoded. Video quality between the left and right views is not always consistent because, for example, each view is encoded at a different bit rate. As a result, the video quality difference between the left and right views degrades the quality of stereoscopic video. However, these characteristics have not been thoroughly studied or modeled. Therefore, it is necessary to better understand how the video quality difference affects stereoscopic video quality and to model the video quality characteristics. To do that, we conducted subjective quality assessments to derive subjective video quality characteristics. The characteristics showed that 3D video quality was affected by the difference in video quality between the left and right views, and that when the difference was small, 3D video quality correlated with the highest 2D video quality of the two views. We modeled these characteristics as a subjective quality metric using a training data set. Finally, we verified the performance of our proposed model by applying it to unknown data sets.

In Peer-to-Peer (P2P) networks, an incentive mechanism is a necessary component to deal with the free-riding behavior. The challenge is that direct reciprocal incentives
e.g., tit-for-tat, which consider the cooperation of peers in a pair-wise manner, are not suited with P2P streaming. In this paper, we propose a new service differentiation mechanism to provide a redistribution incentive for P2P streaming in a hybrid overlay network. The contribution of a peer can be measured from the number of video sub-streams that it uploads to other peers. By sending one request message, the number of sub-streams that each peer can retrieve is varied by its contribution level. An altruistic peer thus has to send less request messages and will experience smoother video quality than a selfish peer. Through simulations, we demonstrate that our solution can provide service differentiation among peers with better streaming quality than the tit-for-tat scheme.

Network resources for mobile users such as available bandwidth fluctuate depending on the level of background traffic. The deployment of many access points enables mobile users to access more network resources on their way to their destination by taking the longcut route rather than the shortcut route. These users pay some extra cost to take the longcut but obtain greater gains, such as in throughput, over the shortcut route. Previous research has shown that the gain-to-cost ratio can be more than double for the longcut route. No considerations, however, have been made for the case in which resource fluctuation is due to network congestion. This paper investigates the Quality of Service (QoS) for a longcut route with fewer resources (real resources) compared with that for one with the total resources (ideal resources) in the longcut route calculation. The results show that, on average, users have gains of approximately twice in real resources of their gains in ideal resources. This finding indicates that users can greatly improve their gains if they could accurately predict the resource fluctuation. Copyright © 2013 ACM.

This paper presents an RTT-fair TCP congestion control using ACK interval measurement and extends the approach to have inter-protocol friendliness, especially with CUBIC TCP. In the previous RTT-fair TCP congestion control, including ours, estimation of RTTs of a competing flow had been a problem. We try to solve this problem by measuring ACK arrival intervals, which are observable parameters by an end host. This approach enables estimation of congestion window behaviors, in addition to RTTs, of a competing flow. We then extend our congestion control to have friendliness to CUBIC-TCP, in addition to classical TCP-Reno, in an adaptive manner. Extensive experiment results are shown for simulations and implementations and effectiveness of our approach is confirmed.

This paper presents a TCP differentiation based on TCP version identification and IEEE 802.11e EDCA for low delay multimedia streaming over wireless LAN. It has been known that delay-based TCP can achieve low delay transport as long as it does not compete with loss-based TCP. However, when competition happens, it seriously decreases its rate by itself due to RTT increase. In order to alleviate this problem, we consider combination of TCP version identification and prioritized transport of delay-based TCP by EDCA at an access point. We evaluate this approach by simulations and implementations, and confirm its effectiveness. © 2013 IEEE.

In vehicle-to-vehicle communication, a message named BSM (Basic Safety Message) has a major role to inform a driver about surrounding condition. A vehicle periodically sends the BSM which includes the information of itself. Traffic load issues caused by BSMs would often be raised on heavily congested roads. Therefore, BSM congestion controls to avoid traffic congestion are challenging. This paper proposes a new BSM congestion control named BRAEVE. BRAEVE adapts its BSM generation rate according to the number of neighbor vehicles in communication range and then controls overall network traffic load. Our simulation evaluation shows that BRAEVE provides more uniform recognition of surrounding vehicles than an existing method, which brings much better safety assurance to a driver.

Objective picture quality measures cannot estimate the effect of blocking artifacts caused by video compression sufficiently. In this paper, we apply cepstrum analysis to quantify the blocking artifacts. We show experimental results for some test sequences using different coding schemes and prove effectiveness of our approach.

Quality of image super-resolution (SR) degrades by noise component. However it should not be eliminated because it is important for high definition impression for digital cinema. Therefore we propose an image SR method by synthesis of resolution-enhanced signal and noise components respectively after dividing an original image into signal and noise components. The signal component is resolution-enhanced using image registration between the signal component and its wavelet multi-scale components with resolution-enhanced parameter optimization. © 2013 IEEE.

Most network applications nowadays are related to the distribution of contents from a source to a number of users. Several developments such as CDN and P2P networking have been used to provide access to contents at the network edges. However, they have no ability to leverage caching at the network level and still rely on a host-based communication model. Recently, Content Centric Networking (CCN) has been emerged as a new networking paradigm. In CCN, the communication focuses on the named content instead of the IP address. This paper provides a framework for enabling video streaming over CCN. We performed experiments using CCNx on the PlanetLab to explore the effects of an overlay delivery path and a chunk size to the streaming quality. Furthermore, we study the deployment of MPEG-DASH to achieve adaptive bit-rate streaming over CCN, which reveals additional challenge. © 2013 IEEE.

We present a robust cellular network (RoCNet) that combines a cellular and an opportunistic networks for spatial uplink mobile data offloading, which focuses on the spatial difference of the traffic load among areas (e. g., business district and residential area in the daytime). RoCNet realizes the spatial data offload by leveraging the store-carry-forward routing mechanism. In the area where traffic load is high, delay-tolerant data originated from a mobile terminal is directly forwarded to a nearby terminal using Bluetooth or wireless LAN instead of being transmitted to a congested cellular base station. When the data is carried by the nearby terminal to other area where the traffic load is low, the data is forwarded to a cellular base station. To enhance the offload effect, it is necessary for data to be forwarded to a terminal that moves to a low traffic load area. In this paper, we use the particle filter to predict user behavior. Before forwarding data between mobile terminals, the terminals exchange prediction results and decide whether or not the data should be forwarded. We conducted a computer simulation whose result shows RoCNet can spatially offload uplink traffic in a traffic concentration area to non-congested areas. As a result, RoCNet can suppress peak traffic by about 20 percent in a traffic-congested base station by distributing traffic to vicinity base stations.

Applications and malware affecting them are dramatically changing. It isn't certain whether the currently used features can classify normal traffic or malware traffic correctly. In this paper, we evaluated the features used in previous studies while taking into account secular changes to classify normal traffic into the normal category and anomalous traffic into the anomalous category correctly. A secular change in this study is a difference in a feature between the date the training data were caputred and the date the test data were captured in the same circumstance. The evaluation is based on the Euclidean distance between the normal codebook or anomalous codebook made by vector quantization and the test data. We report on what causes these secular changes and which features with little or no secular change are effective for malware detection. © 2013 Springer-Verlag Berlin Heidelberg.

Application-layer multicast has been considered as a promising method for streaming live media to users in P2P networks. However, the inefficient bandwidth utilization and the vulnerability to peer dynamics remain important issues. In this paper, we introduce a new approach to constructing an efficient overlay leveraging the strong features of tree and mesh methods. A core overlay is built in a fashion of multiple sub-trees, which allows each peer to receive data from other peers located in any different sub-trees. We propose a set of auxiliary connections, which connect between two nodes that do not have permanent parent/child relationship, to achieve a push-based data transmission from multiple parents without redundancy. Simulations through ns2 demonstrate that our scheme provides a good streaming quality with low end-to-end delay. Also, the proposed auxiliary links is beneficial in improving the resiliency of the overlay and utilizing the bandwidth of peers.

The use of mobile terminals has been expanding dramatically in recent years as they evolve from a means of dispatching and gathering information to a highly functional tool that supports personal lifestyles and behavior. A mobile terminal is likely to store various kinds of personal information such as a calendar and contact information as well as key data to carry out online transactions. Losing one's mobile terminal therefore creates the possibility that one's personal information may fall into the wrong hands and be used for malicious purposes.
We therefore propose a method of personal authentication using sensor data in a mobile terminal. First, we applied the LPC cepstrum to this authentication and checked for validity. We also evaluated the effectiveness of gait authentication using several frames.

We propose a novel image super-resolution method from digital cinema to 8K ultra high-definition television using registration of wavelet multi-scale components with affine transformation. The proposed method features that an original image is divided into signal and noise components by the wavelet soft-shrinkage with detection of white noise level. The signal component enhances resolution by registration between a signal component and its wavelet multi-scale components with affine transformation and parameters optimization. The affine transformation enhances super-resolution image quality because it increases registration candidates. The noise component enhances resolution with power control considering cinema noise representation. Super-resolution image outputs by synthesis of super-resolved signal and noise components. Experiments show that the proposed method has objectively better PSNR measurement and subjectively better appearance in comparison with conventional super-resolution methods. © 2013 IEEE.

This paper focuses on RTT-fairness of multiple TCP flows over the Internet, and proposes a new TCP congestion control named "HRF (Hybrid RTT-Fair)-TCP". Today, it is a serious problem that the flows having smaller RTT utilize more bandwidth than others when multiple flows having different RTT values compete in the same network. This means that a user with longer RTT may not be able to obtain sufficient bandwidth by the current methods. This RTT fairness issue has been discussed in many TCP papers. An example is CR (Constant Rate) algorithm, which achieves RTT-fairness by multiplying the square of RTT value in its window increment phase against TCP-Reno. However, the method halves its windows size same as TCP-Reno when a packet loss is detected. This makes worse its efficiency in certain network cases. On the other hand, recent proposed TCP versions essentially require throughput efficiency and TCP-friendliness with TCP-Reno. Therefore, we try to keep these advantages in our TCP design in addition to RTT-fairness. In this paper, we make intuitive analytical models in which we separate resource utilization processes into two cases: utilization of bottleneck link capacity and that of buffer space at the bottleneck link router. These models take into account three characteristic algorithms (Reno, Constant Rate, Constant Increase) in window increment phase where a sender receives an acknowledgement successfully. Their validity is proved by both simulations and implementations. From these analyses, we propose HRF-TCP which switches two modes according to observed RTT values and achieves RTT fairness. Experiments are carried out to validate the proposed method. Finally, HRF-TCP outperforms conventional methods in RTT-fairness, efficiency and friendliness with TCP-Reno.

Previous our research has introduced an optimal longcut route problem and a number of solutions to this problem. However, to date, the literature has only presented evaluations based on a single theoretical radio propagation model. Because throughput characteristics derived from radio propagation models significantly affects longcut route performance, this study introduces three simple models by referring to actual measurement values. The results of our evaluation show that there are large differences in performance between the three models. The Non-Line-of-Sight model can have an improvement ratio approximately five times greater than that of the Line-of-Sight Indoor model and six times greater than that of the previous theoretical model. © 2012 IEEE.

In Peer-to-Peer (P2P) networks, a great number of peers are reluctant to contribute to the system. An incentive mechanism thus becomes a necessary component to motivate the cooperation of peers. Moreover, due to the continuous nature of data transmission, the P2P media streaming requires different incentive strategies from the P2P file-sharing. In this paper, we propose a new service differentiation mechanism to provide a redistribution incentive for P2P streaming in a hybrid overlay network. The contribution amount of a peer can be measured from the number of video sub-streams it transmits, instead of the quantity of transferred data. By sending one request message, the number of sub-streams that each peer can retrieve is varied by its contribution amount. An altruistic peer thus has to send less request messages and experiences smoother video quality than a selfish peer. Through simulations, we demonstrate that the proposed mechanism can provide service differentiation among peers according to their contribution amount. © 2012 IEEE.

The growing popularity of mobile data communication has been leading a growing lack of radio resource on a cellular network. Therefore, to avoid the data traffic from exceeding the capacity limit of a cellular base station (BS), cellular operators have been trying to install cellular femto cells (local BSs) or wireless LAN access points (APs) in restaurants, cafes, user's houses, etc., to divert data traffic from the cellular radio access link to the fixedline such as x-DSL and FTTH. However, the traffic levels extremely change in a day, even locally, due to the mobility of users. While peak data traffic at certain times of a day appears locally, cellular BSs as well as the femto cells and the APs would sit underutilized for the rest of the day. Thus, we have started designing a robust cellular network (RoCNet) that combines infrastructure-based network (e.g., cellular network) and opportunistic networking for spatial mobile data offloading, which focuses on the difference of the data traffic level among areas (e.g., business district and residential area). The RoCNet facilitates the data traffic offloading leveraging the storecarry-forward feature considering a degree of traffic congestion. In this paper, we give an overview of the RoCNet and show a simulation result. The result shows RoCNet can spatially offload uplink traffic in a traffic concentration area to non-congested areas. In other words, RoCNet can suppress peak traffic in a traffic-congested base station by distributing traffic to vicinity base stations. Copyright © 2012 ACM.

Bit-depth scalable coding method is an approach that generates multiplexed bit-streams that can be decoded to two video sequences, for Standard Dynamic Range (SDR) environment and for High Dynamic Range (HDR) environment. This paper presents a bit-depth scalable coding method that is considered as gradation restoration in inter-layer prediction process. Our proposed inter-layer prediction uses a histogram interpolation method that enables to generate more mid-gray brightness levels than traditional inverse tone mapping methods with one-to-one correspondence.

This paper presents a chord recognition method from music signals using chroma vectors and musical knowledge known as Doubly Nested Circle of Fifths (DNCOF). DNCOF represents the relationships of major and minor chords where the neighboring two triads are similar. We obtain a novel feature from chroma vectors by mapping them onto two-dimensional DNCOF coordinate, which we call DNCOF vectors. We expect that the DNCOF vectors can contribute to correcting false recognition obtained by the chroma vectors when their mapped positions are apart from one another in the DNCOF coordinate. In this research, we evaluated our proposal using the Beatles' datasets and showed its effectiveness.

Feature extraction of images can be applied to image matching, image searching, object recognition, image tracking etc. One of the effective methods to extract features of images is Scale-Invariant Feature Transform (SIFT) [1]. In this paper, we indicate problems of SIFT and propose a method to improve its performance by applying Bilateral Filter [2]. In addition, we implement is acceleration by GPGPU (general purpose GPU), apply this method to generic object recognition and perform a comparison experiment. We compare the proposed method with the original method using SIFT and confirm improvement of the identification rate by the proposed method.

We propose a novel wavelet domain image super-resolution method from digital cinema to ultrahigh definition television considering cinema noise component. The proposed method features that spatial resolution of an original image is expanded by synthesis of super-resolved signal and noise components respectively after dividing an original image into signal and noise components. Dividing noise component uses spatio-temporal wavelet decomposition based on frequency spectrum analysis of cinema noise. And super-resolution parameters are optimized by comparing size-reduced super-resolution images with an original image. Experimental results showed that a super-resolution image using the proposed method has a subjectively better appearance and an objectively better peak signal-to-noise ratio measurement than conventional methods.

This paper presents a music part detection method incorporating chroma vector analysis for use with music TV programs. Results show that envelopes of chroma components of music signals tend to have horizontal (i.e. temporal) correlation in time-frequency representation because music signals have a periodic chord sequences. Based on this fact, we analyze time series of chroma components and attempt to segment music parts in music TV programs from other parts. Experimental results show an F-measure of 0.78, which is better than that obtained using the previous method.

In this paper, we propose a system which automatically generates slideshows for music, by utilizing images retrieved from photo sharing web sites, based on query words extracted from song lyrics. The proposed system consists of two major steps: (1) query extraction from song lyrics, (2) image selection from web image search results. Moreover, in order to improve the display duration of each image in the slideshow, we adjust image transition timing by analyzing the duration of each lyric line in the input song. We have conducted subjective evaluation experiments, which prove that the proposal can generate impressive music slideshows for any input song. © 2010 International Society for Music Information Retrieval.

In this paper, we introduce a hybrid approach for overlay construction and data delivery in an application-layer multicast. We combine the strong points of a tree-based structure and a mesh-based data delivery to form ToMo, a two-layer hybrid overlay. We try to reduce the number of replicated packets at a source, and reduce an effect when slow connection peers are located near the source. The overlay is constructed in the fashion of a mesh layer over a tree layer. This structure allocates the source to multicast each piece of the packet to a specific group of child peers only. Different from other approaches, we employ only push-based data delivery in order to minimize the latency. The redundancy is avoided by defining a set of well-organized mesh connections. Furthermore, in our approach, the isolated peers affected by parent departure are not facing data loss during the rejoin process since they still receive data from their neighbors via mesh connections. Simulations through ns2 demonstrate the efficiency of this solution.

The hybrid Application Layer Multicast (ALM) has been shown its efficiency by leveraging the conventionally main structures of application layer multicast, tree-based and mesh-based. However, how to select the proper node to construct the overlay and how to establish the connection between any two nodes are still unsolved. Therefore, this paper is to design a novel construction algorithm for the hybrid AIM to resolve the above two issues. Firstly, by carrying out the analysis of nodes&apos; characteristics, all nodes are divided into groups and a node priority is proposed to select the super node within each group. Secondly, by using the selected super node, all nodes are hierarchically controlled and different kinds of connections are carried out in the AIM, where the connection between super node and other nodes is set to be a tree to enhance the efficient utilization of network resource while the connection between other normal nodes is decided to be a mesh to reduce the overhead. Simulation results show that the proposal outperforms other conventional methods.

With the development of broadband technologies, more and more multimedia contents such as video or bulk file are being distributed over the Internet. However, an important issue to be resolved is how to construct an efficient overlay to provide users with the acceptable throughput and delay, while receiving multimedia contents. Therefore, this paper designs a tree construction algorithm by distributing the layered steaming over the ALM in order to improve both throughput and user delay. Firstly, to improve the throughput, based on theory analysis, we define a layered degree, out/in-degree and the corresponding constraints to manage the layered streaming and nodes. Secondly, a novel method, called Bi-cast, is proposed to reduce user delay during data-transmission. Thirdly, by using the defined degrees and the Bi-cast, we present a tree construction algorithm. Both the simulation and the implementation are carried out, and the results show that our proposal can obtain better performance than other conventional methods.

This paper presents a hybrid congestion control algorithm supporting throughput efficiency, TCP friendliness and RTT fairness, in order to provide highly smart multimedia streaming over the Internet. Recent studies on TCP and TFRC are evolving to achieve better throughput efficiency and RTT fairness. Examples are Hybrid TCP congestion control, VTP and TCP-Libra, where the former two improve throughput efficiency of TCP and TFRC, respectively, and the last one improves RTT fairness of TCP. This paper integrates these ideas and propose a new congestion control algorithm, called Hybrid TFRC, for multimedia streaming. Experiments are carried out and effectiveness of our proposal is verified. ©2010 IEEE.

Service capacity and contribution of the peers have been proven to be the important factors for selecting a serving peer in the P2P overlay networks. In this paper, we study how to design a multi-parent overlay network in a contribution aware manner. Our idea is to utilize the outbound bandwidth of all overlay nodes efficiently. Different from other multisource streaming approaches, we employ push-based data delivery in order to minimize the latency. The core of our overlay is conceived in a tree structure rooted at a source. The auxiliary connections with redundancy avoidance capability are proposed to increase the resilience of the tree. Furthermore, we introduce an overlay joining algorithm where the parent whose degrees are exhausted responses the new peer with a list of potential peers having residual degrees so that the new peer can join the overlay with minimal requests. Simulations through ns2 demonstrate the efficiency of our solution. ©2010 IEEE.

With the development of recent vehicular communication technologies, distributing multimedia contents in the vehicular communication networks (VCNs) has become more and more popular, to provide conveniences and entertainment services during the time of driving. However, as multimedia contents are changed and updated dynamically, how to keep the consistency between the original and these replicas in VCNs is very important. Therefore, this paper designs a novel algorithm to control the consistency for the VCNs. In our proposal, after the analyses of the status of road-side units, on-board units and local geographical information, we divide all replicas into different groups, where one is necessary for update and the other are not. Then, we compare the cost to update replicas by using wireless and wired connection, and propose a method to make selection between them. The performance of our proposal is tested by simulation experiments. And the results show that our method can obtain the low delay successfully.

This paper proposes a new TCP congestion control supporting loss-fairness, which also contributes to improving RTT fairness. It has been well known that smaller RTT flows grab more bandwidth than larger ones when they compete on the shared link. This is called RTT unfairness and is caused by congestion control mechanism of the current TCP. To solve this problem, many protocols have been proposed. TCP-Libra is one of them, which formulates its window increase rate as a function of RTT and brings the same throughput irrespective of different RTT values. However, we indicate TCP-Libra's formulation holds only when the congestion window is halved upon packet losses similar to TCP-Reno. When we apply different (smarter) window control schemes, the formulation has to be modified accordingly. Therefore, in this paper, we consider a hybrid congestion control supporting RTT fairness which switches two modes according to buffering state at the bottleneck link. In decrement phase upon packet loss, it decreases window size to clear the buffering packets instead of halving the window size. Furthermore, we introduce the concept of loss-fairness for our window increase formulation, which is equivalent to RTTfairness. Experiments are carried out to validate the proposed method by both implementation experiments and simulations, and much better performances are achieved against conventional methods. ©2010 IEEE.

This research presents a formal user evaluation of a typical visualization method for content-based music information retrieval (MIR) systems, and also proposes a novel interface to improve MIR usability. Numerous interfaces to visualize content-based MIR systems have been proposed, but reports on user evaluations of such proposed GUIs are scarce. This research aims to evaluate the effectiveness of a typical 2-D visualization method for content-based MIR systems, by conducting comparative user evaluations against the traditional list-based format to present MIR results to the user. Based on the observations of the experimental results, we next propose a 3-D visualization system, which features a function to specify sub-regions of the feature space based on genre classification results, and a function which allows users to select features that are assigned to the axes of the 3-D space. Evaluation of this GUI conclude that the functions of the 3-D system can significantly improve both the efficiency and usability of MIR systems. © 2009 International Society for Music Information Retrieval.

This paper presents a novel key estimation method of sound sources based on the music theory known as "circle of fifths". We firstly overview music theory and formulate the musical key analysis by vector operations. In detail, we separate music sources into small pieces and calculate FFT-based chroma vectors. They are converted to tonality vectors and COF (circle-of-fifth) vectors are calculated from the tonality vectors, which are mapped onto the circle of fifths coordinate. As a result, each music source can be represented by traces of COF vectors, which usually stay inside a single key region on the circle of fifths. Finally, HMM is applied to the traces of COF vectors in order to detect keys and their boundaries. Experiments using music databases are also carried out.

In order to achieve highly accurate content-based music information retrieval (MIR), it is necessary to compensate the various bit rates of encoded songs which are stored in the music collection, since the bit rate differences are expected to apply a negative effect to content-based MIR results. In this paper, we examine how the bit rate differences affect MIR results, propose methods to normalize MFCC features extracted from encoded files with various bit rates, and show their effects to stabilize MIR results.

With the growing use of dynamic content by mobile content distribution systems, how to manage dynamically changing files has become an important issue, since the cached replicas on different mobile sites must be updated if the originals have been changed. Therefore, this paper proposes a priority based selection method to enhance the efficient utilization of network resource and support the client mobility for Mobile Contents Delivery Network (M-CDN). On one hand, a consistency priority is calculated by analyzing the characteristics of mobile surrogates. If a given content which has been changed on its original node, only the replicas with the high consistency priority instead of all replicas are updated. On the other hand, an update priority is also proposed. If a replica has been selected for update, the latest version will be sent from the site decided by the update consistency. Simulation results show that the proposed new approach outperforms other conventional methods. © 2009 IEEE.

This paper presents TCP-Fusion supporting RTT (Round Trip Time) fairness in addition to throughput efficiency and friendliness to TCP-Reno. When multiple TCP flows having different RTT values compete, more bandwidth is unfairly allocated to the flow having smaller RTT. This means that a user with longer RTT may not be able to obtain sufficient bandwidth by the current methods. On the other hand, recent studies on the TCP congestion control to achieve RTT fairness and throughput efficiency are evolving actively. An example for RTT fairness is TCP-Libra and an example for throughput efficiency is Hybrid TCP congestion control. This paper focuses on Hybrid TCP (exploiting residual link capacity when TCP-Reno drops its rate) and improves its RTT fairness by incorporating the ideas of TCP-Libra (congestion window increase in proportion to square of RTT) to its loss mode. Experiments are carried out to validate the proposed method and much better performances in RTT fairness and throughput are provided against conventional methods.

In this paper, we introduce a new mesh-based approach for data delivery which is organized over multiple tree-shaped core routes. Given that both tree and mesh approaches have their own strong points, we simply combine them together. We evaluate the proposal through ns-2 simulator. The simulation results demonstrate that our approach can provide higher average received quality and has acceptable data delivery delay when compared with a single tree method. We also show that, over a static overlay, the push-based data delivery on mesh can provide the received quality close to pull-based data delivery method with less latency. As well, it has lower control overhead than the pull-based method when the peer number is large.

Recently it has been shown that scalable (layered) image is appropriate for the Internet because of its better flexibility and functionality. At the same time, the pre-fetching technology also proves to be an efficient method to improve the performance of image-delivery with the low delay and adjustable quality. However, how to deploy the pre-fetching to deliver scalable contents has not been mentioned. Therefore, this paper presents an analysis of the pre-fetching algorithm to deliver layered image contents over the Internet. Based on the analysis of web request probability and the relations between different layers in the scalable imaging delivering system, a new pre-fetching algorithm is proposed to drastically improve cache performance. This proposal is a request frequency-based approach, in which a progressive image format can be used.

In order to deliver multimedia contents to the end-users with low delay and acceptable quality, the pre-fetching technologies have been shown the efficiency if we can predict user&apos;s future requests and prefetch the data needed previously. However, because both the network topology and Web access are changed dynamically, whether the pre-fetching can get the stable performance under dynamic environment becomes an important issue. Therefore, this paper talks about the robustness of the pre-fetching algorithm, which has been proposed by us to deliver layered image contents over the Internet. Based on the web access distribution, the request pattern in the scalable imaging delivering system is analyzed in this paper. Furthermore, we check the robustness of our proposal by changing related parameters. Simulation results show that Our proposal is stable against variable conditions even if access patterns are dynamically changed.

The total amount of information to process or analyze is jumping sharply with the quick spread of computers and networks. Our project, «Highly scalable monitoring architecture for information explosion», develops a monitoring system allows observing systems, merging the system logs, and discovering intelligence to share. More concretely, the project builds the total system to maintain, optimize, and protect autonomically. This paper reports the outcomes of the project after first-half of the development period.The rest of the paper is organized as follows. Section 2 describes the concept and details of the monitoring system on a single node, and Section 3 addresses the aggregation of the collected information in distributed environments. Section 4 and Section 5 introduce applications of the monitoring systems. Section 6 summarizes the project and mentions future plans. © 2009 IEEE.

In this paper, we introduce a new mesh-based approach for data delivery which is organized over multiple tree-shaped core routes. Given that both tree and mesh approaches for multicast overlay construction have their own strong points, we simply combine them. During session, the mesh connections are formed dynamically to facilitate the data delivery on tree-shaped core routes which may experience bandwidth bottlenecks. Unlike existing mesh-based approaches, our proposal employs push-based data delivery along mesh connection. A well-organized mesh construction scheme is defined to avoid data redundancies. We evaluated the proposal through ns-2 simulator. The simulation results demonstrate that our approach can provide higher average received quality and has acceptable control overhead when compared to a single tree method. We also show that, over a static overlay, the push-based data delivery on mesh can provide the received quality close to supposed pull-based data delivery method with less latency. Moreover, it has lower control overhead than the pull-based method when the peer number is large.

This paper presents a denoising method of intra-coded pictures using motion estimation and pixel shift. Firstly, we show that pixel-aligned mixture of distorted images which are spatially shifted and differently encoded brings reduction of quantization errors. We show that this effect can be formulated as a special case of Wiener-Hopf equation and independence of quantization errors affects the performance. We then consider its application to denoising of intra-coded pictures by using motion estimation and pixel shift. Experiments using actual image sequences verify that motion estimation is effective in moving regions, pixel shift is effective in static regions and favorable PSNR gains are achieved.

With the development whereby mobile content distribution systems can manage dynamically changing files, an important issue to be resolved is consistency management, which means the cached replicas on different mobile sites must be updated if the originals change. This paper is to design an integrated consistency-control algorithm for mobile contents delivery network (M-CDN) to enhance the efficient utilization of network resource and support the client mobility. Firstly, by carrying out an analysis of mobile surrogates' characteristics, for a given content which has been changed on its original node, only a limited number of its replicas instead of all replicas are updated. Secondly, if a replica has been selected for update, the latest version will be sent from an algorithm-decided site instead of from its original server. Simulation results show that the proposal outperforms other conventional methods.

This paper presents simple analytical models of hybrid TCP congestion controls, which switch loss-based mode and delay-based mode adaptively, and tries their performance tuning. We firstly present ideal behavior models of three kinds of TCP congestion controls (loss-based, delay-based and hybrid). We then give abstracted models of the actual hybrid TCP s and consider their performance tuning. Finally, experiments validate analytical expectations and effectiveness of the hybrid TCP.

This paper is to design an integrated consistency-control algorithm for mobile contents delivery network (M-CDN) to enhance the efficient utilization of network resource and support the client mobility. Firstly, by carrying out an analysis of mobile surrogates characteristics, for a given content which has been changed on its original node, only a limited number of its replicas instead of all replicas are updated. Then, if a replica has been selected for update, the latest version will be sent from an algorithm-decided site instead of from its original server.

Content delivery network improves end-user performance by replicating Web contents on a group of geographically distributed sites interconnected over the Internet. However, with the development whereby content distribution systems can manage dynamically changing files, an important issue to be resolved is consistency management, which means the cached replicas on different sites must be updated if the originals change. In this paper, based on the analytical formulation of object freshness, web access distribution and network topology, we derive a novel algorithm as follows: (1) For a given content which has been changed on its original server, only a limited number of its replicas instead of all replicas are updated. (2) After a replica has been selected for update, the latest version will be sent from an algorithm-decided site instead of from its original server. Simulation results verify that the proposed algorithm provides better consistency management than conventional methods with the reduced the old hit ratio and network traffic.

With the development whereby mobile content distribution systems can manage dynamically changing files, an important issue to be resolved is consistency management, which means the cached replicas on different mobile sites must be updated if the originals change. This paper is to design an integrated consistency-control algorithm for mobile contents delivery network (M-CDN) to enhance the efficient utilization of network resource and support the client mobility. Firstly, by carrying out an analysis of mobile surrogates&apos; characteristics, for a given content which has been changed on its original node, only a limited number of its replicas instead of all replicas are updated. Secondly, if a replica has been selected for update, the latest version will be sent from an algorithm-decided site instead of from its original server. Simulation results show that the proposal outperforms other conventional methods.

The purpose of this study is to construct a robust and stable overlay tree of ALM (Application Layer Multicast) for real-time video transmission. Firstly, we propose a proactive route maintenance which enables fast recovery of the overlay tree against node departures and failures. By forcing free node degrees for route backup, signaling overheads for route maintenance can be reduced in a scalable manner. Secondly, we improve performance of the proposed method by introducing layered video coding. Smooth layer management instead of coarse degree management contributes to reduction of the depth of overlay tree (i.e. delay) as well as to efficient bandwidth utilization. Thirdly, we introduce session records of each node into the overlay tree construction. Long-term history of user access records is expected to bring more robust overlay trees. Finally, we carried out extensive evaluation experiments. Simulations and implementations demonstrate that our methods lead to fast recovery of the overlay tree against node departures and reduction of the signaling overheads. Furthermore, introduction of layered video coding is proved to be efficient to reduce delays, to improve bandwidth utilization and to avoid severe degradation of picture quality. Final experiments show that incorporation of node stability reduces the number of nodes affected by parent node departures by promoting adequate nodes to upper layer of the ALM tree. ©2007 IEEE.

Content Delivery Overlays improves end-user performance by replicating Web contents on a group of geographically distributed sites interconnected over the Internet. However, with the development whereby overlay systems can manage dynamically changing flies, an important issue to be resolved is consistency management, which means the cached replicas on different sites must be updated if the originals change. In this paper, based on the analytical formulation of object freshness, web access distribution and network topology, we derive a novel algorithm as follows: (1) For a given content which has been changed on its original server, only a limited number of its replicas instead of all replicas are updated. (2) After a replica has been selected for update, the latest version will be sent from an algorithm-decided site instead of from its original server. Simulation results verify that the proposed algorithm provides much better consistency management than conventional methods with the reduced the old hit ratio and network traffic. © 2007 IEEE.

A robust replication algorithm is very important to retrieve Web objects from the Content delivery network (CDN), where both the network topology and Web access are changed dynamically. However, current replica strategies in CDN are to statically and repeatedly keep the complete replica of the original object on many content servers. This method is weak in adapting the dynamic network, resulting in too much consumed server space and a waste of the storage cost. It becomes more serious by replicating some large-sized objects such as streaming media, which are being distributed over the Internet,more and more. In this paper, we discuss a robust replication strategy for scalable video strew-ning in CDN to reduce user response and storage cost as follows: 1) For content servers, which layer of which video contents should be selected to replicate? 2) How to test the robustness under different network environments? Simulation results show that the proposed algorithm is robust and can provide much better performance than the conventional methods.

Content Delivery Overlays improves end-user performance by replicating Web contents on a group of geographically distributed sites interconnected over the Internet. However, with the development whereby overlay systems can manage dynamically changing files, an important issue to be resolved is consistency management, which means the cached replicas on different sites must be updated if the originals change. In this paper, based on the analytical formulation of object freshness, web access distribution and network topology, we derive a novel algorithm as follows: (1) For a given content which has been changed on its original server, only a limited number of its replicas instead of all replicas are updated. (2) After a replica has been selected for update, the latest version will be sent from an algorithm-decided site instead of from its original server. Simulation results verify that the proposed algorithm provides much better consistency management than conventional methods with the reduced the old hit ratio and network traffic.

The purpose of this study is to maintain efficient backup routes for reconstructing overlay trees quickly. In most conventional methods, after a node leaves the trees, its child nodes start searching for the new parents. In this reactive approach, it takes a lot of time to find a new parent. In this paper, we propose a proactive approach to finding a next parent as the backup route node over the overlay tree before the current parent leaves. A proactive approach allows a node to find its new parent node immediately and switch to the backup route node smoothly. In our proposal, the structure of the overlay tree using a redundant degree can decide a backup route node without so much overhead. Simulations demonstrate our proactive approach can recover from node departures 2 times faster than reactive approaches, and can construct overlay trees with lower overheads than another proactive method. Additionally we carried out experiments over actual networks and their results support the effectiveness of our approach. We confirmed that our proposal achieved better streaming quality than conventional approaches.

We propose an on-demand multipath routing algorithm in a mobile ad hoc network for video transmission and evaluate its real world performance in video streaming application. There have been a number of multipath routing protocols extended from AODV which is a well-known single path routing protocol. Multipath routing protocols indicate good performance in the reduction of route discovery latency and unnecessary routing packets in simulations. We show that the route establishment using source route lists provided by us (Hu and Johnson, 2002) can reduce the route discovery latency, select stable routes automatically, and work well for live video streaming without limitation of the hop count based approaches. We evaluate this proposed method compared with the original AODV by using eight laptop PCs and demonstrate live streaming experiments.

The current video switching/composition scheme employed is fundamentally subject to a variable delay of 0-1 frame at each stage due to the use of a frame synchronizer. We can fix and minimize this variable delay by controlling the video sync phase of the distributed video sources at the start of the communication session if they are driven by either the same video sync frequency or independent but very accurate sync frequencies. Furthermore, delivering the necessary phase controlling signals between the terminal and the video switching or composing point can be relatively easy because of the widely available IP networks. Based on .these ideas, we determined the performance of an experimental system we constructed that used video phase synchronization of distributed video sources. We describe the system's architecture, discuss the sync source dependency of the system, and report the delay observed in existing and proposed schemes of video composition using real time hardware experiments. The results obtained indicate that the proposed video sync controlling system is effective and has practical applications for fixing and minimizing the video switching/composition delay.

This paper proposes a feature space modification method for feature extraction of music, which is effective for the development of a content-based music information retrieval (MIR) system based on user preferences. The proposed method conducts clustering of all songs in the music collection, and utilizes the resulting cluster IDs as training data for feature space modification. and is capable to automatically generate a feature space which is suitable to the content of any music collection. Experiment results prove that the proposed method improves accuracy of user preference based MIR.

Content Distribution Overlays improves end-user performance by replicating Web contents on a group of geographically distributed sites interconnected over the Internet. However, with the development whereby overlay systems can manage dynamically [2] changing files, an important issue to be resolved is consistency management, which means the cached replicas on different sites must be updated if the originals change. In this paper, based on the analytical formulation of object freshness time, web access distribution and network topology, we derive a novel algorithm as follows: (1) For a given content which has been changed at its original server, only a limited number of its replicas instead of all replicas are updated. (2) After a replica has been selected for update, the latest version will be sent from an algorithm-decided site instead of from its original server. Simulation results verify that the proposed algorithm provides much better consistency management than conventional methods with the reduced update overhead and network traffic.

This paper proposes a feature space modification method for feature extraction of music, which is effective for the development of a content-based music information retrieval (MIR) system based on user preferences. The proposed method conducts clustering of all songs in the music collection, and utilizes the resulting cluster IDs as training data for feature space modification, and is capable to automatically generate a feature space which is suitable to the content of any music collection. Experiment results prove that the proposed method improves accuracy of user preference based MIR.

The purpose of this study is to maintain efficient backup routes for reconstructing overlay trees quickly. In most conventional methods, after a node leaves the trees, its children start searching for a new parent. In this reactive approach, it takes a lot of time to find a new parent. In this paper, we propose a proactive approach to finding a new parent over the overlay trees before the current parent leaves. A proactive approach allows a node to find its new parent immediately and switch to the backup route smoothly. In our proposal, the structure of the overlay tree using a redundant degree can decide a new parent without so much overhead. Simulations demonstrate our proactive approach can recover from node departures 2 times faster than reactive approaches, and can construct overlay trees with lower overheads than another proactive method. Additionally we carried out experiments over actual networks and their results support the effectiveness of our approach. We confirmed that our proposal achieved better streaming quality than conventional approaches.

This paper presents a novel approach for efficient tree construction in ALM (Application Layer Multicast) using layered video coding. This system is suitable for the users in heterogeneous environment such as xDSL and CATV, where upload and download speeds are different. We point out some problems in these environment of the ALM and show their solutions at two stages. First, we redefine the "degree parameter" which originally specifies the number of single rate streams the host can transmit, to reflect asymmetric link properties and to accommodate layered streams. Second, we develop an efficient tree construction algorithm using the renewed degree parameter, which maximizes total throughput of all hosts. Simulations using NS-2 prove improvement in throughput, delay and overhead by our proposal. Software implementation is also carried out, and subjective quality improvement is verified. © 2005 IEEE.

Content Distribution Overlays improves end-user performance by replicating Web contents on a group of geographically distributed sites interconnected over the Internet. However, with the development whereby overlay systems can manage dynamically changing files, an important issue to be resolved is consistency management, which means the cached replicas on different sites must be updated if the originals change. In this paper, based on the analytical formulation of object freshness time, web access distribution and network topology we derive a novel algorithm as follows: (1) For a given content which has been changed at its original server, only a limited number of its replicas instead of all replicas are updated. (2) After a replica has been selected for update, the latest version will be sent from an algorithm-decided site instead of from its original server. Simulation results verify that the proposed algorithm provides much better consistency management than conventional methods with the reduced update overhead and network traffic.

Content Distribution Overlays improves end-user performance by replicating Web contents on a group of geographically distributed sites interconnected over the Internet. However, with the development whereby overlay systems can manage dynamically changing files, an important issue to be resolved is consistency management, which means the cached replicas on different sites must be updated if the originals change. In this paper, based on the analytical formulation of object freshness time, web access distribution and network topology we derive a novel algorithm as follows: (1) For a given content which has been changed at its original server, only a limited number of its replicas instead of all replicas are updated. (2) After a replica has been selected for update, the latest version will be sent from an algorithm-decided site instead of from its original server. Simulation results verify that the proposed algorithm provides much better consistency management than conventional methods with the reduced update overhead and network traffic.

This paper presents a novel approach for efficient tree construction in ALM (Application Layer Multicast) using layered video coding. This system is suitable for the users in heterogeneous environment such as xDSL and CATV, where upload and download speeds are different. We point out some problems in these environment of the ALM and show their solutions at two stages. First, we redefine the "degree parameter" which originally specifies the number of single rate streams the host can transmit, to reflect asymmetric link properties and to accommodate layered streams. Second, we develop an efficient tree construction algorithm using the renewed degree parameter, which maximizes total throughput of all hosts. Simulations using NS-2 prove improvement in throughput, delay and overhead by our proposal. Software implementation is also carried out, and subjective quality improvement is verified.

TCP Reno versions are widely used in current network, however it has been actualized that their throughput deteriorates in high-speed network and wireless environments. To overcome these problems of TCP Reno versions, a number of protocols have been proposed. In these researches, friendliness with TCP Reno becomes important. TCP Westwood is an example that is based on end-to-end bandwidth estimate, and brings higher efficiency performances. However it will be shown that the friendliness with TCP Reno is deteriorated according to network situations such as the buffer size of a bottleneck link router. In this paper, we quantify the buffer size that TCP Reno and TCP Westwood perform friendly to each other, and then propose an improved version of TCP Westwood that achieves friendliness with TCP Reno by estimating the buffer size of a bottleneck link router by using bandwidth estimation technique, RCE, and by updating congestion control parameters under the constraint of throughput estimation models. We confirm effectiveness of our proposal scheme by extensive simulation experiments. Simulation results show that our proposed scheme performs friendly to TCP Reno in various network situations independently of router buffer sizes.

The purpose of this study is to maintain efficient backup routes for restoring overlay trees. In most conventional methods, after a node leaves the trees, its children start searching for a new parent. In this reactive approach, it takes a lot of time to find a new parent. In this paper, we propose a proactive approach to find a new parent over the overlay trees before the current parent leaves. A proactive approach can find respective new parents immediately and switch to the backup route smoothly. In our proposal, the structure of the overlay tree using a redundant degree enables to decide a new parent without so much overhead information. Simulations demonstrate our proactive approach can recover from node departures 2 times faster than reactive approaches, and can construct overlay trees with lower overheads than another proactive method. Additionally we carried out experiments over actual networks and their results support the effectiveness of our approach. We confirmed that our proposal achieved better streaming quality than conventional approaches.

This paper proposes a novel on-demand multipath routing protocol for a mobile ad hoe network. By applying a newly developed route update procedure with combined metrics of delay, hop count and disjointness, each intermediate node deliberately selects multipath candidates while contributing to suppression of unnecessary routing packets. Extension of RREQ/RREP packets with a source route list is also incorporated, not only to alleviate limitation of the hop-count based approaches but rather to provide more efficient multiple routes. Computer simulations using ns-2 simulator are performed with comparison to conventional methods and effectiveness of the proposed methods is quantitatively validated.

CDN (Content Delivery Networks) improves end-user performance by replicating web contents on a group of geographically distributed servers. However, repeatedly keeping the entire replica of the original objects into many content servers consumes too much server resource. This problem becomes more serious for the large-sized objects such as streaming media, e.g. high quality video. In this paper, we therefore propose an efficient replication method for layered video streams in CDN, which can reduce user response delays and storage costs simultaneously. Based on an analytical formulation of the cooperative replication of layers and segments of each video stream, we derive a replication algorithm which solves next three problems quantitatively. (1) How many servers should be selected to replicate a given video stream? (2) For a single video stream, how many layers and segments should be stored in a given server? (3) After selecting a group of servers for each video stream, how do we allocate the replication priority (i.e. order) to each server? Simulation results verify that the proposed algorithm efficiently resolves the above problems and provides much better performance than conventional methods.

In this paper, an efficient algorithm to look for backup parents in preparation of parent leaving is proposed for application layer multicasting whose topology is constituted in the shape of a tree from a single source node. In most conventional methods, each child node starts searching for its new parent after its parent node leaves from a multicasting tree. This reactive operation often causes long interruption period. In our proposal, each node holds its parent candidate proactively over redundant overlay trees. Proactive route maintenance leads to smooth switching to a new parent after node leaving and failure, and redundant structure of a multicasting tree avoids exhausting search of a backup parent. Computer simulations are also carried out and effectiveness of the proposed approach is verified.

With the advance of network technologies, availability and popularity of streaming media contents over the P2P (Peer-to-Peer) Networks have grown rapidly in recent years. However, how to efficiently search a requested steaming media among P2P peers is still a problem which causes a serious user delay and limited hit ratio. This paper presents an efficient search method for streaming media in P2P, which reduces user response delays and exchange overhead simultaneously. Based on an analytical formulation of both streaming media and P2P peers' characteristics, we derive a search algorithm which solves the next two problems quantitatively. (1) How to decide the number of walkers (queries) at each step of search? (2) How to decide the length of walkers (queries) at each step of search? Simulation results verify that the proposed algorithm efficiently resolves the above problems and provides much better performance than conventional methods.

CDN (Content Delivery Networks) improves end-user performance by replicating web contents on a group of geographically distributed servers. However, repeatedly keeping the entire replica of the original objects into many content servers consumes too much server resource. This problem becomes more serious for the large-sized objects such as streaming media, e.g. high quality video. In this paper, we therefore propose an efficient replication method for layered video streams in CDN, which can reduce user response delays and storage costs simultaneously. Based on an analytical formulation of the cooperative replication of layers and segments of each video stream, we derive a replication algorithm which solves next three problems quantitatively. (1) How many servers should be selected to replicate a given video stream? (2) For a single video stream, how many layers and segments should be stored in a given server? (3) After selecting a group of servers for each video stream, how do we allocate the replication priority (i.e. order) to each server? Simulation results verify that the proposed algorithm efficiently resolves the above problems and provides much better performance than conventional methods.

TCP is a popular internet protocol for reliable end-to-end data delivery, but it cannot be directly applied to wireless networks in which packet loss may be induced by higher BER or handover than congestion. TCP assumes that such packet loss is caused by network congestion and initiates congestion control procedures. In this paper, we present a novel protocol using Explicit Handover Notification to improve TCP performance over wireless links. Additionally, we execute computer simulations using network simulator and compare with other various protocols.

This paper presents a method of gender classification by integrating facial, hairstyle, and clothing images. Initially, input images are separated into facial, hairstyle and clothing regions, and independently learned PCAs and GMMs based on thousands of sample images are applied to each region. The classification results are then integrated into a single score using some known priors based on the Bayes rule. Experimental results showed that our integration strategy significantly reduced error rate in gender classification compared with the conventional facial only approach.

In this paper, we propose a motion vector search algorithm that employs the Sum of Absolute Difference (SAD) minimization in DCT domain as a distortion measure. In addition, block based motion estimation is carried out with multiple modes in this frequency domain, and a mode is selected according to the Lagrangian cost function [1, 2, 3]. Four modes are designed. The first two modes are a mode with 1 motion vector (INTER1V) and a mode with 4 motion vectors (INTER4V) per macroblock that are similar to the H.263 [4]. The last two modes use 4 motion vectors per macroblock (INTER1V+) or per block (INTER4V+), where the 8×8 DCT coefficients are firstly separated into 4 groups and a motion vector is independently assigned to each group. The average SNR (Signal to Noise Ratio) gains between 0.3 and 1.0dB are obtained for tested sequences when compared with TMN. Our proposal method provides true motion vectors for low frequency and minimizes distortion for higher frequency. Therefore our proposed method also reduces subjective visual degradation.

CDN (Content Delivery Networks) improves end-user performance by replicating web contents on a group of geographically distributed content servers. Replication Algorithm plays an important role in helping users to retrieve Web objects from the content servers. If a user can directly get the requested objects from the content server, he need not to contact the remote origin server and the user delay can be reduced. However, current replica strategies in CDN are to simply and repeatedly keep the complete replica of the original object on many content servers. This method has some disadvantages, including too much consumed server space and a waste of the storage cost. It is more serious for replicating some large-sized objects such as streaming media, which are being distributed over the Internet more and more. In this paper, we discuss a replication strategy for scalable video streaming in CDN to reduce user response and storage cost. Based on theoretical analysis, assuming layered video coding, we propose a novel replication algorithm which deals with following three problems. (1) How many content servers should be selected to replicate a given video content? (2) For a single video content, how many layers should be kept in a given content server? (3) After selecting a group of content servers for each video content, how do we decide the replication priority for each content server? Simulation results show that the proposed algorithm can efficiently resolve the above problems, and provide much better performance than the conventional methods.

With the advance of high-speed network technologies, availability and popularity of streaming media contents over the Internet has grown rapidly in recent years. Because of their distinct statistical properties and user viewing patterns, traditional delivery and caching schemes for normal web objects such as HTML files or images can not be efficiently applied to streaming media such as audio and video. In this paper, we therefore propose an integrated caching scheme for streaming media with segment-based caching and hierarchically distributed proxies. Firstly, each stream is divided into segments and their caching algorithms are considered to determine how to distribute the segments into different level proxies efficiently. Secondly, by introducing two kinds of segment priorities, segment replacing algorithms are proposed to determine which stream and which segments should be replaced when the cache is full. Finally, a Web-friendly caching scheme is proposed to integrate the streaming caching with the conventional caching of normal web objects. Performance of the proposed algorithms is verified by carrying out simulations.

The efficient distribution of stored information has become a major concern in the Internet. Since the web workload characteristics show that more than 60% of network traffic is caused by image documents, how to efficiently distribute image documents from servers to end clients is an important issue. Proxy cache is an efficient solution to reduce network traffic. And it has been shown that an image caching method (Graceful Caching) based on hierarchical coding format performs better than conventional caching schemes in recent years. However, as the capacity of the cache is limited, how to efficiently allocate the cache memory to achieve a minimum expected delay time is still a problem to be resolved. This paper presents an integrated caching algorithm to deal with the above problem for image databases, web browsers, proxies and other similar applications in the Internet. By analyzing the web request distribution of the Graceful Caching, both replacing and pre-fetching algorithms are proposed. We also show that our proposal can be carried out based on information readily available in the proxy server; it flexibly adapts its parameters to the hit rates and access pattern of users&#039; requesting documents in the Graceful Caching. Finally we verify the performance of this algorithm by simulations.

The emergence of the Internet as a pervasive communication medium has led to the rise of many applications of streaming media. However, because of their distinct statistical properties and user viewing patterns, traditional delivery and caching schemes for web objects such as HTML files or images can not be efficiently applied to streaming media such as audio and video. In this paper, we therefore propose an integrated caching scheme for streaming media with segment-based caching and with hierarchically distributed proxies. Firstly, each stream is divided into segments and their caching algorithms are considered to determine how to distribute the segments into different level proxies efficiently. Then, by introducing two kinds of segment priorities, segments replacing algorithms are proposed to determine which stream and which segments should be replaced when the cache is full. Finally, a web friendly caching scheme is proposed to integrate the streaming caching with the conventional caching of normal web objects. Performance of the proposed algorithms is verified by carrying out simulations.

The efficient distribution of stored information has become a major concern in the Internet. Since the web workload characteristics shows that more than 60% of network traffic is caused by image, documents, how to efficiently distribute image documents from servers to end clients is an important issue. Proxy cache is an efficient solution to reduce network traffic. An it has been shown that an image caching method (Graceful Caching) based on hierarchical coding format showed better performance than conventional. caching schemes in recent years. However, as the capacity of the cache is limited, how to efficiently allocate the cache memory to achieve a minimum expected delay time is still a problem to be resolved.
This paper presents an integrated caching algorithm to deal with the above problem in the Internet. By analyzing the web request distribution of Graceful Caching, both replacement and pre-fetching algorithms are proposed. We also show that our proposal can be carried out based on information readily available in the proxy server and it flexibly adapts its parameters to the hit rates and access pattern of users' requesting documents in the Graceful Caching. We finally verify the performance of this algorithm by simulations.

This paper presents a cache scheme for image databases, web browsers, proxies and other similar applications in Internet The current cache system employs a hard strategy: either the image Is stored in a cache or not even its data is quite big. We have proposed a caching scheme featuring human memory mechanism (image database system featuring graceful oblivion), which is called Graceful Caching. The previous work shows that Graceful Caching reduces user response time by assigning a variable amount of memory to each image. Several recent studies suggest that prefetching, techniques could be employed to further improve the cache performance, by anticipating and prefetching future client requests. Some approaches based on request frequency have been proposed to the conventional caching (hard system) and got better results. In this paper, we try to apply a request frequency based approach. to Graceful Caching system, in which a progressive image format is used. Based on the analysis of web request probability in Graceful Caching, we go on to propose a new prefetching algorithm that drastically improves cache performance of system. We verify the performance of this algorithm by simulations.

This paper presents a novel framework for Euclidean structure recovery utilizing a scaled orthographic view and perspective views simultaneously. A scaled orthographic view is introduced in order to automatically obtain camera parameters such as camera positions, orientation, and focal length. Scaled orthographic properties enable all camera parameters to be calculated implicitly and perspective properties enable a Euclidean structure to be recovered. The method can recover a Euclidean structure with at least seven point correspondences across a scaled orthographic view and perspective views. Experimental results for both computed and natural images verify that the method recovers structure with sufficient accuracy to demonstrate potential utility. The proposed method can be applied to an interface for 3D modeling, recognition, and tracking.

This paper presents a system architecture for synthetic/natural hybrid coding toward future visual services, Scene-description capability, terminal architecture, and network architecture are discussed by taking account of recent standardization activities: MPEG, VRML, ITU-T, and IETF, A consistent strategy to integrate scene-description capability and streaming technologies is demonstrated. Experimental results are also shown, in which synthetic/natural integration is successfully carried out.

A new wavelet coding method is presented to exploit both intra/inter-band correlations. While the traditional zerotree approach can exploit correlation between subbands, a wavelet decomposition has properties more suitable for the runlength coding method, which can exploit spatial correlation. The proposed method classifies wavelet coefficients into groups according to their parents' magnitude. Each group is chosen in order of the magnitude and scanned to keep spatial correlation. It outperforms the zerotree-based coding approach especially at low bit rate.

This paper describes novel methods to improve coding efficiency of the wavelet transform from the viewpoint of scanning techniques. Firstly, statistical properties of the wavelet transform are examined using real image sequences. They indicate energy distribution or correlations of wavelet transformed coefficients. Taking them into account, efficient scanning methods are investigated for two different approaches. The first approach treats transform coefficients, located in spatially neighboring position, in 8 × 8 blocks. This is called block based approach, and its optimum scanning path is examined similar to the zigzag scan for DCT (discrete cosine transform). Furthermore, a new scanning algorithm utilizing correlation between transform coefficients is proposed. The second approach carries out scanning over the entire subimage created by the wavelet transform. This is called subband based approach, in which scanning is applied to each subband. A new scanning algorithm is presented utilizing a block-wise non-linear filter which removes isolated non-zero coefficients. Simulations are carried out, and their results show the effectiveness of the above approaches especially for low bit-rate applications.

This paper presents novel algorithms exploiting multiple camera inputs and segmentation techniques, which can be applied to image fusion, disparity detection and object extraction. Differently focused images, stereo pairs and both of them are used for fusion, disparity detection and object extraction, respectively. Firstly, image fusion is done by segmentation of each image and determination of focused regions per segment. An efficient decision criterion is developed taking a method of auto-focus into consideration. Secondly, disparity detection is executed by recursively applying segmentation and disparity detection per segment. A new clustering criterion is proposed in order to achieve good segmentation and high compression ratio of disparity maps simultaneously. Finally, object extraction is carried out by utilizing both the fusion result and the disparity map. Experiments are carried out, and they show us effectiveness of the proposed algorithms.

This paper presents a novel framework for structure recovery utilizing a scaled orthographic view and perspective views simultaneously. Perspective views lead to precise recovery based on the triangulation principle; however, many parameters such as camera positions, camera poses and focal length, must be measured beforehand. Thus, a scaled orthographic view is introduced as a subsidiary system to achieve them automatically. Camera parameters ate calculated implicitly owing to the scaled orthographic properties, and then the structure recovery is done by the orthogonality of camera coordinate systems and the triangulation principle. The experimental results provide sufficient accuracy of structure recovery.

This paper presents a novel framework which proves the superiority of overlapped motion compensation. Window design problem is revised by introducing a statistical model of motion estimation process. The result clarifies relationship between the optimum window and image characteristics in an explicit formula and quantifies prediction error reduction achieved by overlapped motion compensation. Experimental results using real image sequences support the proposed theory and demonstrate its superiority. Overlapping in warping prediction is also considered and its effectiveness is shown.

This paper presents mathematical frameworks on temporal predictive processing in the MPEG video compression standard. Firstly, a coding gain is derived based on traditional prediction theories. The optimum ordering of three different picture types (I,P,B-pictures) is clarified according to image source characteristics. Secondly, a novel framework on the target bit assignment is presented with some experimental backgrounds. The solution consists of simple formulae, but brings drastic SNR gains to the conventional TM5 algorithm.

This paper describes theories on overlapped motion compensation and their applications to a wavelet codec aimed for very low bit-rate environment below 64 kb/s. The theories are concerned with the evaluation of prediction efficiency improved by overlapped motion compensation and also its smoothing effect on the discontinuities at block boundaries encountered when motion vectors do not coincide among neighboring blocks. They contribute to determine the optimum window shape for overlapped motion compensation in a developed wavelet codec, which suffers from reduction of coding efficiency when there are such discontinuities in a signal to be transformed. Regarding the wavelet codec, a new scanning method of transform coefficients and alternate use of normal and spatially-reversed basis functions for cascaded wavelet transform are introduced. Finally, the implementation of the codec on a Video Image Signal Processor (VISP) is discussed. © 1994 IEEE

© 1992 IEEE. New parallel filter banks allowing perfect reconstruction which are categorized in coding intermediate between subband coding and predictive coding are presented. M-channel filter banks allowing perfect reconstruction are represented mathematically, and filter banks categorized in the above coding are discovered. Three types of three-channel filter banks are designed. The coding gains of these filter banks are then maximized, i.e., the filter coefficients are optimized from the viewpoint of energy compaction property. Computer simulation shows that the designed filter bank performs well.

This paper presents a filter design algorithm in order to realize high efficient image codec system based on subband decomposition, and also describes unified frameworks to interpret various image coding techniques. First, various image coding techniques based on the linear transform are dealt with in an unified manner, where both the matrix representation in time domain and the multirate filter bank concept are introduced. Reconsideration about the DPCM is stressed here because the subband coding and the transform coding have been already formulated in some common frameworks. Neither the filter&#039;s frequency response nor its orthogonality are taken into consideration because they don&#039;t necessarily play an important role in our formulation, particularly in case of the DPCM and the SSKF. Secondly, statistical optimization of short tap and linear phase PR (perfect reconstruction) filter banks is considered. The UCG (unified coding gain) that we proposed at the VSIP &#039;91 as a performance measure of energy compaction properties of the multirate filter bank is expanded to multiple layer cases, and several new examples are presented. The short tap structure leads to low computational complexity, and the linear phase property contributes not only to remove phase distortion but also to solve the so-called border problem. Simulation results are also shown, and the validity of our approach is confirmed.

In this paper, two analytical methods for evaluating the coding efficiency of subband coding are proposed, and optimization of filter coefficients of the perfect reconstruction FIR filter banks is considered, based on a new performance measure called unified coding gain. First, matrix representation of the subband coding in the time domain is considered, and conventional subband filter banks are classified into orthogonal ones such as the QMF and nonorthogonal ones such as the SSKF. For the orthogonal filter banks, the coding gain shown by Jayant and Noll is introduced, and their theoretical performance evaluation is carried out. However, this first method cannot be applied to nonorthogonal filter banks any longer because the coding gain is defined on the assumption of filter orthogonality. Therefore, an optimum bit allocation problem for subband coding is considered, and the unified coding gain, which can be applied to arbitrary subband filter banks, is derived as a new performance measure to take the place of the coding gain. This second method enables us to estimate the coding efficiency of arbitrary transform techniques as well as the subband approaches, and its result suggests that the SSKF(5 × 3) outperforms the QMF as long as the number of subbands is not too large, even though its filter length is much shorter. This result encourages us to find filter coefficients that maximize the unified coding gain according to filter length. In addition, new perfect reconstruction FIR filter banks which have not only low computational complexity but also good energy compaction properties are presented. © 1991.

One of the merits with the introduction of variable bit-rate coding to moving pictures is the constancy of picture quality. However, considering the fact that conventional SNR cannot reflect the result of subjective assessment sufficiently, other quality measures should be considered to design the variable bit-rate coders. In this paper, weighted SNR (WSNR), which is calculated by weighting coding errors with the spatial frequency response of human vision, is introduced as a quality measure in place of SNR, and the variable bit-rate coder based on this measure is considered. Then, subjective assessment is carried out and the effectiveness of our approach is confirmed. © 1991.

In this paper, two analytical methods to evaluate coding performance of subband coding are proposed, and optimization of its filter coefficients from the viewpoint of energy compaction property is considered. The first method is based on matrix representation of subband coding in time domain, where the coding gain given by Jayant and Noll is introduced as a performance measure for filter banks with orthogonal property. The second method is based on the optimum bit allocation problem for subband coding (multirate filter bank), where the unified coding gain is derived as a new performance measure which can be applied to arbitrary transform techniques. We then try to find filter coefficients which maximize the unified coding gain according to input characteristics. This approach leads to optimization of filter coefficients from the viewpoint of energy compaction property.

As a promising means of realizing broadband ISDN, the ATM (asynchronous transfer mode), which is based on fast packet switching, is considered interesting. In video communication by ATM, the constancy and improvement of picture quality by variable rate coding is expected, while the quality deterioration by cell loss is a serious problem. This paper proposes a video coding method introducing hierarchical coding to cope with the cell loss problem. The proposed hierarchical coding extracts the low‐frequency components of the image by iterating the hierarchical processing based on the spatial filter. Then it is possible to suppress the effect of the quality deterioration caused by cell loss, as long as the low‐frequency components are retained. First, the configuration of the proposed system is discussed. Simulations are executed to observe the effect of the cell loss and the coding characteristics, demonstrating the effectiveness of the proposed method in coping with the cell loss. Then discussions are made on the optimal number of hierarchical processings to cope with the cell loss and the protection level for hierarchical information. Furthermore, it is shown that a new type of video communication can be realized by introducing hierarchical coding. Copyright © 1990 Wiley Periodicals, Inc., A Wiley Company