Abstract

Mirror and glass are ubiquitous materials in the 3D indoor living environment. However, the existing vision system always tends to neglect or misdiagnose them since they always perform the special visual feature of reflectivity or transparency, which causes severe consequences, i.e., a robot or drone may crash into a glass wall or be wrongly positioned by the reflections in mirrors, or wireless signals with high frequency may be influenced by these high-reflective materials. The exploration of segmenting mirrors and glass in static images has garnered notable research interest in recent years. However, accurately segmenting mirrors and glass within dynamic scenes remains a formidable challenge, primarily due to the lack of a high-quality dataset and effective methodologies. To accurately segment the mirror and glass regions in videos, this paper proposes key points trajectory and multi-level depth distinction to improve the segmentation quality of mirror and glass regions that are generated by any existing segmentation model. Firstly, key points trajectory is used to extract the special motion feature of reflection in the mirror and glass region. And the distinction in trajectory is used to remove wrong segmentation. Secondly, a multi-level depth map is generated for region and edge segmentation which contributes to the accuracy improvement. Further, an original dataset for video mirror and glass segmentation (MAGD) is constructed, which contains 9,960 images from 36 videos with corresponding manually annotated masks. Extensive experiments demonstrate that the proposed method consistently reduces the segmentation errors generated from various state-of-the-art models and reach the highest successful rate at 0.969, mIoU (mean Intersection over Union) at 0.852, and mPA (mean Pixel Accuracy) at 0.950, which is around 40% - 50% higher on average on an original video mirror and glass dataset.

Abstract

A 1 millisecond (1‐ms) vision system that guarantees high efficiency and timely response for tomato defect detection is essential for factory automation. Because of various defect appearances, recently many existing researches focus on CNN based defect detection, but few of them attempt to reach high processing speed to adapt to the factorial assembly line. This paper proposes a global to multi‐scale local based parallel architecture with hardwired CNN for tomato defect detection. This architecture breaks down image‐wise detection into pixel‐wise localization and block‐wise classification. The pixel‐wise localization utilizes tomato‐aware information as constraints for localization performance. The block‐wise classification uses a fully pipelined network structure to obtain the classification result for each block as the pixel stream moves through the network. The classification network has a six‐layer lightweight network structure with quantization for hardwired type implementation on FPGA. The experiment results show that the proposed architecture processes 1000 FPS images with 0.9476 ms/frame delay. And for detection performance, this architecture keeps at 80.18%, only 1.31% lower than ResNet50 based detection system.

Abstract

In the volleyball game, estimating the 3D pose of the spiker is very valuable for training and analysis, because the spiker’s technique level determines the scoring or not of a round. The development of computer vision provides the possibility for the acquisition of the 3D pose. Most conventional pose estimation works are data-dependent methods, which mainly focus on reaching a high level on the dataset with the controllable scene, but fail to get good results in the wild real volleyball competition scene because of the lack of large labelled data, abnormal pose, occlusion and overlap. To refine the inaccurate estimated pose, this paper proposes a motion-aware and data-independent method based on a calibrated multi-camera system for a real volleyball competition scene. The proposed methods consist of three key components: 1) By utilizing the relationship of multi-views, an irrelevant projection based potential joint restore approach is proposed, which refines the wrong pose of one view with the other three views projected information to reduce the influence of occlusion and overlap. 2) Instead of training with a large amount labelled data, the proposed motion-aware method utilizes the similarity of specific motion in sports to achieve construct a spike model. Based on the spike model, joint and trajectory matching is proposed for coarse refinement. 3) To finely refine, a point distribution based posterior decision network is proposed. While expanding the receptive field, the pose estimation task is decomposed into a classification decision problem, which greatly avoids the dependence on a large amount of labelled data. The experimental dataset videos with four synchronous camera views are from a real game, the Game of 2014 Japan Inter High School of Men Volleyball. The experiment result achieves 76.25%, 81.89%, and 86.13% success rate at the 30mm, 50mm, and 70mm error range, respectively. Since the proposed refinement framework is based on a real volleyball competition, it is expected to be applied in the volleyball analysis.

Abstract

Receive action pose analysis is very meaningful in volleyball games for training and strategy. Because receive action is lack high-quality labeled data sets and with problems like occlusion, body overlap, and abnormal pose. Conventional work fails to obtain high accurate pose results. This paper proposes visible joint refinement and receive action template matching for volleyball receive action analysis. Firstly, the visible joint is using pixel color feature and the potential constraint features between space and joints to classify the visible joint and refine the error visible joint. Secondly, the template is to realize the pose segment refinement at 3D level by matching. It is based on a multi-view system for a real volleyball competition scene. The dataset video is from the Game of 2014 Japan Inter High School of Men Volleyball. The experiment result achieves 95.33 %, 96.92 %, and 98.43 % success rate at the 30 mm, 50 mm, and 70 mm error ranges.

Abstract

Mirror and glass are ubiquitous materials that heavily influence the transmission of indoor 5G signals with extremely high frequency. The existing vision system always tends to neglect them or misdiagnose them, which is unsuitable for 5G signal analysis in the 3D indoor environment. This paper proposes key points trajectory distinction and predicted-real frames distinction to detect mirror and glass regions in video sequences. Firstly, key points trajectory is used to extract the special motion information of reflection in mirror and glass region. Secondly, predicted and real frames distinction is used to remove the wrong detection region at the pixel level. Extensive experiments demonstrate that the proposed method achieves 40 - 50 % accuracy higher than another related state-of-the-art method for mirror and glass detection in the general 3D living environment.

Abstract

Data Volley is one of the most widely used sports analysis software for professional volleyball statistics analysis. To develop the automatic data volley system, the vision-based game data acquisition is a key technology, which includes the 3D multiple objects tracking, event detection and quality evaluation. This paper combines temporal and spatial features of the game information to achieve the game data acquisition. First, the time-vary fission filter is proposed to generate the prior state distribution for tracker initialization. By using the temporal continuity of image features, the variance of team state distribution can be approximated so that the initial state of each player can be filtered out. Second, the team formation mapping with sequential motion feature is proposed to deal with the detection of event type, which represents the players’ distribution from the spatial concept and the temporal relationship. At last, to estimate the quality, the relative spatial filters are proposed by extracting and describing additional features of the subsequent condition in different situations. Experiments are conducted on game videos from the Semifinal and Final Game of 2014 Japan Inter High School Games of Mens Volleyball in Tokyo Metropolitan Gymnasium. The results show 94.1% rounds are successfully initialized, the event type detection result achieves the average accuracy of 98.72%, and the success rate of the events’ quality evaluation achieves 97.27% on average.

Abstract

Competitive figure skaters perform successful jumps with critical parameters, which are valuable for jump analysis in athlete training. Driven by recent computer vision applications, recovering 3D pose of figure skater to obtain the meaningful variables has become increasingly important. However, conventional works have suffered from getting 3D information based on the corresponding 2D information directly or leaving the specificity of sports out of consideration. Issues such as self-occlusion, abnormal pose, limitation of venue and so on will result in poor results. Motivated by these problems, this paper proposes a multi-task architecture based on a calibrated multi-camera system to facilitate jointly 3D jump pose of figure skater. The proposed methods consist of three key components: Likelihood distribution and temporal smoothness- based discrete probability points selection filter out the most valuable 2D information; Multi-perspective and combinations unification-based large-scale venue 3D reconstruction is proposed to deal with the multi-camera; multi-constraint-based human skeleton estimation decides the final 3D coordinate from the candidates. This work is proved can be applied to 3D animated display and motion capture of the figure skating competition. The success rate of the independent joint is: 93.38% of 70 mm error range, 92.57% of 50 mm error range and 91.55% of 30 mm error range.

<p>Significant challenges in ball tracking of sports analysis by computer vision technology are: 1) accuracy of estimated 3D ball trajectory under difficult conditions; 2) external forces added by players lead to irregular motions of the ball; 3) unpredictable situations in the real game, i.e. the ball occluded by players and other objects, complex background and changing lighting condition. With the goal of multi-view 3D ball tracking, this paper proposes an abrupt motion adaptive system model, an anti-occlusion observation model, and a spatial density-based automatic recovery based on particle filter. The system model combines two different system noises that cover the motion of the ball both in general situation and situation subject to abrupt motion caused by external force. Combination ratio of these two noises and number of particles are adaptive to the estimated motion by weight distribution of particles. The anti-occlusion observation model evaluates image feature of each camera and eliminates influence of the camera with less confidence. The spatial density, which is calculated based on 3D ball candidates filtered out by spatial homographic relationship between cameras, is proposed for generating new set of particles to recover the tracking when tracking failure is detected. Experimental results based on HDTV video sequences (2014 Inter High School Men's Volleyball Games, Japan), which were captured by four cameras located at each corner of the court, show that the success rate achieved by the proposals of 3D ball tracking is 99.42%.</p>

<p>The ball state tracking and detection technology plays a significant role in volleyball game analysis, whose performance is limited due to the challenges include: 1) the inaccurate ball trajectory; 2) multiple numbers of the ball event category; 3) the large intra-class difference of one event. With the goal of broadcasting supporting for volleyball games which requires a real time system, this paper proposes a ball state based parallel ball tracking and event detection method based on a sequential estimation method such as particle filter. This method employs a parallel process of the 3D ball tracking and the event detection so that it is friendly for real time system implementation. The 3D ball tracking process uses the same models with the past work [8]. For event detection process, a ball event change estimation based multiple system model, a past trajectory referred hit point likelihood and a court-line distance feature based event type detection are proposed. First, the multiple system model transits the ball event state, which consists the event starting time and the event type, through three models dealing with different ball motion situations in the volleyball game, such as the motion keeping and changing. The mixture of these models is decided by estimation of the ball event change estimation. Secondly, the past trajectory referred hit point likelihood avoids the processing time delay between the ball tracking and the event detection process by evaluating the probability of the ball being hit at certain time without using future ball trajectories. Third, the feature of the distance between the ball and the specific court line are extracted to detect the ball event type. Experimental results based on multi-view HDTV video sequences (2014 Inter High School Men's Volleyball Games, Japan), which contains 606 events in total, show that the detection rate reaches 88.61% while the success rate of 3D ball tracking keeps more than 99%.</p>

<p>Establishing local visual correspondences between images taken under different conditions is an important and challenging task in computer vision. A common solution for this task is detecting keypoints in images and then matching the keypoints with a feature descriptor. This paper proposes a robust and low-dimensional local feature descriptor named Adaptively Integrated Gradient and Intensity Feature (AIGIF). The proposed AIGIF descriptor partitions the support region surrounding each keypoint into sub-regions, and classifies the sub-regions into two categories: edge-dominated ones and smoothness-dominated ones. For edge-dominated sub-regions, gradient magnitude and orientation features are extracted; for smoothness-dominated sub-regions, intensity feature is extracted. The gradient and intensity features are integrated to generate the descriptor. Experiments on image matching were conducted to evaluate performances of the proposed AIGIF. Compared with SIFT, the proposed AIGIF achieves 75% reduction of feature dimension (from 128 bytes to 32 bytes); compared with SURF, the proposed AIGIF achieves 87.5% reduction of feature dimension (from 256 bytes to 32 bytes); compared with the state-of-the-art ORB descriptor which has the same feature dimension with AIGIF, AIGIF achieves higher accuracy and robustness. In summary, the AIGIF combines the advantages of gradient feature and intensity feature, and achieves relatively high accuracy and robustness with low feature dimension.</p>

<p>High-dynamic-range imaging (HDRI) technologies aim to extend the dynamic range of luminance against the limitation of camera sensors. Irradiance information of a scene can be reconstructed by fusing multiple low-dynamic-range (LDR) images with different exposures. The key issue is removing ghost artifacts caused by motion of moving objects and handheld cameras. This paper proposes a robust ghost-free HDRI algorithm by visual salience based bilateral motion detection and stack extension based exposure fusion. For ghost areas detection, visual salience is introduced to measure the differences between multiple images; bilateral motion detection is employed to improve the accuracy of labeling motion areas. For exposure fusion, the proposed algorithm reduces the discontinuity of brightness by stack extension and rejects the information of ghost areas to avoid artifacts via fusion masks. Experiment results show that the proposed algorithm can remove ghost artifacts accurately for both static and handheld cameras, remain robust to scenes with complex motion and keep low complexity over recent advances including rank minimization based method and patch based method by 63.6% and 20.4% time savings averagely.</p>

For creating new tactics of sports like volleyball, the analysis of player motion in real games becomes more and more important. However, since motion data needed for the analysis is captured by human observation currently, an automatic capturing system from video camera is highly expected to gather many useful data easily. This paper proposes an action detection algorithm of volleyball players using motion features based on clustering and aggregation of body trajectories. Since the body trajectories of arms and legs are similar, the clustering utilizes shape, location and density of their trajectories. Furthermore, the clustered feature values are aggregated by means of their mean and variance. Experimental results by using the motion detection system based on the proposed algorithm show that it averagely attains 0.9539 AUC of the ROC curve for the detection of four basic motions (block, receive, spike and toss) from the volleyball game video captured by high-definition cameras. This is 0.014775 higher than conventional methods.

High efficiency video coding (HEVC) is a video compression standard that outperforms the predecessor H.264/AVC by doubling the compression efficiency. To enhance the coding accuracy, HEVC adopts sample adaptive offset (SAO), which reduces the distortion of reconstructed pixels using classification based non-linear filtering. In the traditional coding tree unit (CTU) grain based VLSI encoder implementation, during the pixel classification stage, SAO cannot use the raw samples in the boundary of the current CTU because these pixels have not been processed by deblocking filter (DF). This paper proposes a hardware-oriented category determination algorithm based on estimating the deblocking strengths on CTU boundaries and selectively adopting the promising samples in these areas during SAO classification. Compared with HEVC test mode (HM11.0), experimental results indicate that the proposed method achieves an average 0.13%, 0.14%, and 0.12% BD-bitrate reduction (equivalent to 0.0055dB, 0.0058dB, and 0.0097dB increases in PSNR) in CTU sizes of 64 × 64, 32 × 32, and 16 × 16, respectively.

<p>Recently, cloud systems have started to be utilized for services which analyze user's data in the field of computer vision. In these services, keypoints are extracted from images or videos, and the data is identified by machine learning with a large database in the cloud. To reduce the number of keypoints which are sent to the cloud, Keypoints of Interest (KOI) extraction has been proposed. However, since its computational complexity is large, hardware implementation is required for real-time processing. Moreover, the hardware resource must be low because it is embedded in devices of users. This paper proposes a hardware-friendly KOI algorithm with low amount of computations and its real-time hardware implementation based on dual threshold keypoint detection by gradient histogram and parallelization of connectivity of adjacent keypoint-utilizing register counters. The algorithm utilizes dual-histogram based detection and keypoint-matching based calculation of motion information and dense-clustering based keypoint smoothing. The hardware architecture is composed of a detection module utilizing descriptor, and grid-region-parallelization based density clustering. Finally, the evaluation results of hardware implementation show that the implemented hardware achieves Full-HD (1920x1080)-60 fps spatio-temporal keypoint extraction. Further, it is 47 times faster than low complexity keypoint extraction on software and 12 times faster than spatio-temporal keypoint extraction on software, and the hardware resources are almost the same as SIFT hardware implementation, maintaining accuracy.</p>

Scalable Video Coding (SVC) is an extension of H.264/AVC, aiming to provide the ability to adapt to heterogeneous networks or requirements. It offers great flexibility for bitstream adaptation in multi-point applications such as videoconferencing. However, transcoding between SVC and AVC is necessary due to the existence of legacy AVC-based systems. The straightforward re-encoding method requires great computational cost, and delay-sensitive applications like videoconferencing require much faster transcoding scheme. This paper proposes a 3-stage fast SVC-to-AVC transcoder with medium-grain quality scalability (MGS) for videoconferencing applications. Hierarchical-P structured SVC bitstream is transcoded into IPPP structured AVC bitstream with multiple reference frames. In the first stage, mode decision is accelerated by proposed SVC-to-AVC mode mapping scheme. In the second stage, INTER motion estimation is accelerated by an optimized motion vector (MV) conjunction method to predict the MV with a reduced search range. In the last stage, hadamard-based all zero block (AZB) detection is utilized for early termination. Simulation results show that proposed transcoder achieves very similar coding efficiency to the optimal result, but with averagely 89.6% computational time saving.

Scale-Invariant Feature Transform (SIFT) has lately attracted attention in computer vision as a robust keypoint detection algorithm which is invariant for scale, rotation and illumination changes. However, its computational complexity is too high to apply in practical real-time applications. This paper proposes a low complexity keypoint extraction algorithm based on SIFT descriptor and utilization of the database, and its real-time hardware implementation for Full-HD resolution video. The proposed algorithm computes SIFT descriptor on the keypoint obtained by corner detection and selects a scale from the database. It is possible to parallelize the keypoint detection and descriptor computation modules in the hardware. These modules do not depend on each other in the proposed algorithm in contrast with SIFT that computes a scale. The processing time of descriptor computation in this hardware is independent of the number of keypoints because its descriptor generation is pipelining structure of pixel. Evaluation results show that the proposed algorithm on software is 12 times faster than SIFT. Moreover, the proposed hardware on FPGA is 427 times faster than SIFT and 61 times faster than the proposed algorithm on software. The proposed hardware performs keypoint extraction and matching at 60fps for Full-HD video.

H.264 video codec system requires big capacity and high bandwidth of Frame Store (FS) for buffering reference frames. The up-to-date three dimensional (3D) stacked Phase change Random Access Memory (PRAM) is the promising approach for on-chip caching the reference signals, as 3D stacking offers high memory bandwidth, while PRAM possesses the advantages in terms of high density and low leakage power. However, the write endurance problem, that is a PRAM cell can only tolerant limited number of write operations, becomes the main barrier in practical applications. This paper studies the wear reduction techniques of PRAM based FS in H.264 codec system. On the basis of rate-distortion theory, the content oriented selective writing mechanisms are proposed to reduce bit updates in the reference frame buffers. With the proposed control parameter a, our methods make the quantitative trade off between the quality degradation and the PRAM lifetime prolongation. Specifically, taking a in the range of [0.2,2], experimental results demonstrate that, our methods averagely save 29.9-35.5% bit-wise write operations and reduce 52-57% power, at the cost of 12.95-20.57% BDBR bit-rate increase accordingly.

Scalable Video Coding (SVC) is an extension of H.264/AVC, aiming to provide the ability to adapt to heterogeneous networks or requirements. It offers great flexibility for bitstream adaptation in multi-point applications such as videoconferencing. However, transcoding between SVC and AVC is necessary due to the existence of legacy AVC-based systems. The straightforward re-encoding method requires great computational cost, and delay-sensitive applications like videoconferencing require much faster transcoding scheme. This paper proposes an ultra-low-delay SVC-to-AVC MGS (Medium-Grain quality Scalability) transcoder for videoconferencing applications. Transcoding is performed in pure frequency domain with partial decoding/encoding in order to achieve significant speed-up. Three fast transcoding methods in frequency domain are proposed for macroblocks with different coding modes in non-KEY pictures. KEY pictures are transcoded by reusing the base layer motion data, and error propagation is constrained between KEY pictures. Simulation results show that proposed transcoder achieves averagely 38.5 times speed-up compared with the re-encoding method, while introducing merely 0.71dB BDPSNR coding quality loss for videoconferencing sequences as compared with the re-encoding algorithm.

As an extension of H.264/AVC, Scalable Video Coding (SVC) provides the ability to adapt to heterogeneous networks and user-end requirements, which offers great scalability in multi-point applications such as videoconferencing. However, transcoding between SVC and AVC becomes necessary due to the existence of legacy AVC-based systems. The straightforward full re-encoding method requires great computational cost, and the fast SVC-to-AVC spatial transcoding techniques have not been thoroughly investigated yet. This paper proposes a low-complexity hybrid-domain SVC-to-AVC spatial transcoder with drift compensation, which provides even better coding efficiency than the full re-encoding method. The macroblocks (MBs) of input SVC bitstream are divided into two types, and each type is suitable for pixel- or transform-domain processing respectively. In the pixel-domain transcoding, a fast re-encoding method is proposed based on mode mapping and motion vector (MV) refinement. In the transform-domain transcoding, the quantized transform coefficients together with other motion data are reused directly to avoid re-quantization loss. The drift problem caused by proposed transcoder is solved by compensation techniques for I frame and P frame respectively. Simulation results show that proposed transcoder achieves averagely 96.4% time reduction compared with the full re-encoding method, and outperforms the reference methods in coding efficiency.

Scalable Video Coding (SVC) was standardized as an extension of H.264/AVC with the intention to provide flexible adaptation to heterogeneous networks and different end-user requirements, which provides great scalability in multi-point applications such as video conferencing. However, due to the existence of H.264/AVC-based systems, transcoding between AVC and SVC becomes necessary. Most existing works focus on temporal transcoding, quality transcoding or SVC-to-AVC spatial transcoding while the straightforward re-encoding method requires high computational cost. This paper proposes a low-complexity AVC-to-SVC spatial transcoder based on coarse-level mode mapping for video conferencing scenes. First, to omit unnecessary motion estimations (ME) for layers with reduced resolution, an ME skipping scheme based on AVC mode distribution is proposed with an adaptive search range. Then a probability-profile based scheme is proposed for further mode skipping. After that 3 coarse-level mode-mapping methods are presented for fast mode decision and the adaptive usage of the 3 methods is discussed. Finally, motion vector (MV) refinement is introduced for further lower-layer time reduction. As for the top layer, direct encapsulation is proposed to preserve better quality and another scheme involving inter-layer predictions is also provided for bandwidth-crucial applications. Simulation results show that proposed transcoder achieves up to 92.6% time reduction without significant coding efficiency loss compared to re-encoding method.

As an extension of AVC, SVC provides the ability to adapt to heterogeneous environments. However, transcoding between SVC and AVC becomes necessary due to the existence of legacy AVC-based systems. This paper proposes a low-complexity SVC-to-AVC CGS transcoder in the pixel domain, which achieves approximately the same coding efficiency as the MI reencoding method. The output AVC bitstream modes are deduced by the proposed mode-mapping strategy. Different from previous works which simply use full search for SVC modes, the SVC modes are also examined and refined to generate suitable AVC modes in the proposed method. Simulation results show that proposed transcoder achieves 10 times speed-up with only 2% BDBR increase and 0.08 dB BDPSNR loss, compared with the full re-encoding method.

Object tracking is a key process for various image recognition applications, and many algorithms have been proposed in this field. Especially, particle filter has possibility for tracking objects steadily thanks to prediction using many particles. However, other objects that are a similar color or shape with a tracking object hijack a tracking region if there were such objects nearby the tracking object. It is a critical problem. This paper proposes a recovery method based particle filter by focusing a feature regions attached to an object. This proposal tracks both a feature region and an object including the region at once. This proposal utilizes a recovery method that pulls a tracking region back to an appropriate position using the prior frame's distance and angle between the two tracking regions when the tracking region is hijacked by other objects. Some video sequences including complex environment have been tested for evaluating this proposal. The experimental results show that this proposal can track a specified person in the sequences, while conventional method cannot track the person. This result represents that recovery method of proposal effectively works when other objects hijack the tracking region.

Hand gesture interfaces are more intuitive and convenient than traditional interfaces. They are the most important parts in the relationship between users and devices. Hand tracking for hand gesture interfaces is an active area of research in image processing. However, previous works have limits such as requiring the use of multiple camera or sensor, working only with single color background, etc. This paper proposes a real-time both hands tracking algorithm based on "CAMshift (Continuous Adaptive Mean Shift Algorithm)" using only a single camera in multi-color backgrounds. In order to track hands robustly, the proposed algorithm uses " motion mask" to combine color and movement probability distributions and " probability reduction" for multi-hand tracking in non-limiting environments. Experimental results demonstrate that this algorithm can precisely track both hands of an operator in multi-color backgrounds and process the VGA size input sequences from a web camera in real time (about 25 fps).

Scale-Invariant Feature Transform (SIFT) has lately attracted attention in computer vision as a robust keypoint detection algorithm which is invariant for scale, rotation and illumination change. However, its computational complexity is too high to apply practical real-time applications. This paper proposes a low complexity keypoint extraction algorithm based on SIFT descriptor and utilization of the database, and its real-time hardware implementation for Full-HD resolution video. The proposed algorithm computes SIFT descriptor on the keypoint obtained by corner detection and selects a scale from the database. It is possible to parallelize the keypoint detection and descriptor computation modules in the hardware. These modules do not depend on each other in the proposed algorithm in contrast with SIFT that computes a scale. The processing time of descriptor computation in this hardware is independent of the number of keypoints because its descriptor generation is pipelining structure of pixel. Evaluation results show that the proposed algorithm on software is 12 times faster than SIFT. Moreover, the proposed hardware on FPGA is 427 times faster than SIFT and 61 times faster than the proposed algorithm on software. The proposed hardware performs keypoint extraction and matching at 60 fps for Full-HD video.

Fidelity range extensions of H.264/AVC adopt variable block size (VBS) transform techniques to employ 8× 8/4×4 Hadamard transforms adaptively during the fractional motion estimation. In this literature, the hardwired VBS Hadamard transform accelerator is developed with the following contributions: 1) developed a hardware reusing scheme between 8×8 and 4×4 transforms within the architecture design
2) devised the intermediate bit-truncation algorithm to reduce the hardware cost while maintaining the computational precision well
and 3) reduced the bit-width of sum of absolute transformed differences (SATD) value as compared to the primitive implementation, resulting in optimization in both power and hardware cost for the SATD generator implementation. With TSMC 0.18 μm CMOS technology, the experiments demonstrate that for each VBS Hadamard transform engine, 13.030.4% saving in hardware cost and 12.632.4% saving in power consumption are achieved, whereas the incurred coding quality loss is less than 0.2089 dB in terms of BDPSNR. From the aspect of the whole encoder implementation, and considering the parallelism in searching factional pixel candidates, the proposed strategies garner 2.03.9% overall gate count reduction. © 2011 IEEE.

Previous research illustrates that LRU replacement policy is not efficient when applications exhibit a distant re-reference interval. Recently RRIP policy is proposed to improve the performance for such kind of workloads. However, the lack of access recency information in RRIP confuses the replacement policy to make the accurate prediction. To enhance the robustness of RRIP for recency-friendly workloads, we propose an Dynamic Adaptive Insertion and Re-reference Prediction (DAI-RRP) policy which evicts data based on both re-reference prediction value and the access recency information. DAI-RRP makes adaptive adjustment on insertion position and prediction value for different access patterns, which makes the policy robust across different workloads and different phases. Simulation results show that DAI-RRP outperforms LRU and RRIP. For a single-core processor with a 1MB 16-way set last-level cache (LLC), DAI-RRP reduces CPI over LRU and Dynamic RRIP by an average of 8.1% and 2.7% respectively. Evaluations on quad-core CMP with a 4MB shared LLC show that DAI-RRP outperforms LRU and Dynamic RRIP (DRRIP) on the weighted speedup metric by an average of 8.1% and 15.7% respectively. Furthermore, compared to LRU, DAI-RRP consumes the similar hardware for 16-way cache, or even less hardware for high-associativity cache. In summary, the proposed policy is practical and can be easily integrated into existing hardware approximations of LRU.

One high profile intra predictor generation engine is proposed in this paper. Firstly, hardware level algorithm optimization for intra 8 × 8 (I8MB) mode is introduced. The original candidate pixels for generating prediction samples of I8MB are replaced with boundary pixels of intra 4 × 4 (I4MB) blocks. Based on this adoption, full data reuse between predictors of I4MB and filtered samples of I8MB can be achieved with almost no quality loss. Secondly, one lossless two-4 × 4-block based parallel predictor generation flow is proposed. The original predictor generation flow is optimized from 16 stages to 10 stages for I4MB and Intra 16 × 16 (I16MB), which saves 37.5% processing cycles. For I8MB, similar methodology with different processing order of 4 × 4 scaled blocks is introduced. Thirdly, fully utilized hardwired engines for I4MB, I16MB and I8MB are proposed in this paper. Except DC (direct current) and plane modes, full data reuse among all intra modes of high profile can be achieved. Fourthly, for DC mode, one combined predictor generation process is introduced and predictor generation of I16MB's DC mode is merged into the process of I4MB's DC mode. Moreover, by configuring proposed hardwired engines, predictor generation of I16MB's plane mode and chrominance plane mode can be accomplished with only 50% cycles of original design. Totally, when compared with original full-mode design and latest dynamic mode reused design, the proposed predictor generation engine can achieve 89.5% and 73.2% saving of processing cycles, respectively. Synthesized by TSMC 0.18µm technology under worst work conditions (1.62V, 125°C), with 380MHz and 37.2k gates, the proposed design can handle real-time high profile intra predictor generation of Super Hi-Vision 4k × 4k@60fps. The maximum work frequency of our design under worst condition is 468MHz.

Different encoding modes for variable block size are available in the H.264/AVC standard in order to offer better coding quality. However, this also introduces huge computation time due to the exhaustive check for all modes. In this paper, a fast spatial DIRECT mode decision method for profiles supporting B frame encoding (main profile, high profile, etc.) in H.264/AVC is proposed. Statistical analysis on multiple video sequences is carried out, and the strong relationship of mode selection and rate-distortion (RD) cost between the current DIRECT macroblock (MB) and the co-located MBs is observed. With the check of mode condition, predicted RD cost threshold and dynamic parameter update model, the complex mode decision process can be terminated at an early stage even for small QP cases. Simulation results demonstrate the proposed method can achieve much better performance than the original exhaustive rate-distortion optimization (RDO) based mode decision algorithm by reducing up to 56.8% of encoding time for IBPBP picture group and up to 67.8% of encoding time for IBBPBBP picture group while incurring only negligible bit increment and quality degradation.

Particle filter is well known as a robust object tracking algorithm based on prediction with many distributed particles and is widely used for many practical applications. However, since conventional methods use one state transition model, the tracking accuracy is decreased for the objects with irregular motion. This paper proposes a dual model particle filter based on two state transition models which targets for irregular moving object tracking. By using two state transition models which have different properties each of them, the proposed method makes it possible to track stably even if the object suddenly change its direction. Evaluation results with a software simulation shows that the proposed method attains high tracking accuracy for a irregular moving scene, for example bounding ball on floor or wall, compared with conventional ones. © 2011, The Institute of Image Electronics Engineers of Japan. All rights reserved.

Intuitive man-machine interface based on a gesture with a touchpad device is becoming common. In the near future, the importance of gesture recognition using input from a video camera is expected to be high in order to widen applicable information terminals and their applications. Conventional works, however, use a complex input device combining plural cameras and sensors. Moreover, since most of their algorithms need high computational complexity and is good for images with a simple background, it's difficult to apply practical systems. This paper proposes a real-time single-input object tracking algorithm which can trace both hands precisely under a complex background. It makes it possible to attain both high accuracy and low complexity by applying a technique combining frame difference and color and decision of feature point gathering into the KLT (Kanade-Lucas-Tomasi) tracker, a kind of an optical flow. Software based evaluation results using a wide variety of test sequences (e.g. complex background and object shape change) show that the proposed algorithm achieves higher tracking accuracy compared with conventional ones. Furthermore, a processing performance is 13-16 frame per second, which means both hands can be tracked in real-time. © 2011, The Institute of Image Electronics Engineers of Japan. All rights reserved.

With the increasing demand of high video quality and large image size, adaptive interpolation filter (AIF) addresses these issues and conquers the time varying effects resulting in increased coding efficiency, comparing with recent H.264 standard. However, currently most AIF algorithms are based on either frame level or macroblock (MB) level, which are not flexible enough for different video contents in a real codec system, and most of them are facing a severe time consuming problem. This paper proposes a content based coarse to fine AIF algorithm, which can adapt to video contents by adding different filters and conditions from coarse to fine. The overall algorithm has been mainly made up by 3 schemes: frequency analysis based frame level skip interpolation, motion vector modeling based region level interpolation, and edge detection based macroblock level interpolation. According to the experiments, AIF are discovered to be more effective in the high frequency frames, therefore, the condition to skip low frequency frames for generating AIF coefficients has been set. Moreover, by utilizing the motion vector information of previous frames the region level based interpolation has been designed, and Laplacian of Gaussian based macroblock level interpolation has been proposed to drive the interpolation process from coarse to fine. Six 720p and six 1080p video sequences which cover most typical video types have been tested for evaluating the proposed algorithm. The experimental results show that the proposed algorithm reduce total encoding time about 41% for 720p and 25% for 1080p sequences averagely, comparing with Key Technology Areas (KTA) Enhanced AIF algorithm, while obtains a BDPSNR gain up to 0.004 and 3.122 BDBR reduction.

Currently, 8x8 integer transformation was added to H.264/AVC for enhancing the performance of high profile, while much more complexity increased. All Zero Block (AZB) detection algorithms can help to early terminate DCT and quantization parts to reduce complexity with almost no PSNR loss. Due to the great complexity of 8x8 integer transform and quantization, however, the previous works are only concentrated on the 4×4 SAD all zero block detection. This paper proposes SAD and SATD based 8×8 AZB detection algorithms: By using the statistical analysis instead of complicated matrix derivation, through 17 QCIF sequences, 12 CIF sequences and 7 sequences of 720p test, which covers most video application, the 8×8 SAD value based all zero block threshold has been calculated. While using the SATD as the block matching criterion, the relationship between the 8x8 SAD and SATD has been derived theoretically from the energy point of view. Finally, according to this relationship and proposed SAD thresholds, the 8×8 SATD AZB thresholds have been derived. The proposed SAD value based AZB detection algorithm has saved nearly from 5% to 40% DCT and quantization calculation for QCIF, CIF and 720p sequences, while the SATD value based AZB detection algorithm achieved about from 10% to 43% calculation reduction. © 2010, The Institute of Image Electronics Engineers of Japan. All rights reserved.

One Super Hi-Vision (SHV) 4k×4k@60fps fractional motion estimation (FME) engine is proposed in our paper. Firstly, two complexity reduction schemes are proposed in the algorithm level. By analyzing the integer motion cost of sub blocks in each inter mode, the mode reduction based mode pre-filtering scheme can achieve 48% clock cycle saving compared with previous algorithm. By further check the motion cost of search points around best integer candidate, the motion cost oriented directional one-pass scheme can provide 50% clock cycle saving and 36% reduction in the number of processing units (PU). Secondly, in the hardware level, two parallel improved schemes namely 16-Pel processing and MB-parallel scheme are given out in our paper, which reduces design effort to only 145MHz for SHV FME processing. Also, quarter sub-sampling is adopted in our design and 75% hardware cost is reduced for each PU. Thirdly, one unified pixel block loading scheme is proposed. About 28.67% to 86.39% pixels are reused and the related memory access is saved. Furthermore, we also give out one parity pixel organization scheme to solve memory access conflict of MB-parallel scheme. By using TSMC 0.18µm technology in worst work conditions (1.62V, 125°C), our FME engine can achieve real-time processing for SHV 4k×4k@60fps with 412k gates hardware.

H.264/AVC encoder employs rate control to adaptively adjust quantization parameter (QP) to enable coded video to be transmitted over a constant bit-rate (CBR) channel. In this topic, bit allocation is crucial since it is directly related with actual bit generation and the coding quality. Meanwhile, the rate-distortion-optimization (RDO) based mode-decision technique also affects performance a lot for the strong relation among mode, bits, and quality. This paper presents a multi-stage rate control scheme for R-D optimized H.264/AVC encoders under CBR video transmission. To enhance the precision of the complexity estimation and bit allocation, a frequency-domain parameter named mean-absolute-transform-difference (MATD) is adopted to represent frame and macroblock (MB) residual complexity. Second, the MATD ratio is utilized to enhance the accuracy of frame layer bit prediction. Then, by considering the bit usage status of whole sequence, a measurement combining forward and backward bit analysis is proposed to adjust the Lagrange multiplier λ_MODE on frame layer to optimize the mode decision for all MBs within the current frame. On the next stage, bits are allocated on MB layer by proposed remaining complexity analysis. Computed QP is further adjusted according to predicted MB texture bits. Simulation results show the PSNR improvement is up to 1.13dB by using our algorithm, and the stress of output buffer control is also largely released compared with the recommended rate control in H.264/AVC reference software JM13.2.

Surveillance camera systems play an important role for creating safe and secure society. Especially, real-time motion detection is a key to detect abnormal scenes. So, we picked up KLT (Kanade-Lucas-Tomasi) tracker and tried to implement a system. However, there are still many problems in accuracy and system cost. This paper proposes a score control by weighted mask and an adaptive feature point interval algorithms to increase accuracy of object detection. Moreover, to implement these algorithms onto a low cost FPGA, hardware architectures, such as weighted value generation circuit, insert position calculation circuit and feature point data update circuit, are proposed. Evaluation results shows that the proposed algorithm can detect motion vectors with high accuracy for various surveillance scenes. Moreover, hardware implementation results show that the proposed architecture attains real-time processing with around 20% FPGA resources. © 2010, The Institute of Image Electronics Engineers of Japan. All rights reserved.

Object tracking is one of the most important applications in the field of computer vision. One of the common problems in object tracking is object occlusions. Especially in the presence of long-term full occlusion, or called long-lived full occlusion, during which the target remains invisible for tens of frames, the tracking is more difficult. This paper proposes an occlusion handling scheme based on particle filter. Compared with the conventional particle filter which usually utilizes color as tracking cue, multiple likelihood models: HSV color and gradient orientation likelihoods, are employed in the observation model during occlusion. The incorporation of these two features makes the target distinguishable even if it is occluded by a similar colored object in the background. Also, multiple state noises are introduced to ensure the redetection of the target at the end of full occlusion as well as keeping tracking accuracy under occlusion. Experimental results under different occlusion conditions show that the proposed particle filter achieves robust and accurate performance compared with the particle filter with appearance adaptive models and the color particle filter, even in the condition of long-lived full occlusion. © 2010, The Institute of Image Electronics Engineers of Japan. All rights reserved.

Variable block size motion estimation developed by the latest video coding standard H.264/AVC is the efficient approach to reduce the temporal redundancies. The intensive computational complexity coming from the variable block size technique makes the hardwired accelerator essential, for real-time applications. Propagate partial sums of absolute differences (Propagate Partial SAD) and SAD Tree hardwired engines outperform other counterparts, especially considering the impact of supporting variable block size technique. In this paper, the authors apply the architecture-level and the circuit-level approaches to improve the maximum operating frequency and reduce the hardware overhead of Propagate Partial SAD and SAD Tree, while other metrics, in terms of latency, memory bandwidth and hardware utilization, of the original architectures are maintained. Experiments demonstrate that by using the proposed approaches, at 110.8MHz operating frequency, compared with the original architectures, 14.7% and 18.0% gate count can be saved for Propagate Partial SAD and SAD Tree, respectively. With TSMC 0.18µm 1P6M CMOS technology, the proposed Propagate Partial SAD architecture achieves 231.6MHz operating frequency at a cost of 84.1k gates. Correspondingly, the maximum work frequency of the optimized SAD Tree architecture is improved to 204.8MHz, which is almost two times of the original one, while its hardware overhead is merely 88.5k-gate.

The H.264/AVC coding standard processes variable block size motion-compensated prediction with multiple reference frames to achieve a pronounced improvement in compression efficiency. Accordingly, the computation of motion estimation increases in proportion to the product of the number of reference frame and the number of intermode. The mathematical analysis in this paper illustrates that the motion-compensated prediction errors are mainly determined by the detailed textures in the source image. The image block being rich in textures contains numerous high-frequency signals, which make variable block size and multiple reference frame techniques essential. On the basis of rate-distortion theory, in this paper, the spatial homogeneity of an image block is made as a relative concept with respect to the current quantization step. For the homogenous block, its futile reference frames and intermodes can be eliminated efficiently. It is further revealed that the sum of absolute differences value of an image block is mainly determined by the sum of its edge gradient amplitude and the current quantization step. Consequently, the image content-based early termination algorithm is proposed, and it outperforms the original method adopted by JVT reference software. Moreover, the dynamic search range algorithm based on the edge gradient amplitude of source image block is analyzed. One eminent advantage of the proposed edge-based algorithms is their efficiency to the macroblock-pipelining architecture, and another desirable feature is their orthogonality to fast block-matching algorithms. Experimental results show that when these algorithms are integrated with hybrid unsymmetrical-cross multi-hexagongrid search, an averaged 31.4-60.0% motion estimation time can be saved, whereas the averaging BDPSNR loss is 0.0497 dB for all tested sequences.

H.264 is the latest HDTV video compression standard, which provides a significant improvement in coding efficiency at the cost of huge computation complexity. After transform and quantization, if all the coefficients of the block's residue data are zero, this block is called all-zero block (AZB). Provided that an AZB can be detected early, the process of transform and quantization on an AZB can be skipped, which reduces significant redundant Computations. In this paper, a theoretical analysis is performed for the sufficient condition for AZB detection. As a result, a partial sum of absolute difference (SAD) based 4 x 4 AZB detection algorithm is derived. And then, a hardware-oriented AZB detection algorithm is proposed by modifying the order of SAD calculation. Furthermore, a quantization parameter (QP) oriented 8 x 8 AZB detection algorithm is proposed according to the AZB's statistical analysis. Experimental results show that the proposed algorithm outperforms the previous methods in all cases and achieves major improvement of computation reduction in the range from 6.7% to 42.3% for 4 x 4 blocks, from 0.24% to 79.48% for 8 x 8 blocks. The computation reduction increases as QP increases.

In this paper, we propose a partially-parallel irregular LDPC decoder for IEEE 802.11n standard targeting high throughput applications. The proposed decoder has several merits: (i) The decoder is designed based on a novel deltavalue based message passing algorithm which facilitates the decoding throughput by redundant computation removal. (ii) Techniques such as binary sorting, parallel column operation, high performance pipelining are used to further speed up the message-passing procedure. The synthesis result in TSMC 0.18 CMOS technology demonstrates that for (648,324) irregular LDPC code, our decoder can achieve 8 times increasement in throughput, reaching 418 Mbps at the frequency of 200 MHz.

A low-bandwidth Integer Motion Estimation (IME) module is proposed for MPEG-2 to H.264 transcoding. Based on bandwidth reduction method proposed in Ref. 1), a ping-pang memory control scheme combined with Partial Sum of Absolute Differences (SAD) Variable Block Size Motion Estimation (VBSME) architecture are realized. Experiment results show bandwidth of the proposed architecture is 70.6% of H.264 regular IME (Level C+ scheme, 2 Macro Block (MB) stitched vertically), while the on-chip memory size is 11.7% of that. © 2009 Information Processing Society of Japan.

A macroblock (MB) feature based adaptive propagate partial SAD architecture is proposed in this paper. Firstly, by using edge detection operator, the homogeneous MB is detected before motion estimation and three hardware friendly subsampling patterns are adaptively selected for MB with different homogeneity. The proposed architecture uses four different processing elements to realize adaptive subsampling scheme. Secondly, in order to achieve data reuse and power reduction in memory part, the reference pixels in search window are reorganization into two memory groups, which output pixel data interactively for adaptive subsampling. Moreover, a compressor tree based circuit level optimization is included in our design to reduce hardware cost. Synthesized with TSMC 0.18 um technology, averagely 10 k gates hardware can be reduced for the whole IME engine based on our optimization. With 481 k gates at 110.5 MHz, an 720-p, 30-fps HDTV integer motion estimation engine is designed. Compared with previous work, our design can achieve 39.8% reduction in power consumption with only 3.44% increase in hardware. © 2009 Information Processing Society of Japan.

One VLSI friendly fast motion estimation (ME) algorithm is proposed in this paper. Firstly, theoretical analysis shows that image rich of sharp edges and texture is regarded as high frequency abundant image and macroblocks (MBs) in such image will express large pixel difference. In our paper, we apply adaptive subsampling method during ME process based on pixel difference analysis, so the computation complexity of full pixel pattern can be reduced. Secondly, statistic analysis shows that for MBs with static feature, the ratio of selecting previous reference frame as best one is very high and multiple reference frame technique is not required for these MBs. Based on this analysis, we give out a block overlapping method to pick out static MBs and apply MRF elimination process. Thirdly, since many redundant search positions exist in MB with small motion trend and large search range is only contributive to MB with big motion, we extract motion feature after ME on first reference frame and use it to adjust search range for rest ME process. So, the computation complexity of redundant search positions is eliminated. Experimental results show that, compared with hardware friendly full search algorithm, our proposed algorithm can reduce 71.09% to 95.26% ME time with negligible video quality degradation. Moreover, our fast algorithm can be combined with existing fast ME algorithms like UMHexagon method for further reduction in complexity and it is friendly to hardware implementation.

This paper describes a new real-time approach for detecting motions in the video streams taken from stationary cameras. This method combines a temporal recording scheme with the adaptive background model subtraction scheme. To save the computation brought from conventional Gaussian Mixture Models (GMM) and achieve real-time processing, an adaptively adjusted mechanism is proposed. On the other hand, illumination changes, shadow influence, and ghost in scene, these three important problems which result in low segmentation quality are settled down by utilizing proposed features and temporal information from video streams. The experimental results validate the improvement of detection accuracy. Meanwhile, the execution time for each component per frame is calculated and compared with that of conventional Gaussian Mixture Models. © 2009 IEEE.

This paper presents a reconfigurable SAD Tree (RSADT) architecture based on adaptive sub-sampling algorithm for HDTV application. Firstly, to obtain the the feature of HDTV picture, pixel difference analysis is applied on each macroblock (MB). Three hardware friendly sub-sampling patterns are selected adaptively to release complexity of homogeneous MB and keep video quality for texture MB. Secondly, since two pipeline stages are inserted, the whole clock speed of RSADT structure is enhanced. Thirdly, to solve data reuse and hardware utilization problem of adaptive algorithm, the RSADT structure adopts pixel data organization in both memory and architecture level, which leads to full data reuse and hardware utilization. Additionally, a cross reuse structure is proposed to efficiently generate 16 pixel scaled configurable SAD (sum of absolute difference). Experimental results show that, our RSADT architecture can averagely save 61.71% processing cycles for integer motion estimation engine and accomplish twice or four times processing capability for homogeneous MBs. The maximum clock frequency of our design is 208MHz under TSMC 0.18µm technology in worst work conditions(1.62V, 125°C). Furthermore, the proposed algorithm and reconfigurable structure are favorable to power aware real-time encoding system.

One fast inter mode decision algorithm is proposed in this paper. The whole algorithm is divided into two stages. In the pre-stage, by exploiting spatial and temporal information of encoded macrobocks (MBs), a skip mode early detection scheme is proposed. The homogeneity of current MB is also analyzed to filter out small inter modes in this stage. Secondly, during the block matching stage, a motion feature based inter mode decision scheme is introduced by analyzing the motion vector predictor's accuracy, the block overlapping situation and the smoothness of SAD (sum of absolute difference) value. Moreover, the rate distortion cost is checked in an early stage and we set some constraints to speed up the whole decision flow. Experiments show that our algorithm can achieve a speed up factor of up to 53.4% for sequences with different motion type. The overall bit increment and quality degradation is negligible compared with existing works.

Intra coding in H.264/AVC has significantly enhanced video compression efficiency. However, computation complexity increases by the rate-distortion (RD) based mode decision. This paper proposes a novel fast mode decision algorithm in H.264/AVC intra prediction and its VLSI architecture. A novel edge-detection pattern is proposed and both edge-detection technique and spatial mode prediction technique are combined together to reduce the number of intra 4 x 4 candidate modes from 9 to an average of 2.50. VLSI architecture of intra mode decision module is designed with TSMC 0.18 mu m CMOS technology. The maximum frequency of 285 MHz is achieved and 13.1k NAND gates are required. High frequency, efficient processing cycle reduction and small area make this design to be an excellent accelerator for HDTV 1080p@30 fps real time encoder.

In this paper, we propose a partially-parallel irregular LDPC decoder based on IEEE 802.1 In standard targeting high throughput and small area applications. The design is based on a novel sum-delta message passing algorithm characterized as follows: (i) Decoding throughput is greatly improved by utilizing the difference value between the updated and the original value to remove redundant computations. (H) Registers and memory are optimized to store only the frequently used messages to decrease the hardware cost. (iii) Techniques such as binary sorting, parallel column operation, high performance pipelining are used to further speed lip the message passing procedure. The synthesis result in TSMC 0.18 CMOS technology demonstrates that for (648,324) irregular LDPC code, our decoder achieves 7.5X improvement in throughput, which reaches 402 Mbps at the frequency of 200 MHz, with 11% area reduction. The synthesis result also demonstrates the competitiveness to the fully-parallel regular LDPC decoders in terms of the tradeoff between throughput, area and power.

In H.264/AVC standard, many new techniques Such as variable block size (VBS) and multiple reference frame (MRF) are used in motion estimation (ME) part to achieve superior coding performance, However, the use of new techniques will also cause great burden on computation complexity, which leads to problems in low power hardware implementation. Many software based fast ME algorithms are proposed to reduce complexity. For real-time hardwired encoder, the huge throughput of fractional motion estimation (FME) and integer motion estimation (IME) makes pipeline stage a must. In this case. IME is arranged in a single stage. which deteriorates the efficiency of many software based algorithms. Based on the hardware data flow, this paper provides a complexity reduction algorithm which speeds up ME procedure through three schemes. Firstly, the proposed algorithm executes similarity analysis to detect big mode MB and apply early termination in IME stage. Secondly, for normal MB. motion feature is extracted after IME of each frame and a 6-ring based search range adjustment scheme is introduced to remove redundant search positions. Thirdly, for MBs which have large motion feature, the pixel difference is very small due to the blur effect on video sensor. So, we use sub-sampling technique to reduce computation complexity for such MBs. Experimental results show that. compared with hardware friendly full search algorithm, the proposed fast ME algorithm can reduce 52.63% to 83.21% ME time with negligible video quality degradation. Furthermore, since the proposed algorithm works in a hardware friendly way, it can be embedded into 3-stage real-time hardwired video encoder to achieve low power design.

With the ubiquitous application of Internet and wireless networks, H.264 video communication becomes more and more common. However, due to the high-efficiently predictive coding and the variable length entropy coding, it is more sensitive to transmission errors. The current error concealment (EC) scheme, which utilizes the spatial and temporal correlations to conceal the corrupted region, produces unsatisfied boundary artifacts. In this paper, first we propose variable block size error concealment (VBSEC) scheme inspired by variable block size motion estimation (VBSME) in H.264. This scheme provides four EC modes and four sub-block partitions. The whole corrupted macro-block (MB) will be divided into variable block size adaptively according to the actual motion. More precise motion vectors (MV) will be predicted for each sub-block. Then MV refinement (MVR) scheme is proposed to refine the MV of the heterogeneous sub-block by utilizing three step search (TSS) algorithm adaptively. Both VBSEC and MVR are based on our directional spatio-temporal boundary matching algorithm (DSTBMA). By utilizing these schemes, we can reconstruct the corrupted MB in the inter frame more accurately. The experimental results show that our proposed scheme can obtain better objective and subjective EC quality, respectively compared with the boundary matching algorithm (BMA) adopted in the JM 11.0 reference software, spatio-temporal boundary matching algorithm (STBMA) and other comparable EC methods.

Transmission of compressed video over error prone channels may result in packet losses or errors, which can significantly degrade the image quality. Therefore an error concealment scheme is applied at the video receiver side to mask the damaged video. Considering there are 3 types of MBs (Macro Blocks) in natural video frame, i.e., Textural MB. Edged MB, and Smooth MB, this paper proposes an adaptive spatial error concealment which can choose 3 different methods for these 3 different MBs. For criteria of choosing appropriate method, 2 factors are taken into consideration. Firstly, standard deviation of our proposed edge statistical model is exploited. Secondly, some new features of latest video compression standard H.264/AVC. i.e.. intra prediction mode is also considered for criterion formulation. Compared with previous works. which are only based on deterministic measurement, proposed method achieves the best image recovery. Subjective and objective image quality evaluations in experiments confirmed this.

Compared with previous standards, H.264/AVC adopts variable block size motion estimation (VBSME) and multiple reference frames (MRF) to improve the video quality. Full search motion estimation algorithm (FS), which calculates every search candidate in the search window for 7 block type with multiple reference frames, consumes massive computation power. Mathematical analysis reveals that the aliasing problem of subsampling algorithm comes from high frequency signal components. Moreover, high frequency signal components are also the main issues that make MRF algorithm essential. As we know, a picture being rich of texture must contain lots of high frequency signals. So based on these mathematical investigations, two fast VBSME algorithms are proposed in this paper, namely edge block detection based subsampling method and motion vector based MRF early termination algorithm. Experiments show that strong correlation exists among the motion vectors of those blocks belonging to the same macroblock. Through exploiting this feature, a dynamically adjustment of the search ranges of integer motion estimation is proposed in this paper. Combing our proposed algorithms with UMHS almost saves 96-98% Integer Motion Estimation (IME) time compared to the exhaustive search algorithm. The induced coding quality loss is less than 0.8% bitrate increase or 0.04 dB PSNR decline on average.

The key to high performance in video coding lies on efficiently reducing the temporal redundancies. For this purpose, H.264/AVC coding standard has adopted variable block size motion estimation on multiple reference frames to improve the coding gain. However, the computational complexity of motion estimation is also increased in proportion to the product of the reference frame number and the intermode number. The mathematical analysis in this paper reveals that the prediction errors mainly depend on the image edge gradient amplitude and quantization parameter. Consequently, this paper proposes the image content based early termination algorithm, which outperforms the original method adopted by JVT reference software, especially at high and moderate bit rates. In light of rate-distortion theory, this paper also relates the homogeneity of image to the quantization parameter. For the homogenous block, its search computation for futile reference frames and intermodes can be efficiently discarded. Therefore, the computation saving performance increases with the value of quantization parameter. These content based fast algorithms were integrated with Unsymmetrical-cross Multihexagon-grid Search (UMHexagonS) algorithm to demonstrate their performance. Compared to the original UMHexagonS fast matching algorithm, 26.14-54.97% search time can be saved with an average of 0.0369 dB coding quality degradation.

In the state-of-the-art video coding standard&apos;, H.264/AVC, the encoder is allowed to search for its prediction signals among a large number of reference pictures that have been decoded and stored in the decoder to enhance its coding efficiency. Therefore, the computation complexity of the motion estimation (ME) increases linearly with the number of reference picture. Many fast multiple reference frame ME algorithms have been proposed, whose performance, however, will be considerably degraded in the hardwired encoder design due to the macroblock (MB) pipelining architecture. Considering the limitations of the traditional four-stage MB pipelining architecture, two fast multiple reference frame ME algorithms are proposed here. First, on the basis of mathematical analysis, which reveals that the efficiency of multiple reference frames will be degraded by the relative motion between the camera and the objects, for the slow-moving MB, the authors adopt the multiple reference frames but reduce their search range. On the other hand, for the fast-moving MB, the first previous reference frame is used with the full search range during the ME processing. The mutually exclusive feature between the large search range and the multiple reference frames makes the computation saving performance of the proposed algorithm insensitive to the nature of video sequence. Second, following the Hadamard transform coefficient-based all_zeros block early detection algorithm, two early termination criteria are proposed. These methods ensure the pronounced computation saving efficiency when the encoded video has strong spatial homogeneity or temporal stationarity. Experimental results show that 72.7%-93.7% computation can be saved by the proposed fast algorithms with an average of 0.0899 dB coding quality degradation. Moreover, these fast algorithms can be combined with fast block matching algorithms to further improve their speedup performance.

Video surveillance systems have a huge market, as indicated by the number of installed cameras, particularly for low-power systems. In this paper, we propose a low-power quadtree video encoder for video surveillance systems. It features a low-complexity motion estimation algorithm, an application-specific ME-MC processor, a dedicated quadtree encoder engine and a processor control-based clock-gating technique. A chip capable of encoding 30 fps VGA (640 x 480) at 80 MHz is fabricated using 0.18 mu m CMOS technology. A total of 153 K gates with 558 kbits SRAM have been integrated into a 5.0 mm x 3.5 mm die. The power consumption is 40.87 mW at 80 MHz for VGA at 30 fps and 1.97 mW at 3.3 MHz for QCIF at 15 fps.

Variable Block Size Motion Estimation (VBSME) costs a lot of computation during video coding. Search range reduction algorithm is widely used to reduce computational cost of motion estimation. Current VBSME designs are not suitable for this algorithm. This paper proposes a reconfigurable design of VBSME which can be efficiently used with search range reduction algorithm. While using proposed design, n x m reference MBs form an MB array which can be processed in parallel. n and m can be configured according to the new search range shape calculated by algorithm. In this way, the parallelism of proposed design is very flexible and can be adapted to any search range shape. The hardware resource is also fully used while performing VBSME. There are two primary reconfigurable modules in this design: PEGA (PE Group Array) and SAD comparator. By using TSMC 0.18 mu m standard cell library, the implementation results show that the hardware cost of design which uses 16 PEGs (PE Groups) is about 179 K Gates, the clock frequency is 167 MHz.

Reducing the power dissipation for LDPC code decoder is a major challenging task to apply it to the practical digital communication systems. In this paper, we propose a low power LDPC code decoder architecture based on an intermediate message-compression technique which features as follows: (i) An intermediate message compression technique enables the decoder to reduce the required memory capacity and write power dissipation. (H) A clock gated shift register based intermediate message memory architecture enables the decoder to decompress the compressed messages in a single clock cycle while reducing the read power dissipation. The combination of the above two techniques enables the decoder to reduce the power dissipation while keeping the decoding throughput. The simulation results show that the proposed architecture improves the power efficiency up to 52% and 18% compared to that of the decoder based on the overlapped schedule and the rapid convergence schedule without the proposed techniques respectively.

With the increase of key length used in public cryptographic algorithms such as RSA and ECC, the speed of Montgomery multiplication becomes a bottleneck. This paper proposes a high speed design of Montgomery multiplier. Firstly, a modified scalable high-radix Montgomery algorithm is proposed to reduce critical path. Secondly, a high-radix clock-saving dataflow is proposed to support high-radix operation and one clock cycle delay in dataflow. Finally, a hardware-reused architecture is proposed to reduce the hardware cost and a parallel radix-16 design of data path is proposed to accelerate the speed. By using HHNEC 0.25 mu m standard cell library, the implementation results show that the total cost of Montgomery multiplier is 130 KGates, the clock frequency is 180 MHz and the throughput of 1024-bit RSA encryption is 352 kbps. This design is suitable to be used in high speed RSA or ECC encryption/decryption. As a scalable design, it supports any key-length encryption/decryption up to the size of OD-chip memory.

Comparing with search pattern motion estimation (ME) algorithms, adaptive search range (ASR) algorithms are more fundamental, regular and flexible. In variable block size motion estimation (VBSME), ASR algorithms can be applied whether on a whole frame (frame level), or on an entire macroblock which includes up to forty-one blocks (macroblock level), or just on a single block (block level). In the other hand, in H.264/AVC, not the motion vectors (MVs) but the motion vector differences (MVDs) are coded and the median motion vector predictors (median-MVPs) are used to place the search centers. In this sense, it can be thought that the search windows (SWs) are centered at the positions pointed by median-MVPs, the search ranges (SRs) play the role of limiting MVDs. Thus it is reasonable for considering using MVDs to predict SRs. In this paper, one of the MB level and two of the block level, at all three MVD based SR prediction algorithms are proposed. VBSME based experiments are carried out to assess the proposed algorithms. Comparisons between the proposed three algorithms and the previously proposed one given in [8] are done in terms of encoding quality and computational complexity.

One hardware efficient and high speed architecture for variable block size motion estimation (VBSME) in H.264 is presented in this paper. By improving the pipeline structure and processing element (PE) circuits, the system latency and hardware cost is reduced, which makes this structure more hardware efficient than the original Propagate Partial SAD architecture. For small and middle frame size picture's coding, the proposed structure can save 12.1% hardware cost compared with original Propagate Partial SAD structure. In the case of HDTV, since small inter modes trivially contribute to the coding quality, we remove modes below 8 x 8 in our design. By adopting mode reduction technique, when the set number of PE array is less than 8, the proposed mode reduction based Propagate Partial SAD structure can work at faster clock speed and consume less hardware cost than widely used SAD Tree architecture. It is more robust to the high speed timing constraint when parallel processing is considered. With TSMC 0.18 mu m technology in worst work conditions (1.62 V, 125 degrees C), its peak throughput of 8-set PE array structure is 720p@30 Hz with 128 x 64 search range and 5 reference frames. 12 k gates hardware cost can be reduced by our design compared with the parallel SAD Tree architecture.

H.264/AVC is the newest video coding standard. There are many new features in it which can be easily used for video encryption. In this paper, we propose a new scheme to do video encryption for H.264/AVC video compression standard. We define Unequal Secure Encryption (USE) as an approach that applies different encryption schemes (with different security strength) to different parts of compressed video data. This USE scheme includes two parts: video data classification and unequal secure video data encryption. Firstly, we classify the video data into two partitions: Important data partition and unimportant data partition. important data partition has small size with high secure protection, while unimportant data partition has large size with low secure protection. Secondly, we use AES as a block cipher to encrypt the important data partition and use LEX as a stream cipher to encrypt the unimportant data partition. AES is the most widely used symmetric cryptography which can ensure high security. LEX is a new stream cipher which is based on AES and its computational cost is much lower than AES. In this way,our scheme can achieve both high security and low computational cost. Besides the USE scheme, we propose a low cost design of hybrid AES/LEX encryption module. Our experimental results show that the computational cost of the USE scheme is low (about 25% of naive encryption at Level 0 with VEA used). The hardware cost for hybrid AES/LEX module is 4678 Gates and the AES encryption throughput is about 50 Mbps.

Transmission of compressed video over error prone channels may result in packet losses or errors, which can significantly degrade the image quality. Therefore an error concealment scheme is applied at the video receiver side to mask the damaged video. Considering there are 3 types of MBs (Macro Blocks) in natural video frame, i.e., Textural MB, Edged MB, and Smooth MB, this paper proposes an adaptive spatial error concealment which can choose 3 different methods for these 3 different MBs. For criteria of choosing appropriate method, 2 factors are taken into consideration. Firstly, standard deviation of our proposed edge statistical model is exploited. Secondly, some new features of latest video compression standard H.264/AVC, i.e., intra prediction mode is also considered for criterion formulation. Compared with previous works, which are only based on deterministic measurement, proposed method achieves the best image recovery. Subjective and objective image quality evaluations in experiments confirmed this.

In this paper we introduce an area and power efficient fully-parallel LDPC decoder design, which keeps the BER performance while consuming less hardware resources and lower power compared with conventional decoders. For this decoder, we firstly propose two improved simplified min-sum algorithms, which enable the decoder to reduce the hardware implementation complexity and area: hardware consumption of check operation module is reduced by 40%, while achieving a negligible performance loss compared with the general min-sum algorithm. To reduce the power dissipation of the decoder, we also proposed a power-saved strategy according to which the message evolution halts as the parity-check condition is satisfied. This strategy reduces more than 50% power under food channel condition. The synthesis result in 0.18 mu m CMOS technology shows our decoder based on (648,540) irregular LDPC code of WLAN (802.11n) protocol achieves 8 10 [Mbps] throughput with 283 [mW] power consumption.

A H.264/AVC baseline-profile real-time encoder for HDTV-1080p at 30fps is implemented with the dedicated hardware engines and one 32-bit Media embedded Processor (MeP) equipped with hardware extensions. The 11.5Gbps 64Mb Svstem-in-Silicon DRAMA is embedded to alleviate the external memory bandwidth. With TSMC 0.18 mu m CMOS technology, the SoC core occupies 27.1mm(2) die area and consumes 1.41W at 200MHz in typical work conditions.

This paper presents simple and effective method to further reduce the search points in multilevel successive elimination algorithm (MSEA). Because the calculated sea values of those best matching search points are much smaller than the current minimum SAD, we can simply increase the calculated sea values to increase the elimination ratio without much affecting the coding quality. Compared with the original MSEA algorithm, the proposed strict MSEA algorithm (SMSEA) can provide average 6.52 times speedup. Compared with other lossy fast ME algorithms such as TSS and DS, the proposed SMSEA can maintain more stable image quality. In practice, the proposed technique can also be used in the fine granularity SEA (FGSEA) algorithm and the calculation process is almost the same.

In H.264/AVC standard, motion estimation can be processed on multiple reference frames (MRF) to improve the video coding performance. The computation is also increased in proportion to the reference frame number. Many software oriented fast multiple reference frames motion estimation (MRF-ME) algorithms have been proposed. However, for the VLSI real-time encoder, the heavy computation of fractional motion estimation (FME) makes the integer motion estimation (IME) and FME must be scheduled in two macro block (MB) pipeline stages, which makes many fast MRF-ME algorithms inefficient. In this paper, one edge gradient detection based algorithm is provided to reduce the computation of MRF-ME. The image being rich of texture and sharp edges contains much high frequency signal and this nature makes MRF-ME essential. Through analyzing the edges' gradient, we just perform MRF-ME on those blocks with sharp edges, so the redundant ME computation can be efficiently reduced. Experimental results show that average 26.43% computation can be saved by our approach with the similar coding quality as the reference software. This proposed algorithm is friendly to hardwired encoder implementation. Moreover, the provided fast algorithms can be combined with other fast ME algorithms to further improve the performance. © 2007 EURASIP.

This paper presents two hardware-friendly low-power oriented fast motion estimation (ME) algorithms and their VLSI implementations. The basic idea of the proposed partial distortion sorting (PDS) algorithm is to disable the search points which have larger partial distortions during the ME process, and only keep those search points with smaller ones. To further reduce the computation overhead, a simplified local PDS (LPDS) algorithm is also presented. Experiments show that the PDS and LPDS algorithms can provide almost the same image quality as full search only with 36.7% computation complexity. The proposed two algorithms can be integrated into different FSBMA architectures to save power consumption. In this paper, the 1-D inter ME architecture [12] is used as an detailed example. Under the worst working conditions (1.62 V, 125 degrees C) and 166 MHz clock frequency, the PDS algorithm can reduce 33.3% power consumption with 4.05 K gates extra hardware cost, and the LPDS can reduce 37.8% power consumption with 1.73 K gates overhead.

The coexistence of MPEG-2 and its powerful successor H.264/AVC has created a huge need for MPEG-2/H.264 video transcoding. However, a traditional transcoder where an MPEG-2 decoder is simple cascaded to an H.264 encoder requires huge computational power due to the adoption of a complicated rate-distortion based mode decision process in H.264. This paper proposes a 2-D Sobel filter based motion vector domain method and a DCT domain method to measure macroblock complexity and realize content-based H.264 candidate mode decision. A new local edge based fast INTRA prediction mode decision method is also adopted to boost the encoding efficiency. Simulation results confirm that with the proposed methods the computational burden of a traditional transcoder can be reduced by 20% similar to 30% with only a negligible bit-rate increase for a wide range of video sequences.

Traffic sign detection is a valuable part of future driver support system. In this paper, we present a novel framework to accurately detect traffic signs from a single color image by analyzing geometrical, physical and text/symbol features of traffic signs. First, we utilize an elaborate edge detection algorithm to extract edge map and accurate edge pixel gradient information. Then, we extract 2-D geometric primitives (circles, ellipses, rectangles and triangles) efficiently from image edge map. Third, the candidate traffic sign regions are selected by analyzing the intrinsic color features, which are invariant to different illumination conditions, of each region circumvented by geometric primitives. Finally, a text and symbol detection algorithm is introduced to classify true traffic signs. Experimental results demonstrated the capabilities of our algorithm to detect traffic signs with respect to different size, shape, color and illumination conditions.

Since JPEG 2000 is selected as the standard of Digital Cinema, it attracts a great deal of attention. However, JPEG 2000 coding requires huge amount of computational complexity for a high definition image with more than 200 million pixels required in Digital Cinema standard. Especially, EBCOT is a bottleneck because of its difficulty in parallel processing. This paper proposes a parallel coefficient bit modeling (CBM) algorithm which can process a bit plane, a stripe and a pass in parallel and a CBM architecture based on it. Hardware evaluation results show that a 733k Gate (TSMC 0.18 µm) circuit can process the code block (block size 32 × 32, 30 bit planes) which satisfies Digital Cinema standard in real time. © 2007, The Institute of Image Electronics Engineers of Japan. All rights reserved.

This paper proposes an interpolation algorithm and LSI architecture for a camera with a fish-eye lens. Since the fish-eye lens has around 180 degree field of view and a deep depth of field, it's very promising for a surveillance camera, a video conference system, and so on. Combination of a cubic spline interpolation and a direct interpolation from a RAW image enable to obtain high quality image. A three-stage linear interpolation scheme and a dedicate operators boost up processing performance. Evaluation results base on a subjective test show that our proposed algorithm can generate high-quality interpolation image. Furthermore, a dedicated image processing core with 290k gates can process HDTV (1920x1080, 60 fps) image by using 0.18 µm CMOS technology. © 2007, The Institute of Image Electronics Engineers of Japan. All rights reserved.

Contour detection is a fundamental step to scene analysis and interpretation. However, because contours often locate in rich texture background it is still a difficult task in realistic vision. Through multi-scale analysis, it becomes clear that edge responses of real object contours are relatively stable across scales, while those from noise or texture background are not. In this paper, a salient contour detection method is proposed based on the scale invariance of piecewise linear approximation of real object contours. Firstly, an image pyramid is efficiently constructed by repeatedly smoothing and sub-sampling the image. Secondly, the piecewise linear approximation of contours in multiple scales are extracted and then a collinear line grouping process is implemented to improve the connectivity of the contours. Thirdly, the new salient line segments are generated based on the analysis on the stability of line segments across scales. Experimental results show that the proposed method can effectively improve the connectivity and saliency of the contour detection compared with the former method. © 2007, The Institute of Image Electronics Engineers of Japan. All rights reserved.

Compared with previous video standards, the computational complexity of H.264/AVC is extraordinarily high due to newly adopted tools such as variable block size motion estimation (VBSME). In VBSME, to each macroblock (MB) there are at all 41 blocks corresponded. These 41 blocks are sized from 4 x 4 through 16 x 16. As each block of these 41 blocks accounts for one independent search window (SW), thus to each MB there exist 41 SWs corresponded. In the other hand, adaptive search range (ASR) algorithms are defined as algorithms which adaptively adjust the search ranges (SRs) and hence result in dynamically reduced SWs. While considering whether 41 SWs got changed in the same way or not, "macroblock level ASR (MB-ASR) algorithms" are defined as algorithms which adaptively change the search range all in-once and hence result in the 41 SWs got changed in the same way. "41 blocks level ASR (block - ASR) algorithms" are defined as algorithms that the changing ways of the SRs are specific to each block of the 41 blocks. Since in MB - ASR algorithms for each MB the SR is changed in same way, it is reasonable to consider that the control overhead of MB - ASR is less than that of block - ASR. In this paper, a MB-ASR algorithm is proposed and assessed from being compared with a previously proposed block - ASR algorithm given in [5]. According to experimental results, it is proved the proposed algorithm provides almost the same encoding quality and even a little better SW reducing efficiency (SW-RE) while compared with [5]. Meanwhile, as a MB - ASR one, the proposed algorithm is considered having less control overhead than block - ASR one - [5].

In H.264 standard, multiple reference frame motion estimation (MRF-ME) can help to generate small residues and improve the performance. However, MRF-ME is also a computation intensive task for video coding system. Many software oriented fast algorithms have been proposed to shorten MRF-ME process. For hardwired real-time encoder, the division of ME part into two pipeline stages degrades the efficiency of many fast algorithms. This paper gives one hardware friendly MRF-ME algorithm to reduce computation complexity in MRF-ME procedure. The proposed algorithm is based on the analysis of macroblock&apos;s (MB&apos;s) feature and restricts search range for static background MB. Through experiment results, with negligible video quality degradation, the proposed background analysis based MRF-ME algorithm can averagely reduce 41.75% ME time for sequences with static background. Moreover, the proposed algorithm is compatible to other fast algorithms and friendly to hardware implementation of H.264 real-time encoder.

In this paper, we propose a power-efficient LDPC decoder architecture based on an accelerated message-passing schedule. The proposed decoder architecture is characterized as follows: (i) Partitioning a pipelined operation not to read and write intermediate messages simultaneously enables the accelerated message-passing schedule to be implemented with single-port SRAMs. (H) FIFO-based buffering reduces the number of SRAM banks and words of the LDPC. decoder based on the accelerated message-passing schedule.. The proposed LDPC decoder keeps a single message for each non-zero bit in a parity check matrix as well as a classical schedule while achieving the accelerated message-passing schedule. Implementation results in 0.18 [mu m] CMOS technology show that the proposed decoder architecture reduces an area of the LDPC decoder by 43% and a power dissipation by 29% compared to the conventional architecture based on the accelerated message-passing schedule.

In this paper, we propose a partially-parallel LDPC decoder which achieves a high-efficiency message-passing schedule. The proposed LDPC decoder is characterized as follows: (i) The column operations follow the row operations in a pipelined architecture to ensure that the row and column operations are performed concurrently. (ii) The proposed parallel pipelined bit functional unit enables the column operation module to compute every message in each bit node which is updated by the row operations. These column operations can be performed without extending the single iterative decoding delay when the row and column operations are performed concurrently. Therefore, the proposed decoder performs the column operations more frequently in a single iterative decoding, and achieves a high-efficiency message-passing schedule within the limited decoding delay time. Hardware implementation on an FPGA and simulation results show that the proposed partially-parallel LDPC decoder improves the decoding throughput and bit error performance with a small hardware overhead.

Power efficiency and real-time processing capability are two major issues in today's mobile video applications. We proposed a novel Motion Estimation (ME) engine for power-efficient real-time MPEG-4 video coding based on our previously proposed content-based ME algorithm. By adopting Full Search (FS) and Three Step Search (TSS) alternatively according to the nature of video contents, this algorithm keeps the visual quality very close to that of FS with only 3% of its computational power. We designed a flexible Block Matching (BM) Unit with 16-PE SIMD data path so that the adaptive ME can be performed at a much lower clock frequency and hardware cost as compared with previous FS based work. To reduce the energy cost caused by excessive external memory access, on-chip SRAM is also utilized and optimized for parallel processing in the BM Unit. The ME engine is fabricated with TSMC 0.18μm technology. When processing QCIF (15fps) video, the estimated power is 2.88mW @ 4.16MHz (supply voltage: 1.62V). It is believed to be a favorable contribution to the video encoder LSI design for mobile applications.

Text detection in color images has become an active research area in the past few decades. In this paper, we present a novel approach to accurately detect text in color images possibly with a complex background. The proposed algorithm is based on the combination of connected component and texture feature analysis of unknown text region contours. First, we utilize an elaborate color image edge detection algorithm to extract all possible text edge pixels. Connected component analysis is performed on these edge pixels to detect the external contour and possible internal contours of potential text regions. The gradient and geometrical characteristics of each region contour are carefully examined to construct candidate text regions and classify part non-text regions. Then each candidate text region is verified with texture features derived from wavelet domain. Finally, the Expectation maximization algorithm is introduced to binarize each text region to prepare data for recognition. In contrast to previous approach, our algorithm combines both the efficiency of connected component based method and robustness of texture based analysis. Experimental results show that our proposed algorithm is robust in text detection with respect to different character size, orientation, color and language and can provide reliable text binarization result.

This paper proposes a parallel LSI architecture for LDPC decoder which improves a message-passing schedule. The proposed LDPC decoder is characterized as follows: (i) The column operations follow the row operations in a pipelined architecture to ensure that the row and column operations are performed concurrently, (ii) The proposed parallel pipelined bit functional unit enables the decoder to perform every column operation using the messages which is updated by the row operations. These column operations can be performed without extending the single iterative decoding delay. Hardware implementation and simulation results show that the proposed decoder improves the decoding throughput and bit error performance with a small hardware overhead. © 2006 IEEE.

In this paper, we propose a real-time object detection LSI for mobile videophone. The LSI achieves real-time processing speed by stereo vision algorithm using temporal data relationship between frames. The algorithm uses a last frame result to reduce the detection area and search range to the edge of the object and the background. We have designed object detection LSI by using this algorithm. The LSI can process QCIF stereo images 30 frames/sec and logic area is 15182 gates. © 2006, The Institute of Image Electronics Engineers of Japan. All rights reserved.

Straight lines are important geometric features for aerial image understanding tasks like man-made object detection. As image scene becomes more complex, traditional method like Hough Transform may produce false detections and cannot work efficiently. In this paper, we propose a new multi-scale line detection approach that can efficiently detect semantic lines in aerial image with complex scene. Firstly, a method called "Trichotomy Line Extraction" detects reliable line segments locally. Then multi-scale image system is constructed by wavelet decomposition, from which global information is obtained to detect semantic lines. Experimental results show that proposed method can extract accurate linear features on complex scene aerial images in a robust and efficient way.

With the increase of commercial multimedia applications using digital video, the security of video data becomes more and more important. Although several techniques have been proposed in order to protect these video data, they provide limited security or introduce significant overhead. This paper proposes a video security scheme for MPEG video compression standard, which includes two methods: DCEA (DC Coefficient Encryption Algorithm) and "Event Shuffle." DCEA is aim to encrypt group of codewords of DC coefficients. The feature of this method is the usage of data permutation to scatter the ciphertexts of additional codes in DC codewords. These additional codes are encrypted by block cipher previously. With the combination of these algorithms, the method provides enough security for important DC component of MPEG video data. "Event Shuffle" is aim to encrypt the AC coefficients. The prominent feature of this method is a shuffling of AC events generated after DCT transformation and quantization stages. Experimental results show that these methods introduce no bit overhead to MPEG bit stream while achieving low processing overhead to MPEG codec.

This paper present a simple and effective method to further reduce the search points in the multilevel successive elimination algorithm (MSEA). Because the sea values for most of the best matching search positions are much smaller than the current minimum SAD, we can simply increase the calculated sea value to increase the elimination ratio without much affecting the coding efficiency. Compared with the MSEA algorithm, experiments show that the proposed strict MSEA algorithm (SMSEA) can provides almost 6.5 times speedup with stable image quality, which is better than diamond search (DS). In practice, the proposed technique can also be used in the fine granularity SEA (FGSE) algorithm and the calculation process can be traced by analogy.

Many parallel Fast Fourier Transform (FFT) algorithms adopt multiple stages architecture to increase performance. However, data permutation between stages consumes volume memory and processing time. One FFT array processing mapping algorithm is proposed in this paper to overcome this demerit. In this algorithm, arbitrary 2^k butterfly units (BUs) could be scheduled to work in parallel on n=2^s data (k=0, 1, ..., s-1). Because no inter stage data transfer is required, memory consumption and system latency are both greatly reduced. Moreover, with the increasing of BUs, not only does throughput increase linearly, system latency also decreases linearly. This array processing orientated architecture provides flexible tradeoff between hardware cost and system performance. In theory, the system latency is (s×2^<s-k>)×t_<clk> and the throughput is n/(s×2^<s-k>×t_<clk>), where t_<clk> is the system clock period. Based on this mapping algorithm, several 18-bit word-length 1024-point FFT processors implemented with TSMC0.18μm CMOS technology are given to demonstrate its scalability and high performance. The core area of 4-BU design is 2.991×1.121mm^2 and clock frequency is 326MHz in typical condition (1.8V, 25℃). This processor completes 1024 FFT calculation in 7.839μs.

This paper presents a highly parallel architecture for deblocking filter in H.264/AVC. We adopt various parallel schemes in memory sub-system and datapath. A 2-dimensional parallel memory scheme is employed to support efficient parallel access in both horizontal and vertical directions in order to speed up the whole filtering process. This parallel memory also eliminates the need for a transpose circuit. In the datapath, an algorithm optimization is performed to implement parallel filtering with hardware reuse. Pipeline techniques are also adopted to improve the throughput of filtering operations. Our design is implemented under TSMC 0.18 mu m technology. Results show that the core size is 0.82 x 1.13 mm(2) when the maximum frequency is 230 MHz. Compared to other existing architectures, our design has advantages in both speed and area.

This paper proposes a reconfigurable adaptive FEC system based on Reed-Solomon (RS) code with interleaving. In adaptive FEC schemes, error correction capability t is changed dynamically according to the communication channel condition. For given error correction capability t, we can implement an optimal RS decoder composed of minimum hardware units for each t. If the hardware units of the RS decoder can be reduced for any given error correction capability t, we can embed as large deinterleaver as possible into the RS decoder for each.t. Reconfiguring the RS decoder embedded with the expanded deinterleaver dynamically for each error correction capability t allows us to decode larger interleaved codes which are more robust error correction codes to burst errors. In a reliable transport protocol, experimental results show that our system achieves up to 65% lower packet error rate and 5.9% higher data transmission throughput compared to the adaptive FEC scheme on a conventional fixed hardware system. In an unreliable transport protocol, our system achieves up to 76% better bit error performance with higher code rate compared to the adaptive FEC scheme on a conventional fixed hardware system.

In adaptive motion estimation, spatial-temporal correlation based motion type inference has been recognized as an effective way to guide the motion estimation strategy adjustment according to video contents. However, the complexity and the reliability of those methods remain two crucial problems. In this paper, a motion vector field model is introduced as the basis for a new spatial-temporal correlation based motion type inference method. For each block, Full Search with Adaptive Search Window (ASW) and Three Step Search (TSS), as two search strategy candidates, can be employed alternatively. Simulation results show that the proposed method can constantly reduce the dynamic computational cost to as low as 3% to 4% of that of Full Search (FS), while remaining a closer approximation to FS in terms of visual quality than other fast algorithms for various video sequences. Due to its efficiency and reliability, this method is expected to be a favorable contribution to the mobile video communication where low power real-time video coding is necessary.

A real-time moving object extraction and tracing algorithm is proposed for videophone application. Semantic objects, mainly moving objects in this specified case, are extracted by block based change detection from first several frames. Improved active contour model is then utilized for object tracing. Since there is little difference between continuous two frames, only limited pixels need to be searched instead of whole frame scan so that computing complexity is greatly decreased. Therefore, it is able to be used for real-time applications such as that in videophone. However, previous active contour model is sensitive to noise. Error originating from noise traps accumulates and deteriorates the tracing results. So gradient projection (GP) and jump back (JB) mechanisms are introduced to suppress noise trap effect and error accumulation in this paper. Experiment results demonstrate that proposed algorithm is an efficient noise tolerable real-time approach.

The discrete-time cellular neural network (DTCNN) is a promising computer paradigm that fuses artificial neural networks with the concept of cellular automaton (CA) and has many applications to pixel-level image processing. Although some architectures have been proposed for processing DTCNN, there are no compact, practical computers that can process real-world images of several hundred thousand pixels at video rates. So, in spite of its great potential, DTCNN's are not being used for image processing outside the laboratory. This paper proposes a DTCNN processing method based on a highly parallel two-dimensional (2-D) cellular automata called CAM 2. CAM 2 can attain pixel-order parallelism on a single PC board because it is composed of a content addressable memory (CAM), which makes it possible to embed enormous numbers of processing elements, corresponding to CA cells, onto one VLSI chip. A new mapping method utilizes maskable search and parallel and partial write commands of CAM 2 to enable high-performance DTCNN processing. Evaluation results show that, on average, CAM 2 can perform one transition for various DTCNN templates in about 12 microseconds. And it can perform practical image processing through a combination of DTCNN's and other CA-based algorithms. CAM 2 is a promising platform for processing DTCNN. © 1998 IEEE.

This paper proposes a distributed built-in self-test (BIST) technique and its test design platform for VLSIs. This BIST has lower hardware overhead pattern generators, compressors and a controller. The platform cuts down on the number of complicated operations needed for the BIST insertion and evaluation, so the BIST implementation turn-around-lime (TAT) is dramatically reduced. Experimental results for the 110 k-gate arithmetic execution blocks of an image-processing LSI show that using this BIST structure and platform enables the entire BIST implementation within five days. The implemented BIST has a 1% hardware overhead and 96% fault coverage. This platform will significantly reduce testing costs for time-to-market and mass-produced LSIs.