Studying real-time rate control in perceptual, modeling and efficient aspects

Standard video systems such as H.261/3/4 and MPEG exploit the spatial, temporal and statistical redundancies in the source video. Since the level of redundancy changes from frame to frame, the number of bits per frame is variable, even if the same quantization parameters are used for all frames. Therefore, a buffer is required to smooth out the variable video output rate and provide a constant video output rate. The rate control is used to prevent the buffer from over-flowing (resulting in frame skipping) or/and under-flowing (resulting in low channel utilization) in order to achieve good video quality. In our rate control, there are six schemes ((1) 'Perceptual rate control', (2) 'Frame-layer rate control', (3) 'PID-based rate control', (4) 'Linear rate control', (5) '3-linear rate control' and (6) 'Fast rate control'). Different schemes have different purposes. In the framework of our schemes, there are three categories. The first is 'Perceptual', the second is 'Modeling' and the third is 'Efficient'. In the first category 'Perceptual', the description of our rate control schemes are based on the human visual system (HVS). In the spatial domain (i.e. within a frame), PSNR is usually a common quality performance measure. In fact, PSNR is not a good quality measure for the human visual system. In 'Perceptual rate control', perceptual PSNR is defined and the scheme has better perceptual spatial video quality. Besides spatial quality, temporal quality should also be addressed. Humans prefer consistent quality over frames rather than a large quality fluctuation over frames. There are two schemes to ensure better tradeoff between spatial quality and temporal quality. One scheme, called 'Frame-layer rate control', changes the frame rate whereas another scheme, called 'PID-based rate control', changes the target bit of each frame. In the second category 'Modeling', our rate control schemes use different rate and distortion models described in the literature. When our models are used, the PSNR is higher and the rate prediction is more accurate. There are two schemes. One, called 'Linear rate control', is target at low bit rates whereas the other, called '3-linear rate control', is target at both low and high bit rates. In the third category 'Efficient', our rate control scheme, called 'Fast rate control', has a lower complexity but a higher PSNR compared with TMN8 rate control