This paper presents a novel colour quantization algorithm for compressed video data. The proposed algorithm extracts DC coefficients and motion vectors of blocks in a shot to estimate a cumulative colour histogram of the shot and, based on the estimated histogram, design a colour palette for displaying the video sequence in the shot. It significantly reduces the complexity of the generation of a palette by effectively reducing the number of training vectors used in training a palette without sacrificing the quality. The palette obtained can provide a good display quality even if zooming and panning exist in a shot. The experimental results show that the proposed method can achieve a significant SNR improvement as compared with conventional ...