An FPGA Implementation for Non-Evasive Video Latency Measurement

Video latency plays a critical role in applications that require real time delivery of video content. Key areas include video teleconferencing, remote control, cloud gaming, and live news gathering. Given the value latency plays into these markets, video latency measurement remains an important topic. With the advent of artificial intelligence and deep learning neural networks, reliable low latency transmission becomes critical for automated real time control. Human judgements and one-off manual procedures are no longer reliable. Frame freezes, frame jumps, slow clock drifts, or even frame level corruption can bring instability into the tight control loop, resulting in jitter, drifts, and noise to the remotely controlled target. There exist several classes of video latency measurement approaches, including watermark embedding, out-of-band metadata, and passive feature extraction. In addressing the integrity monitoring of latency for the existing video delivery network base, this paper presents a novel approach following the passive feature extraction class. The solution extracts unique characteristics within each frame of the video stream in question. The characteristics must be unique between frames in the stream, within the range of the maximum delay window. The same feature extraction method is then applied to the delayed stream. By comparing the features between any two frames, the measurement can be carried out by finding the best match between the two streams. The proposal was implemented on an Xilinx Artix family FPGA, utilizing only 20KLUTS of resources. Given the memory size required to store the reference frames, external DDR memory is necessary to support 8K video formats.