Real-time multimedia coding and transmission

Abstract

This paper sums up relevant topics covered by the special issue titled ‘Real-Time Multimedia Coding and Transmission’, including efficient content representation, multimedia transmission, hardware and software acceleration, and transcoding techniques.

1 Introduction

Applications such as online gaming, video conferencing, video surveillance systems, industrial visual inspection systems, and many other embedded image and video processing applications will require very fast image and video processing capabilities due to their intrinsic real-time temporal requirements. Nowadays, full-high-definition video resolution is a reality in common smartphones, video cameras, and so on while multimedia market trends are focused on 4K ultra-high-definition resolution. In fact, several companies have recently presented their new devices which are able to visualize 4K video content like the new organic light-emitting diode Sony monitor or the Christie D4K3560 projector. These devices evidence the limitations of current real-time multimedia coding and transmission techniques due to the huge volume of information to be processed under real-time constraints. Furthermore, most of these applications run on devices with constrained resources (computational and/or power resources) like smartphones, tablets, laptops, embedded systems, and still/video cameras, thus making more challenging the design and deployment of new multimedia coding and transmission techniques.

2 Specific advances

In this special issue, we have covered fundamental aspects related to image and video multimedia coding and transmission techniques with special emphasis on those techniques suitable for resource-constrained devices and applications with real-time temporal restrictions. Three fundamental topics are covered in 11 outstanding research articles regarding the aforementioned topic:

• 1. Efficient techniques for multimedia content representation and transmission as well as challenges found when transmitting multimedia content over modern networks.

• 2. Hardware/software acceleration techniques for multimedia coding.

• 3. Intelligent optimization techniques for transcoding.

The first topic is covered by the following contributions:

• In the article entitled ‘Relationship between Sampling and Multirate Filterbanks in the Linear Canonical Transform Domain’ by Feng Zhang, Ran Tao, and Yue Wang, the authors explore the relationship between sampling theorems and multi-rate filter banks in the linear canonical transform (LCT) domain. Both sampling identity and interpolation identity for band-limited signals in the LCT domain are employed to obtain a discrete-time implementation for band-limited signals in the LCT domain from their multichannel samples.

• In the article entitled ‘SCTP as Scalable Video Coding Transport’ by Jordi Ortiz, Eduardo Martínez Graciá, and Antonio F. Skarmeta, the authors present an evaluation of the stream transmission control protocol (SCTP) for the transport of the scalable video codec (SVC) proposed by the moving picture experts group as an extension to H.264/advanced video coding (AVC). On the one hand, SVC allows for easily splitting the bit stream into sub-streams which carry different video layers, each with different importance according to the reconstruction of the complete video sequence at the receiver’s end. On the other hand, SCTP includes features such as the multi-streaming and multi-homing capabilities, thus permitting robust and efficient transportation of the SVC layers. Simulations show that CMT-SCTP outperforms transmission control protocol and real-time transmission protocol in error-prone networking environments.

• In the article entitled ‘Evaluation of H.264/AVC over IEEE 802.11p Vehicular Networks’ by Ismael Rozas-Ramallal, Tiago M. Fernández-Caramés, Adriana Dapena, and José A. García-Naya, the authors present an FPGA-based testbed developed for evaluating H.264/AVC video transmission over vehicular networks. The testbed covers some of the most common situations in vehicle-to-vehicle and roadside-to-vehicle communications, and it is highly flexible, allowing for the performance evaluation of different vehicular standard configurations. In order to improve the received video quality performance, the authors propose to substitute the convolutional channel encoder used in IEEE 802.11p for a low-density parity-check code encoder.

• In the article entitled ‘Scheduling Schemes with Adaptive Blind Detection for Code Reuse in Multiuser MIMO Systems’ by Horacio Mendoza and Graciela Corral-Briones, the authors study the performance improvement of using jointly opportunistic scheduling of multi-users and interference cancellation together with adaptive multi-user detection techniques for enabling the utilization of the same set of spreading codes to transmit two data streams in closed-loop multi-user MIMO systems. Results show that the performance of an adaptive blind receiver is not degraded when scheduled users share the same spreading code.

• In the article titled ‘Research on Mud Pulse Signal Data Processing in MWD’ by Bing Tu, De Sheng Li, En Huai Lin, and Miao Miao Ji, the authors introduce the so-called measure while drilling (MWD) system including the data transmission format for the underground mud pulse signal. MWD is a well-known technique in oil and gas exploration and consists of providing data measurements to guide the drilling process similarly to the data provided to a pilot by flight instrumentation. In this work, the authors propose a set of algorithms to reduce noise and interference disrupting the so-called mud pulse signals, i.e., the wireless signals employed for communication between the sensors located close to the drilling machinery and the so-called ground-collection box, which is in charge of processing all received data. The proposed algorithms were tested satisfactorily by means of simulations and also in an oil field in the north of China.

In addition to channel and network optimization techniques for multimedia transmission, both hardware and software acceleration techniques for multimedia coding have also been covered in this special issue. This topic is discussed in the following articles:

• The authors Vicente Galiano, Otoniel López, Manuel P. Malumbres, and Héctor Migallón in the article entitled ‘Multicore-based 3D-DWT Video Encoder’ present a three-dimensional discrete wavelet transform (3D-DWT) video encoder based on a fast run-length coding engine. Furthermore, the authors introduce several multi-core optimizations to speed up the 3D-DWT computation. Results show that the proposed encoder obtains good rate/distortion results for high-resolution video sequences with nearly in-place computation using only the memory needed to store a group of pictures while being able to compress a full HD video sequence in real time.

• In the article entitled ‘Multi-GPU based on Multicriteria Optimization for Motion Estimation System’ by Carlos Garcia, Guillermo Botella, Fermin Ayuso, Manuel Prieto, and Francisco Tirado, the authors present a graphics processing unit (GPU)-based version of a neuromorphic motion estimation algorithm with low memory consumption. An evolutionary algorithm was used to find the best configuration, which is a trade-off solution between consumption of resources, parallel efficiency, and accuracy. Both a grain level (by means of multi-GPU systems) and a finer level (by data parallelism) have been exploited in the proposed solution.

• Finally, Kuang-Shyr Wu in the article entitled ‘A Secret Image Sharing Scheme for Light Images’ presents a new (r,n)-threshold secret image sharing scheme with low information overhead for images having a low distortion rate, being more applicable for light images. In the proposed solution, a secret image is encoded into n noise-like shadow images to satisfy the condition that any r of the n shadow images (also denoted as shares) can be used to reveal the secret image, whereas no information on the secret can be revealed from any r−1 or fewer shadow images. The proposed method reduces the size of the shadow images for further storage or transmission.

With respect to the intelligent optimization for transcoding topic, it has been covered by the following articles:

• In the article entitled ‘Region-of-Interest Determination and Bit-rate Conversion for H.264 Transcoding’ by Shu-Fen Huang, Mei-Juan Chen, Kuang-Han Tai, and Mian-Shiuan Li, the authors present a video bit-rate transcoder for the baseline profile in H.264/AVC. The objective is to fit the available channel bandwidth for the client when transmitting video bit streams via communication channels. In order to maintain visual quality for low bit-rate video, they analyze the decoded information in the transcoder and propose a Bayesian-based region-of-interest (ROI) determination algorithm in such a way that the transcoded video will conform to the target bit rate by re-quantization according to those models. The ROI-based transcoder allocates more coding bits to ROI regions and reduces the complexity of the re-encoding procedure for non-ROI regions, not only keeping the coding quality, but also improving transcoding efficiency, thus making real-time transcoding more practical.

• In the article entitled ‘Temporal Scalable Mobile Video Communications based on an Improved WZ-to-SVC transcoder’ by Alberto Corrales-García, José Luis Martínez, Gerardo Fernández-Escribano, and Francisco José Quiles, the authors present a WynerZiv (WZ) coding approach to SVC transcoding. Applications such as low-power sensor networks, video surveillance cameras, or mobile communications present a different framework in which low-cost senders transmit video bit streams to a central receiver. In order to manage this kind of application efficiently, WZ coding proposes a solution in which most of the complexity is moved from the encoder to the decoder. Despite the advantages of the video transcoding framework, the transcoder accumulates high complexity and must be reduced in order to avoid excessive delays in communication. Thus, in order to reduce that delay, the information generated during the WZ stage is reused during the SVC stage. Consequently, the time taken by the transcoding is reduced by around 77.77%, with a negligible rate-distortion penalty.

• Finally, Rosario Garrido-Cantos, Jan De Cock, José Luis Martínez, Sebastian Van Leuven, Pedro Cuenca, and Antonio Garrido in the article entitled ‘Low Complexity Transcoding Algorithm from H.264/AVC to-SVC using Data Mining’ propose a low-complexity algorithm to convert an H.264/AVC bit stream without scalability to scalable bit streams with temporal scalability in the baseline and main profiles by accelerating the mode decision task of the SVC encoding stage using machine learning tools. When these techniques are applied, the complexity is reduced by 87% while maintaining coding efficiency.