Introduction

MPEG has given humans the means to add significant more effectiveness and enjoyment to their lives. If billions of people, however, can stream video to anywhere at any time of the day, global energy consumption can only increase. Enhanced experiences provided by newer features such as High Dynamic Range further adds energy consumption in the display. More sophisticated compression algorithms consume more energy, even though this is mitigated by more advanced circuit geometry.

In 2013 MPEG issued a Call for Proposal on “Green MPEG” requesting technologies that enable reduction of energy consumption in video codecs and 3 years later MPEG released ISO/IEC 23001-11 Green Metadata. This was followed by a number of ancillary activities.

It should be clear that Green Metadata should not be seen as an attempt at solving the global problem of energy consumption. More modestly Green Metadata seeks to reduce power consumption in the encoding, decoding, and display process while preserving the user’s quality of experience (QoE). Green Metadata can also be used to reduce the QoE in a controlled way.

The standard does not require changing the operation of a given encoder or decoder (i.e. changing the video coding standard). It just requires to be able to “access” and “influence” appropriate operating points of any or the encoder, decoder or display functionalities.

A systems view

Green Metadata target metadata suitable to influence the video encoding, decoding and display processes. The framework, however, could be easily generalised by replacing “video” and “display” with “media” and “presentation”. However, the numerical results obtained in the video case cannot be directly extrapolated to other media.

Let’s start from Figure 25 representing a conceptual diagram of a green encoder-decoder pair.

Figure 23 – Conceptual diagram of a green encoder-decoder pair

The Green Video Encoder (GVE), is a regular video encoder that generates a compressed video bitstream and also a stream of metadata (G-Metadata) for use by a Green Video Decoder (GVD) to reduce power consumption. When a return channel is available (e.g. on the internet), the GVD may generate feedback information (G-Feedback) that the GVE may use to generate a compressed video bitstream that demands less power for the GVD to decode.

To understand what is actually standardised by Green Metadata, it is worth digging a little bit in the following high-level diagram and see what is the new “green component” that is added. Figure 24 helps to understand such green components.

Figure 24 – Inside a green encoder-decoder pair

The GVE generates G-Metadata packaged by the G-Metadata Generator for transmission to a GVD. The GDV G-Metadata Extractor extracts the G-Metadata payload and passes the GVE G-Metadata to the GVD Power Manager along with G-Metadata coming from the GVD. The GVD Power Manager, based on the two G-Metadata streams and possibly other input such as user’s input (not shown in figure), may send

1. Power Control data to the Video Decoder to change its operation
2. G-Feedback data to the G-Feedback Generator to package it for transmission to the GVE.

At the GVE side, G-Feedback Extractor extracts the G-Feedback data and passes them to the GVE Power Manager. This may send Power Control data to the Video Encoder to change its operation.

To examine a bit more in detail how G-Metadata can be used, it is helpful to dissect the Video Encoder and Decoder pair as shown in Figure 25.

Figure 25 – Inside the encoder and decoder

The Video Encoder is composed of a Media Preprocessor (e.g. a video format converter) and a Media Encoder. The Video Decoder is made of a Media Decoder and a Presentation Subsystem (e.g. to drive the display). All subsystems send G-Metadata and receive Power Control. The Presentation Subsystem only receives Power Control.

What is standardised in Green Metadata? As always in MPEG, the minimum that is required for interoperability. This means the Encoder Green Metadata and the Decoder Green Feedback (in red in  REF _Ref7628989 \h  \* MERGEFORMAT Figure 24) that are exchanged by systems which are potentially manufactured by different entities. Other data formats inside the GVE and the GVD are a matter for GVE and GVD manufacturers to decide because they do not affect interoperability but may affect performance. In particular, the logic of the Power Manager that generates Power Control can be the differentiating factor between implementations.

Achieving reduced power consumption

In the following the 3 areas positively affected by the use of the Green Metadata standard – encoder, decoder and display – will be illustrated.

1. Encoder. By using a segmented delivery mechanism (e.g. DASH), encoder power consumption can be reduced by encoding video segments with alternate high/low quality. Low-quality segments are generated by using lower-complexity encoding (e.g. fewer encoding modes and reference pictures, smaller search ranges etc.). Green Metadata include the quality of the last picture of each segment. The video decoder enhances the low-quality segment by using the metadata and the last high-quality video segment.
2. Decoder. Lowering the frequency of a CMOS circuit implementing a video decoder reduces power consumption because this roughly increases linearly with the clock frequency and quadratically with the voltage applied. In a software decoder picture complexity can be used to control the CPU frequency. One type of Green Metadata signals the duration and degree of complexity of upcoming pictures. This can be used to select the most appropriate setting and offer the best QoE for a desired power-consumption level.
3. Display. The display adaptation technique known as backlight dimming reduces power consumption by dimming the LCD backlight while RGB values are scaled in proportion to the dimming level (RGB values do not have a strong influence on power consumption).

Green Metadata need to be carried

ISO/IEC 23001-11 only specifies the Green Metadata. The way this information is transported depends on the specific use scenarios (some of them are described in Context, Objectives, Use Cases and Requirements for Green MPEG).

Two transports have been standardised by MPEG. In the first, Green Metadata is transported by a Supplementary Enhancement Information (SEI) message embedded in the video stream. This is a natural solution since Green Metadata are due to be processed in a Green Video Decoder that includes a regular video decoder. In this case, however, transport is limited to decoder metadata, not display metadata. In the second, suitable for a broadcast scenario, all Green Metadata is transported in the MPEG-2 Transport Stream.

Conclusions

Power consumption is a dimension that had not been tackled by MPEG, but the efforts that have led to the Green Metadata standard have been rewarded: with the currently standardised metadata 38% of video decoder power and 12% of video encoder power can be saved without affecting QoE and up to 80% of power can be saved with some degradation. Power saving data were obtained using the Google Nexus 7 platform and the Monsoon power monitor, and a selection of test material.

Interested readers can know more by visiting the MPEG web site and, more so, by purchasing the Green Metadata standard from the ISO website or from a National Body.