Can MPEG cope with new media technologies?
Convenor, ISO/IEC JTC1/SC29/WG11 (MPEG)
This paper scans the almost 20 year of MPEG life to discover the movements of the industry, as represented by the standards produced, and analyses the current status..
Formally the Moving Pictures Experts Group (MPEG) is Working Group 11 (WG 11) Coding of moving pictures and audio” of Subcommittee 29 (SC 29) “Coding of audio, picture, multimedia and hypermedia information" of the Joint Technical Committee 1 (JTC 1) “Information Technologies” of the International Organisation for Standardisation (ISO) and the International Electrotechnical Commission (IEC).
Working Groups are typically established by a Subcommittee to deal with some specific technical issue. Once the issue has been resolved working groups are typically disbanded.
This has not been the case of MPEG. Born in 1988 as an experts group of the then WG 8 of SC 2 MPEG was driven by a vision whose main points can be summarised as below:
2. The bread and butter
This standard was developed in the early days when it was not really clear what the impact of digital would be. Indeed MPEG-1 meant having the technology to handle audio and video bits for the first time with the magic word of “interactivity”.
In those days anything that implied digital audio and video had to confront with issues of hardware – i.e. integrated circuit (IC) – implementation. So all decisions were weighed against possible impacts on IC design.
The most serious problem faced by MPEG-1 was one of the actual use of digital technologies based on the standard. The Digital Compact Cassette (DCC), meant to be a user of MPEG-1 Audio Layer I was a flop, the Digital Audio Broadcasting (DAB) standard, meant to be a user of MPEG-1 Audio Layer II, is not really flying even now and Copact Disc Interactive (CD-i), meant to be a user of the complete set of MPEG-1 technologies, is long dead.
Still MPEG-1 is well alive and kicking: Video CD uses the complete set of MPEG-1 technologies and MP3 uses MPEG-1 Audio Layer III. Both need no introduction.
Once it was proved that the machine could work it was possible to make true the long-held dream of television made digital. What MPEG added was the definition of a truly universal digital video format (audio, as it well known, is not part of the truly universal digital television format).
The development of MPEG-2 Video was obsessed by the issues of hardware implementation. In the early 1990s designing an MPEG-1 Video chip was becoming possible, but an MPEG-2 chip was another story. Hardware problems existed not only for the signal processing part but also for the Random Access Memory (RAM) that was required to store decoded pictures. A special profile (Simple) was designed to avoid use of too much RAM, although with some picture quality loss.
The issue of patents came to the fore with all its importance. The Consumer Electronics (CE) industry had its ways to handle the issue of patents in its standards: the IPR of a successful device belonged to the company that had originally conceived it, but with MPEG quite a few companies held IPR on MPEG-2 Video. The first patent pool after many years was established to handle the over 100 patents that the patent pool administrator has found to be relevant to implementing the MPEG-2 standard. The royalty model followed the established CE practice: a fixed amount per device and Digital Versatile Disc (DVD) disc carrying MPEG-2 Video encoded content.
MPEG-4 was conceived as a unified multimedia coding standard. Indeed MPEG-4 Video has been designed to cover all bitrates and all video formats and MPEG-4 Audio is capable of handling music and speech.
MPEG-4, however, is also capable of handling synthetic content, such as 2D and 3D graphics, human faces and bodies, synthetic music etc. Moreover it has a powerful technology to handle the composition of natural and synthetic information.
MPEG-4 has been extremely prolific in Visual extensions and profiles. The basic Advanced Audio Coding (AAC) technology inherited and extended in MPEG-4 has undergone many extensions.
MPEG-4 did not develop a transport mechanism for MPEG content but only a generic interface to transport. It did develop, however, a Real Time Protocol (RPT) payload – and after untold vicissitudes got it adopted by the Internet Engineering Task Force (IETF) – for generic content.
MPEG-4 has been an innovator in several respects. The first is the fact that MPEG-4 has provided the first example to manage Intellectual Property Rights (IPR) in a complete fashion through its Intellectual Property Management and Protection (IPMP).
The second form of innovation was brought by MPEG-4 as the convergence point of the traditional hardware-based and the new software-based audio-visual industry. This prompted the transformation of the reference software from a nice-to-have add on to the formal status of standard on par with the textual form of the standard. The development process of the reference software was also codified – independently – along the lines of the Open Source Software tradition.
Another innovation was made by MPEG-4 in patent licensing. While MPEG-2 Video has a licensing model that is based on physical objects, be they set top boxes (STB) or DVD discs, MPEG-4 Visual also has a licensing form that charges the use of content per unit time.
3. New land
The MPEG-1/-2/-4 standards have each played a role in shaping the digital audio and video industry as we know it today. MPEG-7 has the potential to reshape the industry in even more substantial ways.
The natural observation – a tautology today when we use a search engine to find information on the web – is that content is important but more so is the description of the content so that it can be found and organised.
The idea of MPEG-7 was triggered by the realisation that as much as a standard form of audio and video coding had created horizontal markets of content, a standard form of audio and video descriptions would create horizontal markets of descriptions.
The needed standard technology has all been developed. There are visual descriptions, audio descriptions and multimedia descriptions. It is also possible to define Multimedia Description Schemes (MDS) using the MPEG-7 Description Definition Language (DDL).
Today the MPEG-7 vision has still some way to go. The main reason is that digital technologies have removed value from “free” content. Premium content, for which value is still high, is offered as proprietary package using Digital Rights Management (DRM) technologies.
We are not exactly in a horizontal market and this will remain so for as long as there will be no interoperable DRM (iDRM) solution broadly adopted by the industry. iDRM is the only way for content to retain its value thereby providing an incentive for companies to add more features to their offers.
As much as MPEG-7 is not an audio and video coding standard, MPEG-21 is also not a “signal processing” standard. Its need was triggered by the realisation that compression and description are two important components of “Multimedia” but to enable a complete multimedia market where content can be traded at the atomic level there are more disparate technologies to integrate.
To achieve this goal a number of new technologies are required:
4. New horizons
Some 19 years after MPEG began its existence the situation today is that there is a lot of multimedia technologies out there both from MPEG and from other sources, but making successful multimedia applications is as hard as ever – just too many choices.
On the other hand MPEG has an impressive repository of multimedia tools and has the know how to define the appropriate combination of technologies that enable an application.
This is the rationale for the MPEG-A (A as Application) standard for which the following has been done or is under way
The MPEG Multimedia Middleware (M3W) is another non-coding and non-signal processing but very much multimedia oriented standard.
The standard will be eventually made up of the following parts
The M3W API is intended to be a generic API suitable for use in multiple products with different capabilities that have significant overlap in the functionality they require from the platform, but there are also differences.
5. The situation today
While MPEG is running towards the end of its second decade it is possible to make a few assessments regarding its field of work.
In its life span close to 2 decades MPEG has played a key role in moving the incumbent (20 years ago) audio and video industry to the digital shore and creating the new converged digital audio and video industry that we see around us.
Many things have changed in these years and the industry is far from reaching a stable configuration. As the state of technology continues to progress the need for standards will continue.
MPEG will continue its role of anticipating the standardisation needs of the industry by continuing to stick to its original vision, relying on the considerable technology assets developed so far and designing new standards in a less structured world.