[Matroska-devel] VC-1 codec support in matroska - take two

Mike Chen mike.chen82 at googlemail.com
Wed May 30 13:51:25 CEST 2007


	I would like to discuss (again!) support for next-gen HD video and
audio formats (namely VC1 for start) in matroska. A month ago I sent a
message asking if the support is planned and what is the way to do it.
We discussed much of it off the DL with Michael (Манцев), since he
already implemented it in Haali splitter. However, from my point of
view his implementation (as of today) will bring more trouble to
everyone in a long run, so that's the reason I would like to discuss
it here.

	There are actually three things to discuss: Codec spec in general,
reason  for adding VC1, and VC1 format itself.

Codec spec.

As of today, there are two code standards (at least for mpeg-like
codecs) in matroska. One published at matroska spec page and at
http://haali.cs.msu.ru/mkv/codecs.pdf (and this standard frankly has
nothing to do with a real life), and a de-facto standard implemented
by all major readers (which of course is different from a published
one). By major readers I mean VLC, ffmpeg and haali. For example,
MPEG2 description reads "the stream itself does not contain any
sequence/gop headers anymore". Had it been implemented that way, it
would be pretty inconvenient to write a splitter for mpeg - one had to
decode I frames and apply sequence header from CodecPrivate (or add
sequence header to each frame) in a splitter. However de-facto
standard just requires mpeg stream unchanged, split at frame
boundaries (sequence and gop header merged with the next frame). This
makes writing a splitter a very easy job - just pass frames among with
PTS down the chain, and decoder will understand it. VLC and ffmeg
splitters work exactly like that. In fact nor VLC nor mplayer not even
haali would play an MPEG stream formatted by "official specification".
Funny enough it looks like muxer in mkvtools did produce stripped
stream some time ago, but later the code that adds sequence header to
EVERY frame was added. This produces non-conformant mpeg stream
(sequence headers should not be present within GOP boundaries), but
all decoders seem to ignore additional sequence headers.

The reason I wrote about this is that sometimes this spec is used as
an example. In his current implementation of VC1 codec Michael
requires sequence headers (and even startcodes!!!) to be stripped from
a stream and cites MPEG documentation as an example, claiming that is
how MPEG is implemented. That complicates discussion. So, here is my
question - how one adds changes to matroska spec? Is there any
voting/discussion/etc, or is it just kind of anarchy? I would very
much like official spec to be in sync with a de-facto standard (which
is much more simpler and elegant then current "official" one).

Reasons to add VC1 codec.

VC1 has three profiles - simple, main and advanced. Simple and main
can be (and are as of today) supported via fourcc encapsulation, so
there are no special changes required. Thanks to Michael who pointed
this out. Advanced profile requires adding a codec, since it has B
frames (and even field-based stream, when one can have a B-field
followed by P-field !!!)and as Steve mentioned it usually comes from
mpeg PS/TS streams. VC1 is a very much like x264 and mpeg, so
supporting it and adding a codec seems natural. Changes in  decoders
should be minimal.

Proposed format for VC1.

In my first email I proposed basically to keep ES stream unchanged,
split at frame boundaries and PTS-timestamped. This is how all
mpeg-like streams are encoded today, making a matroska essentially an
ES-container. With this format changes to decoders are minimal (ffmpeg
would require one line change and VLC - 3 lines :) ). The very first
VC1 codec was of course from MS. It required the following codec
private area - one byte len, followed by sequence header, followed by
entrypoint header, all with startcodes. Both headers are required for
decoding but first byte seems unnecessary, but that was original MS
format. Later ffmeg adopted this format in its decoder module. That's
what I was proposing to keep in codec private. This way no additional
changes are required and in fact after doing this 4-line source code
change both VLC and mplayer played VC1 streams flawlessly (VLC played
with both ffmpeg and DMO).

Michael, however does not agree with this proposal, and actually
implemented VC1 in haali differently, using his ability to change it
as he sees fit. He proposed to strip startcodes from each frame (!!!)
, strip all headers and put sequence/entrypoint without startcodes
into CodecPrivate. There are two problems with this design, in my
opinion. Firstly, it creates an unneeded complexity - now the decoder
should reconstruct the stream by adding startcodes and headers,
decoder should also keep track on which kind of frame it is processing
to apply required header. Also CodecPrivate area cannot be passed
directly to codec by decoder in this design. All of this will probably
make VC1 the ugliest implemented codec in matroska. Second, all these
optimizations will achieve nothing. There is not much value in saving
5 bytes per 15 frames with a 15mbit bitstream. As for stripping
startcodes, this may be achieved even today just by using header
compression, without any need to write additional code.

	So here is my question - is there any formal way to add a codec spec
to matroska? Or should we keep everything as is - try to implement
codecs without any discussion differently in different software and
then create a format mess/war in Matroska?

	The reason I'm writing all this is not because I have to fulfill my
own ego but because I want to make matroska more mature and ready for
HD video. Our company believes in Matroska and in fact our product
will promote it very highly. Our muxer is superior to any other I
personally know of, and we are going to release muxer core under LGPL.
So is there finally any way to contribute important stuff without
hearing "I already did it my way, go away"? Approach like this won't
help matroska to become a mature format I fear. I honestly believe
that we can avoid situation when proprietary patented formats are
designed cleanly and reasonably, and open-source ones are a mess. I

			Yours, Mike.

More information about the Matroska-devel mailing list