[Matroska-devel] Re: Dirac Video Codec
Paul Bryson
paul at msn.com
Fri Oct 1 18:20:40 CEST 2004
"Steve Lhomme" wrote...
Tim Borer a écrit :
> It's all on the website, but if you need more precision you can use our
> mailing list or our IRC channel (#matroska on irc.corecodec.com).
The mailing list is best for writing out points that require lengthy
explanations. IRC on the other hand is very effective for a quick
discussion about some point. You can usually get quick comments from
multiple developers at once.
> > I guess one thing we need to know is what we need to do to integrate
> > Dirac with Matroska. We already have a C interface for the decoder and
> > are working on one for the encoder. These are only alpha and so the
> > details will change but the principle should be about right. We have
> > modeled them on C interfaces used for codecs in other libraries (e.g.
> > ffmpeg).
There are currently several independently written Matroska writing libs that
are free for use through GPL/LGPL licenses. You could certainly write
directly to a file using one of these. Although in the long run I would
imagine that it would probably be better to find a way to integrate into an
existing media framework. Steve mentioned GStreamer and it probably has the
best chances to be a real cross platform media encoding/playback system.
But as also mentioned, you will likely need another tool to add some of the
more advanced features.
> > >I couldn't tell about MXF. All I know is that it's a professional class
> > >container, but maybe not >good for everyday/basic use.
> >
> > Its a professional container. From our point of view it has problems in
> > getting additions agreed with the standards body (SMPTE). There are also
I had a look at MXF around a year and a half ago and even exchanged some
emails with some of the people in the group. For some reason though they
stopped responding and I became busy with other things. I had looked
through some of their specification drafts and it looked like a good thing
for large scale data imports. Basically just a good way to transfer data
from one system to another. (Much like XML might be used to transfer data
between different types of databases) When I went back to look at some
things around 8 months ago I couldn't find any of their documents that I
could look at for free. Maybe this has changed or maybe I just wasn't
looking in the right place, but this made the format seem much less 'open'.
(To me 'open' means anyone can look at the specs, not just people with
money.) Fortunately I still had a copy of the drafts and was able to refer
to that for what I was interested in. Regardless, MXF seems like overkill
for what you want as it looks like you need a playback/authoring format.
In regards to Ogg, it is a good format for basic streaming, but not much
more. Also, any format that is put into Ogg basically has to be more or
less approved by Xiph as there is not a 'standard' way of adding formats.
In many ways it is the polar opposite of Matroska in philosophy in that
almost no information is stored at the container level whereas Matroska
stores as much information at the container level. For instance, you don't
know the timecode of the data packet you are looking at without the codec in
Ogg.
> > issues about the availability of open source libraries for it and their
> > limitations.
> >
> > Open source, freely available, cross platform is where we want to be.
> >
> > One question is can Matroska be used commercially as well as personally,
> > or are there intellectual property issues that would cause problems?
>
> Yes, all the code we (the core team) write is either LGPL. The code of
> other people (like the C implementation in GStreamer) is usually LGPL too.
> Mkvtoolnix and AVI-Mux GUI are GPL.
There are several independently written reading libraries that are all open.
The official library libmatroska (C++), AVI-Mux GUI (Delphi), GStreamer (C),
Mplayer which is somehow based on GStreamer (C), as well as some ones in
development such as Javatroska (Java), RbMatroska (Ruby). There are
probably others that I'm forgetting.
> > Sounds like this might be useful, although I though we needed to handle
> > this ourselves in an elementary stream to ensure that the stream could
> > be stand alone if required. Do you have a URL that would point to
>
> Yes. But it is very important to have this information available to the
> container (too). It's needed for precise seeking. It's also needed for
> "lossless" or "blind" editing (not modifying the codec data when
> inserting/cutting frames in a file).
As he says, it is very useful to have as much data as possible available at
the container level for authoring purposes. This is part of the philosophy
of Matroska.
> > I am wary about using time code. It works well for video at 24, 25 and
> > 30 frames per second. For NTSC at 30000/1001 (aprox 29.97) frames it is
> > bodged using drop frame. There is also confusion about streams which say
> > they are 29.97 when they may mean 30000/1001. The next result is that my
> > experience is that timecode is not as reliable as I would like.
> > Therefore for Dirac we are planning to use sequential frame numbers
> > rather than time code. then you can keep accurate track of frames and
> > convert to a time code if required. What is you opinion on this?
This is what MPEG currently does. It can only display a constant framerate
and does not allow for dropped frames. Things get very interesting when you
mix framerates. For instance, Star Trek TNG was filmed using standard NTSC
speeds of around 30fps. But all of the scenes in space that were rendered
on computers were rendered at film's 24fps. So when storing on a DVD, they
use 30fps and have the 24fps images interpolated to 30fps. (Maybe you
already know how they do this, if it's important to you I could explain it.)
In our view this seems a little silly.
During playback, all systems convert to a timecode anyway. DirectShow uses
a 100ns accuracy timecode. Matroska can handle accuracy up to 1ns
internally, but typical Matroska video files are stored with 1ms accuracy.
(It is fully adjustable for what is needed.) Granted this will produce
timecodes that have error of <1ms from what was originally used (Ex: 5370 *
30000/1001 for the timecode of frame 5370). But keep in mind that the
effect is not cumulative. Each frames timecode is calculated individually.
So no timecode would have an error that is more than <1ms for typical 1ms
accuracy files.
While the idea of introducing a level of 'error' into the timecode can seem
like a bad thing, having a frame off by 0.5ms is less than the total time
that it would be off due to typical display processes. There are also some
advantages.
VFR (Variable Frame Rate). Vorbis is Xiph's free general audio codec. Ogg
was designed for it, but it is also usable in Matroska. The number of
samples in a Vorbis packet are variable. One might have 1000 samples and
another might have 300. (I can't recall real numbers you would see.) So the
length of time that one packet covers is not the same as another. This
introduces issues in systems that demand a constant rate of packets, or
CFR(Constant Frame Rate). Because Matroska simply attaches a timecode to
the packet, this is not an issue.
Now to extend this idea a little further. This may require a bit of thought
as the world of video has been a CFR place for so long that it is difficult
for people to grasp the idea that it could be done differently. Because of
timecodes, we don't have to be dependant on a specific framerate. Instead
we can record video frames as they come in. If one video frame lasts 37ms,
and the next lasts 283ms, its not an issue. This also gives you the option
that if you have a very low motion scene in video, you could dither the
frames together and instead of storing them at 30fps, you could store them
at 23fps. Completely optional depending on your codec and the level of
compression you are seeking.
This also gives many options for authoring as well. If you want to speed up
a sequence of frames or slow them down, there is no reason to reencode them
and lose efficiency. Simply adjust the timecodes to put the frames closer
together or further apart. I have a tool that lets me edit Matroska files
by cutting, offsetting audio/video, and appending streams. Because all of
the data is at the container level, the tool is able to do this no matter
what codec was used on the audio or video.
This is also a huge boon in the ability to maintain audio and video
synchronization in the event of an error. If there is data corruption in a
Matroska file, the next audio and video packets to be found will have
timecodes. Instead of the application having to guess at the timing of the
packets relative to each other, it can instead know for certain because it
knows exactly when a packet should be displayed or played. While I have
seen many AVI and MPEG files lose A/V sync for various reasons, I have never
seen this happen to a Matroska file.
Here is an example video sequence that shows the exact same sequence of
frames, but with the timecodes adjusted so that the video moves at different
speeds.
http://commo.de/3blocks-vfr.mkv
Paul Bryson
More information about the Matroska-devel
mailing list