[Matroska-devel] Re: Dirac Video Codec
paul at msn.com
Tue Oct 5 04:00:03 CEST 2004
"Arioch /BDV/" wrote...
> The stars so gaily glistened... (Fri, 1 Oct 2004 11:20:40 -0500 @722)
> ...while the fading voice of Paul whispered through the darkness:
> PB> As he says, it is very useful to have as much data as possible
> PB> available at the container level for authoring purposes. This is part
> PB> of the philosophy of Matroska.
> I guess what is file size overhead here, comparing to plain MPEG or OGG or
> even AVI
There are some good overhead comparisons for AVI, OGM, and Matroska here:
MPEG is a little more difficult. I'm not sure about the MP4 container, but
preliminary results for MPEG-2 the other overhead appears to be slightly
lower in Matroska. I just remember that being mentioned and I'm afraid I
can't recall what stream type was being used or why there was lower
overhead. Perhaps spyder482 could give more information.
> PB> in a Matroska file, the next audio and video packets to be found will
> PB> have timecodes. Instead of the application having to guess at the
> PB> timing of the packets relative to each other, it can instead know for
> In other words, each packet is having non-relative timecode, which is
> again longer than relative timecode with shorter value?
> Or if it is relative - then application seems to meet almost the same
It is relative within a small temporal segment.
> If i got it, in MKV timestamps are relative to upper EBML block, so within
> one block, all sub-blocks might be taken as they were absolutely stamped,
> but will this upper-level block be damaged, then all sub-blocks inside
> will lost its timestamps (read: sync) ?
Correct. It would theoretically be possible to decode the audio and
determine the length of the audio packets, and thus what the damaged portion
All Blocks are contained in a Cluster. A Block uses a relative timecode
stored in a SINT16. This gives it a range of 65536 (+/-32k). A Cluster has
a timecode stored in UINT that can be any size. The Block's timecode is
relative to the Cluster's timecode. Because of the size of the Block's
timecode, the Cluster can have a maximum length of 65 seconds (+/- 32s) when
using the default value of 1ms for timecode accuracy. However, in
practice no tool makes a Cluster of more than a few seconds long as the
overhead benefits drop off quickly.
Also, while the timecodes would have lost their time positioning relative to
the rest of the file, they would still be completely accurate relative to
each other. It would still be possible to simply continue to play back the
Blocks in the same order until the next fully valid Cluster timecode is
> So in desync is not eliminated, it is just more varied in severity of
> damage after the random place, where the file was corrupt.
Correct. It would be possible to lose synch for a few seconds in the
extremely improbable event that only the Cluster's timecode is damaged by a
single bit flip. However, data errors tend to be more catastrophic than a
single bit flip. I would expect that in the event of an error you are
probably looking at enough data loss to take out a few Blocks or possibly
most of a Cluster. In any event, any loss of synch or playback would be
localized to the time covered by the damaged Cluster, a few seconds at most.
As a point of interest, I obtained a file that was badly damaged. I believe
it was from a P2P application where only some of the data was obtained,
resulting in a file filled mostly with zeros where there should have been
real data. The file was still playable, it simply played only the Clusters
that were actually in the file. All synch was kept.
Bit flipping is only one type of error though. There is also the
possibility that data from the file could be lost resulting in a file that
is smaller than it should be. (Data being added is possible, but much more
rare.) In the event of data being lost it is possible to lose most of the
data within the affected Cluster without doing a data analysis to find Block
headers for resync. This would also make the seek tables useless beyond the
point of the data loss. While seek in Matroska files that don't have a
usable seek table, it is much slower than with. However, remuxing of the
file along with probable recovery of most of the data in the damaged Cluster
would be trivial. Again, a few seconds lost at most.
Compare this with MP4. I am told that MP4 stores an index of frames in the
file. All it has to do is store the number of samples that the frame lasts
for (typically one sample) and the offset of the frame within the file. As
this data is highly repetitive, it can be compressed to take less space. In
the event of bit flips within the index, you could expect minor odd
behavior. The greatest would be a frame lasting for maybe 10x the amount of
time that it should and causing the rest of the stream to be that much out
of synch. However, a data loss from the index would result in catastrophic
failure of the file as all of the off the offset information would point to
the incorrect offset within the file. It is possible for the index to be
spread through an MP4 file to minimize the effects of these failures, but I
do not know that any utility yet does this.
Certainly all formats have their benefits and drawbacks, and Matroska is no
exception. But data loss is certainly not a problem for Matroska. Besides
localizing errors, Matroska also has the possibility of adding CRC32 data at
different levels. I have only seen it done at the Cluster level, but this
lets you know if a Cluster has been damaged, even if all of the data appears
to be valid. And while it has not been implemented yet, there is also the
possibility of adding error recovery data in the same way to be able to
actually recover damaged data. It is possible that the required overhead
for this could be prohibitive so as to make this a moot point. Also, most
information like the Segment Information and the Track data is repeatable
throughout the file with little impact on required overhead.
More information about the Matroska-devel