[Matroska-devel] Re: Dirac Video Codec

Paul Bryson paul at msn.com
Tue Oct 5 04:00:03 CEST 2004


"Arioch /BDV/" wrote...
> The stars so gaily glistened... (Fri, 1 Oct 2004 11:20:40 -0500 @722)
> ...while the fading voice of Paul whispered through the darkness:
> PB> As he says, it is very useful to have as much data as possible
> PB> available at the container level for authoring purposes.  This is part
> PB> of the philosophy of Matroska.
>
> I guess what is file size overhead here, comparing to plain MPEG or OGG or 
> even AVI

There are some good overhead comparisons for AVI, OGM, and Matroska here:
http://www-user.tu-chemnitz.de/~noe/Video-Zeug/AVIMux%20GUI/en_overhead_comparison.html
and here:
http://www-user.tu-chemnitz.de/~noe/Video-Zeug/containers.pdf

MPEG is a little more difficult.  I'm not sure about the MP4 container, but 
preliminary results for MPEG-2 the other overhead appears to be slightly 
lower in Matroska.  I just remember that being mentioned and I'm afraid I 
can't recall what stream type was being used or why there was lower 
overhead.  Perhaps spyder482 could give more information.

> PB> in a Matroska file, the next audio and video packets to be found will
> PB> have timecodes.  Instead of the application having to guess at the
> PB> timing of the packets relative to each other, it can instead know for
>
> In other words, each packet is having non-relative timecode, which is 
> again longer than relative timecode with shorter value?
> Or if it is relative - then application seems to meet almost the same 
> troubles.

It is relative within a small temporal segment.

> If i got it, in MKV timestamps are relative to upper EBML block, so within 
> one block, all sub-blocks might be taken as they were absolutely stamped, 
> but will this upper-level block be damaged, then all sub-blocks inside 
> will lost its timestamps (read: sync) ?

Correct.  It would theoretically be possible to decode the audio and 
determine the length of the audio packets, and thus what the damaged portion 
should be.

All Blocks are contained in a Cluster.  A Block uses a relative timecode 
stored in a SINT16.  This gives it a range of 65536 (+/-32k).  A Cluster has 
a timecode stored in UINT that can be any size.  The Block's timecode is 
relative to the Cluster's timecode.  Because of the size of the Block's 
timecode, the Cluster can have a maximum length of 65 seconds (+/- 32s) when 
using the default value of 1ms for timecode accuracy.    However, in 
practice no tool makes a Cluster of more than a few seconds long as the 
overhead benefits drop off quickly.

Also, while the timecodes would have lost their time positioning relative to 
the rest of the file, they would still be completely accurate relative to 
each other.  It would still be possible to simply continue to play back the 
Blocks in the same order until the next fully valid Cluster timecode is 
reached.

> So in desync is not eliminated, it is just more varied in severity of 
> damage after the random place, where the file was corrupt.

Correct.  It would be possible to lose synch for a few seconds in the 
extremely improbable event that only the Cluster's timecode is damaged by a 
single bit flip.  However, data errors tend to be more catastrophic than a 
single bit flip.  I would expect that in the event of an error you are 
probably looking at enough data loss to take out a few Blocks or possibly 
most of a Cluster.  In any event, any loss of synch or playback would be 
localized to the time covered by the damaged Cluster, a few seconds at most.

As a point of interest, I obtained a file that was badly damaged.  I believe 
it was from a P2P application where only some of the data was obtained, 
resulting in a file filled mostly with zeros where there should have been 
real data.  The file was still playable, it simply played only the Clusters 
that were actually in the file.  All synch was kept.

Bit flipping is only one type of error though.  There is also the 
possibility that data from the file could be lost resulting in a file that 
is smaller than it should be.  (Data being added is possible, but much more 
rare.)  In the event of data being lost it is possible to lose most of the 
data within the affected Cluster without doing a data analysis to find Block 
headers for resync.  This would also make the seek tables useless beyond the 
point of the data loss.  While seek in Matroska files that don't have a 
usable seek table, it is much slower than with.  However, remuxing of the 
file along with probable recovery of most of the data in the damaged Cluster 
would be trivial.  Again, a few seconds lost at most.

Compare this with MP4.  I am told that MP4 stores an index of frames in the 
file.  All it has to do is store the number of samples that the frame lasts 
for (typically one sample) and the offset of the frame within the file.  As 
this data is highly repetitive, it can be compressed to take less space.  In 
the event of bit flips within the index, you could expect minor odd 
behavior.  The greatest would be a frame lasting for maybe 10x the amount of 
time that it should and causing the rest of the stream to be that much out 
of synch.  However, a data loss from the index would result in catastrophic 
failure of the file as all of the off the offset information would point to 
the incorrect offset within the file.  It is possible for the index to be 
spread through an MP4 file to minimize the effects of these failures, but I 
do not know that any utility yet does this.

Certainly all formats have their benefits and drawbacks, and Matroska is no 
exception.  But data loss is certainly not a problem for Matroska.  Besides 
localizing errors, Matroska also has the possibility of adding CRC32 data at 
different levels.  I have only seen it done at the Cluster level, but this 
lets you know if a Cluster has been damaged, even if all of the data appears 
to be valid.  And while it has not been implemented yet, there is also the 
possibility of adding error recovery data in the same way to be able to 
actually recover damaged data.  It is possible that the required overhead 
for this could be prohibitive so as to make this a moot point.  Also, most 
information like the Segment Information and the Track data is repeatable 
throughout the file with little impact on required overhead.


Atamido 






More information about the Matroska-devel mailing list