[matroska-devel] Re: EDC/ECC

Steve Lhomme steve.lhomme at free.fr
Tue Feb 18 17:30:10 CET 2003


En réponse à thana <thanatos666 at gmx.net>:

> hi again..

Hi thana,
 
> so far the best FEC (forward error correction) algorithm still seems to
> be
> the Reed-Solomon code that is used on CD for EDC and ECC (CIRC - Cross
> Interleaved Reed-Solomon Code) and also on DVD (RSPC - Reed Solomon
> Product

Did you check the webpages of Frank Klemm that deal with FEC ? I'm not sure he's
using the RS algorithm.

> network EDC-checking should be enough, so that corrupted parts gets
> redownloaded, which requires support of the p2p-program - but i think
> this
> has already been mentioned some times here.

Well, there are 2 things that are handled differently. The ECC inside matroska
and the ECC outside matroska.

In the case of a P2P download, for the P2P program to be able to download a
corrupted part it only needs EDC from the matroska file and the portion of data
to download. If internal ECC is used, you can reconstruct the data if they are
corrupted (that's why keeping the EDC is a good idea), but you'll never have
100% recovery. I'm not sure if ECC is necessary in this case.

The ECC outside matroska is dealt at the transport level : HD, TCP stream, UDP
stream, etc. Some of them (like UDP) are not very reliable. And they all have
different unreliability characteristics (UDP is more subject to data loss than
data corruption, unlike the other ones). That's why it's not covered by the
matroska specs.

We've started discussing about a UDP transport layer based on EBML + ECC + EDC
but we have freezed it for now because that's a huge project.
 
> a good thing of RS-code is the ability to scale quite well.. you can use
> any
> arbitrary amount of parity bytes.. f.e. a 255/223 RS-code (255 bytes
> total /
> 223 bytes data / 32 bytes Parity) will give us exactly 100MB overhead on
> a
> 700MB file.. 255/251 will give only 12,5 MB overhead.. so 1 byte
> parity
> overhead equates to 2,73-3,12MB, which should be good enough regarding
> granularity to fill an XCD..

I think the system of Frank Klemm is very flexible too.

> regarding the placing of the ECC-elements in a matroska-file: should we
> put
> them after every block, every cluster or every n clusters (f.e. after
> every
> complete GOP-sequence, before the next keyframe)? or any of them
> depending
> on the size of the blocks/clusters? are there upper/lower limits on how
> big
> a block/cluster element can be? as i think an ECC-element for less data
> than
> 223 bytes would be quite inefficient..

The CRC can be placed every where in the file and applies for all the following
data at the level where it's found.
For example if you have :
Cluster
  CRC
  Timecode
  BlockGroup
   Block
end

The CRC is computed with the EBML head+data of Timecode and Blockgroup (and
everything inside).

There are virtually no limit to a Cluster, although we advise not to make it
bigger than 2MB when possible.

For the CRC, we have 2 options (at least) :
- create it at fixed position (start of a Cluster) everywhere in the file
- post process an "unprotected" file and add CRC at all key positions in the
file to fit a specified final size

The second can apply to the first file too...
The first one probably makes sense during live capture/editing. While the second
is better to prepare a file for storing/streaming. (ECC could also be a good
addition there).
http://www.matroska.org



More information about the Matroska-devel mailing list