[Matroska-devel] EBML buffer construction
mike at haali.net
Thu Nov 15 00:06:42 CET 2007
Yann Renard wrote:
> As some of you may know, I have decided to use EBML in my current
> project so to encode several types of stream. We will later use matroska
> as a container for our dumped streams. However, my implementation of the
> EBML spec for the buffer construction part is basically based on a
> single callback taking a void* pointer and an integer size with the
> bytes to append to the buffer. The EBML spec encodes a node in three
> parts : an identifier, the node size and the node data or children. The
> two first parameters are encoded on a variable length, depending on
> their value. The total size of a node can only be computed when all of
> its children sizes are known. Thus the buffer construction can only be
> performed when the top most node is closed. Now, when this particular
> node is closed, the buffer construction starts, calling repeatedly the
> callback function, first with the first node's id, the first node size,
> and so on for its children...
> What I want to discuss with you tonight, is a way to improve this buffer
> construction. First, I would like to avoid so many calls to the callback
> function. For this to be done, the writer could prepare an internal
> buffer which is finally sent in a single call to the callback. Second, I
> would like it to be as fast as possible, thus trying to avoid the
> internal bufferization. Problem is I can not guess the total size of the
> buffer a priori.
> Do any of you have any impressions to share about this ?
In Matroska the data is split into reasonably sized clusters that can be
held in memory. Alternatively you can use an inefficient, but fixed size
length encoding, and the topmost element (Segment) can have an undefined
length that extends until the end of file. Choose the method that you like
best and implement it. My matroska writer uses a combination of these
methods, clusters + a fixed size length of the segment (not undefined though).
More information about the Matroska-devel