[Matroska-devel] Re: Dirac Video Codec

[Steve Lhomme]

Tim Borer a écrit :

> Thanks for your quick reply. It looks like Matroska might be a good 
> thing to use - though we will have to read your documentation more 
> thoroughly. I've embedded some comments below.

It's all on the website, but if you need more precision you can use our 
mailing list or our IRC channel (#matroska on irc.corecodec.com).

> I guess one thing we need to know is what we need to do to integrate 
> Dirac with Matroska. We already have a C interface for the decoder and 
> are working on one for the encoder. These are only alpha and so the 
> details will change but the principle should be about right. We have 
> modelled them on C interfaces used for codecs in other libraries (e.g. 
> ffmpeg).

For the moment there is only VirtualDubMod that can encode and save in 
matroska at the same time. You might be able to do it with GStreamer, 
but it's not (yet) guarantee to work. What we usually do it take an AVI 
file, RM or whatever and remux it to Matroska. There are 2 main tools 
for matroska that can do that :
- mkvtoolnix : http://www.bunkus.org/videotools/mkvtoolnix/
- Avi-Mux GUI : 
http://www-user.tu-chemnitz.de/~noe/Video-Zeug/AVIMux%20GUI/index-eng.html

But if you want to use some matroska specific features, you probably 
want to save to Matroska directly... You could make a plugin for 
GStreamer and use the Matroska muxer there. But as of now I'm sure the 
matroska features you need are not supported there... So the other 
option is to make a tool using your codec that can read/save natively in 
Matroska. Then mkvtoolnix and Avi-Mux GUI could be used to add extra 
features (that you probably don't want to support in an encoding tool) 
like chapters, tags, menus, etc.

As libmatroska/libebml is in C++, using the STL it can probably 
integrated easily with your code.

> At 12:23 01/10/2004 +0200, you wrote:
>  >I couldn't tell about MXF. All I know is that it's a professional 
> class container, but maybe not >good for everyday/basic use.
> 
> Its a professional container. From our point of view it has problems in 
> getting additions agreed with the standards body (SMPTE). There are also 

Yes, that's one of the problems with big systems. They are not very 
responsive and sometimes not willing to do things for reasons other than 
technical ones :( We try to stay away from such problems as possible.

> issues about the availability of open source libraries for it and their 
> limitations.
> 
> Open source, freely available, cross platform is where we want to be.
> 
> One question is can Matroska be used commercially as well as personally, 
> or are there intellectual property issues that would cause problems?

Yes, all the code we (the core team) write is either LGPL. The code of 
other people (like the C implementation in GStreamer) is usually LGPL 
too. Mkvtoolnix and AVI-Mux GUI are GPL.

About IPs, the format is free of use. We never checked for any patent. 
Maybe we infringe some, probably not. We don't have the money and/or 
time to check all patents in the world to be sure it's patent free. But 
since EBML is the basis of all we do and using a patent-free system 
(like Unicode encoding) I think it's quite safe.

>> It seems you somehow use the same usual IPB frames principle but with 
>> additions on the possibilities. That's great because the frame 
>> reference we have designed allows the possibilities you are looking 
>> for. It can take care, at the container side, about complex references 
>> that are not possible with basic IPB frames (and therefore with most 
>> existing containers, if not all).
> 
> 
> Sounds like this might be useful, although I though we needed to handle 
> this ourselves in an elementary stream to ensure that the stream could 
> be stand alone if required. Do you have a URL that would point to 

Yes. But it is very important to have this information available to the 
container (too). It's needed for precise seeking. It's also needed for 
"lossless" or "blind" editing (not modifying the codec data when 
inserting/cutting frames in a file).

> information on this? I am in the process of drafting the Dirac bit 
> stream spec (what we use at present is ad hoc), so I've been thinking 
> about these issues.

I don't think it's described well somewhere. It was done on a previous 
mailing lists which archives are not available anymore... And I never 
took the time to explain it in the specs, because it's quite a complex 
thing. Maybe it's time to do it...

The elements involved in the specs 
http://www.matroska.org/technical/specs/ :
- ReferencePriority : the priority of the element in the "cache", it can 
only replace elements of a bigger or same priority (or smaller, I never 
remember which one it is)
- ReferenceBlock : can be multiple
- MinCache
- MaxCache

>> In short, we can handle the number of frames that should be kept "in 
>> cache" and reference any frame that is in that cache during playback. 
>> The reference system is based on the timecode, to avoid problems in 
>> damaged streams. The frames have to be stored in coding order (always 
>> known before being referenced). And the seeking possibilities also 
>> take care of having the reference frames cached/seen, even though it's 
>> not displayed at the seek point.
> 
> 
> I am wary about using time code. It works well for video at 24, 25 and 
> 30 frames per second. For NTSC at 30000/1001 (aprox 29.97) frames it is 
> bodged using drop frame. There is also confusion about streams which say 
> they are 29.97 when they may mean 30000/1001. The next result is that my 
> experience is that timecode is not as reliable as I would like. 

We don't care about frame rates. There is no frame rate in Matroska. 
That allows us to support VFR (Variable Frame Rate) easily, and dropped 
frames.

> Therefore for Dirac we are planning to use sequential frame numbers 
> rather than time code. then you can keep accurate track of frames and 
> convert to a time code if required. What is you opinion on this?

Well. My opinion is that a video codec should not have any knowledge 
about time. But in the other hand it's now needed for efficiency, 
because codecs now use it for motion compensation and stuff like that...

In the other hand it's only needed at encoding time, not decoding (but 
maybe I'm wrong). So from the codec side, you only need to know which 
frame is used as a reference and which references a frame uses... That's 
the kind of thing that the container can give you, because it has to 
have such info (for reasons mentioned above like seeking and editing)... 
But I understand that the timecode format may not be efficient and 
usefull for you. Especially is the data are shifted in time for whatever 
reasons. So it makes sense for you to have your own reference system... 
And that's what MPEG codecs do when saved in Matroska.

I suggest that your internal system should be designed to allow removing 
some frames from the stream and still be playable. You might want to 
keep a time reference to support VFR. So that when removing frames, you 
can still know which timecode you need to get for a reference frame...

When saved in Matroska such a stream that would be edited (or decimated) 
would keep the needed frames intact (even if they shouldn't be 
displayed) and would feed the decoder with the right (only needed) 
frames when playing and/or seeking.