[Matroska-devel] Re: Multi-Angle (summary) + Multi-resolution files

Paul Bryson paul at msn.com
Mon Sep 20 10:23:09 CEST 2004

"Steve Lhomme" wrote...
> It probably doesn't make sense to have different codec for different 
> angles (well, I think it does but let's not consider this case). So both 
> solutions are probably valid. But there is another problem. It's very 
> common on the DVD to have the menu in 4:3 and the movie in 16:9. And I'm 
> sure anyone would want to crop the content of the 16:9 part for better 
> encoding efficiency. So the question is: should we use the same video 
> track for the menu and the movie ? Or not...

I would say not if they are two different resolutions.  It is much simpler 
to use the existing tracks system.  I don't know how DVD players do it, but 
consider this image for a moment:


Here we have a file with three video streams.  The first is the menu, say 
encoded at 600x400.  The second is the movie itself, encoded at 700x240. 
The third is just there for future discussion purposes.

As the first two video streams were encoded with different resolutions, and 
likely different encoding settings, the codecprivate data will be completely 
different, even though it is the same codec being used.  Also, the output of 
the two will be dramatically different as they will be two different 
resolutions.  I have heard that you cannot make changes to the resolution of 
the video renderer once the graph is built, so I am going on that 

When the graph is built, the Video Stream Switcher connects to all of the 
video streams from the Matroska Splitter.  It then initializes a decoder for 
all of the streams using the appropriate init data.  The decoders all 
connect to the Video Converter/Resizer with their RAW video outputs.  The 
video resizer can see the maximum vertical and horizontal size required by 
all of the video streams and set the video renderer to that size.  In this 
case, 700x400.

On playback, the Video Stream Switcher only gives video packets to the 
decoder that should be being used.  The decoding codec then passes the raw 
video on to the Video Converter/Resizer which pads the video with the amount 
required to make it fit on the Video Renderer.  So while decoding the menu 
in the first stream, it would pad the video by 50 pixels of black on the 
left and the right sides.  During this time, the other two decoders sit 
doing nothing.

Once the movie is selected for playback, the Switcher start handing packets 
from the second stream to the second decoder.  The Resizer then pads the top 
and the bottom with 80 pixels each.

1. The video renderer never needs to change resolutions as the videos are 
always just padded.
2. Switching should be very fast because all of the decoders are initialized 
on start up and can start decoding video as soon as they are handed packets.
3. Only once video stream at a time getting decoded so CPU is not to high.

1. Do Decoders like sitting there without being handed any packets?
2. You have to force the Video Converter/Resizer into place, and this is 
probably best left at the playback application level.
3. Different decoders output into different colorspaces so you would 
probably have to do some colorspace conversions inside of the 

The Switcher would probably be built into the Splitter in real life.  And 
the Converter/Resizer would probably be an application thing, not its own 
filter.  It would probably also have to get info from the Switcher on which 
decoder to take data from.


More information about the Matroska-devel mailing list