[matroska-devel] Re: b-frames

Cyrius suiryc at yahoo.com
Sat Jun 28 14:31:21 CEST 2003


--- Steve Lhomme <steve.lhomme at free.fr> wrote:
> Moritz Bunkus wrote:
> > Heya.
> > 
> > Although I'm at home I'm still pondering the B
> frame handling. So here
> > are some of my questions.
> > 
> > 1. Let's assume you have the typical sequence of
> IBP (display order?),
> >    timestamps 0, 40 and 80ms. In order to store
> that in Matroska WITH
> >    libmatroska I have to first AddFrame(I), then
> AddFrame(P, I) and then
> >    AddFrame(B, I, P). Otherwise I don't have the
> BlockGroup for the P
> >    frame that the AddFrame(B...) needs. Correct?
> So in a Matroska file
> >    the frames would be stored as IPB.
> Yes, in coding order. That's also probably the
> preferred way for the codec.

Yup coding order (we have no choice atm anyway -
unless by cheating - as mentionned Mosu :p).

> > 2. Reading those frames and refpriorities. For a
> file containing B
> >    frames I have a MinCache of 2. I frames have a
> prio of 0, P have 1
> >    and B have a prio of 2. Correct? So let's take
> the example above. I
> Yes and no. You can also have P frames with prio of
> 0 because they are
> used the same as I frames in MPEG (for references).

One day someone told me I should read and follow the
specs instead of what he was saying (/me whistle).
So to remind you the specs :

"ReferencePriority : This frame is referenced and has
the specified cache priority. In cache only a frame of
the same or higher priority can replace this frame. A
value of 0 means the frame is not referenced."

So what does that mean ? Somewhat the contrary of what
both of you are saying here :p
First that B frames should have a priority of 0 since
they aren't referenced (it is the case in avs2matroska
Chris ;)), and that I and P frames would have a
priority of 1 or 2.
Currently when there are only I and P frames nobody
uses priorities (why ? well maybe because using the
same priority number for each frame would be enough
since in this case we only need the previous decoded
frame - whatever it may be - to be able to decode the
next one if it isn't a new I frame, so setting all
frames to a priority of 1 would just be a waste of

So let's see what is actually done with avs2matroska.
Here is the order in which frames are (display order)
(I will not use a common pattern to see what would
happen in many cases)

I1 B1 B2 P1 B3 B4 P2 P3 B5 I2 P4 ...
(what isn't common in this sequence : you find 2
consecutive P frames - P2 & P3 - while generally you
wouldn't break the BBP sequence; you find an I frame -
I2 - as forward reference of a B frame, while it seems
generally it wouldn't happen - maybe I'm wrong here)

Here is how those frames would be stored (coding
order) :

I1 P1 B1 B2 P2 B3 B4 P3 I2 B5 P4 ...

You can consider we obtained this sequence by shifting
the B frames to the right, moving the replaced frame
to the left :

I1 B1 B2 P1 B3 B4 P2 P3 B5 I2 P4

   B1 B2    B3 B4       B5       ==>
I1       P1       P2 P3    I2 P4

      B1 B2    B3 B4       B5
I1     <-P1     <-P2 P3  <-I2 P4

      B1 B2    B3 B4       B5
I1 P1       P2       P3 I2    P4

= I1 P1 B1 B2 P2 B3 B4 P3 I2 B5 P4

How does that look in a pseudo algorithm ?
Let's imagine you have a list of 2 elements, first
element is named ref1, second element is named ref2.
When you add a new element in the list
(queue_as_reference()), it replaces ref2, which
replaces ref1, which is 'lost' (we don't use it
anymore). (ref1 and ref2 will point to KaxBlockGroup
elements actually ;))
Let's say you also have a list of pending b-frames.

while(frame_received) {
  if(b-frame) {
  if(i-frame) {
  if(p-frame) {
    add_to_cluster(using ref2 as reference)

where :

flush_queued_bframes() {
  for each frame in the b-frame list {
    add_to_cluster(using ref1 and ref2 as reference)

There is still the question of what priorities to use.
Actually avs2matroska uses priorities of 0 for
B-frames (right, since they aren't referenced), and 2
for the other frames).
Is this good ?
Let's take our sequence

Frames : I1 P1 B1 B2 P2 B3 B4 P3 I2 B5 P4
Refs:    2  2  0  0  2  0  0  2  2  0  2

Cache (frames start to be rendered when it is full)
[   ,   ]
[   ,I1 ] I1, priority 2
[I1 ,P1 ] P1, priority 2
[I1*,P1 ] B1, priority 0 => can't replace a frame in
the cache, the cache is full, render the frames we
can, i.e. I1 and B1 (because it has a timestamp lower
than P1, and thanks to the referenced frames I1 and
[I1*,P1 ] B2, priority 0 => idem, render B2 (timestamp
lower than P1)
[P1*,P2 ] P2, priority 2 => replace P1 which replaces
I1* (older timestamp), render P1 since it has a
timestamp lower than P2
[P1*,P2 ] B3, priority 0 => B3 rendered
[P1*,P2 ] B4, priority 0 => B4 rendered
[P2*,P3 ] P3, priority 2 => P2 rendered
[P3*,I2 ] I2, priority 2 => P3 rendered
[P3*,I2 ] B5, priority 2 => B5 rendered
[I2*,P4 ] P4, priority 2 => I2 rendered

So frames are rendered in this order so far :
I1 B1 B2 P1 B3 B4 P2 P3 B5 I2 ...

which is good :)

Do you Yahoo!?
SBC Yahoo! DSL - Now only $29.95 per month!

More information about the Matroska-devel mailing list