And what about my first question, what is the purpose of buffering the frames ? The frames are stored in coding order in matroska and the decoder needs the frames in this order, so you don't need to change the order... http://www.matroska.org