[Matroska-devel] Opus in Matroksa Cont.

Steve Lhomme slhomme at matroska.org
Tue Apr 9 04:58:24 CEST 2013


Hi,

Just an addition to the conversation for now as I didn't see it mentioned.
One of the major use case of Opus (+VP8) in Matroska/Webm would be to
stream a WebRTC stream from a website as a read only video. Either live or
for later replay. That would also be a good way to store a multi party
WebRTC session in one file.

If it's live it would be important to minimize the playback delay to a
minimum. That plays against having a timecode shift applied by default.
On Apr 1, 2013 9:44 PM, "Frank Galligan" <frankgalligan at gmail.com> wrote:

>
>
>
> On Sun, Mar 31, 2013 at 7:22 AM, Steve Lhomme <slhomme at matroska.org>wrote:
>
>> Hi everyone, sorry being late to the party, I don't have much "me time"
>> lately.
>>
> me time? Never heard of it.
>
>
>>
>>
>> On Fri, Mar 22, 2013 at 10:24 PM, Frank Galligan <frankgalligan at gmail.com
>> > wrote:
>>
>>> *
>>>
>>> Open issues from [1] I want to address:
>>>
>>> 1. Seeking in Opus streams requires a pre-roll.
>>>
>>> 2. How does the OpusHead pre-skip field interact with the timestamps?
>>> *
>>>
>>
>> First question that doesn't seem to be answered from past conversations:
>> is the pre-roll a constant time, a contact packet of data or not even
>> something constant ? It seems to be a constant time, given this but I want
>> to make sure we're on the same page.
>> http://tools.ietf.org/html/draft-terriberry-oggopus-01#section-4.2
>>
>
> Pre-roll is a constant at least samples 3840. As Opus is only 48000Hz,
> this translates to at least 80 milliseconds.
> See http://tools.ietf.org/html/draft-terriberry-oggopus-01#section-4.5
>
> Also pre-roll and pre-seek were mentioned. Are they actually the same
>> thing ? I assume they are. (if not my following comments are only on
>> pre-seek)
>>
> No, they are different.
>
>>
>> If it's a constant then all timecodes of the Opus track can be shifted of
>> that value to have the data to "render" or to "decode" (so it renders
>> properly at the time needed). I think for simplicity it's easier if that
>> shift is not visible at the muxer/demuxer level. That will avoid all kinds
>> of possible issues. While shifting time at the codec level is a very simple
>> operation. So I think the cleanest way to do that is let the codec do its
>> sample/timecode shift internally, by setting the pre-roll time in the
>> CodecPrivate, with a format to decide.
>>
> It is a constant time that Opus in Ogg shifts each timestamp (granule
> pos). I agree with you that shifting all timecodes in a stream in Matroska
> at the muxer/demuxer level will lead to issues (IMO, I think I can make it
> work. But until you actually try to implement it, you are not going to know
> what the issues are)
>
> If we decide to not shift all the timestamps by a constant time (I.e.
> pre-skip ), then the decoder/encoder does not need to re-write any
> timestamps.
>
>>
>> *
>>>
>>> 2.2 The pre-skip data must be contained in the first audio Block (or
>>> maybe in the CodecPrivate) with non-pre-skip encoded data.
>>>
>>> *
>>>
>> *
>>>
>>> Pros:
>>>
>>> - No demuxer changes to handle Opus audio. Only the decoder will need to
>>> know about the pre-skip data.
>>>
>>> - Block timestamps will match decoded timestamps for all time T.
>>>
>>> - No PreSkip element in the TrackHeader. The decoder will still have
>>> pke-skip from the CodecPrivate data.
>>>
>>> - No negative timestamps.
>>>
>>> Cons:
>>>
>>> - Data may need to be added to all Opus Blocks to delimit the pre-skip
>>> data.
>>> *
>>>
>>
>> If it's just a timecode shift there is no additional data (apart from
>> when decoding starts) compared to the normal decoder flow.
>>
> In 2.2 we are not talking about a timecode shift.
>
>>
>>
>>> *
>>>
>>> - Does not help future formats that need a PreSkip.
>>> *
>>>
>>
>> If a codec needs data pre-prended or non constant pre-skip then we may
>> not have a generic enough solution anyway.
>>
>>
>>> *
>>>
>>> - Muxers may have to special case the pre-skip data.
>>> *
>>>
>>
>> I don't see how. The timecode shift is transparent outside of the codec.
>> The CodecPrivate data HAS to be passed to the codec, nothing is new here.
>>
> In 2.2 we don't have a timecode shift.  In a later email I referenced
> 2.2.2 as putting the pre-skip data in the CodecPrivate. This should work
> fine on initial playback, like other codecs, as the Opus decoder can decode
> the pre-skip data when it is initialized. But, unlike other codecs (as far
> a I know), the Opus decoder must decode the pre-skip data every time the
> player seeks to the first normal audio Block.
>
>
>>
>>> *
>>>
>>> Questions:
>>>
>>> Can the decoder assume that a packet of timestamp 0 will have to decode
>>> and throw out pre-skip samples? Then we wouldn’t need to delimit all of the
>>> Opus Blocks.
>>> *
>>>
>>
>> I think the codec should not output the pre-skip samples, ie samples
>> before 0.
>>
> I agree, but unless we change the Opus decoder (Opus experts, does the
> Opus decoder know it is decoding pre-skip data?). Or we implement something
> like 2.2.3, but then we could run into the issue that some decoders may be
> able to handle Opus data from Ogg files but not Mkv files, and vice versa.
>
>  After seeking the same should apply: samples that are in the pre-skip
>> timeframe should not be sent to the renderer as they are just there to get
>> the codec started internally and to avoid having garbage if decoding was
>> started right away.
>>
> Right, how are you specifically advocating we lay out the pre-skip data in
> a Matroska file?
>
>
>
>> --
>> Steve Lhomme
>> Matroska association Chairman
>>
>> _______________________________________________
>> Matroska-devel mailing list
>> Matroska-devel at lists.matroska.org
>> http://lists.matroska.org/cgi-bin/mailman/listinfo/matroska-devel
>> Read Matroska-Devel on GMane:
>> http://dir.gmane.org/gmane.comp.multimedia.matroska.devel
>>
>
>
> _______________________________________________
> Matroska-devel mailing list
> Matroska-devel at lists.matroska.org
> http://lists.matroska.org/cgi-bin/mailman/listinfo/matroska-devel
> Read Matroska-Devel on GMane:
> http://dir.gmane.org/gmane.comp.multimedia.matroska.devel
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.matroska.org/pipermail/matroska-devel/attachments/20130409/dd1c893e/attachment.html>


More information about the Matroska-devel mailing list