[Matroska-devel] [Cellar] ReferenceTimecode tracks in Matroska proposal

[Jerome Martinez]

Le 10/01/2016 23:57, Dave Rice a écrit :
> Hi all,
>
> Here is an early draft of Elements for managing ReferenceTimecode tracks. Matroska uses the term 'Timecode' to reference presentation timestamps, whereas other formats (QuickTime, MXF) use 'Timecode' to reference a time reference that is added to the timeline which is inpendent of the presentation (i.e. a QuickTime file with a timecode track that starts at one hour aka 1:00:00;00 would not cause a one hour delay before playback starts). In the context of Matroska I am using ReferenceTimecode (for now) to refer to QuickTime/MXF style timecode tracks.
>
> TrackType:
> Add to the TrackType definition to list "0x13: timecode,"
>
> CodecID:
> Add the following timecode Codec IDs.
> Codec ID        | Name               | Description
> T_QUICKTIME     | QuickTime Timecode | Timecode data as stored in QuickTime, e.q. tmcd. The CodecPrivate contains all additional data that is stored in the 'tcmd' (timecode) atom in the QuickTime file.
> Others? LTC? VITC?
>
> Proposed elements under TrackEntry:
>
> Element Name: ReferenceTimecode
> Level:        3
> ID:           [EF]
> Mandatory:    -
> Multiple:     -
> Default:      -
> Type:         m
> Description:  ReferenceTimecode settings
>
> Element Name: ReferenceTimecodeForm
> Level:        4
> ID:           [EF][A1]
> Mandatory:    mand.
> Multiple:     -
> Range:        0-2
> Default:      0
> Type:         u
> Description:  Indicates the form of expression used to present the timecode (using HH for hours, MM for minutes, SS for seconds, mmm for milliseconds, ff for frames).
>                0: HH:MM:SS.mmm
>                1: HH:MM:SS:ff (non-drop frame)
>                2: HH:MM:SS;ff (drop frame)

I don't see the purpose of such flag.
a time code is not a time stamp, so no "HH:MM:SS.mmm" in the stream 
directly. From my point of view, it is the role of the player to propose 
either HH:MM:SS:ff or HH:MM:SS.mmm display formats for the same content. 
the container should not contain such detail.
non-drop frame vs drop frame: it is already in the private data (not 
only for T_QUICKTIME, such flag is in all timecode formats I have ever 
seen). Is there a need to repeat it?
If you want such piece of information for being able to compute the 
timecode without the private data, you also need the frame rate (as an 
integer).

>
> Element Name: ReferenceTimecode24HourMax
> Level:        4
> ID:           [EF][A2]
> Mandatory:    mand.
> Multiple:     -
> Range:        0-1
> Default:      0
> Type:         u
> Description:  Indicates if the timecode should revert to a zero-value at the time reference of 24 hours. Set to 1 if the ReferenceTimecode should use a maximum value of 24 hours and wrap values back to zero hours at that point.

it is already in the private data. Is there a need to repeat it?

>
> For storage of ReferenceTimecode values in Blocks, I suggest storing a signed integer (ReferenceTimecode values may be negative). The stored signed integer would be the frame number

which format of the integer? EBML? if "32-bit signed integer", the issue 
is with (theoretical) huge values: if QuickTime settings are with 
NegativeTimes=0, you can not store the big numbers.

Actually, I don't like the idea to "standardize" the storage, because we 
define the CodecID as "T_QUICKTIME", so we should just raw copy the 
stream from a .mov file, without modification (and without specifying 
it, as we don't specify the AVC stream for video. Reference is QuickTime 
specification).
And with "T_QUICKTIME", we don't define our own format, so adding 
"increased by the duration of the corresponding video frame" in the 
definition does not make any sense: either we define a new timecode 
format, either we have the goal to use QuickTime format, but mixing 
"T_QUICKTIME" and our own definition will create a lot of confusion.

>   (players supporting ReferenceTimecode should convert the frame number accordingly to a drop-frame or non-drop-frame display). So if ReferenceTimecodeForm=1, the Block stores 0x00BC, and the frame duration is NTSC, then the displayed value would be 00:00:06:08. The next value 00:00:06:09, etc.
>
> I propose that one block per ReferenceTimecode not be required for simple incrementing ReferenceTimecode, but that the ReferenceTimecode value of the next video frame be considered to be either 1 greater than the previous value (if ReferenceTimecodeForm is 1 or 2) or increased by the duration of the corresponding video frame.

Actually, technically and except when the timecode is "burned" in the 
video frame, the timecode stream does not depend of any "video frame" or 
"video stream". You could have (in theory, totally useless in the 
reality, I know) a 24 fps timecode stream with a 25 fps video stream 
(actually, in .mov file, the time code stream has a frame rate defined 
by the time scale and duration of a frame, and also a "An 8-bit integer 
that contains the number of frames per second for the timecode format. 
If the time is a counter, this is the number of frames for each counter 
tick." so you can have a 24 fps time code at 30 fps beside a video 
stream at 25 fps)
So linking this way timecode stream and video stream is risky. linking 
duration ("time stamp") and timecode is also risky.

I propose to not try to "adapt" QuickTime time code, just dump it as it 
is, without trying to "limit" the possibilities offered by QuickTime: so 
a timecode stream has a value (32-bit signed or unsigned integer) with a 
timestamp (Matroska "timecode") and a duration (useful if we want to 
know if there is only one value or several values).

Example of metadata from a QuickTime file
- time scale from container (mdhd): 30000
- total duration from container (mdhd) (in time scale units): 335335
- sample duration from container (stts): 335335
- tmcd time scale: 30000
- tmcd frame duration (in time scale units): 1001
- tmcd number of frames (time code "frame rate"): 30
- tmcd drop frame flag: no
(note: I don't know the behavior of QuickTime player in case of tmcd 
time scale != mdhd time scale)

so we need to keep such data in Matroska, e.g.:
- CodecPrivate: tcmd content
- no need of extra setting in the track header (drop frame flag, 24H 
flag are in tmcd)
- Matroska TimecodeScale: 30000
- Matroska Timecode of the unique frame: 0
- Matroska BlockDuration of the unique frame: 335335
- Block content: the 4 bytes from QuickTime timecode stream.

>
> If the ReferenceTimecode is non-continuous then a new ReferenceTimecode value in a new Block with a new Timecode (Matroska meaning of Timecode) could be inserted at the point of the discontinuity.

It is not so easy to insert a new block and being sure it will be 
correctly displayed, due to interlacement (when can we consider that the 
the next block is not present?) without BlockDuration.

Jérôme