From joranopi at hotmail.com Wed Sep 3 21:12:48 2003 From: joranopi at hotmail.com (Chris McMillian) Date: Wed, 03 Sep 2003 15:12:48 -0400 Subject: [Matroska-devel] Anamorphic Encoding Message-ID: An HTML attachment was scrubbed... URL: From chris at matroska.org Wed Sep 3 21:21:11 2003 From: chris at matroska.org (Christian HJ Wiesner) Date: Wed, 03 Sep 2003 21:21:11 +0200 Subject: [Matroska-devel] Re: [Matroska-users] Anamorphic Encoding In-Reply-To: References: Message-ID: <3F563F27.8030507@matroska.org> Chris McMillian wrote: > Well I went into the Matroska tag to try and change the video size > and it said no EBML header found. Where am I going wrong? Am I > supposed to specify in VDubMod the desired size, or after? Anyhelp > you could give I would greatly app reciate. Keep up the good work! Use Jory 'jcsston' Stone's great matroska shell extension from http://matroska.free.fr/downloads/cdl , called 'matroskaprop_xxx' , right click on your file, Properties, 'set display size' and enter the desired display size. Christian From moritz at bunkus.org Thu Sep 4 10:18:11 2003 From: moritz at bunkus.org (Moritz Bunkus) Date: Thu, 4 Sep 2003 10:18:11 +0200 Subject: [Matroska-devel] libebml 0.5.2 Message-ID: <20030904081811.GT27554@bunkus.org> Hi. I'm ill at the moment, so don't expect me to pop up in IRC today, maybe not even tomorrow. I've just browsed through the conversations quickly, and I'm absolutely fine with releasing 0.5.2. -- ==> Ciao, Mosu (Moritz Bunkus) From steve.lhomme at free.fr Thu Sep 4 19:47:19 2003 From: steve.lhomme at free.fr (Steve Lhomme) Date: Thu, 04 Sep 2003 19:47:19 +0200 Subject: [Matroska-devel] libebml 0.5.2 In-Reply-To: <20030904081811.GT27554@bunkus.org> References: <20030904081811.GT27554@bunkus.org> Message-ID: <3F577AA7.5000306@free.fr> Moritz Bunkus wrote: > Hi. > > I'm ill at the moment, so don't expect me to pop up in IRC today, maybe > not even tomorrow. I've just browsed through the conversations quickly, > and I'm absolutely fine with releasing 0.5.2. OK, I just tagged the release and made the tar.gz and it's now in http://matroska.free.fr/downloads/libebml/ From moritz at bunkus.org Sun Sep 7 00:36:20 2003 From: moritz at bunkus.org (Moritz Bunkus) Date: Sun, 7 Sep 2003 00:36:20 +0200 Subject: [Matroska-devel] new codecids Message-ID: <20030906223620.GU27554@bunkus.org> Heya. As some of you may be aware I'm coding a Quicktime reader at the moment. It works so far, and now seems a good time to talk about the CodecIDs and PrivateData structs to use. (For those who don't know how QuickTime is layed out: All data is stored in atoms. Each atom has a size, a four byte identifier which is usually something like 'vide', 'moov' etc and contents - either other atoms or data. All atoms are in one big tree, but there are more than one level 0 atoms. Just like Matroska ;)) 1) Video This is the more complex thingy. The QuickTime DLLs seem to need the contents of one complete atom, the 'stsd' atom (stream sample description). This atom contains the following struct: typedef struct { uint32_t size; char fourcc[4]; uint8_t reserved[6]; uint16_t data_reference_index; uint16_t version; uint16_t revision; uint32_t vendor; uint32_t temporal_quality; uint32_t spatial_quality; uint16_t width; uint16_t height; uint32_t horizontal_resolution; // 32bit fixed-point number uint32_t vertical_resolution; // 32bit fixed-point number uint32_t data_size; uint16_t frame_count; char compressor_name[32]; uint16_t depth; uint16_t color_table_id; } video_stsd_atom_t; This structure might be followed by codec specific initialization data (similar to WMV9-in-AVI). This is the case for size > sizeof(video_stsd_atom_t). Yes, the complete structure is needed. The fourcc field contains the - surprise! - FourCC of the codec used, e.g. 'SVQ3' for Sorenson 3 which is used most often in current QuickTime files. For this reason I suggest we do something similar to what we've done with AVI and RM - we use some 'compatibility mode'. My proposed CodecID for this is V_QUICKTIME. The complete structure including the optional codec initialization data is stored in CodecPrivate. This is basically how it's done for AVI and RM, too. Endianess: QuickTime uses big endian numbers. We should NOT convert the structure's members in any way. Again, this is how we've done it for AVI and RM. 2) Audio This case is a bit simpler. The audio codecs don't need such a structure, but the codec might still have some initialization data. This initialization data should be stored in CodecPrivate, cause that's what it's there for ;) As the audio codecs don't need such structures we don't need a compatibility mode for audio. We should therefore use separate CodecIDs for separate codecs. The most common codecs for QuickTime are MP3 which we already cover and the two QDesign codecs, FourCC QDMC and QDM2 ('QDesign Music' and 'QDesign Music V2'). I don't have any other sample files (well I have some with uncompressed audio, but we've already covered that) atm, so I suggest to use the following CodecIDs for them: QDMC: A_QDESIGNMUSIC QDM2: A_QDESIGNMUSIC/V2 Or something similar. At the moment I only have some basic support for QuickTime (well, not SOOOO basic anymore, e.g. I can handle compressed headers), but some trailers can already be converted to Matroska, and my local mplayer version can play those ;) I'd like some feedback :) -- ==> Ciao, Mosu (Moritz Bunkus) From chris at matroska.org Sun Sep 7 11:46:37 2003 From: chris at matroska.org (Christian HJ Wiesner) Date: Sun, 07 Sep 2003 11:46:37 +0200 Subject: [Matroska-devel] CHIP article about TCMP and matroska Message-ID: <3F5AFE7D.8040300@matroska.org> Hi, just informing you guys that the CHIP article is up now, in latest issue 10 / 2003. I was scanning it and uploaded to http://www.wiesneronline.net/downloads/matroska/Chip102003 Its in German, but pls. dont expect me to have the time to translate it. For matroska the relevant page is nr. 7, but it was mentioned in many other pages of the player comparison also, the players even got minus points if OGM and MKV were not supported :) ! Enjoy ! Christian P.S. I was buying 2 issues of the magazine ( one for robux4 ), but both had no CD coming with it :( From chris at matroska.org Tue Sep 9 14:38:36 2003 From: chris at matroska.org (Christian HJ Wiesner) Date: Tue, 09 Sep 2003 14:38:36 +0200 Subject: [Matroska-devel] Next steps for matroska - where do we go from now ?? Message-ID: <3F5DC9CC.7000604@matroska.org> HI guys, vacation is over :P !! Everybody wake up and start working again :D Here some ideas where we should progress from where we are : 1. EDC / ECC / mode2 form2 / XCD / hardware profiles http://forum.doom9.org/showthread.php?s=&postid=369721#post369721 . I am writing in this thread : ' For MKV hardware support : One of the next things we will focus on is the inherent error detection and correction in matroska files for enhanced mode2 form2 / XCD support, and the same tools will have an automatic feature integrated to reorder the elements in matroska files such that hardware devices will have a much easier life reading them. In short, the MKV files will be written in a new, mandatory way for improved hardware support, and this element order will be a part of the matroska hardware profiles specs ....' 2. Image subtitles, compressed and uncompressed My contact with the PNG/MNG mailing list ( most of the emails were at least copied to the matroska lists, i hope you realized the origin ) had as a result that maybe the speed advantage of lzo was only because it has no CRC32 routines coming with it by default, while all the other standard compression libs had, so maybe Mosu's test was not really fair. I have no idea how to disable CRC32 for the other libs, you guys will know. In any case, this is a great new feature we should implement asap IMHO, users are waiting for it and keen on using it, and it would underline our enhanced subs support capabilities. 3. FLAC / Wavpack ( lossless audio ) support With VirtualVCR-matroska from Belgabor and AVT2000 from Czechia there are 2 capturing apps now that will allow capturing into MKV files, and i get positive replies from all sides about matroska's very good performance for capturing, especially if compression is used during capturing ( uncompressed files result in LOOOONG parsing times in VdubMod ) and the capture is running for several hours. FLAC or Wavpack support would offer new possibilities here, and stregthen our 'market position' for the analog capture people IMHO. 4. Menues Yes, we should get that going and nothing can stop matroska anymore !!!! Ideas, suggestions ? Christian From chris at matroska.org Tue Sep 9 15:51:34 2003 From: chris at matroska.org (Christian HJ Wiesner) Date: Tue, 09 Sep 2003 15:51:34 +0200 Subject: [Matroska-devel] Re: [Matroska-general] Next steps for matroska - where do we go from now ?? In-Reply-To: <3F5DD200.6050202@faireal.net> References: <3F5DC9CC.7000604@matroska.org> <3F5DD200.6050202@faireal.net> Message-ID: <3F5DDAE6.8070702@matroska.org> Liisachan wrote: > Christian HJ Wiesner wrote: > >> 2. Image subtitles, compressed and uncompressed > > This shoud be a long-wated feature :) :) .. thats what i think also ! >> 3. FLAC / Wavpack ( lossless audio ) support > > Any eta for MusePack support? Many people would be happy with .mpc in > Matroska, i think > > *Liisachan* According to Frank Klemm, the main musepack developer, we should not try to implement MPC SV7 into MKV. Unfortunately progress on SV8 seems to have come to a halt :( . No idea how to handle this wicked situation now. We certainly dont want to act against Frank's advice, but we want MPC in MKV so very badly .... Christian From chris at matroska.org Tue Sep 9 16:02:21 2003 From: chris at matroska.org (Christian HJ Wiesner) Date: Tue, 09 Sep 2003 16:02:21 +0200 Subject: [Matroska-devel] new codecids In-Reply-To: <20030906223620.GU27554@bunkus.org> References: <20030906223620.GU27554@bunkus.org> Message-ID: <3F5DDD6D.607@matroska.org> Moritz Bunkus wrote: >Heya. > >As some of you may be aware I'm coding a Quicktime reader at the >moment. It works so far, and now seems a good time to talk about the >CodecIDs and PrivateData structs to use. > > Will a MP4 parser be resulting from this also maybe ??? >(For those who don't know how QuickTime is layed out: All data is stored >in atoms. Each atom has a size, a four byte identifier which is usually >something like 'vide', 'moov' etc and contents - either other atoms or >data. All atoms are in one big tree, but there are more than one level 0 >atoms. Just like Matroska ;)) > LOL !! How can we prove Steve and Frank had no idea about how MOV is working when designing MKV :D ?? >1) Video >This is the more complex thingy. The QuickTime DLLs seem to need the >contents of one complete atom, the 'stsd' atom (stream sample >description). This atom contains the following struct: >typedef struct { > uint32_t size; > char fourcc[4]; > uint8_t reserved[6]; > uint16_t data_reference_index; > uint16_t version; > uint16_t revision; > uint32_t vendor; > uint32_t temporal_quality; > uint32_t spatial_quality; > uint16_t width; > uint16_t height; > uint32_t horizontal_resolution; // 32bit fixed-point number > uint32_t vertical_resolution; // 32bit fixed-point number > uint32_t data_size; > uint16_t frame_count; > char compressor_name[32]; > uint16_t depth; > uint16_t color_table_id; >} video_stsd_atom_t; >This structure might be followed by codec specific initialization data >(similar to WMV9-in-AVI). This is the case for size > >sizeof(video_stsd_atom_t). >Yes, the complete structure is needed. The fourcc field contains the - >surprise! - FourCC of the codec used, e.g. 'SVQ3' for Sorenson 3 which >is used most often in current QuickTime files. >For this reason I suggest we do something similar to what we've done >with AVI and RM - we use some 'compatibility mode'. My proposed CodecID >for this is V_QUICKTIME. The complete structure including the optional >codec initialization data is stored in CodecPrivate. This is basically >how it's done for AVI and RM, too. > > Yes, this was always the plan !! I 100% agree .... >Endianess: QuickTime uses big endian numbers. We should NOT convert the >structure's members in any way. Again, this is how we've done it for AVI >and RM. > Agreed >2) Audio >This case is a bit simpler. The audio codecs don't need such a >structure, but the codec might still have some initialization data. This >initialization data should be stored in CodecPrivate, cause that's what >it's there for ;) >As the audio codecs don't need such structures we don't need a >compatibility mode for audio. We should therefore use separate CodecIDs >for separate codecs. >The most common codecs for QuickTime are MP3 which we already cover and >the two QDesign codecs, FourCC QDMC and QDM2 ('QDesign Music' and >'QDesign Music V2'). I don't have any other sample files (well I have >some with uncompressed audio, but we've already covered that) atm, so I >suggest to use the following CodecIDs for them: >QDMC: A_QDESIGNMUSIC >QDM2: A_QDESIGNMUSIC/V2 >Or something similar. > > Here my recommendation : If there are more audio formats used in QT than these 2, and if those are ONLY used in QT, i vote for not giving them independent codec IDs but to use a QT compatibility mode for audio also. If only MP3 and the formats above are possible in QT files, and if any of the used audio formats can be appearing in other containers/sources as well, specific codec IDs are the way to go IMHO. >At the moment I only have some basic support for QuickTime (well, not >SOOOO basic anymore, e.g. I can handle compressed headers), but some >trailers can already be converted to Matroska, and my local mplayer >version can play those ;) I'd like some feedback :) > /me gives some Feedback to Mosu, in a box with some nice decoration .... :D .. :P LOL Christian From spyder at matroska.org Tue Sep 9 16:51:18 2003 From: spyder at matroska.org (John Cannon) Date: Tue, 9 Sep 2003 09:51:18 -0500 Subject: [Matroska-devel] new codecids References: <20030906223620.GU27554@bunkus.org> <3F5DDD6D.607@matroska.org> Message-ID: <000901c376e1$d47432d0$01c8c8c8@JOHNC2> I agree with everything but the last suggestion by Chris. If the codecs have any chance of being decoded without quicktime's framework, then we should make the ID's independant of QT. Everything else seems right to me. Good work BTW :) Spyder From jcsston at toughguy.net Tue Sep 9 17:30:56 2003 From: jcsston at toughguy.net (Jory) Date: Tue, 9 Sep 2003 10:30:56 -0500 Subject: [Matroska-devel] Next steps for matroska - where do we go from now?? References: <3F5DC9CC.7000604@matroska.org> Message-ID: <001201c376e7$e22c68b0$0200a8c0@jcsston> > 3. FLAC / Wavpack ( lossless audio ) support > > With VirtualVCR-matroska from Belgabor and AVT2000 from Czechia there > are 2 capturing apps now that will allow capturing into MKV files, and i > get positive replies from all sides about matroska's very good > performance for capturing, especially if compression is used during > capturing ( uncompressed files result in LOOOONG parsing times in > VdubMod ) and the capture is running for several hours. FLAC or Wavpack > support would offer new possibilities here, and stregthen our 'market > position' for the analog capture people IMHO. > In case anybody didn't already know, I did add FLAC muxing to VDubMOD via libflac. But it was buggy and slow because the only way to extract the frames with libflac was to feed the decoder one byte at a time, storing the data start and length whenever a decompressed frame was produced. On top of that the timecodes were not always correct, some files would play too fast others too slow. Same encode settings, same channel, sample rate, bit-depth. So I started a flac parser from scratch and made good progress until I hit the UTF encoded integers. http://flac.sourceforge.net/format.html I have the code in the coreflac CVS if anybody wants to look at it. http://corecodec.org/projects/coreflac/ See Ya, Jory From moritz at bunkus.org Wed Sep 10 13:26:35 2003 From: moritz at bunkus.org (Moritz Bunkus) Date: Wed, 10 Sep 2003 13:26:35 +0200 Subject: [Matroska-devel] new codecids In-Reply-To: <000901c376e1$d47432d0$01c8c8c8@JOHNC2> References: <20030906223620.GU27554@bunkus.org> <3F5DDD6D.607@matroska.org> <000901c376e1$d47432d0$01c8c8c8@JOHNC2> Message-ID: <20030910112635.GD1946@bunkus.org> Hi. After some more work on the stuff I found out that for the audio codecs mentioned, QDesignMusic v1 and v2, the private data contains QuickTime like structs as well. Therefore I'll go for... A_QUICKTIME/QDMC and A_QUICKTIME/QDM2 This is how we've done it for RealAudio as well. -- ==> Ciao, Mosu (Moritz Bunkus) From moritz at bunkus.org Wed Sep 10 13:41:38 2003 From: moritz at bunkus.org (Moritz Bunkus) Date: Wed, 10 Sep 2003 13:41:38 +0200 Subject: [Matroska-devel] new codecids In-Reply-To: <20030910112635.GD1946@bunkus.org> References: <20030906223620.GU27554@bunkus.org> <3F5DDD6D.607@matroska.org> <000901c376e1$d47432d0$01c8c8c8@JOHNC2> <20030910112635.GD1946@bunkus.org> Message-ID: <20030910114138.GE1946@bunkus.org> Hi. Codec specs page updated. -- ==> Ciao, Mosu (Moritz Bunkus) From tronic2 at sci.fi Wed Sep 10 15:46:40 2003 From: tronic2 at sci.fi (=?ISO-8859-1?Q?Lasse_K=E4rkk=E4inen_/_Tronic?=) Date: Wed, 10 Sep 2003 13:46:40 +0000 Subject: [Matroska-devel] Next steps for matroska - where do we go from now ?? In-Reply-To: <3F5DC9CC.7000604@matroska.org> References: <3F5DC9CC.7000604@matroska.org> Message-ID: <3F5F2B40.6080302@sci.fi> > ' For MKV hardware support : One of the next things we will focus on is > the inherent error detection and correction in matroska files for > enhanced mode2 form2 / XCD support, and the same tools will have an > automatic feature integrated to reorder the elements in matroska files > such that hardware devices will have a much easier life reading them. In > short, the MKV files will be written in a new, mandatory way for > improved hardware support, and this element order will be a part of the > matroska hardware profiles specs ....' Problems with Mode2 in general: -Not compatible with anything, without adding special device drivers or other software that accesses hardware on low level. -Not really required, because burnable DVD is just around the corner and already a lot cheaper than CD-Rs. Additional problems with Mode2 extra ECC: -Even less different from ISO CDROM (Mode1) than pure Mode2, but still completely incompatible with it, possibly even totally unusable on some hardware. -Very strong (much stronger than Mode1) still isn't secure enough, because a scratch on the disc headers makes the entire disc unreadable (this happens!). Because of this, I propose that absolutely no work is done on extending XCD any further. If you were talking about XCD features, you got the wrong mailing-list. But I guess it was about Matroska, so ... Additional problems, when you do that at file level (Matroska), instead of disc level (XCD): -Cannot take full advantage of the ECC already present on the lower layers (because files are NOT supposed to know where they are stored and how) -Not automatically removed or changed when the media is changed. Transmitting the file over a radio link with no error-handling would require the file to have very strong ECC and transmitting over TCP/IP doesn't have use for any. -Intelligent remuxer required for changing the ECC on a file (at least with Matroska; MCF *might* be able to do this will less changes, but it still is painful). Because of this, I propose that no containers (movies or otherwise) should use any ECC at all. This belongs to lower levels (hardware and the very lowest level of software). > 2. Image subtitles, compressed and uncompressed Why uncompressed too? Btw, technically they are titles, not subtitles ;) > My contact with the PNG/MNG mailing list ( most of the emails were at > least copied to the matroska lists, i hope you realized the origin ) had > as a result that maybe the speed advantage of lzo was only because it > has no CRC32 routines coming with it by default, while all the other > standard compression libs had, so maybe Mosu's test was not really fair. > I have no idea how to disable CRC32 for the other libs, you guys will know. > In any case, this is a great new feature we should implement asap IMHO, > users are waiting for it and keen on using it, and it would underline > our enhanced subs support capabilities. CRC32 can be calculated at several hundred megaoctets per second (257 Mo/s on 1.1 GHz Duron with SDR memory, 423 Mo/s on 1.8 GHz P4-M with DDR memory). This cannot be a limiting factor. In case you really had some trouble decompressing PNG fast enough, I recommend checking out the code for bugs. I have really hard time believing it could be nothing more than that.. Remember also that the algorithms used in many open-source compression libs are only meant to be educational and thus are extremely slow. Try to ask the developers about the speed. > 4. Menues I still recommend using the framework designed for MCF as basis. It needs work, but is half done! It's like a simple programming language and can do much more than very simple menus. http://mcf.sourceforge.net/oldpage/trackt.htm#0x00 http://mcf.sourceforge.net/oldpage/control.htm Changes to that system are accepted and if they still follow MCF design guidelines (most importantly: simple extendable solution is better than a bucketful of nifty features with complex structure), will be used in MCF too (and we all want compatibility, right?). No matter which menu system you are using, you must support either and both of the following UI options: -Four arrow buttons + OK button (and no, mouse emulation is not OK) -Mouse or other pointing device with one button/like (must work like a mouse is supposed to, "four arrows" emulation is not OK) Additionally, this should be supported in most menus: -[1-9, 0] numbered menu items, that are instantly selected when the number is clicked on. 0 should always be a "up a level" operation. The menu option is not only selected, but also instantly activated on numeric input. If too many items, only the first 9 and the "return" menu can be selected via these hotkeys! -Cancel button: "up a level" (maybe ESC button on PCs?) No binding of other buttons and especially no binding of other mouse buttons. This cannot work in a cross-platform way. Numeric/text input fields might be an exception, but I don't think we can really support those yet (cross-platform again; the profiles WON'T solve this). This is not actually related to this, but also "top menu" etc. special shortcut buttons should be supported. It is not related to menues because it actually belongs to chapters. If there are no chapters, "top menu" should jump to the beginning. - Tronic - From moritz at bunkus.org Thu Sep 11 21:41:35 2003 From: moritz at bunkus.org (Moritz Bunkus) Date: Thu, 11 Sep 2003 21:41:35 +0200 Subject: [Matroska-devel] VobSub in Matroska Message-ID: <20030911194134.GF31148@bunkus.org> Heya, another RFC (request for comments). Preface ------- Today's topic: VobSub embedding in Matroska. NOT general image subs! VobSub is the 'raw' subtitle stream from the DVD enriched with two additional files. Usually there are three files with the extensions .sub (subtitle data stream), .idx (index file with some settings and the timecodes), .ifo (dunno...). The .ifo may be absent if I recall correctly. Actually I don't know all that much about this format, but as Gabest is not here atm... Anyway. Parts of the .idx file and the complete .ifo file belong into CodecPrivate while the contents of the .sub file are put into blocks, of course. The .idx file ------------- This is a text file that contains comments (# bla bla...), settings (alpha: 100%) and index entries (timestamp: 00:02:53:873, filepos: 000002000). We need the index entries for reading the .sub file, but we don't have to keep those in the Matroska file. Therefore we can keep the comments and the setting lines but discard the index lines. The .ifo file ------------- This is a binary file. It should be kept intact. CodecPrivate ------------ We do something similar to how Vorbis stores its three header packets in CodecPrivate: We use a lace like structure. First byte is the number of files in CodecPrivate - 1. If the .ifo file is present, this byte is 1 (because there are two files), if the file ain't present, then it is 0 (because there's only the stripped down .idx). The following bytes are lace codec lengths of the .idx file, but only if the .ifo file exists. You can always get the length of the last file (.ifo if .ifo exists, .idx otherwise) by looking at the length of the complete CodecPrivate data. Next is the contents of the stripped down .idx file followed by the contents of the .ifo file if it exists. The .sub file ------------- According to the index entries read from the .idx file each line is transformed into one Matroska block. CodecIDs -------- The CodecID should be "S_VOBSUB". If compression is used (see below) then the compression scheme is appended to the CodecID. I've implemented three compression algorhythms: libzo, zlib, bz2 (and none, of course). The resulting CodecIDs are "S_VOBSUB/LZO", "S_VOBSUB/Z", "S_VOBSUB/BZ2" and "S_VOBSUB", of course. Compression ----------- Each block is compressed independently from all the other blocks. This ensures seekability, even though the compression ratio suffers. Here are some real life examples that I've just created: 2446311 2003-09-11 21:16 n-none.mkv 994944 2003-09-11 21:16 n-lzo.mkv 950935 2003-09-11 21:18 n-bz2.mkv 906374 2003-09-11 21:16 n-zlib.mkv mkvmerge prints out the compression ratio of the raw data. The results are: lzo 38.2%, bz2 36.3% and zlib 34.4%. The CodecPrivate data has not been compressed in this example. The stripped down .idx file is usually very small (ca. 1200 bytes in this case), but the .ifo file may be a lot bigger (about 70kb in this case). This might make compressing the CodecPrivate contents attractive. Recommendations --------------- Size is not everything. Maybe you remember the decompression speed test I've done a couple of weeks ago which included these three libraries. The results were that lzo is about 10 times as fast as zlib which is in case about 3 to 4 times faster than bz2. The fact is that all libs should be fast enough on modern computers to decompress the small amounts of data. Example: all blocks in this VobSub file are either 2048 or 4096 bytes big. zlib was capable of decompressing over 7000 of these blocks per second. My recommendation: a) Use the CodecIDs like I've introduced them above. b) Make zlib the compression library of choice for use. It is spread widely, very well known and available for pretty much each and every platform. Speed may be gained by disabling any CRC algorhythm that might be present in zlib. c) Maybe compress the .ifo in the CodecPrivate data with the same library. This must be put into the specs. Either it is ALWAYS compressed or NEVER. Comments? -- ==> Ciao, Mosu (Moritz Bunkus) From moritz at bunkus.org Thu Sep 11 22:00:45 2003 From: moritz at bunkus.org (Moritz Bunkus) Date: Thu, 11 Sep 2003 22:00:45 +0200 Subject: [Matroska-devel] VobSub in Matroska In-Reply-To: <20030911194134.GF31148@bunkus.org> References: <20030911194134.GF31148@bunkus.org> Message-ID: <20030911200045.GG31148@bunkus.org> /me searches his big rubber to get rid of all the spelling mistakes... 'algorhythm' should be 'algorithm', of course. > CodecPrivate > ------------ ... > The following bytes are lace codec lengths of the .idx file... should read: "... lace CODED lengths ..." > mkvmerge prints out the compression ratio of the raw data. The results > are: lzo 38.2%, bz2 36.3% and zlib 34.4%. Meaning that the compressed data is 38.2% of the uncompressed data. So we saved 61.8% space. -- ==> Ciao, Mosu (Moritz Bunkus) From chris at matroska.org Fri Sep 12 09:13:41 2003 From: chris at matroska.org (Christian HJ Wiesner) Date: Fri, 12 Sep 2003 09:13:41 +0200 Subject: [Matroska-devel] VobSub in Matroska In-Reply-To: <20030911194134.GF31148@bunkus.org> References: <20030911194134.GF31148@bunkus.org> Message-ID: <3F617225.7070804@matroska.org> Moritz Bunkus wrote: >Heya, another RFC (request for comments). > It seems the guys are still on vacation :P Hey, i said one WEEK, not one MONTH !!!! :) >The .idx file >------------- >This is a text file that contains comments (# bla bla...), settings >(alpha: 100%) and index entries (timestamp: 00:02:53:873, filepos: >000002000). We need the index entries for reading the .sub file, but we >don't have to keep those in the Matroska file. Therefore we can keep the >comments and the setting lines but discard the index lines. > > Question : if we extract a VobSub from a MKV file, can we recreate everything that is in the .idx file, or is this done by the DVD authoring program for every new DVD in this case ? There maybe some uses for matroska where a complete DVD is TEMPORARILY stored in a huge MKV file, to make a smaller DVD-5 from it later, and it would be a pitty if the VobSubs coming from MKV could not be used anymore because we thrown away some bytes. >The .ifo file >------------- > >This is a binary file. It should be kept intact. > I dont think we need the .ifo at all. The .ifo will normally contain a lot of info about a DVD, like how many audio tracks it has, how many subtitles tracks, and what language they are. It should be left to the subs handling program if it will read the DVD .ifo file to know about the language and number of the tracks it will mux into the MKV, but we dont have to store this IMHO. >CodecPrivate >------------ > >We do something similar to how Vorbis stores its three header packets in >CodecPrivate: We use a lace like structure. >First byte is the number of files in CodecPrivate - 1. If the .ifo file >is present, this byte is 1 (because there are two files), if the file >ain't present, then it is 0 (because there's only the stripped down >.idx). >The following bytes are lace codec lengths of the .idx file, but only if >the .ifo file exists. You can always get the length of the last file >(.ifo if .ifo exists, .idx otherwise) by looking at the length of the >complete CodecPrivate data. >Next is the contents of the stripped down .idx file followed by the >contents of the .ifo file if it exists. > See above, i dont think we need a .ifo, its very specific to the source DVD, and all the info it contains can normally easily be read from the corresponding tags in the MKV file. >The .sub file >------------- >According to the index entries read from the .idx file each line is >transformed into one Matroska block. > Cool :) >CodecIDs >-------- > >The CodecID should be "S_VOBSUB". If compression is used (see below) >then the compression scheme is appended to the CodecID. I've implemented >three compression algorhythms: libzo, zlib, bz2 (and none, of >course). The resulting CodecIDs are "S_VOBSUB/LZO", "S_VOBSUB/Z", >"S_VOBSUB/BZ2" and "S_VOBSUB", of course. > Ok, it is wise to NOT use S_IMAGE, as meanwhile we know that VobSubs are not really images at all, but using a special, complex coding table to store subs, so it makes sense to use S_VOBSUB . But i dont agree to allow the use of different compression algorithms, we absolutely have to standardize one of them ! Just think of the matroska parser filters, to be able to support all possible compressed VobSubs they have to implement all of these libs !! Definitely, we have to make a choice here, and this choice must be based on - speed - compression ratio - license form of the lib ( only BSD should be used, else no closed source software can support VobSub in MKV, or they have to recode the complete lib ) ! What do we know about the license form of the libs you were mentioning ? >Compression >----------- >Each block is compressed independently from all the other blocks. This >ensures seekability, even though the compression ratio suffers. >Here are some real life examples that I've just created: >2446311 2003-09-11 21:16 n-none.mkv > 994944 2003-09-11 21:16 n-lzo.mkv > 950935 2003-09-11 21:18 n-bz2.mkv > 906374 2003-09-11 21:16 n-zlib.mkv >mkvmerge prints out the compression ratio of the raw data. The results >are: lzo 38.2%, bz2 36.3% and zlib 34.4%. >The CodecPrivate data has not been compressed in this example. The >stripped down .idx file is usually very small (ca. 1200 bytes in this >case), but the .ifo file may be a lot bigger (about 70kb in this >case). This might make compressing the CodecPrivate contents attractive. > > About speed, did you see the comments from the guys on the PNG list commenting your test, stating that LZO was only faster because there is no inherent CRC32 in the bitstream ? Looking at the small differences between those 3, i recommend we really concentrate on speed and license type of the libraries to make our choice. >Recommendations >--------------- >Size is not everything. Maybe you remember the decompression speed test >I've done a couple of weeks ago which included these three >libraries. The results were that lzo is about 10 times as fast as zlib >which is in case about 3 to 4 times faster than bz2. The fact is that >all libs should be fast enough on modern computers to decompress the >small amounts of data. Example: all blocks in this VobSub file are >either 2048 or 4096 bytes big. zlib was capable of decompressing over >7000 of these blocks per second. > > See above, if the PNG guys were right then your speed test maybe suffered from some differences in the data handling. I get you a link : http://article.gmane.org/gmane.comp.graphics.png.general/6 >My recommendation: >a) Use the CodecIDs like I've introduced them above. > Agreed :) , but only one compression please :) >b) Make zlib the compression library of choice for use. It is spread >widely, very well known and available for pretty much each and every >platform. Speed may be gained by disabling any CRC algorhythm that might >be present in zlib. > :) >c) Maybe compress the .ifo in the CodecPrivate data with the same >library. This must be put into the specs. Either it is ALWAYS compressed >or NEVER. > Leave the .ifo away, we dont need it, and if a new DVD is authored there will be a new .ifo created anyhow, and done compress the codec private .... just my 2 cents :) Christian From moritz at bunkus.org Fri Sep 12 09:40:49 2003 From: moritz at bunkus.org (Moritz Bunkus) Date: Fri, 12 Sep 2003 09:40:49 +0200 Subject: [Matroska-devel] VobSub in Matroska In-Reply-To: <3F617225.7070804@matroska.org> References: <20030911194134.GF31148@bunkus.org> <3F617225.7070804@matroska.org> Message-ID: <20030912074049.GH31148@bunkus.org> Hi. > Question : if we extract a VobSub from a MKV file, can we recreate > everything that is in the .idx file, Yes. What I've thrown away is the index entries. These entries only contain two things: a) the timestamp (which we get from the Matroska block) and the file position where this block is stored in the .sub (which we can get by adding up all previous blocks' sizes). > I dont think we need the .ifo at all. The .ifo will normally contain a > lot of info about a DVD, like how many audio tracks it has, how many > subtitles tracks, and what language they are. It should be left to the > subs handling program if it will read the DVD .ifo file to know about > the language and number of the tracks it will mux into the MKV, but we > dont have to store this IMHO. If we can leave the IFO out (I'd like some input from Gabest here) then CodecPrivate should only contain the contents of the stripped down .idx file without that lacing structure. > Ok, it is wise to NOT use S_IMAGE, as meanwhile we know that VobSubs are > not really images at all, but using a special, complex coding table to > store subs, so it makes sense to use S_VOBSUB . But i dont agree to > allow the use of different compression algorithms, we absolutely have to > standardize one of them ! I don't want to allow them :) I just wanted to do some real life testing, so I implemented them all, that's all. The option to switch between them is not documented, and I'll even remove it once we've settled on one compression lib being the standard. > - speed lzo wins hands down, even if I use the "safe" function. Rough numbers: In my initial test (without the "safe" implementation as the PNG guys noticed) lzo was about 14 times faster than zlib. With the use of the "safe" function lzo is still 10 times faster. > - compression ratio zlib beats the other two. > - license form of the lib ( only BSD should be used, else no closed > source software can support VobSub in MKV, or they have to recode the > complete lib ) ! > > What do we know about the license form of the libs you were mentioning > ? lzo: GPL zlib: Not one of the 'classic' licenses, but it includes the following text: ---------------------- Permission is granted to anyone to use this software for any purpose, including commercial applications, and to alter it and redistribute it freely, subject to the following restrictions: 1. The origin of this software must not be misrepresented; you must not claim that you wrote the original software. If you use this software in a product, an acknowledgment in the product documentation would be appreciated but is not required. 2. Altered source versions must be plainly marked as such, and must not be misrepresented as being the original software. 3. This notice may not be removed or altered from any source distribution. --------------------- meaning it should be ok bz2: again no classic license but this text: --------------------- Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. The origin of this software must not be misrepresented; you must not claim that you wrote the original software. If you use this software in a product, an acknowledgment in the product documentation would be appreciated but is not required. 3. Altered source versions must be plainly marked as such, and must not be misrepresented as being the original software. 4. The name of the author may not be used to endorse or promote products derived from this software without specific prior written permission. ------------------- > About speed, did you see the comments from the guys on the PNG list > commenting your test, stating that LZO was only faster because there is > no inherent CRC32 in the bitstream ? So what? :) Anyway I wouldn't recommend using lzo anymore (worst compression ratio, GPL). > See above, if the PNG guys were right then your speed test maybe > suffered from some differences in the data handling. I get you a link : > http://article.gmane.org/gmane.comp.graphics.png.general/6 They had the valid objection that I used the "unsafe" lzo decompression routine which might crash the application if damaged data is fed to the decoder. However, as I've written above, I've redone my test with the safe decompression function, and lzo is still 10 times faster. > Agreed :) , but only one compression please :) Sure :) Then I'd vote for S_VOBSUB/Z or S_VOBSUB/ZLIB > >b) Make zlib the compression library of choice for use. It is spread > >widely, very well known and available for pretty much each and every > >platform. Speed may be gained by disabling any CRC algorhythm that might > >be present in zlib. > > > :) ...and it's free to use in both free and closed source projects. > Leave the .ifo away, we dont need it, and if a new DVD is authored there > will be a new .ifo created anyhow, and done compress the codec private > .... just my 2 cents :) As I said I'd like some input from Gabest here, but thanks for your thoughts. I hadn't considered licensing issues... -- ==> Ciao, Mosu (Moritz Bunkus) From yusaku at anime.cz Sat Sep 13 15:53:36 2003 From: yusaku at anime.cz (Yusaku) Date: Sat, 13 Sep 2003 15:53:36 +0200 Subject: [Matroska-devel] Re: VobSub in Matroska References: <20030911194134.GF31148@bunkus.org> Message-ID: > The .idx file > ------------- > > This is a text file that contains comments (# bla bla...), settings > (alpha: 100%) and index entries (timestamp: 00:02:53:873, filepos: > 000002000). We need the index entries for reading the .sub file, but we > don't have to keep those in the Matroska file. Therefore we can keep the > comments and the setting lines but discard the index lines. however, the language IDs and VOB/CELL boundaries are context-sensitive, meaning you will need to replace them with some context information (which stream they belong to) > The .ifo file > ------------- > > This is a binary file. It should be kept intact. IFO file is not part of VobSub subtitle specification for at least one year now; thus can be completelly dropped. The requirement for muxing VobSub into matroska should be v7 subtitles (see first line of idx file). If the version is smaller, user can remux them using SubResync utility from VobSub 2.23 (or those blessed with better videocards using MPC) into v7 format. Generally it should not be a problem though, because any newly created subs will be in v7 format. > CodecIDs > -------- > > The CodecID should be "S_VOBSUB". If compression is used (see below) > then the compression scheme is appended to the CodecID. I've implemented > three compression algorhythms: libzo, zlib, bz2 (and none, of > course). The resulting CodecIDs are "S_VOBSUB/LZO", "S_VOBSUB/Z", > "S_VOBSUB/BZ2" and "S_VOBSUB", of course. because of the speed issues I think it would be best to use only LZO. The size difference is IMHO simply not worth it; and if Christian still haven't abandoned the idea of HW playback, then I think only simpler decompressions are possible there because of limited CPU power. A bit off topic: VobSub subtitles could potentialy be used for storing generic image subs as well (under the limits imposed by DVD subs, but that is not that bad): 1) there is utility to convert SSA formatted text subtitles to format used for DVD Authoring (MaestroSBT on http://www.doom9.org/software2.htm ), which is working remarkably well and has been used on some commercial release DVDs already (e.g. His and Her Circumstances US). This will create text metadata (starttime, endtime, palette and file reference for each sub) and a list of BMP image files referenced from it (which can be manually edited afterwards). 2) use C code from http://www.xs4all.nl/~vielle/video/sublog.html SubLog filter to convert image subs (above) into VobSub .sub MPEG2 stream and encapsulate this into MKV This way there would be possibility to include 4-bit image subs in mkv already... and 16 colors is enough for most practical purposes. For an example of demuxed subtitles and their VobSub counterpart see http://www.anime.cz/temp/SubtitleExample.rar . BMP files contain bitmap images, SON file the timecodes and SPF file the palette. Yusaku From moritz at bunkus.org Sat Sep 13 16:27:43 2003 From: moritz at bunkus.org (Moritz Bunkus) Date: Sat, 13 Sep 2003 16:27:43 +0200 Subject: [Matroska-devel] Re: VobSub in Matroska In-Reply-To: References: <20030911194134.GF31148@bunkus.org> Message-ID: <20030913142742.GO31148@bunkus.org> Hi. > however, the language IDs and VOB/CELL boundaries are context-sensitive, > meaning you will need to replace them with some context information (which > stream they belong to) Ah thanks. I don't know how exactly several languages are stored in the .sub file, but if they can be separated easily then they should be separated into individual Matroska tracks. E.g. the VobSub file contains streams for both English and German subtitles. Then the resulting Matroska file should contain two tracks. That way the language information could be 'dropped' or mapped to Matroska's language tags. > IFO file is not part of VobSub subtitle specification for at least one year > now; thus can be completelly dropped. Good to know :) > The requirement for muxing VobSub into > matroska should be v7 subtitles (see first line of idx file). Aha... Why is that? One of my test files is v3. Any reason that v7 should be the minimum version? > If the version is smaller, user can remux them using SubResync utility > from VobSub 2.23 (or those blessed with better videocards using MPC) > into v7 format. Generally it should not be a problem though, because > any newly created subs will be in v7 format. How exactly do those formats differ? Because I'd like to do that conversion on-the-fly if it's not too complicated. > because of the speed issues I think it would be best to use only LZO. The > size difference is IMHO simply not worth it; and if Christian still haven't > abandoned the idea of HW playback, then I think only simpler decompressions > are possible there because of limited CPU power. I disagree. LZO is GPL and will therefore pose problems for hardware devices... And zlib is not THAT slow. We're talking about very, very small amounts of data per second that have to be uncompressed. > A bit off topic: VobSub subtitles could potentialy be used for storing > generic image subs as well (under the limits imposed by DVD subs, but that > is not that bad): ... Very interesting :) Thanks for your input. -- ==> Ciao, Mosu (Moritz Bunkus) From yusaku at anime.cz Sat Sep 13 17:26:57 2003 From: yusaku at anime.cz (Yusaku) Date: Sat, 13 Sep 2003 17:26:57 +0200 Subject: [Matroska-devel] Re: Re: VobSub in Matroska References: <20030911194134.GF31148@bunkus.org> <20030913142742.GO31148@bunkus.org> Message-ID: > Ah thanks. I don't know how exactly several languages are stored in the > .sub file, but if they can be separated easily then they should be > separated into individual Matroska tracks. E.g. the VobSub file contains > streams for both English and German subtitles. Then the resulting > Matroska file should contain two tracks. That way the language > information could be 'dropped' or mapped to Matroska's language tags. Ok. Only problem associated with this approach (otherwhise definitely valid) is that so far there is not any VobSub subtitle joiner (to join two different languages) that I know of. But considering the structure of vobsub files it is not such a problem (one could do it even in bash script...). (the two languages just have different file offsets in the IDX file for each language stream, very easy to split/join) > > The requirement for muxing VobSub into > > matroska should be v7 subtitles (see first line of idx file). > > Aha... Why is that? One of my test files is v3. Any reason that v7 > should be the minimum version? not particularly... just that people will use mkv mostly for new files and thus will not come over older at all... which will save you some time coding. > How exactly do those formats differ? Because I'd like to do that > conversion on-the-fly if it's not too complicated. They differ only in IDX files. I don't have any V3 files to check, but IIRC palette information was not extracted from IFO in V3, rest was the same; maybe some other minor things. On the fly conversion would require some IFO parsing code to extract that. > I disagree. LZO is GPL and will therefore pose problems for hardware > devices... And zlib is not THAT slow. We're talking about very, very > small amounts of data per second that have to be uncompressed. Ahhh... yeah, hands off of GPL code (in this case) is a good idea. Yusaku From moritz at bunkus.org Sat Sep 13 17:39:12 2003 From: moritz at bunkus.org (Moritz Bunkus) Date: Sat, 13 Sep 2003 17:39:12 +0200 Subject: [Matroska-devel] Re: Re: VobSub in Matroska In-Reply-To: References: <20030911194134.GF31148@bunkus.org> <20030913142742.GO31148@bunkus.org> Message-ID: <20030913153911.GP31148@bunkus.org> Hi. > (the two languages just have different file offsets in the IDX file for each > language stream, very easy to split/join) Great :) Easy to implement, then. And definitely the way to go. > They differ only in IDX files. I don't have any V3 files to check, but IIRC > palette information was not extracted from IFO in V3, rest was the same; > maybe some other minor things. On the fly conversion would require some IFO > parsing code to extract that. ! Ok, so I'll only allow v7 style files. The palette argument is valid, and if that's the only reason the IFO was needed then we should not allow them. As you've said there is a program that can convert older files to new ones, so there's no need to support v3 with IFO. -- ==> Ciao, Mosu (Moritz Bunkus) From paul at msn.com Sat Sep 13 19:39:20 2003 From: paul at msn.com (Pamel) Date: Sat, 13 Sep 2003 12:39:20 -0500 Subject: [Matroska-devel] Re: Re: Re: VobSub in Matroska References: <20030911194134.GF31148@bunkus.org> <20030913142742.GO31148@bunkus.org> <20030913153911.GP31148@bunkus.org> Message-ID: "Moritz Bunkus" wrote... > ! Ok, so I'll only allow v7 style files. The palette argument is valid, > and if that's the only reason the IFO was needed then we should not > allow them. The only information from the IFO that we will store is the palette information, correct? Pamel From steve.lhomme at free.fr Wed Sep 17 21:41:02 2003 From: steve.lhomme at free.fr (Steve Lhomme) Date: Wed, 17 Sep 2003 21:41:02 +0200 Subject: [Matroska-devel] New versions Message-ID: <3F68B8CE.3060506@free.fr> Hi, I've released new versions of libebml (0.6.0) and libmatroska (0.5.2). Libebml has changed of version because of changes in some constructors, and the introduction of a new global method Clone(). Libmatroska has some additions on the WinIO callback and changes in some IDs or additions. From chris at matroska.org Sat Sep 20 22:29:18 2003 From: chris at matroska.org (Christian HJ Wiesner) Date: Sat, 20 Sep 2003 22:29:18 +0200 Subject: [Matroska-devel] GPAC opensource Menue System based on MPEG4 standards Message-ID: <3F6CB89E.7070106@matroska.org> http://gpac.sourceforge.net/ http://www.comelec.enst.fr/osmo4/#samples Seems that MP4 container is not as limited as we thought :( .... Christian From chris at matroska.org Sun Sep 21 00:38:06 2003 From: chris at matroska.org (Christian HJ Wiesner) Date: Sun, 21 Sep 2003 00:38:06 +0200 Subject: [Matroska-devel] VobSub in Matroska In-Reply-To: <20030912074049.GH31148@bunkus.org> References: <20030911194134.GF31148@bunkus.org> <3F617225.7070804@matroska.org> <20030912074049.GH31148@bunkus.org> Message-ID: <3F6CD6CE.4020008@matroska.org> Moritz Bunkus wrote: > As I said I'd like some input from Gabest here, but thanks for your > >thoughts. I hadn't considered licensing issues... > > Some additional info : I just learned from the Videolan guys that most players have included zlib already, as .mov files can have headers compressed with zlib ... Christian From moritz at bunkus.org Sun Sep 21 01:07:41 2003 From: moritz at bunkus.org (Moritz Bunkus) Date: Sun, 21 Sep 2003 01:07:41 +0200 Subject: [Matroska-devel] VobSub in Matroska In-Reply-To: <3F6CD6CE.4020008@matroska.org> References: <20030911194134.GF31148@bunkus.org> <3F617225.7070804@matroska.org> <20030912074049.GH31148@bunkus.org> <3F6CD6CE.4020008@matroska.org> Message-ID: <20030920230741.GI15097@bunkus.org> Hi. > I just learned from the Videolan guys that most players have included > zlib already, as .mov files can have headers compressed with zlib ... I know :) Yet another reason for chosing zlib. I hope I'll be able to continue my work on the muxer (especially support for handling multi-language vobsubs by separating them into individual tracks) today so that I can make the final draft for the specs. -- ==> Ciao, Mosu (Moritz Bunkus) From Liisachan at faireal.net Tue Sep 23 13:33:38 2003 From: Liisachan at faireal.net (Liisachan) Date: Tue, 23 Sep 2003 20:33:38 +0900 Subject: [Matroska-devel] Re: VobSub in Matroska In-Reply-To: References: <20030911194134.GF31148@bunkus.org> Message-ID: <3F702F92.7010801@faireal.net> Yusaku wrote: >A bit off topic: VobSub subtitles could potentialy be used for storing >generic image subs as well (under the limits imposed by DVD subs, but that >is not that bad): > >1) there is utility to convert SSA formatted text subtitles to format used >for DVD Authoring (MaestroSBT on http://www.doom9.org/software2.htm ), which >is working remarkably well and has been used on some commercial release DVDs >already (e.g. His and Her Circumstances US). This will create text metadata >(starttime, endtime, palette and file reference for each sub) and a list of >BMP image files referenced from it (which can be manually edited >afterwards). > > > This is even more offtopic too, but let me make an additional note about this As i am subbing, i m getting more and more convinced that this point of view is important-- font embedding has its forte, but you cannot alwayws do that, especially for CJK languages (a font file can be too big) Probebly, the only way you can use _for sure_ the style you want as softsub, is image sub. But this has a negative side too. It is like making a webpage as image just because you want to use your special font... besides, You couldnt change font-size etc dynamically even tho it is softsub, so image-sub is , semi-hard sub, so to speak While i m looking forward to "generalized" soft image subs, font embedding is the way to go, at least theoretically. Another thing I d like to have, is font matching like in CSS (or in USF) like, "Verdana, Chicago, sans-serif" where the sub renderer is supposed to try to use Verdana, and if it is not available try to use Chicago, and if it is again not available, use a generic font that is sans-serif. In today's SSA, you cannot specify alternative fonts--you can specify only one fontname, and if the system doesnt have it, VSFilter will use default font. This may be ok only if the language is a "normal" one--like english or french, as any generic font should have the glyphs needed. But, atm, I m not sure what is the best way to softsub in Japanese, and, even if the original sub data is in plain text, image sub maybe the best choice, practically. Because, you can replay such image subs on Linux or Mac too, I think, while SSA only works on Windows. Each method has each problem and Im not sure what is the best way... Liisachan From Liisachan at faireal.net Tue Sep 23 13:38:47 2003 From: Liisachan at faireal.net (Liisachan) Date: Tue, 23 Sep 2003 20:38:47 +0900 Subject: [Matroska-devel] MP2 and MP1 in MKV by VDM doesnt play Message-ID: <3F7030C7.90604@faireal.net> Hi! I am not sure which kind of problem this is (a bug in VDM or a prob in MKV spec or a bug in my player) but it seems that MP1 /2 audio in MKV desnt play when muxed by VirtualDubMod. Is this a known problem? The same A+V is ok in OGM or in AVI. This is sample... http://space34.at.infoseek.co.jp/mp2.zip Getting Video Only, no sound (MPC TCMP) + EBML head + Segment |+ Segment information | + Muxing application: libebml-0.5.0 & libmatroska-0.5.0 | + Writing application: VirtualDubMod 1.5.4.1 (build 2092/release) | + Segment UID: 0x62 0xa8 0x31 0xd4 0x71 0x21 0xb9 0x90 0x1d 0xe7 0x8e 0x73 0x30 0x30 0x18 0x18 | + Duration: 31.944s | + Date: Tue Sep 23 10:26:53 2003 UTC |+ Seek head (subentries will be skipped) |+ EbmlVoid |+ Segment tracks | + A track | + Track number: 1 | + Track UID: 3560024162 | + Track type: video | + MinCache: 1 | + Codec ID: V_MS/VFW/FOURCC | + CodecPrivate, length 40 (FourCC: XVID, 0x44495658) | + Lacing flag: 0 | + Default duration: 33.367ms (29.970 fps for a video track) | + Video track | + Pixel width: 480 | + Pixel height: 272 | + Display width: 480 | + Display height: 272 | + Display unit: 0 (pixels) | + A track | + Track number: 2 | + Track UID: 2428051825 | + Track type: audio | + Codec ID: A_MPEG/L2 | + Audio track | + Sampling frequency: 48000.000000 | + Channels: 2 |+ Cluster *Liisachan* From chris at matroska.org Tue Sep 23 17:10:08 2003 From: chris at matroska.org (Christian HJ Wiesner) Date: Tue, 23 Sep 2003 17:10:08 +0200 Subject: [Matroska-devel] Mosu appointed new Project Administrator for the matroska project Message-ID: <3F706250.2010008@matroska.org> HI, It is with great pleasure that i can announce today that Moritz 'Mosu' Bunkus has accepted our offer to become a full-featured, all-mighty project administrator for the matroska opensource project. This step was more than sensible to do from the point of view of the existing admin team, Steve and myself, as Steve is currently getting more and more involved wit his second hobby, making music, and i am still struggling with job and private life, most of you will know the background. Acting as responsible project creators, and in order to make sure the project moves along, we are convinced that Mosu can help pushing matroska forward, so that we can finally provide the community with a powerful open standard container solution. He is certainly the right type of person for that, as he has proven in the past that he is working on mkvtoolnix with great intelligence, consistency and effort, and especially mkvmerge is truely one of the real backbones of the project today. 3 cheers for the new admin :) !! Christian From renekoch at e-divx.at Tue Sep 23 21:52:43 2003 From: renekoch at e-divx.at (=?iso-8859-1?Q?Ren=E9_Koch?=) Date: Tue, 23 Sep 2003 21:52:43 +0200 Subject: [Matroska-devel] MP2 and MP1 in MKV by VDM doesnt play References: <3F7030C7.90604@faireal.net> Message-ID: <007b01c3820c$413c5fc0$5ef698d4@SCRATPC> Hey! It seems that there is a bug. I couldn't hear a sound in your file and I also muxed a mp2-soundfile into a movie from me with VirtualDub Mod 1.5.4.1 and played it with The Core Media Player and there was no sound. Greetings, scrat ----- Original Message ----- From: "Liisachan" To: "Discussion about the current and future development of Matroska" Sent: Tuesday, September 23, 2003 1:38 PM Subject: [Matroska-devel] MP2 and MP1 in MKV by VDM doesnt play > Hi! > > I am not sure which kind of problem this is (a bug in VDM or a prob in > MKV spec or a bug in my player) but > it seems that MP1 /2 audio in MKV desnt play when muxed by VirtualDubMod. > Is this a known problem? > > The same A+V is ok in OGM or in AVI. > This is sample... > http://space34.at.infoseek.co.jp/mp2.zip > > Getting Video Only, no sound (MPC TCMP) > > + EBML head > + Segment > |+ Segment information > | + Muxing application: libebml-0.5.0 & libmatroska-0.5.0 > | + Writing application: VirtualDubMod 1.5.4.1 (build 2092/release) > | + Segment UID: 0x62 0xa8 0x31 0xd4 0x71 0x21 0xb9 0x90 0x1d 0xe7 0x8e > 0x73 0x30 0x30 0x18 0x18 > | + Duration: 31.944s > | + Date: Tue Sep 23 10:26:53 2003 UTC > |+ Seek head (subentries will be skipped) > |+ EbmlVoid > |+ Segment tracks > | + A track > | + Track number: 1 > | + Track UID: 3560024162 > | + Track type: video > | + MinCache: 1 > | + Codec ID: V_MS/VFW/FOURCC > | + CodecPrivate, length 40 (FourCC: XVID, 0x44495658) > | + Lacing flag: 0 > | + Default duration: 33.367ms (29.970 fps for a video track) > | + Video track > | + Pixel width: 480 > | + Pixel height: 272 > | + Display width: 480 > | + Display height: 272 > | + Display unit: 0 (pixels) > | + A track > | + Track number: 2 > | + Track UID: 2428051825 > | + Track type: audio > | + Codec ID: A_MPEG/L2 > | + Audio track > | + Sampling frequency: 48000.000000 > | + Channels: 2 > |+ Cluster > > *Liisachan* > > _______________________________________________ > Matroska-devel mailing list > Matroska-devel at lists.matroska.org > http://lists.matroska.org/cgi-bin/mailman/listinfo/matroska-devel > > From steve.lhomme at free.fr Tue Sep 23 23:30:26 2003 From: steve.lhomme at free.fr (Steve Lhomme) Date: Tue, 23 Sep 2003 23:30:26 +0200 Subject: [Matroska-devel] [Fwd: [Tinyxml-general] New project forum] Message-ID: <3F70BB72.90003@free.fr> In case any of you use TinyXML, tell it to Lee :) -------- Original Message -------- Subject: [Tinyxml-general] New project forum Date: Tue, 23 Sep 2003 13:20:50 -0700 From: Lee Thomason To: All -- I added a new forum - Projects - for collecting projects that use TinyXml. Completely optional, of course. I'm looking forward to seeing what turns up! lee ------------------------------------------------------- This sf.net email is sponsored by:ThinkGeek Welcome to geek heaven. http://thinkgeek.com/sf _______________________________________________ Tinyxml-general mailing list Tinyxml-general at lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/tinyxml-general From suiryc at yahoo.com Wed Sep 24 02:49:52 2003 From: suiryc at yahoo.com (Cyrius) Date: Tue, 23 Sep 2003 17:49:52 -0700 (PDT) Subject: [Matroska-devel] MP2 and MP1 in MKV by VDM doesnt play In-Reply-To: <3F7030C7.90604@faireal.net> Message-ID: <20030924004952.4877.qmail@web12905.mail.yahoo.com> It works in OGM or AVI ... GraphEdit doesn't create the output pin for the audio stream ... Transmuxing the OGM or AVI to MKV (thus keeping the AVI compatibility mode which use A_MS/ACM as codec ID) make the sound work ... All this seems to indicate that the Matroska Splitter just doesn't render streams with the A_MPEG/L2 codec ID. __________________________________ Do you Yahoo!? Yahoo! SiteBuilder - Free, easy-to-use web site design software http://sitebuilder.yahoo.com From moritz at bunkus.org Thu Sep 25 23:53:35 2003 From: moritz at bunkus.org (Moritz Bunkus) Date: Thu, 25 Sep 2003 23:53:35 +0200 Subject: [Matroska-devel] libebml's EbmlMaster missing a feature... Message-ID: <20030925215335.GT1891@bunkus.org> Hi, ...which is insertion at arbitrary points. E.g. in my chapter editor I want to insert an element at a specific location - but there's only PushElement() available. -- ==> Ciao, Mosu (Moritz Bunkus) From steve.lhomme at free.fr Fri Sep 26 09:43:15 2003 From: steve.lhomme at free.fr (Steve Lhomme) Date: Fri, 26 Sep 2003 09:43:15 +0200 Subject: [Matroska-devel] libebml's EbmlMaster missing a feature... In-Reply-To: <20030925215335.GT1891@bunkus.org> References: <20030925215335.GT1891@bunkus.org> Message-ID: <3F73EE13.9050800@free.fr> Moritz Bunkus wrote: > Hi, > > ...which is insertion at arbitrary points. E.g. in my chapter editor I > want to insert an element at a specific location - but there's only > PushElement() available. OK, I'll check that (not until next tuesday). I don't remember is EbmlMaster is directly a public derivative of std::vector. Otherwise you can do all the vector things already. I doubt it because of data consistency in the chapter. From moritz at bunkus.org Fri Sep 26 10:01:53 2003 From: moritz at bunkus.org (Moritz Bunkus) Date: Fri, 26 Sep 2003 10:01:53 +0200 Subject: [Matroska-devel] libebml's EbmlMaster missing a feature... In-Reply-To: <3F73EE13.9050800@free.fr> References: <20030925215335.GT1891@bunkus.org> <3F73EE13.9050800@free.fr> Message-ID: <20030926080153.GW1891@bunkus.org> Hi. protected: std::vector ElementList; Not a derivative - the vector is a data member. We already have almost all we need. What I'd propose are two new functions: void InsertElement(EbmlElement &, int before = 0); void InsertElement(EbmlElement &, EbmlElement &before); Both are trivial to implement, e.g. void EbmlMaster::InsertElement(EbmlElement &elt, int before) { vector::iterator insert_it; assert((before >= 0) && (before <= ElementList.size())); insert_it = ElementList.begin(); insert_it += before; ElementList.insert(elt, insert_it); } We already have operator [], PushElement(), ListSize(), Remove(int). -- ==> Ciao, Mosu (Moritz Bunkus) From chris at matroska.org Fri Sep 26 21:45:49 2003 From: chris at matroska.org (Christian HJ Wiesner) Date: Fri, 26 Sep 2003 21:45:49 +0200 Subject: [Matroska-devel] Re: FLAC + Matroska ( From Hydrogenaudio Forums ) In-Reply-To: <20030926003203.15614.qmail@animus-facticius.org> References: <20030926003203.15614.qmail@animus-facticius.org> Message-ID: <3F74976D.7040902@matroska.org> Forwarding this important info from Josh to the matroska-devel list .... Christian Josh Coalson wrote: >I've been thinking about this thread: > >http://www.hydrogenaudio.org/index.php?showtopic=13079&hl= > >I realized it's possible to save significant time by skipping the inverse FIR filter calculation. The best way to do this is to add a 'skip frame' call in the libFLAC API. Aside from making muxing easier, this will allow other cool things like intelli >gble fast-forward in native FLAC players. > >I have some time now to catch up on things so I plan to put this in the next release (recently I've been finishing up Ogg FLAC seek support). After that I plan on making a 'reflac' utility that will convert a FLAC file to Ogg FLAC without decoding or r >e-encoding, and vice versa. The source for this program will be an ideal example for how to mux native FLAC in Matroska. > >Hope this helps. > >Josh > > From chris at matroska.org Fri Sep 26 22:00:23 2003 From: chris at matroska.org (Christian HJ Wiesner) Date: Fri, 26 Sep 2003 22:00:23 +0200 Subject: [Matroska-devel] Re: email adress of David Bryant for wavpack 4 integration into matroska tools ? In-Reply-To: <01c38327$b5f98e20$0100007f@default> References: <01c38327$b5f98e20$0100007f@default> Message-ID: <3F749AD7.7040009@matroska.org> Hi David, thanks for your email ! I forward this reply of mine to the matroska-devel list, as all the devs might be interested to learn what you have to say in this respect. David Bryant wrote: >Hi Christian, >Roberto forwarded your e-mail to me as you requested. I am a little confused >because we e-mailed back and forth a few weeks ago about MPC, but I guess >you didn't realize I was the same person. > > Yes, i know i contacted you but for whatever reason i couldnt find any emails about this containing your email addy, maybe because we were using PM on HA:org ? >I am interested in doing whatever I can to help WavPack be easier to >integrate into any format or application. Also, since nothing is really cast >into stone yet, this is a good time to let me know of any special >requirements. I have read a little in the threads about FLAC and matroska >and I think that WavPack will be okay. WavPack 4.0 streams will be trivial >to parse (without decoding) and easy to synch up with if you need to start >in the middle of a file. Each block will be completely independent and can >be any number of samples, although the default will be rather large (about >1/2 second) to keep the overhead low. > > Thats good news !! Josh Coalson just sent me an email that he has ideas how to improve his existing lib by skipping the FIR process ( no idea what this is :) ), but we still havent decided what lossless format to implement first. As stated in the original mail, lossless audio has mainly a meaning for us when talking about editing video material in our editing tools, so unfortunately its only used as an interims solution in most cases, the final file will always have a lossy compressed audio format IMHO. Of course, matroska could also be used as a very advanced storage container for people archiving music albums with lossless audio compression formats, you can do fancy things with it like putting a complete album into one single matroska file, each song being a chapter, including lyrics and/or covers as attachement files, use the matroska tagging system and tools to tag the files ( compatible with ID3V1/2 and Ape V2 tags BTW, but not allowing custom tags - we try to fight anarchy :-) ) or even store preferred equalizer settings for each song in it. But, we are not sure if matroska will be accepted by the HA.org community for that purpose, as all seem to be centered around Foobar2k player these days, and Peter seems not to be interedted in matroska at all :-( ..... >BTW, I am also trying to design WavPack 4.0 to be easily decoded in hardware >(at least the faster modes), so perhaps that advantage of FLAC will also be >eliminated eventually. > > One thing that could give you a real advantage IMHO, was to alllow transition from lossless to lossy mode in one file. Dont know if this is possible, but pls. think about it :-) ! >Please let me know of any other special requirements and I'll keep you >up-to-date as things progress here. >Thanks...David > I will ask my devs to email you and send a short description of what is technically necessary for good matroska implementation. And i know Linux support will be on the very top of their list, amongst other things ;-) ..... Christian matroska project admin From jcsston at toughguy.net Sat Sep 27 05:41:31 2003 From: jcsston at toughguy.net (Jory) Date: Fri, 26 Sep 2003 22:41:31 -0500 Subject: [Matroska-devel] Re: FLAC + Matroska ( From Hydrogenaudio Forums ) References: <20030926003203.15614.qmail@animus-facticius.org> <3F74976D.7040902@matroska.org> Message-ID: <000e01c384a9$40708040$0200a8c0@jcsston> To give you an update, I have created a DShow FLAC Encoder filter with the help of Toff and Toff has also made a FLAC decoder filter. http://corecodec.com/projects/coreflac/ It's still in the alpha stage, playback is not always smooth with video and other various bugs. There is no public release yet. Besides the CVS source code. Later, Jory ----- Original Message ----- From: "Christian HJ Wiesner" To: "Hydrogenaudio Forums" ; "Discussion about the current and future development of Matroska" Sent: Friday, September 26, 2003 2:45 PM Subject: [Matroska-devel] Re: FLAC + Matroska ( From Hydrogenaudio Forums ) > Forwarding this important info from Josh to the matroska-devel list .... > > Christian > > Josh Coalson wrote: > > >I've been thinking about this thread: > > > >http://www.hydrogenaudio.org/index.php?showtopic=13079&hl= > > > >I realized it's possible to save significant time by skipping the inverse FIR filter calculation. The best way to do this is to add a 'skip frame' call in the libFLAC API. Aside from making muxing easier, this will allow other cool things like intelli > >gble fast-forward in native FLAC players. > > > >I have some time now to catch up on things so I plan to put this in the next release (recently I've been finishing up Ogg FLAC seek support). After that I plan on making a 'reflac' utility that will convert a FLAC file to Ogg FLAC without decoding or r > >e-encoding, and vice versa. The source for this program will be an ideal example for how to mux native FLAC in Matroska. > > > >Hope this helps. > > > >Josh > > > > > > > _______________________________________________ > Matroska-devel mailing list > Matroska-devel at lists.matroska.org > http://lists.matroska.org/cgi-bin/mailman/listinfo/matroska-devel From chris at matroska.org Sat Sep 27 07:51:35 2003 From: chris at matroska.org (Christian HJ Wiesner) Date: Sat, 27 Sep 2003 07:51:35 +0200 Subject: [Matroska-devel] MP3Pro in MKV Message-ID: <3F752567.3000507@matroska.org> http://forum.doom9.org/showthread.php?s=&postid=377907#post377907 From paul at msn.com Sat Sep 27 16:29:23 2003 From: paul at msn.com (Pamel) Date: Sat, 27 Sep 2003 09:29:23 -0500 Subject: [Matroska-devel] Re: email adress of David Bryant for wavpack 4 integration into matroska tools ? References: <01c38327$b5f98e20$0100007f@default> <3F749AD7.7040009@matroska.org> Message-ID: "Christian HJ Wiesner" wrote... > Of course, matroska could also be used as a very advanced > storage container for people archiving music albums with lossless audio > compression formats, you can do fancy things with it like putting a > complete album into one single matroska file, each song being a chapter, > including lyrics and/or covers as attachement files, use the matroska > tagging system and tools to tag the files ( compatible with ID3V1/2 and > Ape V2 tags BTW, but not allowing custom tags - we try to fight anarchy > :-) ) Actually, we do allow custom tags. It was one of the complaints from the HA folks, so it was added in. However, you have to read carefully through the specs to find it, and we have no intention of making it easier to find. Pamel From chris at matroska.org Mon Sep 29 18:00:29 2003 From: chris at matroska.org (Christian HJ Wiesner) Date: Mon, 29 Sep 2003 18:00:29 +0200 Subject: [Matroska-devel] Re: email adress of David Bryant for wavpack 4 integration into matroskatools ? In-Reply-To: <01c385f3$ba879260$0100007f@default> References: <01c385f3$ba879260$0100007f@default> Message-ID: <3F78571D.7070202@matroska.org> David Bryant wrote: >Hi Christian, > > >I believe what Josh means is that the audio specific processing involved in >decoding FLAC could be skipped if the only purpose was to scan through the >data while staying sychronized. This whole issue surprised me a little >because I was under the impression that FLAC was inherently streamable, but >it now sounds to me that it is only streamable when it's in ogg containers. >One of the other applications I have in mind for WavPack 4.0 is for storing >high resolution audio (i.e. 24/96) using the lossy mode. There are currently >no solutions for lossy storage of this material except for the existing >lossy codecs that are useless for this in the sense that they are based on >psychoacoustic models that would indicate that nothing is required beyond >16/44. > > :) ! While i am a strong fighter for the thesis that audio information above 20 KHz is viable for highest audio quality, i admit i cant see the sense in using a lossly compression on this type of audio file. If i have the choice between a lossy compressed 24/96 audio and a lossless compressed 16/44,1 or 16/48 at same size, i go for the latter :D ..... >>One thing that could give you a real advantage IMHO, was to alllow >>transition from lossless to lossy mode in one file. Dont know if this is >>possible, but pls. think about it :-) ! >> >> > >Actually, this will be easy in WavPack 4.0 because every block of compressed >audio is completely independent. It will also be possible to store >multichannel material with different compression modes for the various >streams. > > :O !!! You mean this is possible already ? Using lossless for complex sound parts, and lossy for say silence and the like, similar to a mixed mode ? Wow !!! >>I will ask my devs to email you and send a short description of what is >>technically necessary for good matroska implementation. And i know Linux >>support will be on the very top of their list, amongst other things ;-) >> >> > >I am planning for the code to be easily portable to Linux and even to the >Mac. I may even try the linux port myself as soon as I get that Linux system >going that I have been planning for years... ;-) >David > > Looking forward to be able to tell our devs that Wavpack 4 code can be used to start adding it to our muxing tools David :) ! About the requirements of your libs for easy implementation, i hope at least one of the lazy matroska devs now takes the ball and writes something sensible back to you :) .... Christian From steve.lhomme at free.fr Mon Sep 29 22:58:27 2003 From: steve.lhomme at free.fr (Steve Lhomme) Date: Mon, 29 Sep 2003 22:58:27 +0200 Subject: [Matroska-devel] libebml's EbmlMaster missing a feature... In-Reply-To: <20030926080153.GW1891@bunkus.org> References: <20030925215335.GT1891@bunkus.org> <3F73EE13.9050800@free.fr> <20030926080153.GW1891@bunkus.org> Message-ID: <3F789CF3.4010001@free.fr> Moritz Bunkus wrote: > Hi. > > protected: > std::vector ElementList; > > Not a derivative - the vector is a data member. We already have almost > all we need. What I'd propose are two new functions: > > void InsertElement(EbmlElement &, int before = 0); > void InsertElement(EbmlElement &, EbmlElement &before); > > Both are trivial to implement, e.g. > > void EbmlMaster::InsertElement(EbmlElement &elt, int before) { > vector::iterator insert_it; > > assert((before >= 0) && (before <= ElementList.size())); > insert_it = ElementList.begin(); > insert_it += before; > ElementList.insert(elt, insert_it); > } OK, I added the functions in CVS but didn't test it :/ From moritz at bunkus.org Mon Sep 29 23:53:05 2003 From: moritz at bunkus.org (Moritz Bunkus) Date: Mon, 29 Sep 2003 23:53:05 +0200 Subject: [Matroska-devel] libebml's EbmlMaster missing a feature... In-Reply-To: <3F789CF3.4010001@free.fr> References: <20030925215335.GT1891@bunkus.org> <3F73EE13.9050800@free.fr> <20030926080153.GW1891@bunkus.org> <3F789CF3.4010001@free.fr> Message-ID: <20030929215305.GF13138@bunkus.org> Hi. Don't worry, I'll test them in the next couple of days. Thanks. -- ==> Ciao, Mosu (Moritz Bunkus) From moritz at bunkus.org Tue Sep 30 10:56:09 2003 From: moritz at bunkus.org (Moritz Bunkus) Date: Tue, 30 Sep 2003 10:56:09 +0200 Subject: [Matroska-devel] format of CodecIDs Message-ID: <20030930085609.GG13138@bunkus.org> Hi. Right now we've reached a critical point in the Matroska development, and we need to make a decision about how to store some additional information. The thing I'm talking about is compression (e.g. zlib compressed VobSubs) and encryption (jcsston has just implemented something with AES encryption). The problem is that we have to store these facts somehow. Here we have two choices: 1) Append the information to the CodecID. 2) Introduce new track information elements. So far I've assumed that we would chose 1) and have acted accordingly and introduced S_VOBSUB/ZLIB as the CodecID for compressed VobSubs. Please note that NO release of mkvtoolnix so far featured this compression, so there are NO files that use this CodecID out in the open. Therefore we can still change it. The drawback with this CodecID is that I've just appended '/ZLIB' to the CodecID. But if we want to do this the right way then we have to separate the content type (S_VOBSUB) from the modifications it went through (compression with ZLIB). A 'standard CodecID' could look like this: 1) Track type, followed by an underscore, 2) Content type, may be grouped, separator is the forward slash '/', 3) Optional flags indicating modifications, separator is some character that won't occur otherwise, e.g. the at sign '@'. 4) Each flag consists of a descriptor and a value, separated by a colon ':'. 5) Order of the flags is important. The first flag to occur is the first modification used when it was put into the container. Demuxers have to apply those counter-modifications from back to front. Example: Normal VobSubs: S_VOBSUB Compressed VobSubs: S_VOBSUB at COMPRESSION:ZLIB VobSubs that have been compressed first and then encoded with AES: S_VOBSUB at COMPRESSION:ZLIB at ENCRYPTION:AES Obvious drawback: The CodecID gets pretty unreadable. And it might not be very Matroska-like. Second approach: introduce new track info elements. This approach is more Matroska-like. Possible names could be KaxContentCompression and KaxContentEncryption, each being an unsigned integer. As I said: we have to be quick about this! Compressed VobSubs should be a reality soon, and we need a concise system by then. Thoughts? Other ideas? -- ==> Ciao, Mosu (Moritz Bunkus)