Intermediate video format survey

[Ignus2]

Hi everyone!

Has been a very long time since I last visited regularly these forums, but if you could spare a few minutes to help me out, I would be really grateful.

I'm developing a new lossless video codec for video post-production/intermediate format and I'm also writing a diploma thesis about it, and I would like to get some opinions about what kind of formats/codecs/software do you use for any kind of post-production work with your videos currently.

I've prepared a short survey with Google Forms, but any opinion is welcome here.

I'm particularly interested in what would be your requests about features missing in current lossless/intermediate codecs (too slow, too few colorspace support, too old, etc.) and what would you like to have as a feature if a new codec would pop up

Thank you for your time, it is much appreciated.

The link to the form: http://goo.gl/E8vAfg

Happy New Year!

EDIT: I'm done with the thesis, so I'm publishing the results. Thanks to everyone who filled out the survey!
Survey results

Greets,
I.

[SquallMX]

No love for Cineform?
Basic version is free, fast has several quality options, Interlaced and 4:2:2 support.

[Sparktank]

Also I had to add in UtVideo Codec Suite int he form as well.
4:2:0/4:2:2
601/709
Interlace support.

Plus it's really, really, really free.
And easy to find and download.

[Ignus2]

Wow, thanks for the detailed answer and all the answers so far!
I added UTVideo, Cineform (don't know why I left that out, was on my list), 4:2:0, interlaced video support and keyframe-only (intra-frame-only) format support to the form options.

My question now is what would you say, if you got a lossless codec, that compresses the same ratio as UTVideo and a little bit faster at compression but at 2-2.5 times faster at decoding speed (and more than 10 times of Lagarith)?

I've read the opinions, but I don't really get yet why BT.709 or BT.601 matters, as for a lossless codec you get in bits and output the same. As I get it, it only matters when converting to/from RGB, which is not particularly the job of the codec, but can be. Or I am missing something...
Also, thanks for all the suggestions!

Greets,
I.

[ChiDragon]

Of course more speed would be great, but what I could personally really use is one that has considerably more compression than Ut Video for 1080p content and fast enough for 60 fps, heh. (Enough compression to capture clean 1080p60 4:2:0 to a single HDD; doesn't have to be completely lossless but preferred.)

Right now I capture 1080i60 and 720p60 fine with Ut Video but I want to upgrade to the new AVerMedia USB 3.0 device without doing RAID or expensive SSD. Actually, for all I know this is already possible with Ut Video but since I sometimes get drops with 720p60 I doubt it.

[Ignus2]

Thanks to everyone for all the replies so far!

To ChiDragon: I see. Doing considerably better lossless compression fast is difficult What is the throughput of that device BTW (MBytes/sec)?

[ChiDragon]

Does interframe compression take a lot more CPU time on the compression side even if the window is only 2 frames?

I don't know the exact specs of the device but it should be around 238MB/sec for 1080p60 since they claim to push uncompressed 4:2:2 across the USB 3.0 bus.

[kolak]

Originally Posted by Ignus2

Wow, thanks for the detailed answer and all the answers so far!
I added UTVideo, Cineform (don't know why I left that out, was on my list), 4:2:0, interlaced video support and keyframe-only (intra-frame-only) format support to the form options.

My question now is what would you say, if you got a lossless codec, that compresses the same ratio as UTVideo and a little bit faster at compression but at 2-2.5 times faster at decoding speed (and more than 10 times of Lagarith)?

I've read the opinions, but I don't really get yet why BT.709 or BT.601 matters, as for a lossless codec you get in bits and output the same. As I get it, it only matters when converting to/from RGB, which is not particularly the job of the codec, but can be. Or I am missing something...
Also, thanks for all the suggestions!

Greets,
I.

Show me a free, working codec (can have the same compression ratio) which is faster than UTVideo.

601/709 does not matter if you not doing RGB<->YUV conversion.

If you want more compression than you have to use intermediate codec and than Cineform is the fastest option.

[SeeMoreDigital]

Personally, I use AVClossless set to the same colour-space as the received source

[Ignus2]

Originally Posted by kolak

Show me a free, working codec (can have the same compression ratio) which is faster than UTVideo.

601/709 does not matter if you not doing RGB<->YUV conversion.

If you want more compression than you have to use intermediate codec and than Cineform is the fastest option.

I have not released it yet, so I cannot show it now
But that is partially what this survey is for, and also for the diploma thesis (I have to finish that first).
I will publish the results of the survey of course. I also posted it on the indietalk forums, and there are already some interesting answers.

[Ignus2]

Originally Posted by ChiDragon

Does interframe compression take a lot more CPU time on the compression side even if the window is only 2 frames?

I don't know the exact specs of the device but it should be around 238MB/sec for 1080p60 since they claim to push uncompressed 4:2:2 across the USB 3.0 bus.

Well, I also thought about that, but in that case, the codec won't be intraframe-only, which I thought could be a huge problem. But I guess I'm wrong here, as in the survey, almost nobody cares about whether the codec is intraframe-only or not. And that is interesting, as I thought the exact opposite.

EDIT: And while we are at it, what is more important? Compression speed or decompression speed? (I left that out of the survey).

--
Greets,
I.

[kolak]

Both are important, but decompression is probably bit more, but some codecs eg. Cineform is almost symmetric due to its wavelet nature.

If you do write one, make it 16bit and just pad with 0 for other, smaller precision (it should not create much of overhead).
UTVideo is faaast- if you can mach it at 16bit than this is already big thing

It can be interframe, but random access has to be easy- not like in case of AVC.

[WorBry]

@Ignus2

Just to be clear. Once you have completed your thesis, and are freed from the bonds of academic non-disclosure, is your intent to release this as a freely available, open-source lossless codec, as I am sure that is the expectation of those who have taken the trouble to complete the survey and made known their wishes for an 'ideal' lossless codec ? And that certainly would be my expectation if I were to participate.

Best to be upfront, I'd say

[Ignus2]

It is just 8 bit yet, intraframe only. In the survey, very few people selected 16 bit, (and those who did, only use it sometimes), so I guess it is not that mainstream yet.

To be honest, it is ready since almost 1,5 years ago, and first I thought to release it in 2 variations free and standard (crippleware in bad terms). I just never had the time to do so. Now I'm thinking to just release it in it's current form for free (not opensource though). Donations will be welcome of course (though I know all too well, that it rarely if ever works).
If all goes well, I hope the release will be around end of january.

EDIT: kolak: By easy random access you mean just using frequent and lot of keyframes?

--
Greets,
I.

[kolak]

It's up to you how it's done, as long as using it in NLE works well (scrubbing is instant).

Utvideo is soooooo fast in eg. Edius (which works in YUV). Interframe is not a problem as long as GOPs are not too long and overall decoding process is not very complex.
We have to move above 8bit- there are many great codecs for 8bit and basically none (except ffv1) for 8bit+.
Codec should also have some SDK around it, being OS independent and available for custom implementation (like Cineform). We don't need just yet another codec, but more like a bit of technology, so people can implement it in their own ways.

Do you have any details about your codec? supported modes, some speed tests, directshow, vfw, QT?

[Ignus2]

Originally Posted by kolak

It's up to you how it's done, as long as using it in NLE works well (scrubbing is instant).

Utvideo is soooooo fast in eg. Edius (which works in YUV). Interframe is not a problem as long as GOPs are not too long and overall decoding process is not very complex.
We have to move above 8bit- there are many great codecs for 8bit and basically none (except ffv1) for 8bit+.
Codec should also have some SDK around it, being OS independent and available for custom implementation (like Cineform). We don't need just yet another codec, but more like a bit of technology, so people can implement it in their own ways.

Do you have any details about your codec? supported modes, some speed tests, directshow, vfw, QT?

Thank you for your suggestions.
About the codec: as I mentioned, it's 8 bit, intraframe only. It is a VFW 32/64 bit, multi-threaded, SSE2 optimized codec. Supports: YUY2, UYVY, YV12, RGB24, RGB32 compression (no conversion support though!), and decompression of these directly to the original colorspace or to RGB32.
The interface is separate from the coder, so it shouldn't be very hard to implement it using other APIs than VFW.
Compression speed is somewhat faster than UTVideo, decompression is about 2-2,5 times faster (in 64 bit mode, 32 bit is slower, only 2x perhaps).
I've written a small command line tool to measure codec compression/decompression speed using the AVIFile API. It measures coding speed separately of disk I/O, I've made my measurements using that (real world speed depends on disk I/O a lot, but I'm talking about coding speed only).

Again, thank you for your time, but something I have to mention, as WorBry requested to be upfront about it, that I don't know how much time will I have to work on it further, or to make the move to 10 bit, and if I do, that version of the codec might end up being non-free. Of course, anyone who helped either by filling out the survey or taking the time to post here (or make any donation, when I will finally have the website up) absolutely will have it for free, but apart from that I really can't say what the future holds. Just to be upfront about it...

--
Greets,
I.

[kolak]

I'm interested in any codec and on both sides- free and pro (including even enterprise usage).
Is your codec inter or intra (I frames only) frame based?

[Ignus2]

Originally Posted by kolak

I'm interested in any codec and on both sides- free and pro (including even enterprise usage).
Is your codec inter or intra (I frames only) frame based?

Intra-only. Nothing fancy really, the "magic" is in the optimization. If you are really interested, I can send it to you in it's current form, provided you don't distribute it

[SeeMoreDigital]

Originally Posted by Ignus2

Intra-only. Nothing fancy really, the "magic" is in the optimization. If you are really interested, I can send it to you in it's current form, provided you don't distribute it

Apple offer the ability to encode H.264 using 'Intra' only frames...

[WorBry]

Originally Posted by Ignus2

Again, thank you for your time, but something I have to mention, as WorBry requested to be upfront about it, that I don't know how much time will I have to work on it further, or to make the move to 10 bit, and if I do, that version of the codec might end up being non-free. Of course, anyone who helped either by filling out the survey or taking the time to post here (or make any donation, when I will finally have the website up) absolutely will have it for free, but apart from that I really can't say what the future holds. Just to be upfront about it...

Hey, it's your creation to do what you want with. I just wasn't at all clear where you were coming from. Your subsequent posts have "cleared the fog" somewhat.

I was going to add a request for MT support, but I see you have that in hand.

Will be interesting to see how it works out.