Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > NLE - Non Linear Editing

Reply
 
Thread Tools Search this Thread Display Modes
Old 16th October 2019, 11:31   #1  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Germany
Posts: 672
Lossless export from Davinci Resolve?

Hi there,
I basically gotta work on the encoding of a documentary.
The whole project has been edited in AVID and it's now gonna be color-corrected in Davinci Resolve.
After that, they're gonna send the file over and I'm gonna have to encode it to make the official Bluray version (I'm gonna use Avisynth and x264 of course), the Motion JPEG2000 version for the DCP for which I'm gonna use Avisynth and ffmpeg/OpenDCP and the MPEG-2 XDCAM version for air-time in our linear channels using Avisynth and ffmbc.
Davinci Resolve is running on Windows 10 and they wanted to know how they have to export it. I don't have access to that computer, so I can't check which codecs it supports on export.
I do know, however, that it cannot export in AppleProRes 'cause it's running on Windows.
Anyway, since my encode it's gonna be a lossy one, I'd like to make them export the whole thing in lossless, so the question to you folks is: which lossless codecs does Davinci Resolve support on Windows?
And, if it doesn't support any lossless codec, which is the "less lossy" codec and settings I can ask them to export the file as?

The final project is gonna be a BT709 SDR 100 nit FULL HD 8bit yv12, even though it has been shot in UHD 10bit 4:4:4 planar in log with a number of nits that were actually enough to make an HLG HDR content. I know you may think that it's "madness" to go for a FULL HD SDR 8bit in 2019 (almost 2020), but the documentary is full of old archive footages coming from very old 35mm and it took a lot of time to remaster it, upscale it to FULL HD with NNEDI and Spline64 with 16bit precision and convert it to BT709, so bringing it to UHD and BT2020 HLG would be madness and it would also potentially make it look not only worse but also different from how the author wanted it to look like, so...



Anyway, in a nutshell, does Davinci Resolve support lossless codecs?
And if it doesn't which is the less lossy one I can ask them to export?

Thank you in advance as always,
Frank.
__________________
Broadcast Encoder
Avisynth memes: 1 - 2 - 3
Videotek - Audacity XP
FranceBB is offline   Reply With Quote
Old 16th October 2019, 11:52   #2  |  Link
ChaosKing
Registered User
 
Join Date: Dec 2005
Location: Germany
Posts: 1,033
Never used davinci resolve, but here is a list of supported decoder/encoder on each plattform https://documents.blackmagicdesign.c...Codec_List.pdf
__________________
AVSRepoGUI // VSRepoGUI - Package Manager for AviSynth // VapourSynth
VapourSynth Portable FATPACK || VapourSynth Database || https://github.com/avisynth-repository
ChaosKing is offline   Reply With Quote
Old 16th October 2019, 16:29   #3  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 3,977
Uncompressed (MOV, AVI) and image sequences ( EXR , TIFF, DPX) are the only "true" lossless export formats . It does not support 3rd party codecs

If Prores 4444XQ is "good enough" for some situation, DNxHR 12bit 4444 should be "good enough" too , so if you don't need mathematically lossless, that' s what people use on Win version of Resolve for those situation

However, beware that FFMpeg/Libav (and thus avisynth) can have some issues with decoding DNxHR 4444, because of the ACT (adaptive color transform) used in official codec implementations (such as Resolve, Adobe, Avid) , so if subsampled 422 is ok, that 's the variant I'd use if using avisynth or ffmpeg

https://trac.ffmpeg.org/ticket/8157

Last edited by poisondeathray; 16th October 2019 at 16:48.
poisondeathray is offline   Reply With Quote
Old 16th October 2019, 16:44   #4  |  Link
Cary Knoop
Cary Knoop
 
Cary Knoop's Avatar
 
Join Date: Feb 2017
Location: Newark CA, USA
Posts: 208
You also might want to take a look at the Kakadu option.
Cary Knoop is offline   Reply With Quote
Old 16th October 2019, 18:02   #5  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Germany
Posts: 672
Quote:
Originally Posted by poisondeathray View Post
Uncompressed (MOV, AVI)
Whoa, that's gonna make a huge file. It's gonna be impossible to send it over, even using Aspera.

Quote:
image sequences ( EXR , TIFF, DPX) are the only "true" lossless export formats . It does not support 3rd party codecs
Oh, God, no. That reminds me of when I had to deal with shootings made by Blackmagic 6K cameras in .dng lossless. Each and every frame was an image, everything was kept together with metadata and... sure, it was lossless, so maximum quality recorded by the camera with the maximum amount of stops (nits) available, but the size was HUGE and making a mezzanine file took a hell lot of time and it was a nightmare... Really, an absolute nightmare!

Quote:
If Prores 4444XQ is "good enough" for some situation, DNxHR 12bit 4444 should be "good enough" too , so if you don't need mathematically lossless, that' s what people use on Win version of Resolve for those situation
Yep, if they were on OSX, I would have asked them ProRes4444XQ. Wavelet transform at such an high bitrate, 10bit and 4:4:4 can't really make any visible damage, so it doesn't matter if it's not mathematically lossless as long as it's visually lossless.
Anyway, they are on Windows, so they cannot export on ProRes, therefore I'm actually gonna go with DNxHR 4444 12bit. I mean, it still uses the Wavelet transform (which is a good transform by the way), it has 12bit and 4:4:4 so it's definitely gonna be fine.


Quote:
However, beware that FFMpeg/Libav (and thus avisynth) can have some issues with decoding DNxHR 4444, because of the ACT (adaptive color transform) used in official codec implementations (such as Resolve, Adobe, Avid) , so if subsampled 422 is ok, that 's the variant I'd use if using avisynth or ffmpeg
I see...
Well, the thing is that 4:2:2 is definitely gonna be fine for the H.264 yv12 8bit bluray encode and for the MPEG-2 XDCAM yv16 8bit encode, however I would really like not to upscale the chroma for the DCP encode which is gonna be MJPEG2000 4:4:4 12bit. If it's really gonna be an issue when I try to index it with, let's say, FFMpegSource2 or LWLibavVideoSource in Avisynth because of the fact that some blocks were encoded in YUV and some others in RGB, then perhaps I'll ask them to send over a 4:2:2 version and then I'm sadly gonna upscale the chroma for the DCP version. I gotta say that I wasn't aware of the "hybrid" RGB/YUV approach used by the adaptive color transform. It's a shame that it's not decoded properly, though.
__________________
Broadcast Encoder
Avisynth memes: 1 - 2 - 3
Videotek - Audacity XP

Last edited by FranceBB; 16th October 2019 at 18:07.
FranceBB is offline   Reply With Quote
Old 16th October 2019, 22:18   #6  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 3,977
Quote:
Originally Posted by Cary Knoop View Post
You also might want to take a look at the Kakadu option.
This should work ; and it has a lossless option too

ffms2/lsmash decode as RGB48 , but it's a slow decoder for JPEG2K , probably not well optimized
poisondeathray is offline   Reply With Quote
Old 17th October 2019, 17:09   #7  |  Link
kolak
Registered User
 
Join Date: Nov 2004
Location: UK
Posts: 2,412
Quote:
Originally Posted by FranceBB View Post
Whoa, that's gonna make a huge file. It's gonna be impossible to send it over, even using Aspera.

I see...
Well, the thing is that 4:2:2 is definitely gonna be fine for the H.264 yv12 8bit bluray encode and for the MPEG-2 XDCAM yv16 8bit encode, however I would really like not to upscale the chroma for the DCP encode which is gonna be MJPEG2000 4:4:4 12bit. If it's really gonna be an issue when I try to index it with, let's say, FFMpegSource2 or LWLibavVideoSource in Avisynth because of the fact that some blocks were encoded in YUV and some others in RGB, then perhaps I'll ask them to send over a 4:2:2 version and then I'm sadly gonna upscale the chroma for the DCP version. I gotta say that I wasn't aware of the "hybrid" RGB/YUV approach used by the adaptive color transform. It's a shame that it's not decoded properly, though.
If you want this as source for BD then DNxHR HQX or Cinefrom YUV (best setting) are way good enough and can be reliably exported from Resolve on PC (and later processed).

Last edited by kolak; 17th October 2019 at 17:11.
kolak is offline   Reply With Quote
Old 18th October 2019, 13:49   #8  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Germany
Posts: 672
In the end, I asked them to export an DNxHR 4:2:2 planar 12bit and I'm gonna upscale the chroma to 4:4:4 for the MJPEG2000 Cinema encode (Avisynth + ffmpeg and OpenDCP), while I'm gonna downscale the chroma to 4:2:0 planar and then dither everything down to 8bit with the Floyd-Steinber error diffusion for the Bluray release in H.264 (Avisynth + x264 + ffmpeg + Scenarist).

Thanks.
__________________
Broadcast Encoder
Avisynth memes: 1 - 2 - 3
Videotek - Audacity XP
FranceBB is offline   Reply With Quote
Old 18th October 2019, 14:57   #9  |  Link
Cary Knoop
Cary Knoop
 
Cary Knoop's Avatar
 
Join Date: Feb 2017
Location: Newark CA, USA
Posts: 208
Quote:
Originally Posted by FranceBB View Post
...then dither everything down to 8bit with the Floyd-Steinber error diffusion for the Bluray release in H.264
Are you sure you want to use Floyd–Steinberg dithering for H.264?
Cary Knoop is offline   Reply With Quote
Old 18th October 2019, 15:37   #10  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Germany
Posts: 672
Quote:
Originally Posted by Cary Knoop View Post
Are you sure you want to use Floyd–Steinberg dithering for H.264?
That's what I generally use. Shall I send to x264 the 12bit output directly and let it dither it down automatically to 8bit?
Ideally, I'd like to avoid getting banding artifacts while retaining as many details as possible. Since it's a documentary, though, there are many old archive footages that are very grainy (we had a discussion about them when I encoded them to make our internal mezzanine files and the director was against de-graining them as he wanted to keep the original with the original grain). Some other footages, instead, are the ones we shot in studio or travelling across the country but with accurately handled light in log with 17 stops which have then been graded to BT709 100 nits in post-production which is a waste considering that 17 stops are more than enough to get 1000 nits, but we decided to have our entire workflow as SDR for historical reasons. (i.e it didn't make much sense to have 4K HLG interviews perfect in every detail and a crappy 35mm digitalized and upscaled on the next scene).
The rest of the contents which are not interviews or archive footages are just relatively slow pan.
It has been entirely shot at 25fps progressive.
__________________
Broadcast Encoder
Avisynth memes: 1 - 2 - 3
Videotek - Audacity XP

Last edited by FranceBB; 18th October 2019 at 15:41.
FranceBB is offline   Reply With Quote
Old 18th October 2019, 17:05   #11  |  Link
kolak
Registered User
 
Join Date: Nov 2004
Location: UK
Posts: 2,412
Quote:
Originally Posted by Cary Knoop View Post
Are you sure you want to use Floyd–Steinberg dithering for H.264?
Works well, but you need to add tiny amount of noise as well and use fairly high bitrate, eg. 30Mbit+.
I've not found anything else as efficient and 'transparent'.

x264's Sierra-2-4A is also ok.

Last edited by kolak; 18th October 2019 at 17:11.
kolak is offline   Reply With Quote
Old 18th October 2019, 17:18   #12  |  Link
Cary Knoop
Cary Knoop
 
Cary Knoop's Avatar
 
Join Date: Feb 2017
Location: Newark CA, USA
Posts: 208
Quote:
Originally Posted by kolak View Post
Works well, but you need to add tiny amount of noise as well and use fairly high bitrate, eg. 30Mbit+.
I've not found anything else as efficient and 'transparent'.
But isn't Floyd–Steinberg dithering just a tad too good to survive the H.264 "macro-block hammer"?

Here is an example of a Floyd–Steinberg dithered clip. Look at the black patches, they are a bit too restless, using a lower quality dithering seems better for H.264/VP9.

https://www.youtube.com/watch?v=MDtQPJJ20Q8

Last edited by Cary Knoop; 18th October 2019 at 17:23.
Cary Knoop is offline   Reply With Quote
Old 18th October 2019, 17:30   #13  |  Link
kolak
Registered User
 
Join Date: Nov 2004
Location: UK
Posts: 2,412
For typical low bitrate web encodes probably it's. For BD with good encodes at relatively high bitrate not necessarily. I was using Cinemacraft h264 encoder- maybe it behaves differently than x264.
Youtube is no near BBD quality, so it's rather very different scenario.
kolak is offline   Reply With Quote
Old 18th October 2019, 17:31   #14  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 3,977
Quote:
Originally Posted by Cary Knoop View Post
But isn't Floyd–Steinberg dithering just a tad too good to survive the H.264 "macro-block hammer"?

Here is an example of a Floyd–Steinberg dithered clip. Look at the black patches, they are a bit too restless, using a lower quality dithering seems better for H.264/VP9.

https://www.youtube.com/watch?v=MDtQPJJ20Q8
Youtube bitrates and encoding is not going to be representative of blu-ray bitrates with offline encoding
poisondeathray is offline   Reply With Quote
Old 18th October 2019, 17:34   #15  |  Link
kolak
Registered User
 
Join Date: Nov 2004
Location: UK
Posts: 2,412
Quote:
Originally Posted by FranceBB View Post
In the end, I asked them to export an DNxHR 4:2:2 planar 12bit and I'm gonna upscale the chroma to 4:4:4 for the MJPEG2000 Cinema encode (Avisynth + ffmpeg and OpenDCP), while I'm gonna downscale the chroma to 4:2:0 planar and then dither everything down to 8bit with the Floyd-Steinber error diffusion for the Bluray release in H.264 (Avisynth + x264 + ffmpeg + Scenarist).

Thanks.
You should use Cineform RGB export in Resolve. Would preserve 12bit, RGB nature and then would be plenty to make BD version.

Last edited by kolak; 18th October 2019 at 20:14.
kolak is offline   Reply With Quote
Old 18th October 2019, 17:44   #16  |  Link
Cary Knoop
Cary Knoop
 
Cary Knoop's Avatar
 
Join Date: Feb 2017
Location: Newark CA, USA
Posts: 208
Quote:
Originally Posted by poisondeathray View Post
Youtube bitrates and encoding is not going to be representative of blu-ray bitrates with offline encoding
True, but dithering with noise is going to be a challenge even for Blu-ray encodings right?
Cary Knoop is offline   Reply With Quote
Old 18th October 2019, 17:49   #17  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 3,977
Quote:
Originally Posted by Cary Knoop View Post
True, but dithering with noise is going to be a challenge even for Blu-ray encodings right?
Yes - and x264 is good at retaining dither and grain at those bitrates for BD . That's what it's known for
poisondeathray is offline   Reply With Quote
Old 18th October 2019, 18:57   #18  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Germany
Posts: 672
Quote:
But isn't Floyd–Steinberg dithering just a tad too good to survive the H.264 "macro-block hammer"?
I hope it will survive at BD-tier bitrate.

Quote:
For typical low bitrate web encodes probably it's. For BD with good encodes at relatively high bitrate not necessarily.
Ok, that's what I thought.

Quote:
x264 is good at retaining dither and grain at those bitrates for BD . That's what it's known for
Yep, x264 is my de-facto choice for H.264 encodes and official BD releases, not only 'cause it's open source, but because it's the best.
I say it also 'cause it has been proven to be better than closed-source encodes in terms of SSIM/PSNR/VMAF with tests I've made internally in the company I work for.

Quote:
Works well, but you need to add tiny amount of noise as well and use fairly high bitrate, eg. 30Mbit+.
I've not found anything else as efficient and 'transparent'.

x264's Sierra-2-4A is also ok.
Sure, since it's an official BD, I'm gonna use a two-pass encode with x264.


Quote:
You should use Cineamcraft RGB export in Resolve. Would preserve 12bit, RGB nature and then would be plenty to make BD version.
I see. That would be great 'cause this way I won't have to upscale the chroma when I got to 4:4:4 12bit MJPEG2000 for the cinema version and of course it's gonna be more than enough for the yv12 8bit encode.
__________________
Broadcast Encoder
Avisynth memes: 1 - 2 - 3
Videotek - Audacity XP
FranceBB is offline   Reply With Quote
Old 18th October 2019, 20:13   #19  |  Link
kolak
Registered User
 
Join Date: Nov 2004
Location: UK
Posts: 2,412
Quote:
Originally Posted by FranceBB View Post

Yep, x264 is my de-facto choice for H.264 encodes and official BD releases, not only 'cause it's open source, but because it's the best.
I say it also 'cause it has been proven to be better than closed-source encodes in terms of SSIM/PSNR/VMAF with tests I've made internally in the company I work for.


Sure, since it's an official BD, I'm gonna use a two-pass encode with x264.

I see. That would be great 'cause this way I won't have to upscale the chroma when I got to 4:4:4 12bit MJPEG2000 for the cinema version and of course it's gonna be more than enough for the yv12 8bit encode.
If you use relatively high bitrates (30Mbit+) Cinemacrfat can be better than x264. It has better quality distribution between I,P,B frames. At lower bitrates x264 is better. x264 can also produce nasty artefacts at high contrast edges as well (even at 30Mbit+), up to level which makes (otherwise very good encode) unacceptable. As for bigger studio Cinemacraft has also plenty other features which can be useful (segment re-encode/replace, SDI preview, etc). It's also faster than x264 as in order to be at reference level you need to run x264 at slow or even very slow preset. In the same time you can make nice BDs with x264 as well
For interlaced content my choice was actually Sony's Blu-Code encoder, which wasn't that popular. As for today BD authoring business brings no money (well you can make money if you get good contracts, but it's a lot of work), so I moved away from it

Cineform is also good as it's wavelet based, so re-encoding to JPEG2000 will cause less quality loss due to same nature of the encoding technique.

Last edited by kolak; 18th October 2019 at 20:22.
kolak is offline   Reply With Quote
Old 18th October 2019, 20:38   #20  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 3,977
Quote:
Originally Posted by kolak View Post
If you use relatively high bitrates (30Mbit+) Cinemacrfat can be better than x264. It has better quality distribution between I,P,B frames. At lower bitrates x264 is better. x264 can also produce nasty artefacts at high contrast edges as well (even at 30Mbit+), up to level which makes (otherwise very good encode) unacceptable
When you have a noisy, grainy source - that's difficult to compress. 35Mb/s might actually be considered relatively "low" in that scenario depending on how much noise and grain you actually have. 30Mb/s might not be "enough" for some encoders. 35Mb/s might not be enough. That's where a higher compression efficiency encoder helps. Watch out for splotchy ugly grain distribution in some of the other encoders.

Yes, x264 can have problems when using default settings. The AQ redistribution tends to generate a "halo" like effect of reduced grain around edges. When combined with higher psy-rd and psy-trellis, there can be artifacts not obscured by the grain. Also, there can be noticable I/B/P fluctuations. That's why the --tune grain preset has adjusted settings for --aq-strength and IP/PB ratios. You can adjust them farther if it's still a problem .

Quote:
As for bigger studio Cinemacraft has also plenty other features which can be useful (segment re-encode/replace, SDI preview, etc). It's also faster than x264 as in order to be at reference level you need to run x264 at slow or even very slow preset. In the same time you can make nice BDs with x264 as well
Yes, segment reencoding especially is sorely lacking for x264 . Supposedly it was going to get it for a separate special BD version, but never materialized


Quote:
Originally Posted by FranceBB View Post

Yep, x264 is my de-facto choice for H.264 encodes and official BD releases, not only 'cause it's open source, but because it's the best.
I say it also 'cause it has been proven to be better than closed-source encodes in terms of SSIM/PSNR/VMAF with tests I've made internally in the company I work for.
The problem with VMAF/SSIM/PSNR etc... they do not measure noisy grainy sources very well. VMAF is slightly better, but still has demonstratable problems in this 1st generation of models. Be careful not to rely on them to heavily, especially in the noisy/grainy scenario . Double check with your eyes as well

Last edited by poisondeathray; 18th October 2019 at 21:07.
poisondeathray is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 15:17.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.