Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > NLE - Non Linear Editing

Reply
 
Thread Tools Search this Thread Display Modes
Old 29th March 2020, 13:45   #21  |  Link
Stereodude
Registered User
 
Join Date: Dec 2002
Location: Region 0
Posts: 1,436
Quote:
Originally Posted by kolak View Post
Resolve handling of 420/422 was far from correct.
It was enough to export v210 5x back to v210 (which was always going through RGB) and you had red channel shifted few pixels! Then they improved it a bit, but it's still not very good. They will also now pass v210 (and not only) without any processing if you do just cut editing.
So is there any format that it handles better or a workaround?
Stereodude is offline   Reply With Quote
Old 29th March 2020, 14:28   #22  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,340
I mentioned it to you in a previous thread - there is a completely lossless workflow , but it involves EXR sequences and vapoursynth . There were issues before, but Resolve handles EXR float properly now, since v12 or 13 I think . Definitely with v15 it's ok.

Float enables you to convert YUV<=>RGB losslessly when done properly. If avs+ gets a float importer (import EXR that doesn't clip to 16bit int, like ffmpeg) , then it should possible in avs+ too since it can handle float now too.
poisondeathray is offline   Reply With Quote
Old 29th March 2020, 14:54   #23  |  Link
Stereodude
Registered User
 
Join Date: Dec 2002
Location: Region 0
Posts: 1,436
I've never messed with vapoursynth. Not sure of the learning curve of it vs. avisynth. Maybe I'll take a look at it later.

However, I suspect I'm missing the forest for the trees getting lost in the minutia here. The less than idea PNSR result doesn't mean mean it looks bad or has visible artifacts. I didn't do a super exhaustive comparison, but in my quick look at a few frames, I couldn't see any quickly apparent difference in the before and after frame despite the sub 50dB PNSR of the worst spots (using uncompressed video formats).
Stereodude is offline   Reply With Quote
Old 29th March 2020, 15:09   #24  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,340
Quote:
Originally Posted by Stereodude View Post

However, I suspect I'm missing the forest for the trees getting lost in the minutia here. The less than idea PNSR result doesn't mean mean it looks bad or has visible artifacts. I didn't do a super exhaustive comparison, but in my quick look at a few frames, I couldn't see any quickly apparent difference in the before and after frame despite the sub 50dB PNSR of the worst spots (using uncompressed video formats).
Of course, and 50dB is usually great for intermediates.

But it seems the min values are lower than expected. This is qp 1 intra. You should be getting results higher than prores xq or cineform fs2.

PSNR is good for is detecting lossless (or not), and indicating trends, and potential workflow problems

So I'm wondering if something else going on there
poisondeathray is offline   Reply With Quote
Old 29th March 2020, 22:09   #25  |  Link
Stereodude
Registered User
 
Join Date: Dec 2002
Location: Region 0
Posts: 1,436
Quote:
Originally Posted by poisondeathray View Post
Of course, and 50dB is usually great for intermediates.

But it seems the min values are lower than expected. This is qp 1 intra. You should be getting results higher than prores xq or cineform fs2.

PSNR is good for is detecting lossless (or not), and indicating trends, and potential workflow problems

So I'm wondering if something else going on there
I did find two issues.

1) The "retain sub-black and super-white data" in Resolve's export has to be checked.
2) You have to also use z.lib explicitly set to full range, the matrix and transfer coefficients seem to have no effect, but the range certainly does.

I also found that if you're using the internal avisynth functions converting to 8 bit first then to YV12 gives a slightly better PNSR score. I was not able to find a matrix for the internal function that gave similar results.

Here's the best z.lib/avsresize result I was able to get:


Here's the best internal function result I could get:
Stereodude is offline   Reply With Quote
Old 30th March 2020, 05:59   #26  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,340
Quote:
Originally Posted by Stereodude View Post
2) You have to also use z.lib explicitly set to full range, the matrix and transfer coefficients seem to have no effect, but the range certainly does.
Nice, that fixes the earlier observations about low Y PSNR values for avsresize/zimg for that conversion.

My thinking was input=output so it's a no-op, so I omitted the args. Lesson learned!

Quote:
I also found that if you're using the internal avisynth functions converting to 8 bit first then to YV12 gives a slightly better PNSR score.
Interesting. You'd think higher precision @ 10bit operations would produce slightly better results

Quote:
I was not able to find a matrix for the internal function that gave similar results.
Matrix only applies to YUV<=>RGB transforms , so it's not applicable here. That I'm sure of for the internal functions. (I'm questioning my understanding of how avsresize works.)



My results using the clip in avsresize/zimg. Not sure if it was the same clip because it was 5002 frames vs 5000. And the trends seem different w.r.t the frame ranges . My U plane seemed more problematic than V , but your V seemed to have more trouble than your U . Values seem slightly higher too , the min was 56.288 in vqmt , but you have clear V drops below 50
https://postimg.cc/QHcjV8V7

VQMT scales the U,V planes to Y plane size, that might introduce some errors. I tried unchecking that option , but it would crash. It might be a pro version feature. Anyways it's the trends that are important

FWIW this is ffmpeg PSNR , and the abs. min value is higher , and the U,V averages are higher than VQMT - maybe some of the difference is from the VQMT UV scaling, not sure. Often there are slightly different calcs too between programs
PSNR y:65.979368 u:64.950274 v:66.120163 average:65.812176 min:59.375502 max:88.556475

You can combine methods too. If a different method produces higher values in a plane you mix/match with combineplanes. But like you said this is splitting hairs these higher PSNR values. It just sucks to see drops like that in sections
poisondeathray is offline   Reply With Quote
Old 30th March 2020, 13:49   #27  |  Link
Stereodude
Registered User
 
Join Date: Dec 2002
Location: Region 0
Posts: 1,436
Quote:
Originally Posted by poisondeathray View Post
Interesting. You'd think higher precision @ 10bit operations would produce slightly better results
Yes, it seems counter intuitive.

Quote:
Matrix only applies to YUV<=>RGB transforms , so it's not applicable here. That I'm sure of for the internal functions. (I'm questioning my understanding of how avsresize works.)
There is talk about full vs. limited range in the matrix selection section, hence I thought it might have some impact, but apparently it's only used if going between RGB and YUV.

Quote:
VQMT scales the U,V planes to Y plane size, that might introduce some errors. I tried unchecking that option , but it would crash. It might be a pro version feature. Anyways it's the trends that are important
I found the same thing. Insta-crash.

Quote:
FWIW this is ffmpeg PSNR , and the abs. min value is higher , and the U,V averages are higher than VQMT - maybe some of the difference is from the VQMT UV scaling, not sure. Often there are slightly different calcs too between programs
PSNR y:65.979368 u:64.950274 v:66.120163 average:65.812176 min:59.375502 max:88.556475
I did the comparison in ffmpeg and got this: PSNR y:65.974546 u:57.684810 v:51.288692 average:57.718014 min:53.666230 max:88.556475

I also found that you want to make sure the two sources are exactly the same length or ffmpeg will give you erroneous numbers where as VQMT just uses the length of the shorter source for the comparison.
Stereodude is offline   Reply With Quote
Old 30th March 2020, 16:00   #28  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,340
Quote:
Originally Posted by Stereodude View Post

I did the comparison in ffmpeg and got this: PSNR y:65.974546 u:57.684810 v:51.288692 average:57.718014 min:53.666230 max:88.556475

I also found that you want to make sure the two sources are exactly the same length or ffmpeg will give you erroneous numbers where as VQMT just uses the length of the shorter source for the comparison.

Yes, for all types of metrics, you also need frames to align exactly, and frame accuracy.

FFmpeg can tricky because it uses timestamps to guide it. There are some side cases that can still yield incorrect results if you're not careful. eg. If container timebases are off (e.g. MP4 vs. MKV express differently), or jitter in the timestamps, the results can be off. 23.976 is not the same thing as 24000/1001. There are ways to reset timebase and set framerate within ffmpeg to try to get it to align (there is an example in the docs with settb and setpts). But there are still other cases - ffmpeg psnr/ssim/vmaf calcs will drop negative timestamps or open GOP b-frames at the beginning in the calculation. But if you were to decode the same video to rawvideo (ES), and use that as input, it keeps those b-frames. So on one hand ffmpeg keeps the frames, but on the other it drops them. That inconsistency can cause a problem. For example , official vmafossexec (Netflix VMAF) takes rawvideo input . This can cause discrepancies in the reported values, even though they use the same library

If you feed it avs script or vpy script , you can check for alignment (e.g. in avspmod or vsedit) , and be certain the same frames are decoded the same way and have the same timestamps (info , or use AssumeFPS if they differ)

Did you use avs input for ffmpeg ssim ? That 5-6db delta between our avg U,V results is too large for my liking if it was the same clip. And the avg Y values are close, so it doesn't suggest some misalignment issue

Same with VQMT - it can't be exactly the same clip because I had 5002 frames vs. your 5000.
poisondeathray is offline   Reply With Quote
Old 30th March 2020, 17:30   #29  |  Link
Stereodude
Registered User
 
Join Date: Dec 2002
Location: Region 0
Posts: 1,436
Quote:
Originally Posted by poisondeathray View Post
Did you use avs input for ffmpeg ssim ? That 5-6db delta between our avg U,V results is too large for my liking if it was the same clip. And the avg Y values are close, so it doesn't suggest some misalignment issue
Yes, I used two avs inputs into both VQMT and ffmpeg. What did you use?

Quote:
Same with VQMT - it can't be exactly the same clip because I had 5002 frames vs. your 5000.
I had adjusted the avs script to 5002 frames to match the sample I sent you (which was cut on the MPEG-2 GOP), but Resolve apparently didn't notice that I lengthened the intra .h264 input file and only output 5000 frames, so I just kept the comparison to 5000 frames.

However, I just went back and redid the whole thing and made sure I got 5002 frames this time.

PSNR y:65.979373 u:57.686445 v:51.298844 average:57.726204 min:53.681992 max:88.556475
Stereodude is offline   Reply With Quote
Old 30th March 2020, 17:50   #30  |  Link
Stereodude
Registered User
 
Join Date: Dec 2002
Location: Region 0
Posts: 1,436
I also got curious and decided to try something. Simply changing the iDCT algorithm used by MPEG2Source can change the output quite a bit in PSNR terms. All using the same 5002 frame MPEG-2 source.

The three 32 bit routines compared to the IEEE-1180 Reference:
PSNR y:62.449197 u:62.938405 v:62.566806 average:62.546733 min:58.816064 max:90.653474

The 64-bit floating point compared to the IEEE-1180 Reference:
PSNR y:102.999039 u:104.085312 v:104.101176 average:103.334047 min:89.595436 max:inf

SSEMMX (Skal) compared to the IEEE-1180 Reference:
PSNR y:62.376757 u:62.547402 v:62.376608 average:62.404711 min:58.623051 max:90.653474

Simple MMX (XviD) compared to the IEEE-1180 Reference:
PSNR y:65.242412 u:67.925161 v:66.576349 average:65.801940 min:61.771485 max:90.653474

ffms2 vs MPEG2Source (IEEE-1180 Reference):
PSNR y:65.346402 u:66.217016 v:65.103104 average:65.437119 min:61.781376 max:inf

Does H264 have similarly vague iDCT specs that allows different decoders to do different things that give slightly different results?
Stereodude is offline   Reply With Quote
Old 30th March 2020, 18:16   #31  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 5,340
Quote:
Originally Posted by Stereodude View Post
Yes, I used two avs inputs into both VQMT and ffmpeg. What did you use?
Yes, avs for everything. The "original" tested against in all cases is the same mpeg2source script you provided in PM


Quote:
However, I just went back and redid the whole thing and made sure I got 5002 frames this time.

PSNR y:65.979373 u:57.686445 v:51.298844 average:57.726204 min:53.681992 max:88.556475
Something else is going on . That avg U,V delta between our results is too large.


Quote:
Does H264 have similarly vague iDCT specs that allows different decoders to do different things that give slightly different results?
For h264 - I don't think so. LSmash/FFMS2 do not list those types of options. DGDecodeNV, the old DGIndexAVC do not either.

The only decoder side switch you might sometimes see is disable deblocking
poisondeathray is offline   Reply With Quote
Old 30th March 2020, 18:26   #32  |  Link
videoh
Useful n00b
 
Join Date: Jul 2014
Posts: 1,667
poisondeathray is correct. All valid 264 decoders must produce bit identical output. The IDCT is defined in the specification. Disabling deblocking is nonsensical.

Last edited by videoh; 30th March 2020 at 18:34.
videoh is offline   Reply With Quote
Old 30th March 2020, 18:28   #33  |  Link
Stereodude
Registered User
 
Join Date: Dec 2002
Location: Region 0
Posts: 1,436
Quote:
Originally Posted by poisondeathray View Post
Something else is going on . That avg U,V delta between our results is too large.
I'd chalk it up to changes in Resolve or some setting I don't know I need to change in Resolve. I'm using 16.2 and you're using 15. We get the same PSNR results on the intra version of the source.
Stereodude is offline   Reply With Quote
Old 30th March 2020, 18:30   #34  |  Link
Stereodude
Registered User
 
Join Date: Dec 2002
Location: Region 0
Posts: 1,436
Quote:
Originally Posted by videoh View Post
poisondeathray is correct. All valid 264 decoders must produce bit identical output. The IDCT is defined in the specification.
Good to know.
Stereodude is offline   Reply With Quote
Old 23rd April 2020, 03:46   #35  |  Link
Stereodude
Registered User
 
Join Date: Dec 2002
Location: Region 0
Posts: 1,436
I haven't tried it, but nothing in their list of changes would lead me to believe they fixed it.
Stereodude is offline   Reply With Quote
Reply

Tags
chroma bug, resolve

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 08:11.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.