Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
21st November 2022, 18:17 | #321 | Link | ||
Registered User
Join Date: Mar 2021
Location: North Carolina
Posts: 138
|
Quote:
Thank you for the response, I am reading it, I just wanted to make it clear what I'm doing. I'll have to take a look at thread ripper, I've heard a few people talking about it before. Thank you. The maxed-out spec for 4K-UHD-BD is 100 Mbps, but after authoring I don't believe anyone can make it work. It doesn't mux properly from what I've heard. However, I believe the absolute bleeding edge maxed is 98.5 Mbps, but I just round down to 98 Mbps. I've only seen a handful of encodes on 4K-UHD-BD that I believe are absolutely perfect, Django being one of them. That was encoded at 96 Mbps average bit-rate, and the grain never broke up or was ever blurred. It was released in 2021 and the transfer still cannot be beaten today. Quote:
I've encoded Sol Levante from Netflix at 98 Mbs, and optimized the encoding settings to finish the encode with an average QP of 7.60. But, as you say, there's transparent, and then there's different levels of transparency. I have found that the biggest differences that have a meaningful impact of picture quality and detail is...Obviously having enough bits to make the image look good, but no-sao, no-deblock, no-strong-intra-smoothing, subme=7 (more sharpness), aq-mode=3 with aq-strength=1.7 is the best combination for Sol Levante that I've found, and I've tested a lot of combinations. I've also found that ipratio=1.00 & pbratio=1.00 allow for a much more consistent image from a QP point of view. They are all just about equal. I've also found that using b-adapt=0 actually leads to better utilization of bits, because my bframes are higher quality that my I and P frames, and having b frames at 75% cheaper bits is simply better than having the system sometimes use bframes, and then at other times not. I personally think maximizing the use of bframes is extremely important for formats like 4K-UHD-BD because they can only use 3 bframes. I have never found that using b-adapt= (any setting above) leads to better results ever. Obviously digitally made content is easier to get QP levels down lower. Film grain material will probably be around the 20s for QP, but the biggest issue getting film grain consistency to look perfect. Just some thoughts I have. |
||
26th November 2022, 21:51 | #323 | Link | |||
Lost my old account :(
Join Date: Jul 2017
Posts: 324
|
Quote:
Quote:
If thats not the case and the goal is quality, why shot yourself in the foot and comply to uhd-bd? Just set level 5.1 high teir and a low crf and tune for that. You are ofc free to do whatever you want, I'm just trying to understand your use case/rational here, cause I cant really wrap my head around what your trying to achieve. Quote:
Last edited by excellentswordfight; 26th November 2022 at 21:59. |
|||
29th November 2022, 00:52 | #324 | Link |
Moderator
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,770
|
In particular, it seems really unlikely that --subme 7 would do anything to improve detail at those bitrates. I've not been able to demonstrate any visible improvement using 7 even at <500 Kbps bitrates. Not even --preset placebo goes that high, because MCW didn't find it offered quality/speed improvements valuable even at placebo. --preset veryslow only uses 4.
And all >5 does is add an extra half and quarter pel iteration, which only would matter if the second iteration found something better than the first did. Higher subme values can shave a little of QPs, but at a bitrate where you're already probably getting mean QP around 10, QP could go up or down by 2-3 without any visible change. Getting an extra 0.1 QP isn't visible even at very low bitrates. You'd need individual frames getting at least a 1 QP difference before seeing much. Last edited by benwaggoner; 29th November 2022 at 00:57. Reason: Added detail |
30th November 2022, 07:11 | #325 | Link | |
Registered User
Join Date: Mar 2021
Location: North Carolina
Posts: 138
|
Quote:
From my understanding of bit-rate distribution, is, you want low detail so the encoders don't have to try and waste bits on the ultra-high detail that it will be picking out. So, using low subme settings, intra smoothing, sao and deblock should give better results for low bit-rate content. Picking out blades of grass and fine facial detail wouldn't be distributing bits in the best way when you have a very restricted bit-rate. That's how I see it, sounds right to me, unless I'm missing something. I use aqmode 3 for Sol yes, I found that it improved microbanding. I didn't see improvements in picture quality with aqmode 1 or 2 personally even though they typically should be used for this type of content. As you said, even though minor QP changes will not be seen with the eye, aqmode 3 was the lowest I could get vs mode 1 and 2. |
|
30th November 2022, 16:32 | #326 | Link | |
Registered User
Join Date: Feb 2002
Location: San Jose, California
Posts: 4,407
|
Quote:
What happens that is bad when using high subme at low bitrate? Subme is a pure quality v.s. speed option, using low subme as a way to smooth the video is a terrible idea! All it is doing is using a worse prediction, so the codec has to quantize the differences more to maintain the same bitrate. This raises the QP of all predicted blocks, lowering quality across the board, not only dropping details.
__________________
madVR options explained |
|
1st December 2022, 01:52 | #327 | Link | |
Moderator
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,770
|
Quote:
There are some exceptions to what seem to be pure speed/quality tradeoffs. For example, grainy content at 4K can look at lot better with --rd 4 than --rd 6 for reasons I've not yet root caused to my satisfaction. It seems to be related to increased QP fluctuation between adjacent CTUs. And I speculate that it's a bigger issue in 4K + grain as the single-to-noise ratio can be a lot worse than lower resolutions, as the grain may be the only highly detailed element in a GOP. 80's/90's Super 35 titles like Ghostbusters or Men in Black have maybe 720p of actual detail with a 4K overly of temporally and spatially random noise. Downscaling is a powerful low-pass filter, so the same source encoded at 1080p has the same amount of signal but a quarter the noise. |
|
4th December 2022, 06:42 | #328 | Link | |
Registered User
Join Date: Mar 2021
Location: North Carolina
Posts: 138
|
Quote:
|
|
4th December 2022, 06:46 | #329 | Link | |
Registered User
Join Date: Mar 2021
Location: North Carolina
Posts: 138
|
Quote:
This is the link to my encode of Sol Levante from Netflix it's open source, (just letting others know). https://drive.google.com/file/d/1ySV...ew?usp=sharing |
|
4th December 2022, 08:26 | #330 | Link |
Registered User
Join Date: May 2005
Posts: 1,462
|
To answer the question for me, definitely the i9 12900K. It does the encoding near twices as fast as my old i9 10700K. But, it's imperative you use P-Cores only! If not, the E-Cores will become all saturated, leaving all P-Cores near idle (as they're all waiting for the E-Cores to finish their chinks). See:
CPU loads
__________________
Gorgeous, delicious, deculture! |
5th December 2022, 21:04 | #331 | Link | |
Lost my old account :(
Join Date: Jul 2017
Posts: 324
|
Quote:
I had a look at that encode, and it displayed what I suspected, a lot of your unorthodox setting really doesnt do you any good. I compared it to a uhd-bluray encode I did a year ago at about 50Mbps avg bitrate with pretty much just preset slower and some minor teaks. https://screenshotcomparison.com/comparison/30065 (mouse on for my enocode, no tone mapping applied and cropped to 1080p). This is ofc just a screenshot, but I couldnt find any improvments with your encode at higher a bitrate. Most of the more easily compressed sections looks fine (as expected at this bitrate), but its quite obvious that your are doing stuff that hurt compression in the though scenes. How exactly did you come to the conclusion to disable strong-intra-smoothing, sao & deblock for this source? Even keeping SAO fully enabled is great for this source (i've played with selective-sao for it, but its a bit of a tradeoff). I have all those turned on, and still my version is sharper... Last edited by excellentswordfight; 6th December 2022 at 00:01. |
|
6th December 2022, 06:08 | #332 | Link | |
Registered User
Join Date: Mar 2021
Location: North Carolina
Posts: 138
|
Quote:
I'm happy with the way it looks, but if there are any errors with it I'd like to take a look and see what settings would improve it. Last edited by HD MOVIE SOURCE; 7th December 2022 at 17:30. |
|
6th December 2022, 17:00 | #333 | Link | |
Registered User
Join Date: Mar 2021
Location: North Carolina
Posts: 138
|
Quote:
You mentioned --rd 4 / --rd 6 and how it affects film grain. From my understanding the rd setting is trying to lower the bit-rate and to get the perceived quality to be exactly the same, is that correct? If this is the case, would it be true that rd=6 is trying to lower the bit-rate so much it's actually causing it to miss some film grain and cause inconsistencies in the film grain? So for example rd=1 isn't really trying to lower the bit-rates so it's actually retaining much of the image. But, I'm quite new to encoding, and I'm just trying to think why this is happening. Another similar thought is that the algorithm used by x265 is simply not good enough to attempt to lower the bit-rates on film grain content without loss. Would --ipratio=1.00 --pbratio=1.00 -- psy-rd=0.00 -- psy-rdoq=0.00 -- rdoq-level=0 help? The thought process here to use equal QP, and to have zero influence from psy and rdoq, which is typically used on film grain content to increase detail, (from my understanding). I wonder if any of the psy settings are having an effect with the rd setting? If people are worried about film grain detail, there are other settings that can increase film grain detail without going into psy and rdoq. Last edited by HD MOVIE SOURCE; 7th December 2022 at 17:28. |
|
7th December 2022, 20:07 | #334 | Link | ||
Moderator
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,770
|
Quote:
[QUOTE]--amp, --no-amp Enable analysis of asymmetric motion partitions (75/25 splits, four directions). At RD levels 0 through 4, AMP partitions are only considered at TU sizes 32x32 and below. At RD levels 5 and 6, it will only consider AMP partitions as merge candidates (no motion search) at 64x64, and as merge or inter candidates below 64x64. Quote:
(Yes, lowering QP of an IDR or I takes several times more bits than lowering QP of a P frame, but there's generally only one of those frames per GOP, with all frames of a GOP benefiting from a better IDR) (And yes, the biggest IDR benefits apply to variable GOP where the whole GOP is a single continuous shot.) |
||
9th December 2022, 18:23 | #335 | Link | |
Registered User
Join Date: Mar 2021
Location: North Carolina
Posts: 138
|
[QUOTE=benwaggoner;1979471]Good questions. After some rumination, I have a new working theory. The big problem with noisy --rd 6 is increasingly visible differences between CUs in QP and shape. It may not be --rd 6 itself that is doing it, but features of other modes that kick in at --rd >4.
Quote:
I think the biggest thing with film grain content is that it's absolutely consistent from a pixel point of view. What I mean is, take an old movie like Django for instance. It's a constant barrage of film grain all the time. There's things moving all of the time. It never ends. When the encoder sees grain it's almost like movement right? Or it's actually worse than movement? At least with movement, if something pans from left to right the encoder can just move the background from one side to the other side and save a good amount of bits on prediction. The issue with film grain is it's completely unpredictable from teh encoder. If we look back from afar at a screen with film grain on it, it looks generally consistent. But to the encoder, every single pixel is always changing. Having inconsistent QP values is like dialing into a radio station but getting static noise. The I frame is super loud, the P frame is less loud and the B frame is quiet. This is just how I'm picturing this in my head as a way to understand what's happening to the frames. quiet frames would be the b frames and low QPs, and the noisier scenes are louder at higher QPs. Just another thought, I wonder if certain predictions like amp or rect, rd, and there could be others...that actually hinder film grain or have limited effect. I'm not saying that they are specifically, but as you said, maybe some type of combination could be affecting it. Another thought and I say this because I always use CRF. My thought is that CRF could actually help with film grain because it always aims for a certain level of quality rather than bit-rate. Now, I know you're going to say, well, you can't control bit-rate with CRF, but you actually can. You can still use bv-maxrate=**** and vbv-bufsize=**** as I do with my encodes. Even if you set the CRF=0 you can still use bv-maxrate=12000 vbv-bufsize=15000 or so to target 12 Mbps. The biggest issue with this and even 2 pass encoding is available bit-rate. Once you start encoding at really high bit-rates like 90 Mbps film grain really balances out, especially with equal QP. Unfortunately for the streaming world and even people trying to lower the bit-rates of their own movies, it's just not viable to run bit-rates anywhere near that high. I'd like to try and encode some film grain content. I'd like some that isn't Tears of Steel. TOS has a digital-looking grain that's been applied and is very inconsistent. Even the DCP is blurry at times. You can watch the credits at the end, and the grain looks like blurry noise. Do you know of any other open-source content that was either shot on film or has film grain that's pretty consistent looking? I'd like to play around with some settings without going heavy into the psy settings, but taking a look at some other settings and just seeing if they have a visible effect on film grain. Thanks, I really enjoy this conversation, because making film grain look good and consistent with breaking up is a fun challenge, especially with x265, which many say is far harder to make film grain look good vs x264. I wonder, have you run any tests with x265 and found settings that exactly emulate how an image looks vs x264? I wonder if anything could be learned from the way that encoder does things, what settings are good, and applying those ideas to x265? Thanks. |
|
9th December 2022, 22:42 | #336 | Link | ||||||||
Moderator
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,770
|
Quote:
Quote:
Quote:
Quote:
Quote:
Quote:
The FGS decoder module is out of loop of the AV1 decoder, and it appears lots of hardware will make the AV1 FGS it available for other codecs, like VVC. And I think there are some that can do it for HEVC using SEI metadata. Quote:
I can't think of any broadly available very grainy sources, alas. The lack of good grainy sources in standard test libraries is probably one reason issues in grain handling aren't more directly addressed in reference encoders. Quote:
|
||||||||
10th December 2022, 14:53 | #337 | Link | |
Lost my old account :(
Join Date: Jul 2017
Posts: 324
|
Quote:
Code:
--preset slow --profile main10 --level-idc 41 --keyint 240 --min-keyint 24 --rc-lookahead 48 --aq-mode 1 --no-sao --deblock -1:-1 --psy-rd 2.5 --psy-rdoq 4 --colorprim bt709 --transfer bt709 --colormatrix bt709 https://screenshotcomparison.com/comparison/30156 (mouse on for x265) Encoded with crf17 with x264, and crf for x265 is set to match average bitrate. so --aq-mode 1 --no-sao --deblock -1:-1 and a slight psy tweak and you have something similar to x264 with tune film. Last edited by excellentswordfight; 10th December 2022 at 21:37. |
|
10th December 2022, 23:08 | #338 | Link | |
Registered User
Join Date: Mar 2021
Location: North Carolina
Posts: 138
|
Quote:
I running some other tests like rd=3 just to see if rd=4 is the limit of quality when it comes to film grain. Then I might see if there's any other odd interactions with some other settings later. I wonder if this is something that the software guys could actually update in a future build? |
|
12th December 2022, 21:55 | #339 | Link | |||
Moderator
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,770
|
Quote:
Quote:
Quote:
Since the problems are by far the worst at 4K, I suspect there's some threshold of noise detail versus content detail where things go wonky if crossed. |
|||
13th December 2022, 16:58 | #340 | Link |
Lost my old account :(
Join Date: Jul 2017
Posts: 324
|
Finally got a Epyc milan to play with, and its a nice upgrade over Rome.
Amd Epyc "Milan" 7543P - 6,4fps Amd EPyc "Rome" 7402P - 4,86fps Intel Xeon "Cascade Lake Refresh" 2x6226R - 3,97fps All 1u Dell servers in performance mode, doing a 2160p HDR10 transcode of STEM2 with preset slow, CPU utilization is about 50-75%, all system has 64 threads and 8x16GB for Epyc, and 12x8GB RAM for Intel. Last edited by excellentswordfight; 13th December 2022 at 17:05. |
Thread Tools | Search this Thread |
Display Modes | |
|
|