2 Pass min quality settings?

[JoyBell]

I do many encodes of seasons of shows for my archive. I have long favored hour 2 pass balances quality in a given budget. I also find that 2 pass pleases my eyes with vs CRF as to do with balancing movement vs dialog scenes. That's all my taste, now to my question.

Is there a way to encode x265 in two pass with a minimum effective CRF quality for the entire encode or any given scene? That is to say, if I set an encode for 2 pass 1500kbs but never want worse quality than CRF 26, for the total encode or any give scene, at which I am willing to let the encode exceed the 1500kbs setting.


I have tried the --crf-max setting any I tried encodes of a sitcom. I started with --crf-max 30 and did encodes lowering the --crf-max 29, 28, 27, 26 each time. All of which came out to my exact bitrate setting with no change. I then tried --crf-max 25 and that encode came out 4334kbs! These tests where entire episodes not just 1 minute test clips or anything. I feel that I am not understanding something about how this works and am asking for help.


My idea, is. I do an entire season, and I say two pass 1500kbs... .But we know there are always those few episodes that could use a bit more..... so I would be cool if a few came out 1645kbs, or 1721kbs, but most stuck to 1500kbs.
I am looking for an escape clause to break the bitrate budget in an emergency to keep a minimum quality.


To be clear, no I don't want to do CRF.

[RainyDog]

Originally Posted by JoyBell

I do many encodes of seasons of shows for my archive. I have long favored hour 2 pass balances quality in a given budget. I also find that 2 pass pleases my eyes with vs CRF as to do with balancing movement vs dialog scenes. That's all my taste, now to my question.

Is there a way to encode x265 in two pass with a minimum effective CRF quality for the entire encode or any given scene? That is to say, if I set an encode for 2 pass 1500kbs but never want worse quality than CRF 26, for the total encode or any give scene, at which I am willing to let the encode exceed the 1500kbs setting.


I have tried the --crf-max setting any I tried encodes of a sitcom. I started with --crf-max 30 and did encodes lowering the --crf-max 29, 28, 27, 26 each time. All of which came out to my exact bitrate setting with no change. I then tried --crf-max 25 and that encode came out 4334kbs! These tests where entire episodes not just 1 minute test clips or anything. I feel that I am not understanding something about how this works and am asking for help.


My idea, is. I do an entire season, and I say two pass 1500kbs... .But we know there are always those few episodes that could use a bit more..... so I would be cool if a few came out 1645kbs, or 1721kbs, but most stuck to 1500kbs.
I am looking for an escape clause to break the bitrate budget in an emergency to keep a minimum quality.


To be clear, no I don't want to do CRF.

Hi JoyBell, unfortunately there's no way to set a bitrate 'range' in 2pass, it has to be a specific figure. If you do want a minimum quality then there's no other way than to use CRF really as that's what CRF is; a target quality level.

If 1500kbps is a target bitrate that you're generally happy with then I'd suggest cutting a couple of 5 min clips from each planned encode that ideally contain a mixture of high and low motion and light and dark scenes, and run test encodes at 1500kbps. Then look at the average QP's they've come out on the logs and give them a going over with your eyes to see if you're happy with them. And if they look to need a bit more bitrate then bump them up to 1700 or 1800kbps. Or if they look like they could get away with an even lower bitrate then bump it down and run a repeat of one of the 5 mins clips to be sure.

Another idea might be to use CRF but lower qcomp to say 0.5 or even 0.4 which will decrease bits allocated to motion/grain and increase bitrate in low motion scenes. Might also give you that balance you prefer. You can also turn off cutree which will give a more balanced bitrate allocation without over allocating to grain and often under allocating to low motion frames like cutree does.

[Cary Knoop]

Originally Posted by JoyBell

I do many encodes of seasons of shows for my archive. I have long favored hour 2 pass balances quality in a given budget. I also find that 2 pass pleases my eyes with vs CRF as to do with balancing movement vs dialog scenes. That's all my taste, now to my question.

Is there a way to encode x265 in two pass with a minimum effective CRF quality for the entire encode or any given scene? That is to say, if I set an encode for 2 pass 1500kbs but never want worse quality than CRF 26, for the total encode or any give scene, at which I am willing to let the encode exceed the 1500kbs setting.


I have tried the --crf-max setting any I tried encodes of a sitcom. I started with --crf-max 30 and did encodes lowering the --crf-max 29, 28, 27, 26 each time. All of which came out to my exact bitrate setting with no change. I then tried --crf-max 25 and that encode came out 4334kbs! These tests where entire episodes not just 1 minute test clips or anything. I feel that I am not understanding something about how this works and am asking for help.


My idea, is. I do an entire season, and I say two pass 1500kbs... .But we know there are always those few episodes that could use a bit more..... so I would be cool if a few came out 1645kbs, or 1721kbs, but most stuck to 1500kbs.
I am looking for an escape clause to break the bitrate budget in an emergency to keep a minimum quality.


To be clear, no I don't want to do CRF.

I am not sure if you actually know what you want.

Do you need the video file to be of a given average bitrate? Because that is the only reason one ever would need two-pass.

Some people have a false impression that two-pass encoding is somehow better than single pass encoding. After all, two is more than one!

But all the first pass of the two-pass stage is doing is to record the relative 'need' for each frame so that the second pass can spread out this 'need' by giving the needy frames some extra bitrate while reducing it for the less needy frames.

[JoyBell]

Thank you for taking time to respond.

Originally Posted by Cary Knoop

I am not sure if you actually know what you want.

Some people have a false impression that two-pass encoding is somehow better than single pass encoding. After all, two is more than one!

I do know what I want. I have a fairly good grasp on the encoding methods and are asked often why I don't do "just do CRF". I have hundreds of research encodes that all show at my low target bitrates 2 pass is the best option. To get 1000kbs 1080p, one could try and pic CRF 28, even with some low psy settings and even some smoothing. It still often won't keep at the target end file-size goal. And when bits are easy won't give better image quality.

As for why 2 pass isn't just wasting time, and is in fact truly different than if one just guessed CRF and got the same final bitrates.

2-Pass Average Bitrate (2-Pass ABR)
Allowing the encoder to do two passes (or more) makes it possible for it to estimate what’s ahead in time. It can calculate the cost of encoding a frame in the first pass and then, in the second pass, more efficiently use the bits available. This ensures that the output quality is the best under a certain bitrate constraint.

This can be demonstrated in these graphs from Werner Robitza.

You can see that—especially for the first content—ABR (the turquoise line) and ABR+VBV (purple) wrongly estimate the clip’s complexity. In fact, the Big Buck Bunny sequence starts with a fade-in, smooth gradients and low motion, which means that not many bits are needed to compress it with good enough quality. The 2-pass approach correctly starts with a lower bitrate and saves bandwidth. The last third of the video clip contains lots of spatial details, which makes the 2-pass mode use up more of the bits that it saved in the beginning.

https://slhck.info/video/2017/03/01/rate-control.html

This is what I like; There is no guess-work when using 2-pass. In this example not wasting bits on the fade-in and using them later where they are useful.

CRF has the --vbv-maxrate as an limiter for out of control bit-rates.

It would make sense that ABR and 2 Pass ABR would have stops for maintaining a minimum and maximum quality. If such options are lacking then they should be looked into by the developers of the codec.

[Cary Knoop]

Originally Posted by JoyBell

To get 1000kbs 1080p, one could try and pic CRF 28, even with some low psy settings and even some smoothing. It still often won't keep at the target end file-size goal. And when bits are easy won't give better image quality.

1000Kb/s for 1080p?

I could not possibly watch that!

Do you seriously call that archive level?

[poisondeathray]

Originally Posted by JoyBell

It would make sense that ABR and 2 Pass ABR would have stops for maintaining a minimum and maximum quality. If such options are lacking then they should be looked into by the developers of the codec.

You can try setting --qpmax and --qpmin but it probably won't do what you want exactly .

Originally Posted by JoyBell

My idea, is. I do an entire season, and I say two pass 1500kbs... .But we know there are always those few episodes that could use a bit more..... so I would be cool if a few came out 1645kbs, or 1721kbs, but most stuck to 1500kbs.
I am looking for an escape clause to break the bitrate budget in an emergency to keep a minimum quality.

What you're saying about going over the target bitrate doesn't really make any sense - because that's the entire point of using 2 pass rate control - a target bitrate. If you set it at 1500kbps, you're going to get something close to that.

[JoyBell]

Originally Posted by poisondeathray

You can try setting --qpmax and --qpmin but it probably won't do what you want exactly .
What you're saying about going over the target bitrate doesn't really make any sense - because that's the entire point of using 2 pass rate control - a target bitrate. If you set it at 1500kbps, you're going to get something close to that.

I don't believe that --qpmax and --qpmin will do what I am looking for.

Originally Posted by Cary Knoop

1000Kb/s for 1080p?
I could not possibly watch that!
Do you seriously call that archive level?

I could show Diff.pics and clips, but that would get my post deleted. For series on my local archive I go for and achieve equivalent or better than live cable TV or streaming services. This is good enough for my home archive.

Let's looks at the stats of my most recent encode:

x265 [info]: frame I:    354, Avg QP:21.07  kb/s: 12049.65  SSIM Mean: 0.995182 (23.171dB)
x265 [info]: frame P:   9795, Avg QP:28.84  kb/s: 1682.65   SSIM Mean: 0.993233 (21.696dB)
x265 [info]: frame B:  22303, Avg QP:33.45  kb/s: 379.25    SSIM Mean: 0.992825 (21.442dB)
x265 [info]: Weighted P-Frames: Y:3.2% UV:1.4%
x265 [info]: consecutive B-frames: 21.6% 11.6% 14.5% 29.8% 22.4%
x265 [info]: RPS in SPS: 26676 frames (82.20%), RPS not in SPS: 5776 frames (17.80%)
encoded 32452 frames in 3800.70s (8.54 fps), 899.96 kb/s, Avg QP:31.92, SSIM Mean Y: 0.9929739 (21.533 dB)

Video Compression Testing: x264 vs x265 CRF in Handbrake 0.10.5 (labeled with the matching names in Stax)
https://goughlui.com/2016/08/27/vide...dbrake-0-10-5/

x265 - Average Case
CRF Bitrate SSIM     SSIM (dB) PSNR (avg) PSNR (min) fps
8   37545   0.995211 23.197804 51.363782  46.213167  0.40533746
12  18834   0.991562 20.737413 48.847675  43.550076  0.571760862
16  8407    0.987726 19.110261 47.004687  39.892563  0.847564601  Super High 
20  4315    0.983769 17.896507 45.373892  36.558467  1.094531205  Medium 
24  2504    0.978525 16.680696 43.635386  33.687427  1.289760018  Super Low 

For this animation example my low bitrate encode matches the SIMM and dB of CRF 10 or so. I admit most of my SIMM scores fall into the 18-24 equivalent range for my live action TV series.

I do have two labor intensive ideas on how to accomplish my original goal.

Idea #1:
Do each episode in the series first using CRF, doesn't matter which settings. Find the episode with the mean bitrate- call that the base. Then match up any episodes with above average CRF bitrates and bump them up by he ratio. Or use the ratio on all the episodes both higher and lower and the total for the season would match the intended bitrate.


Idea #2:
Mux the entire season into a single file. Encode using 2 pass ABR. Mux apart the encoded season file. I am trying this right now.
Idea #2 could be automated by software.

[benwaggoner]

Actually, syntactically the request can be done, using x265's (rarely used) crf-max and crf-min parameters.

x265.exe --pass 1 --crf-max 28 --bitrate 1500
x265.exe --pass 2 --crf-max 28 --bitrate 1500

That should produce a 2 pass encode with a maximum file size of 1500 Kbps, and lower if the content is easier to encode.

I don't know that the results would be significantly better than just doing a normal crf encode where rc-lookahead=keyint, ala.

x265.exe --crf 28 --vbv-maxrate 1500 --vbv-buffsize <foo>

It's possible that the more advanced multipass refinement feature could make a difference at slower presets. But I'm sure that a 1-pass --preset slower would look better than a 2-pass --preset slow for this scenario, at ballpark similar encoding times.

[benwaggoner]

Originally Posted by Cary Knoop

1000Kb/s for 1080p?

I could not possibly watch that!

Do you seriously call that archive level?

It's definitely not archive level, but I have some samples of 1080p at 1.0, 1.5, and 2.0 Mbps over here:

https://forum.doom9.org/showthread.php?t=175776

[JoyBell]

Originally Posted by benwaggoner

x265.exe --pass 1 --crf-max 28 --bitrate 1500
x265.exe --pass 2 --crf-max 28 --bitrate 1500

That should produce a 2 pass encode with a maximum file size of 1500 Kbps, and lower if the content is easier to encode.

I don't know that the results would be significantly better than just doing a normal crf encode where rc-lookahead=keyint, ala.

But that is backwards, to my original request- I was asking for a min quality that would break 1500kbs and go bigger if necessary.
I wrote that I tried --crf-max settings and nothing happened until --crf-max 25 where the file size tripled.



Are you saying that I have had those settings backwards the whole time?

[Asmodian]

Try very fine changes for --crf-max, e.g. 25.2, 25.2, 25.15, etc. Small changes in crf can result in large changes in bitrate.

To make sure I understand what you want:
Maximum compression is what would be achieved by twopass 1500 Kbps, quality can go above this point but not below.

Because 2pass and crf rate control is so similar a crf-max will not change the output until you use a value below the crf that would give you the same average bitrate as 2pass. To give harder scenes more bits lower qcomp. To avoid lowering the quality for easy scenes too much when hard scenes take more bits use crf. If you then worry about crf compressing an entire video too much use 2pass and a crf-max value, 2pass gives a minimum average bitrate and crf-max gives a minimum quality. All these options will increase the file size.

[foxyshadis]

Have you even tried qcomp? This whole thread is premised on the idea that 2-pass allocates bits a little more to your liking, but ideally, at the same bitrate crf and 2-pass are supposed to be identical, and updates do shift this around. Qcomp, on the other hand, is literally the knob you turn to allocate bits between hard scenes and easy scenes. Since adjusting that all along is what you wanted, you should be testing qcomp, like RainyDog mentioned in the first reply!

[JoyBell]

I started the thread with: I do not want to do CRF. I was looking for a knob for 2 pass, to exceed the chosen bitrate when a minimum quality was not reached.

I am not saying I won't fry Qcomp, just the thread has gotten side tracked.

[JoyBell]

I have completed my encode of an entire season using 2 pass, as a way of matching up the needs of the episodes better. I did this as I mentioned by muxing all the episodes of the season into one large file, encoding using my normal two pass settings, muxing apart the episodes. This gave the season my target bitrate/size while allowing the episodes that needed more bits do so.

ep01 1 140 kbs -4.6%
ep02 1 130 kbs -5.4%
ep03 1 038 kbs -13.1%
ep04 1 283 kbs +7.3%
ep05 1 296 kbs +6.2%
ep06 1 337 kbs +11.9%
ep07 1 256 kbs +5.1%
ep08 1 176 kbs -1.6%
ep09 1 109 kbs -7.2%
ep10 1 123 kbs -6.0%

Unfortunately the total length of the audio was 5 h 1 min, and the video 5 h 0 min. I was unable to resync the audio in this case. Looks like a few seconds where lost per episode. I tried stretching the video by 18060/18000, but that did not help either. It was an experiment and I knew sync issues where a risk of muxing together.

[benwaggoner]

Originally Posted by JoyBell

I started the thread with: I do not want to do CRF. I was looking for a knob for 2 pass, to exceed the chosen bitrate when a minimum quality was not reached.

I am not saying I won't fry Qcomp, just the thread has gotten side tracked.

--crf-min

[JoyBell]

I always read --crf-min as the best quality CRF it will attempt. That is --crf-min 26 would never apply CRF 20 to a frame even if it had plenty of bit-rate available.


So, if I want a 1500kbs two pass string that would never provide worst image quality than CRF 26, I would:

--pass 1 --bitrate 1500 --crf-min 26

[JoyBell]

Originally Posted by RainyDog

Another idea might be to use CRF but lower qcomp to say 0.5 or even 0.4 which will decrease bits allocated to motion/grain and increase bitrate in low motion scenes.

x265.exe" --pass 1 --bitrate 1200 --preset slow --profile main10 --qcomp 0.4

	
	
x265 [error]: qCompress must be between 0.5 and 1.0

x265.exe" --pass 2 --bitrate 1200 --preset slow

	
	
x265 [info]: frame I:     20, Avg QP:26.88  kb/s: 8882.50   SSIM Mean: 0.976305 (16.253dB)
x265 [info]: frame P:    424, Avg QP:30.52  kb/s: 2438.03   SSIM Mean: 0.969470 (15.153dB)
x265 [info]: frame B:    992, Avg QP:35.16  kb/s: 501.24    SSIM Mean: 0.970722 (15.335dB)
x265 [info]: Weighted P-Frames: Y:0.0% UV:0.0%
x265 [info]: consecutive B-frames: 6.3% 3.6% 61.0% 18.5% 10.6%
x265 [info]: RPS in SPS: 1360 frames (94.71%), RPS not in SPS: 76 frames (5.29%)
encoded 1436 frames in 229.04s (6.27 fps), 1189.84 kb/s, Avg QP:33.68, SSIM Mean Y: 0.9704299 (15.291 dB)

x265.exe" --pass 2 --bitrate 1200 --preset slow --crf-min 26

	
	
x265 [info]: frame I:     20, Avg QP:26.88  kb/s: 8880.14   SSIM Mean: 0.976305 (16.253dB)
x265 [info]: frame P:    424, Avg QP:30.52  kb/s: 2438.38   SSIM Mean: 0.969475 (15.153dB)
x265 [info]: frame B:    992, Avg QP:35.16  kb/s: 501.18    SSIM Mean: 0.970722 (15.335dB)
x265 [info]: Weighted P-Frames: Y:0.0% UV:0.0%
x265 [info]: consecutive B-frames: 6.3% 3.6% 61.0% 18.5% 10.6%
x265 [info]: RPS in SPS: 1360 frames (94.71%), RPS not in SPS: 76 frames (5.29%)
encoded 1436 frames in 220.07s (6.53 fps), 1189.87 kb/s, Avg QP:33.68, SSIM Mean Y: 0.9704315 (15.292 dB)

x265.exe" --pass 2 --bitrate 1200 --preset slow --crf-min 22

	
	
x265 [info]: frame I:     20, Avg QP:26.88  kb/s: 8877.71   SSIM Mean: 0.976290 (16.251dB)
x265 [info]: frame P:    424, Avg QP:30.52  kb/s: 2438.37   SSIM Mean: 0.969456 (15.151dB)
x265 [info]: frame B:    992, Avg QP:35.17  kb/s: 501.14    SSIM Mean: 0.970706 (15.332dB)
x265 [info]: Weighted P-Frames: Y:0.0% UV:0.0%
x265 [info]: consecutive B-frames: 6.3% 3.6% 61.0% 18.5% 10.6%
x265 [info]: RPS in SPS: 1360 frames (94.71%), RPS not in SPS: 76 frames (5.29%)
encoded 1436 frames in 261.97s (5.48 fps), 1189.80 kb/s, Avg QP:33.68, SSIM Mean Y: 0.9704145 (15.289 dB)

x265.exe" --crf 26 --preset slow 

	
	
x265 [info]: frame I:     20, Avg QP:23.17  kb/s: 13122.90  SSIM Mean: 0.982502 (17.570dB)
x265 [info]: frame P:    428, Avg QP:25.72  kb/s: 4515.47   SSIM Mean: 0.977090 (16.400dB)
x265 [info]: frame B:    988, Avg QP:31.54  kb/s: 714.04    SSIM Mean: 0.976340 (16.260dB)
x265 [info]: Weighted P-Frames: Y:0.2% UV:0.0%
x265 [info]: consecutive B-frames: 6.0% 3.8% 64.1% 15.8% 10.3%
encoded 1436 frames in 184.55s (7.78 fps), 2019.88 kb/s, Avg QP:29.69, SSIM Mean Y: 0.9766496 (16.317 dB)

The 2 pass with --crf-min 26 came out the same as Stock 2 pass 1200kbs, and doesn't match up at all with CRF 26 Slow encode. Even pushing --crf-min 22 did not change things. The higher bit rate on the CFR 26 encode suggests that it needed more bits than 1200kbs to maintain CRF 26 quality levels.

[excellentswordfight]

Originally Posted by JoyBell

I started the thread with: I do not want to do CRF. I was looking for a knob for 2 pass, to exceed the chosen bitrate when a minimum quality was not reached.

I am not saying I won't fry Qcomp, just the thread has gotten side tracked.

I dont blame them, what you are asking for is in my eyes (and apperently many others) what crf were intended for, and 2pass ABR not...

Anyway, Isnt what you wanna do pretty simple? Just script it, analyze the stat file between passes and redo them with a higher bitrate if not within criteria.

[JoyBell]

The --crf-min doesn't seem to be doing anything. The tip that CRF points at the P frames is helpful. But my listed stats from the encodes still suggests that setting crf-min isn't working.

Please give me a few lines to try test encodes with. Thank you.

[alex1399]

--ipratio 1.0 There seems to be too much bits in I-frame to support the QP. --bframes 5~8 Reduce the appearance of P-frame. --pbratio 1.5 or default. --no-open-gop and --qpfile Insert specific I-frames positions in the concatenated file for accurately segmented files.