Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 24th March 2023, 21:52   #1  |  Link
arranhs
Registered User
 
Join Date: Oct 2022
Posts: 5
Best settings for encoding at very low bitrates

Hey, I have been lurking on Doom9 for a while now and I was hoping I could get some help!

I am looking to encode some 1080p video, after downscaling it to 720p, using x265 at a very low bitrate - between 750 Kbps and 1 Mbps. I have been trying to find the best settings to get the best quality at this bitrate and was hoping for some advice as there doesn't seem to be a lot of low bitrate advice so far on this forum as far as I can see.

I would like to mention that I am already aware that there will be a very significant amount of detail lost in the videos at this bitrate, but as the videos are for reference purposes only and I will have a lot of them, I am more interested in saving space at this point, and the bitrates I have mentioned give approximately the size I am looking for. That being said, I would naturally like to squeeze out as much quality as I can at this bitrate. The videos also have a mixture of light and dark, so this is a consideration.

These are the settings that I have so far that seem to give the best results (that I have tested) on some test footage I have:
2 pass, bitrate 750k, 10-bit
--preset=slow (Slower was a little bit too slow for me - I'm looking for a slow/slower middle ground)
--bframes=8
--ref=6
--rect (the slow default)
--amp
--psy-rd=1 (the slow default)
--psy-rdoq=1 (a larger value seemed to produce a lower quality by my comparison)

Extra notes:
- I tried aq-mode=3/4 but their quality seemed worse than the default aq-mode=2
- Tried increasing rd, this gave a worse output.
- Tried no-sao and deblock=-1:-1 that I have seen mentioned a lot around here, but these settings were definitely being used to retain detail at far higher bitrates than I am targeting.

If there are any suggestions on how I can further improve these settings, the help would be massively appreciated
arranhs is offline   Reply With Quote
Old 24th March 2023, 23:48   #2  |  Link
excellentswordfight
Lost my old account :(
 
Join Date: Jul 2017
Posts: 322
I think that you actually have a rather good baseline there, that is already rather fitting to your target.

Quote:
Originally Posted by arranhs View Post
- I tried aq-mode=3/4 but their quality seemed worse than the default aq-mode=2
In my experience its not "worse", it just a different tradeoff and it will be somewhat source dependent on what would give the best result.
Quote:
Originally Posted by arranhs View Post
- Tried no-sao and deblock=-1:-1 that I have seen mentioned a lot around here, but these settings were definitely being used to retain detail at far higher bitrates than I am targeting.
And that is expected, as SAO is beneficial for very high compression. But instead of trying no-sao, you could play with --selective-sao, although you are at such low bitrates that it might be better just keeping it fully on.

Last edited by excellentswordfight; 24th March 2023 at 23:53.
excellentswordfight is offline   Reply With Quote
Old 25th March 2023, 01:14   #3  |  Link
TDS
Formally known as .......
 
TDS's Avatar
 
Join Date: Sep 2021
Location: On a need to know basis.
Posts: 921
Quote:
Originally Posted by arranhs View Post
Hey, I have been lurking on Doom9 for a while now and I was hoping I could get some help!

I am looking to encode some 1080p video, after downscaling it to 720p, using x265 at a very low bitrate - between 750 Kbps and 1 Mbps. I have been trying to find the best settings to get the best quality at this bitrate and was hoping for some advice as there doesn't seem to be a lot of low bitrate advice so far on this forum as far as I can see.

I would like to mention that I am already aware that there will be a very significant amount of detail lost in the videos at this bitrate, but as the videos are for reference purposes only and I will have a lot of them, I am more interested in saving space at this point, and the bitrates I have mentioned give approximately the size I am looking for. That being said, I would naturally like to squeeze out as much quality as I can at this bitrate. The videos also have a mixture of light and dark, so this is a consideration.

These are the settings that I have so far that seem to give the best results (that I have tested) on some test footage I have:
2 pass, bitrate 750k, 10-bit
--preset=slow (Slower was a little bit too slow for me - I'm looking for a slow/slower middle ground)
--bframes=8
--ref=6
--rect (the slow default)
--amp
--psy-rd=1 (the slow default)
--psy-rdoq=1 (a larger value seemed to produce a lower quality by my comparison)

Extra notes:
- I tried aq-mode=3/4 but their quality seemed worse than the default aq-mode=2
- Tried increasing rd, this gave a worse output.
- Tried no-sao and deblock=-1:-1 that I have seen mentioned a lot around here, but these settings were definitely being used to retain detail at far higher bitrates than I am targeting.

If there are any suggestions on how I can further improve these settings, the help would be massively appreciated
So just out of interest's sake, what encoding app are you using ??
__________________
Long term RipBot264 user.

RB 1.27.1 (std)
RB 1.27.1 (fully loaded, OOD)
PD 7.6 (new)
TDS is offline   Reply With Quote
Old 25th March 2023, 03:29   #4  |  Link
arranhs
Registered User
 
Join Date: Oct 2022
Posts: 5
Quote:
Originally Posted by excellentswordfight View Post
And that is expected, as SAO is beneficial for very high compression. But instead of trying no-sao, you could play with --selective-sao, although you are at such low bitrates that it might be better just keeping it fully on.
Thanks for the helpful suggestion. I'll give selective-sao a go. There's no harm in trying!

Found your point on aq-mode interesting, because I think you are right having relooked. 3 and 4 are not "worse" per say, but the trade-offs definitely makes certain frames look "worse" for my eye.

I wonder if I should give aq-mode=1 a try and add that to the comparison.

Quote:
Originally Posted by TDS View Post
So just out of interest's sake, what encoding app are you using ??
I am using ffmpeg, with libplacebo for scaling (scale and zscale are much lower quality). I am using ewa_lanczos as the downscaled as it is by far the best quality.

If you are interested here are the first and second pass commands I am currently using (I am using bash substitution for the file),

Code:
ffmpeg \
  -init_hw_device vulkan \
  -i "${1}" \
  -map 0:v:0 \
  -c:v libx265 \
  -profile:v main10 \
  -b:v 750k \
  -preset:v slow \
  -x265-params "
    bframes=8:
    ref=6:
    rect=1:
    amp=1:
    psy-rd=1:
    psy-rdoq=1:
    pass=1
  " \
  -vf "
    hwupload,
    libplacebo=
      w=1280:h=-1:
      downscaler=ewa_lanczos:
      deband=true:
      colorspace=bt709:
      color_primaries=bt709:
      color_trc=bt709:
      range=limited:
      format=yuv420p10le,
    hwdownload,
    format=yuv420p10le
  " \
  -an \
  -f null \
  -

ffmpeg \
  -init_hw_device vulkan \
  -i "${1}" \
  -map 0:v:0 \
  -c:v libx265 \
  -profile:v main10 \
  -b:v 750k \
  -preset:v slow \
  -x265-params "
    bframes=8:
    ref=6:
    rect=1:
    amp=1:
    psy-rd=1:
    psy-rdoq=1:
    pass=2
  " \
  -vf "
    hwupload,
    libplacebo=
      w=1280:h=-1:
      downscaler=ewa_lanczos:
      deband=true:
      colorspace=bt709:
      color_primaries=bt709:
      color_trc=bt709:
      range=limited:
      format=yuv420p10le,
    hwdownload,
    format=yuv420p10le
  " \
  -map 0:a:0 \
  -c:a libfdk_aac \
  -profile:a aac_he_v2 \
  -vbr 3 \
  -ac 2 \
  "$2"
arranhs is offline   Reply With Quote
Old 25th March 2023, 03:36   #5  |  Link
HD MOVIE SOURCE
Registered User
 
HD MOVIE SOURCE's Avatar
 
Join Date: Mar 2021
Location: North Carolina
Posts: 138
Quote:
Originally Posted by arranhs View Post
Hey, I have been lurking on Doom9 for a while now and I was hoping I could get some help!

I am looking to encode some 1080p video, after downscaling it to 720p, using x265 at a very low bitrate - between 750 Kbps and 1 Mbps. I have been trying to find the best settings to get the best quality at this bitrate and was hoping for some advice as there doesn't seem to be a lot of low bitrate advice so far on this forum as far as I can see.

I would like to mention that I am already aware that there will be a very significant amount of detail lost in the videos at this bitrate, but as the videos are for reference purposes only and I will have a lot of them, I am more interested in saving space at this point, and the bitrates I have mentioned give approximately the size I am looking for. That being said, I would naturally like to squeeze out as much quality as I can at this bitrate. The videos also have a mixture of light and dark, so this is a consideration.

These are the settings that I have so far that seem to give the best results (that I have tested) on some test footage I have:
2 pass, bitrate 750k, 10-bit
--preset=slow (Slower was a little bit too slow for me - I'm looking for a slow/slower middle ground)
--bframes=8
--ref=6
--rect (the slow default)
--amp
--psy-rd=1 (the slow default)
--psy-rdoq=1 (a larger value seemed to produce a lower quality by my comparison)

Extra notes:
- I tried aq-mode=3/4 but their quality seemed worse than the default aq-mode=2
- Tried increasing rd, this gave a worse output.
- Tried no-sao and deblock=-1:-1 that I have seen mentioned a lot around here, but these settings were definitely being used to retain detail at far higher bitrates than I am targeting.

If there are any suggestions on how I can further improve these settings, the help would be massively appreciated

Could you do me a favor and try psy-rd=1.60 and --psy-rdoq=0.00 with the rdoq level set to 2. This does very well for me in film content.

I personally only use aq mode 3 for animation with no film grain. For film grain I like aq mode 0, basically off, but I'm encoding at high bit-rates.

I don't like sao or deblock, however, I think with low but-rates like yours it may be needed. As some have suggested with sao, you could use selective. The only reason I don't like it is because of blurring or softness.

Default deblock may be useful for low bit-rates I believe. But I'm not sure about how strong it needs to be based on the bit-rates.
HD MOVIE SOURCE is offline   Reply With Quote
Old 25th March 2023, 10:08   #6  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,708
Some suggestions:

--aq-mode 2 is probably better than 1 for low-bitrate encodes. Mode 3 is a definite no-no because it requires more bits due to the dark bias.
You could try lowering aq-strength one or two notches, like 0.8-0.9.

Set --rskip 2 and see if it makes a difference.

At low bitrates, --rd 6 might be useful to couple with --rd-refine.

Increase the amount of b-frames to 10. In some cases, it may help a lot.

Try --deblock 0:0 or 1:1. Your encode will be blurry anyway.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 26th March 2023, 14:59   #7  |  Link
arranhs
Registered User
 
Join Date: Oct 2022
Posts: 5
Thanks for all the replies so far, I have found them incredibly helpful. I've gone away and run some tests, but have only managed to complete 3 so far:

- aq-mode=3, aq-strength=0.8 - This was indeed a big improvement over aq-strength=1. I noticed a big improvement in dark scenes but sadly light scenes were worse. I think these settings would be perfect for if I know the video will be dark, but for general video, probably not as good as the baseline I currently have.

- psy-rd=1.6, psy-rdoq=0, psy-rdoq=2 - These settings gave a very slight (almost unnoticeable tbh) improvement in light scenes, but a significant decrease in dark scene quality. I wonder why this is. Why does psy options affect dark scenes so much? Due to how much it degrades dark scenes I'd be concerned using this over the baseline for the very minor light scene improvement it provides.

- bframes=10 makes dark scene noticibly highter quality. It also slightly increases quality of light scenes but the improvements are mostly negligible. Interestingly however it gets a lower vmaf and psnr score (the same ssim) as my baseline, but I wonder if that is just imperfect nature of these metrics.

Currently working on some tests for rd=6 and rd-refine, rskip=2 and selective-sao (I am going to give 2 a go).

On a separate note, I have seen it mentioned that --nr (noise reduction) could help at low bitrates, but I am unsure at what a reasonable initial setting might be to try. Any suggestions?
arranhs is offline   Reply With Quote
Old 26th March 2023, 16:12   #8  |  Link
microchip8
ffx264/ffhevc author
 
microchip8's Avatar
 
Join Date: May 2007
Location: /dev/video0
Posts: 1,843
if you get banding and/or bad quality in dark frames, you should be increasing psy-rd and psy-rdoq, not lower/disable it. Try something like psy-rd=4 and psy-rdoq=15
__________________
ffx264 || ffhevc || ffxvid || microenc
microchip8 is offline   Reply With Quote
Old 26th March 2023, 16:20   #9  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,708
Quote:
Originally Posted by arranhs View Post
- aq-mode=3, aq-strength=0.8 - This was indeed a big improvement over aq-strength=1. I noticed a big improvement in dark scenes but sadly light scenes were worse. I think these settings would be perfect for if I know the video will be dark, but for general video, probably not as good as the baseline I currently have.
There's jpsdr's modded build available which contains the aq-auto feature. It will automatically switch between various modes depending on the brightness and the edge densities detected. See the big x265 thread here.

Quote:
- bframes=10 makes dark scene noticibly highter quality. It also slightly increases quality of light scenes but the improvements are mostly negligible. Interestingly however it gets a lower vmaf and psnr score (the same ssim) as my baseline, but I wonder if that is just imperfect nature of these metrics.
Metrics really can't be trusted, your eyes are what tell you what you like. Many option decrease the metrics but perceptual quality during playback increases.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 26th March 2023, 16:28   #10  |  Link
arranhs
Registered User
 
Join Date: Oct 2022
Posts: 5
Quote:
Originally Posted by microchip8 View Post
if you get banding and/or bad quality in dark frames, you should be increasing psy-rd and psy-rdoq, not lower/disable it. Try something like psy-rd=4 and psy-rdoq=15
Hmm, that's interesting to hear cos my experimentation and understanding was that the opposite was true. But I have never tried such large values so I might give them a go and see
arranhs is offline   Reply With Quote
Old 26th March 2023, 16:36   #11  |  Link
arranhs
Registered User
 
Join Date: Oct 2022
Posts: 5
I have also just run a test on rskip=2 and sadly it didn't provide any benefit. Dark scenes particularly were very bad
arranhs is offline   Reply With Quote
Old 26th March 2023, 16:53   #12  |  Link
microchip8
ffx264/ffhevc author
 
microchip8's Avatar
 
Join Date: May 2007
Location: /dev/video0
Posts: 1,843
Quote:
Originally Posted by arranhs View Post
Hmm, that's interesting to hear cos my experimentation and understanding was that the opposite was true. But I have never tried such large values so I might give them a go and see
Same. I came up with these values after a lot of experimentation trying to get rid of banding in low-light/dark scenes. These are the values that completely got rid of the banding. Make sure to use rdoq-level=1.

All my encodes are virtually banding-free now, as I always use these values along with aq-mode=1 (at default strength).
__________________
ffx264 || ffhevc || ffxvid || microenc
microchip8 is offline   Reply With Quote
Old 27th March 2023, 15:56   #13  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,727
First, what kind(s) of content are you compressing? How will you be watching it? The more focused the scenario, the finer-tuned the recommendations we can make.

Your average bitrate isn't that low for 720p. My Compression Challenge has had some decent 1080p entries down to 500 Kbps ABR.

Banding issues tend to not be so bad in 10-bit. Have you checked that your preprocessed input is 10-bit and doesn't have banding before it hits x265?

Throwing in some --nr-inter and --nr-intra can help quite a lot at low bitrates. As they are integrated into quantization, they do a smart job of reducing small details that have a bigger bitrate impact, and so can lower QP improving quality overall. I'd try with something like --nr-intra 100 --nr-inter 400 and see if that yields an improvement.

I'd suggest just using --preset slower and iterating from there. You're already manually setting a number of parameters to their slower values anyway. And the presets have pretty well balanced and well-tested combinations of settings. Still, they haven't been refactored for a while, so there are some newer options that are worth adding:
--selective-sao 2: this is a small speed boost; SAO only really helps forward prediction, so 99% of the benefit comes from applying it to I and P frames. I wouldn't ever use 4, and would only use 3 if I was already at placebo.
--rskip 2: faster and more robust with noise than --rskip 1. --rskip-edge-threshold 3 or 2 (default is 5) can improve visual results a bit at the cost of a little quality.
--aq-mode 4: this seems slightly better than 2 for SDR content. You might want to lower --aq-strength a bit when using it, as it is a bit stronger than other modes.
--nr-intra 100
--nr-inter 400
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 27th March 2023, 16:03   #14  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,727
And as you're worried about speed, and 720p is a relatively lower resolution, you might try --pmode if you've got a bunch of idle cores during encoding.

Setting --frame-threads 1 can help improve quality somewhat, but probably not worth it if you're struggling to use --preset slower.

Also, if your source isn't 10-bit, you can also try using --dither.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 1st April 2023, 11:08   #15  |  Link
PatchWorKs
Registered User
 
PatchWorKs's Avatar
 
Join Date: Aug 2002
Location: Italy
Posts: 300
...it would be great to have - if technically possible - such kind of preset for rigaya's HW-accellerated encoders too.
__________________
HYbrid Multimedia Production Suite page @ GitHub
PatchWorKs is offline   Reply With Quote
Old 3rd April 2023, 03:50   #16  |  Link
HD MOVIE SOURCE
Registered User
 
HD MOVIE SOURCE's Avatar
 
Join Date: Mar 2021
Location: North Carolina
Posts: 138
Quote:
Originally Posted by arranhs View Post
Thanks for all the replies so far, I have found them incredibly helpful. I've gone away and run some tests, but have only managed to complete 3 so far:

- aq-mode=3, aq-strength=0.8 - This was indeed a big improvement over aq-strength=1. I noticed a big improvement in dark scenes but sadly light scenes were worse. I think these settings would be perfect for if I know the video will be dark, but for general video, probably not as good as the baseline I currently have.

- psy-rd=1.6, psy-rdoq=0, psy-rdoq=2 - These settings gave a very slight (almost unnoticeable tbh) improvement in light scenes, but a significant decrease in dark scene quality. I wonder why this is. Why does psy options affect dark scenes so much? Due to how much it degrades dark scenes I'd be concerned using this over the baseline for the very minor light scene improvement it provides.

- bframes=10 makes dark scene noticibly highter quality. It also slightly increases quality of light scenes but the improvements are mostly negligible. Interestingly however it gets a lower vmaf and psnr score (the same ssim) as my baseline, but I wonder if that is just imperfect nature of these metrics.

Currently working on some tests for rd=6 and rd-refine, rskip=2 and selective-sao (I am going to give 2 a go).

On a separate note, I have seen it mentioned that --nr (noise reduction) could help at low bitrates, but I am unsure at what a reasonable initial setting might be to try. Any suggestions?
Just wondering, what do use to measure quality? And tell the difference in the quality of light and dark scenes?
HD MOVIE SOURCE is offline   Reply With Quote
Old 3rd April 2023, 17:22   #17  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,727
Quote:
Originally Posted by HD MOVIE SOURCE View Post
Just wondering, what do use to measure quality? And tell the difference in the quality of light and dark scenes?
These bitrates are all about the psychovisually preferable tradeoffs, so eyeballs are the gold standard. x265 will log per frame PNSR and SSIM with --psnr --ssim --csv-log-level 2.

If I have too much content to viably eyeball, I'll parse the .csv files and look for spikes of high QP (higher is more artifacts) or low YUV-PSNR (only built-in metric that looks at chroma) and SSIM over a --keyint rolling average.

VMAF is better than any one of those, but is also luma only and not great at comparing adaptive quantization or penalizing banding as much as its psychovisual impact.

A per-encode metrics score isn't very useful for content longer than a couple of minutes, as subjective quality is much more dependent on how bad the worst shots look than the mean. Hence looking for spikes in rolling average.

And yeah, I have some really big Excel files for full-length movies ! Using .xlsb instead of .xlsx cuts file size in about half.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 3rd April 2023, 17:29   #18  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,727
Quote:
Originally Posted by arranhs View Post
TI am using ffmpeg, with libplacebo for scaling (scale and zscale are much lower quality). I am using ewa_lanczos as the downscaled as it is by far the best quality.
I personally prefer to use spline36 instead of lanczos for downscaling for recompression. Lanczos is "sharper" but increases risk of ringing artifacts. And higher frequencies are intrinsically harder to encode, and so can yield QP increases. Spline36 is quite neutral and handles 25%-400% scaling without aliasing. While it might have a little less apparent detail before encoding, at low bitrates it will generally be equal or better quality as it doesn't have artificial high frequencies to encode.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Reply

Tags
bitrate, low, quality

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 02:14.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.