Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 21st December 2019, 18:04   #1  |  Link
mindphasar
Registered User
 
Join Date: Oct 2017
Location: Italy
Posts: 2
10 hours for 21.8 GiB.

Hi guys,

I would like to reduce the size of this file: 10 hours for 21.8 GiB. The audio part (AAC) will be copied with ffmpeg.

This file is a live event recorded with youtube-dl.


Can anyone give me some settings for x265?



Thanks


Format : MPEG-4
Format profile : Base Media
Codec ID : isom (isom/iso2/avc1/mp41)
File size : 21,8 GiB
Duration : 10 o 8 min
Overall bit rate : 5.135 kb/s
Writing application : Lavf58.33.100

Video
ID : 1
Format : AVC
Format/Info : Advanced Video Codec
Format profile : High@L4.2
Format settings : CABAC / 1 Ref Frames
Format settings, CABAC : Si
Format settings, RefFrames : 1 frame
Format settings, GOP : M=1, N=30
Codec ID : avc1
Codec ID/Info : Advanced Video Coding
Duration : 10 o 8 min
Bit rate : 5.034 kb/s
Width : 1.920 pixel
Height : 1.080 pixel
Display aspect ratio : 16:9
Frame rate mode : Variabile
Frame rate : 42,547 FPS
Minimum frame rate : 0,016 FPS
Maximum frame rate : 62,500 FPS
Original frame rate : 60,000 FPS
Color space : YUV
Chroma subsampling : 4:2:0
Bit depth : 8 bit
Scan type : Progressive
Bits/(Pixel*Frame) : 0.057
Stream size : 21,4 GiB (98%)
Codec configuration box : avcC
mindphasar is offline   Reply With Quote
Old 22nd December 2019, 01:35   #2  |  Link
Forteen88
Herr
 
Join Date: Apr 2009
Location: North Europe
Posts: 482
If you got a fast CPU, then this x265-setting is good,
--profile main10 --output-depth 10 --preset slower --crf 18 --no-sao --aq-mode 3 --rdoq-level 0 --psy-rdoq 0

Use the --crf 18 for high-quality, but if you want to compromise video-quality over file-size, then change to a higher crf-value, like 20.

Also, maybe use you should the Avisynth-script TemporalDegrain2 if the video-source is too grainy for your taste.

That video-source got variable frame-rate, I hope that it will cause no problem.

Last edited by Forteen88; 22nd December 2019 at 15:58. Reason: included 10-bits setting.
Forteen88 is offline   Reply With Quote
Old 22nd December 2019, 03:46   #3  |  Link
RanmaCanada
Registered User
 
Join Date: May 2009
Posts: 160
Youtube is already at optimal bitrate. You're not going to get a beneficial size reductions in regard to the amount of time it will take you to encode it. Just burn this download to a bluray and you're done.
RanmaCanada is offline   Reply With Quote
Old 22nd December 2019, 06:18   #4  |  Link
froggy1
ffx264/ffhevc author
 
froggy1's Avatar
 
Join Date: May 2007
Location: Belgium
Posts: 1,651
Quote:
Originally Posted by Forteen88 View Post
If you got a fast CPU, then this x265-setting is good,
--preset slower --crf 18 --no-sao --aq-mode 3 --rdoq-level 0 --psy-rdoq 0

Use the --crf 18 for high-quality, but if you want to compromise video-quality over file-size, then change to a higher crf-value, like 20.

Also, maybe use you should the Avisynth-script TemporalDegrain2 if the video-source is too grainy for your taste.

That video-source got variable frame-rate, I hope that it will cause no problem.
I find crf 18 rather high. The crf mappings between x264 and x265 are different. A crf 18 in x264 will correspound to a crf 20 or 21 for x265. Since x265 is more efficient in compressing for the same visual quality, for me personally a crf of 21 is visually indistinguishable comparet to crf 18 in x264

But to each his own...

PS: i do 10 bits encodes only so this has an effect too
__________________
ffx264--ffhevc--ffxvid

Desktop: AMD Ryzen 7 3700X
Server: Intel Core i7-7700K
NAS: Intel Pentium G4400
froggy1 is offline   Reply With Quote
Old 22nd December 2019, 11:40   #5  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,529
To save anything at all, since YouTube 1080p is already a fairly low 5mbps, you'd have to start with a crf of at least 25 and go up from there if it still isn't enough. (5mpbs will generally be between 18 and 24, though YMMV by video.) You're already working with mediocre Youtube-quality source, and the only way you have to really save bits is to murder quality even further. How far do you want to go?

Dropping it to 720p might actually help significantly, though.

I'd probably just put some time into editing the 10 hour video down, to save whole swaths of bits that don't need to be encoded at all.
__________________
There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order.
foxyshadis is offline   Reply With Quote
Old 22nd December 2019, 15:57   #6  |  Link
Forteen88
Herr
 
Join Date: Apr 2009
Location: North Europe
Posts: 482
Quote:
Originally Posted by froggy1 View Post
PS: i do 10 bits encodes only so this has an effect too
True, 10 bits is the way to go with x265. Weird, I thought that 10 bits was the default --output-depth value in x265, but I just looked it up, and saw that it's not (at least not in the x265-build I use), default is 8!

You might want to add: --frame-threads 1
in the commandline for better picture-quality, but if you got many cores on your CPU, then it will take longer time to encode (since x265 don't use all cores then).

UPDATE: froggy1, OK, --frame-threads 2 might be a better value to set, for more CPU-core usage.

Last edited by Forteen88; 30th December 2019 at 13:50.
Forteen88 is offline   Reply With Quote
Old 22nd December 2019, 16:43   #7  |  Link
froggy1
ffx264/ffhevc author
 
froggy1's Avatar
 
Join Date: May 2007
Location: Belgium
Posts: 1,651
Quote:
Originally Posted by Forteen88 View Post
True, 10 bits is the way to go with x265. Weird, I thought that 10 bits was the default --output-depth value in x265, but I just looked it up, and saw that it's not (at least not in the x265-build I use), default is 8!

You might want to add: --frame-threads 1
in the commandline for better picture-quality, but if you got many cores on your CPU, then it will take longer time to encode (since x265 don't use all cores then).
What's the point then of having a CPU with enough/many threads and then not being able to utilize it fully during encoding for the sake of quality improvement that is so minor you won't notice it unless you use a magnifying glass to look at still pictures?
__________________
ffx264--ffhevc--ffxvid

Desktop: AMD Ryzen 7 3700X
Server: Intel Core i7-7700K
NAS: Intel Pentium G4400
froggy1 is offline   Reply With Quote
Old 22nd December 2019, 17:55   #8  |  Link
excellentswordfight
Lost my old account :(
 
Join Date: Jul 2017
Posts: 148
Quote:
Originally Posted by Forteen88 View Post
True, 10 bits is the way to go with x265. Weird, I thought that 10 bits was the default --output-depth value in x265, but I just looked it up, and saw that it's not (at least not in the x265-build I use), default is 8!

You might want to add: --frame-threads 1
in the commandline for better picture-quality, but if you got many cores on your CPU, then it will take longer time to encode (since x265 don't use all cores then).
So he should encode an 10h 60p(!) clip with preset slower and frame-threads 1 with crf18? So he can come back to this thread after like 2months of encoding to say that he ended up with another 20Gb file?
excellentswordfight is offline   Reply With Quote
Old 22nd December 2019, 18:43   #9  |  Link
Forteen88
Herr
 
Join Date: Apr 2009
Location: North Europe
Posts: 482
Quote:
Originally Posted by excellentswordfight View Post
So he should encode an 10h 60p(!) clip with preset slower and frame-threads 1 with crf18? So he can come back to this thread after like 2months of encoding to say that he ended up with another 20Gb file?
& froggy1.
He could just encode something else (maybe even 3-4 encodes), simultaneously, if he wants 100% CPU-core use.

Bitrate on the video-source is only 5k, so it seems very compressible, not a high detail source...

TC (mindphasar) should probably better use 2-pass instead of CRF, to get the file-size he wants.

Last edited by Forteen88; 22nd December 2019 at 19:25. Reason: 2-pass instead of CRF...
Forteen88 is offline   Reply With Quote
Old 22nd December 2019, 19:26   #10  |  Link
froggy1
ffx264/ffhevc author
 
froggy1's Avatar
 
Join Date: May 2007
Location: Belgium
Posts: 1,651
Quote:
Originally Posted by Forteen88 View Post
& froggy1.
He could just encode something else (maybe even 3-4 encodes), simultaneously, if he wants 100% CPU-core use.

Bitrate on the video-source is only 5k, so it seems very compressible, not a high detail source...
That doesn't solve the problem of waiting a long time to finish the specific encode. Throwing other encodes too will not make them encode any faster even if the CPU is fully saturated because you run an X amount of them. It will take just as long for all of them since they'll all use 1 frame thread and the quality benefit is still very questionable.

You really need to re-think your strategy. Encoding is all about compromise between how much speed you can or want to give up and how much quality you want to get. The trick is to find the right balance between the two

If you don't mind encoding at 0.1 fps and wait a whole month for a single (or a few encodes if you use others simultaniously to saturate the CPU in case of 1 frame thread) and think this will somehow improve quality so much that it's obvious from an average distance from the TV/monitor, you're only deluding yourself
__________________
ffx264--ffhevc--ffxvid

Desktop: AMD Ryzen 7 3700X
Server: Intel Core i7-7700K
NAS: Intel Pentium G4400
froggy1 is offline   Reply With Quote
Old 22nd December 2019, 20:09   #11  |  Link
Forteen88
Herr
 
Join Date: Apr 2009
Location: North Europe
Posts: 482
froggy1. Personally (with my 6-core CPU), I like setting --frame-threads 1
because if I use all cores, then the computer gets quite unusable (laggy) for anything else.
Setting --frame-threads 1 makes x265 take like 60-70% of my CPU.
I don't encode a lot though, and I've never tried x265-encoding on a 12+ core CPU.

EDIT: froggy1, yeah, thanks, I know about priority (crappy Windows10 that I use got that too), but I just don't bother with that (because I'm in no hurry when I encode, and then I don't need to stress out my CPU).

Last edited by Forteen88; 22nd December 2019 at 20:30. Reason: priority-reply
Forteen88 is offline   Reply With Quote
Old 22nd December 2019, 20:21   #12  |  Link
froggy1
ffx264/ffhevc author
 
froggy1's Avatar
 
Join Date: May 2007
Location: Belgium
Posts: 1,651
Quote:
Originally Posted by Forteen88 View Post
froggy1. Personally (with my 6-core CPU), I like setting --frame-threads 1
because if I use all cores, then the computer gets quite unusable (laggy) for anything else.
Setting --frame-threads 1 makes x265 take like 60-70% of my CPU.
I don't encode a lot though, and I've never tried x265-encoding on a 12+ core CPU.
That's fine, but don't suggest others do it as well. You'll find virtually none here use 1 frame threads for all of their encodes. The quality benefit vs speed just isn't worth it.

I do encodes all the time (previously on an i7-7700K, now on a Ryzen 7 3700X) and spent a few days comparing different options and tweaking things in order to come up with something that improves quality (compared to the i7-7700K) and doesn't sacrifice too much speed on my new Ryzen 7 system. Using 1 frame threads wasn't even a consideration as there's no point in getting a Ryzen 7 for this

As for sluggishness, you can make the encode run at the lowest possible priority and still have a responsive system. Sure, it'll take a bit longer if you do other stuff that uses most or all threads, but it's worth it. As you might guessed from my avatar, I'm on Linux and always run encodes at nice priority of 19 (lowest possible). There is absolutely no sluggishness here. Windows might be different so I can't comment on that
__________________
ffx264--ffhevc--ffxvid

Desktop: AMD Ryzen 7 3700X
Server: Intel Core i7-7700K
NAS: Intel Pentium G4400
froggy1 is offline   Reply With Quote
Old 22nd December 2019, 20:34   #13  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 4,924
On Windows, I use the command start /belownormal encode.bat to run the batch file which contains my job list. Works nicely, no sluggishness and wins over any stuff that is set for idle priority but goes to the background when something more urgent comes up. I just add stuff to the end of the file so my computer is basically encoding 24/7.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 22nd December 2019, 23:28   #14  |  Link
Forteen88
Herr
 
Join Date: Apr 2009
Location: North Europe
Posts: 482
By the way, not everyone here at Doom9 have high --frame-threads,
https://forum.doom9.org/showthread.p...04#post1852904

TC, you can also set --ctu 32 if you want more CPU-usage, --ctu 64 (default for --slower) should be set for UHD-video though.

Last edited by Forteen88; 23rd December 2019 at 11:18.
Forteen88 is offline   Reply With Quote
Old 23rd December 2019, 10:36   #15  |  Link
DotJun
Registered User
 
Join Date: Aug 2014
Posts: 24
On windows you can remove a core using affinity while you are doing other things then re-enable it when you are done.
DotJun is offline   Reply With Quote
Old 29th December 2019, 22:45   #16  |  Link
mindphasar
Registered User
 
Join Date: Oct 2017
Location: Italy
Posts: 2
thank you so much for the great suggestions guys. I don't know which settings are recommended for this huge file. I use debian 10.with ffmpeg, x265 and mkvmerge.

where i find a examples of 2pass with x265?.

if I add --no-sao with CRF 19, it is not passed 19 but CRF-28.0 Why?

x265 [info]: Main 10 profile, Level-3 (Main tier)
x265 [info]: Thread pool created using 4 threads
x265 [info]: Slices : 1
x265 [info]: frame threads / pool features : 2 / wpp(8 rows)
x265 [warning]: Source height < 720p; disabling lookahead-slices
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 1 inter / 1 intra
x265 [info]: ME / range / subpel / merge : star / 57 / 3 / 3
x265 [info]: Keyframe min / max / scenecut / bias: 25 / 250 / 40 / 5.00
x265 [info]: Lookahead / bframes / badapt : 25 / 4 / 2
x265 [info]: b-pyramid / weightp / weightb : 1 / 0 / 0
x265 [info]: References / ref-limit cu / depth : 4 / on / on
x265 [info]: AQ: mode / str / qg-size / cu-tree : 1 / 1.0 / 32 / 1
x265 [info]: Rate Control / qCompress : CRF-28.0 / 0.60
x265 [info]: tools: rect limit-modes rd=4 psy-rd=2.00 rdoq=2 psy-rdoq=1.00
x265 [info]: tools: rskip signhide tmvp strong-intra-smoothing


my goal is to reduce the size of this file, if it is convenient to do a similar action, obviously I have an asus Bluray 16x recorder.

this script is ok?

#!/bin/bash
ffmpeg -i "10 hours.mp4" -pix_fmt yuv420p10 -s 720x408 -aspect 16:9 -strict -1 -an -f yuv4mpegpipe - | x265 --y4m - --preset slow --tune fastdecode --no-high-tier --ref 3 --rc-lookahead 5 -b 3 --profile main10 --crf 19 --deblock -4:-4 --output-depth 10 --dither --wpp -o "raw_video.hevc"
mindphasar is offline   Reply With Quote
Old 30th December 2019, 03:54   #17  |  Link
RanmaCanada
Registered User
 
Join Date: May 2009
Posts: 160
If you have a bluray burner, just burn the file to disc as I suggested. Or even use any of the free authoring programs to create an actual bluray. It will take 1 single layer bluray to keep the file as it is, which typically cost $0.50 when they are bought in a 50 pack. You will waste more money in electricity converting this than the price of a bluray.
RanmaCanada is offline   Reply With Quote
Old 30th December 2019, 13:38   #18  |  Link
Forteen88
Herr
 
Join Date: Apr 2009
Location: North Europe
Posts: 482
Quote:
Originally Posted by mindphasar View Post
...
this script is ok?
...
--rc-lookahead 5 ... --deblock -4:-4 ...
That --rc-lookahead is too low! Set it to at least 25 (I set it to 48).

And you should rather set --deblock -3:-3 or as high as you accept (I usually set it to --deblock -2:-2), since the compression gets better then.

You can use an encoding-GUI like "Simple x264/x265-launcher" (available here at Doom9) to encode 2-pass, but don't forget to update x265 in that package, because that developer doesn't update that program often.

EDIT: Although you didn't really apply any of my advice, so IDC.

Last edited by Forteen88; 30th December 2019 at 13:57.
Forteen88 is offline   Reply With Quote
Old 30th December 2019, 18:41   #19  |  Link
MeteorRain
結城有紀
 
Join Date: Dec 2003
Location: NJ; OR; Shanghai
Posts: 871
10 hours live event 1920x1080 60fps at 5034kbps is fairly low.

I have been encoding some live events from TV broadcasting (source is at much higher bitrate -- less artifacts), and 10-bit 1440x1080 60fps CRF24 costs 7500kbps, and that's after a heavy denoising (which saves a good portion).

So a good chance is you'd come back at a much worse, terrible quality for an even bigger file.

EDIT: Just saw you're re-encoding it to 400p. That would make it small but still it would be at much worse quality.
__________________
Projects
x265 - Yuuki-Asuna-mod Download / GitHub
TS - ADTS AAC Splitter | LATM AAC Splitter | BS4K-ASS
Neo AviSynth+ filters - F3KDB | FFT3D | DFTTest | MiniDeen | Temporal Median

Last edited by MeteorRain; 30th December 2019 at 18:44.
MeteorRain is offline   Reply With Quote
Old 30th December 2019, 19:43   #20  |  Link
froggy1
ffx264/ffhevc author
 
froggy1's Avatar
 
Join Date: May 2007
Location: Belgium
Posts: 1,651
Quote:
Originally Posted by Forteen88 View Post
That --rc-lookahead is too low! Set it to at least 25 (I set it to 48).

And you should rather set --deblock -3:-3 or as high as you accept (I usually set it to --deblock -2:-2), since the compression gets better then.

You can use an encoding-GUI like "Simple x264/x265-launcher" (available here at Doom9) to encode 2-pass, but don't forget to update x265 in that package, because that developer doesn't update that program often.

EDIT: Although you didn't really apply any of my advice, so IDC.
A deblock of -3,-3 is a bit too low and will start to create banding in flat/clean areas. Keep in mind that the deblock filter in HEVC is much improved compared to the one in H.264. I use -1,-1 and while it creates a bit softer image, it masks banding while still preserving enough detail. Also, compression will be better the more you increase the deblocking since it will start to smooth things (= remove more detail) leading to the encoder to spend less bits. The more you lower deblocking, the more detail it'll preserve (and start to introduce banding) thus making the encoder spend more bits on the detail

As for rc-lookahead, the more you increase it, the more efficient adaptive quantization will be. rc-lookahead cannot be greater than the max keyframe. I use a value of 100 here (max-keyframe set to 240). Also rc-lookahead has an impact on latency. Lower values provide better latency at the expense of less effectiveness of adaptive quant (and less accurate scenecut decision)
__________________
ffx264--ffhevc--ffxvid

Desktop: AMD Ryzen 7 3700X
Server: Intel Core i7-7700K
NAS: Intel Pentium G4400

Last edited by froggy1; 30th December 2019 at 19:46.
froggy1 is offline   Reply With Quote
Reply

Tags
x265

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 15:34.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.