Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 28th April 2019, 09:24   #6781  |  Link
jethro
Registered User
 
Join Date: Mar 2006
Posts: 102
Quote:
Originally Posted by Boulder View Post
Thanks, that's exactly what I was looking for since I was experimenting a lot with aq-strength and the psy options to try to figure out what is wrong but was clearly looking in the wrong place.

I did some more testing now that I found those troublesome frames (it seems they evaded me when I pinpointed my go-to settings a long time ago), and using CTU 32 makes the onionize effect go away. I also left max-tu-size and qg-size at their defaults since things looked good.
Encoder Settings:
x265 CRF 28, veryslow, --no-sao --no-strong-intra-smoothing --deblock -1:-1








More bitrate removes the artifacting.
jethro is offline   Reply With Quote
Old 28th April 2019, 12:58   #6782  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,718
In those cases, I had thrown quite a lot of bitrate there. Using CRF18 produced a smaller file than 6900 kbps, which was x264@CRF18. CRF28 is something I'd never use The onion artifacts were visible only in certain frames with very high motion, that's probably why I missed them in my test rounds earlier. I do recall someone complaining that tune grain sometimes did those as well.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 28th April 2019, 22:00   #6783  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,988
I'd caution against doing too much settings tuning using still frame comparisons, especially when evaluating max CU size (--ctu).

I've spent a lot of time doing this. Using --ctu 32 can indeed produce a sharper / more detailed / less blurred image when looking at single frames - relative to keeping the default (--ctu 64) in some cases. However, the trade-off is typically more blocking, which you're more likely to notice in motion. In other words, an inferior user experience unless you're using very high bitrates.

You're kind of performing a brute force non-adaptive psy tuning. x265 will use smaller CUs when it sees fit, you're just forcing it to never consider 64x64, even when coding giant uniform areas.

In my opinion, you're typically best off trusting the extensive tuning and testing that MulticoreWare has done, and just using a preset. They set --ctu to 64 starting with --preset veryfast, and keep it that way all the way through placebo. In fact, in looking at the documentation for this parameter: https://x265.readthedocs.io/en/defau...#cmdoption-ctu it appears the only reason they set this to 32 for superfast and lower is that this increases parallelism and can therefore be faster.

Do what's right for you, of course

Last edited by Blue_MiSfit; 28th April 2019 at 22:02.
Blue_MiSfit is offline   Reply With Quote
Old 29th April 2019, 14:01   #6784  |  Link
tuanden0
Registered User
 
Join Date: Oct 2016
Posts: 111
I got crash when using zone in x265_12bit.
So I tried to use VSEdit to test it and here is result.
Here is x265 file: http://msystem.waw.pl/x265/x265-3.0_..._gcc83-AVX2.7z
Here is everything else: https://i.imgur.com/FFlFylW.png
tuanden0 is offline   Reply With Quote
Old 29th April 2019, 16:32   #6785  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,718
Quote:
Originally Posted by Blue_MiSfit View Post
I'd caution against doing too much settings tuning using still frame comparisons, especially when evaluating max CU size (--ctu).

I've spent a lot of time doing this. Using --ctu 32 can indeed produce a sharper / more detailed / less blurred image when looking at single frames - relative to keeping the default (--ctu 64) in some cases. However, the trade-off is typically more blocking, which you're more likely to notice in motion. In other words, an inferior user experience unless you're using very high bitrates.

You're kind of performing a brute force non-adaptive psy tuning. x265 will use smaller CUs when it sees fit, you're just forcing it to never consider 64x64, even when coding giant uniform areas.

In my opinion, you're typically best off trusting the extensive tuning and testing that MulticoreWare has done, and just using a preset. They set --ctu to 64 starting with --preset veryfast, and keep it that way all the way through placebo. In fact, in looking at the documentation for this parameter: https://x265.readthedocs.io/en/defau...#cmdoption-ctu it appears the only reason they set this to 32 for superfast and lower is that this increases parallelism and can therefore be faster.

Do what's right for you, of course
I've always considered 64x64 to be quite a lot for 720p video, and also the difference in performance is substantial. I can see that the average QP is lower with 32x32 than 16x16 though so it could be that there are situations where the encoder could squeeze some more out of the video. Of course, I could always run two simultaneous encodes to keep the CPU as busy as possible.

As you apparently have been testing things quite a lot, do you have any opinion regarding AQ-mode? I'm still quite unsure if the new default of mode 2 is the best general option for film.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 29th April 2019, 17:12   #6786  |  Link
excellentswordfight
Lost my old account :(
 
Join Date: Jul 2017
Posts: 322
Quote:
Originally Posted by Boulder View Post
I've always considered 64x64 to be quite a lot for 720p video, and also the difference in performance is substantial. I can see that the average QP is lower with 32x32 than 16x16 though so it could be that there are situations where the encoder could squeeze some more out of the video. Of course, I could always run two simultaneous encodes to keep the CPU as busy as possible.

As you apparently have been testing things quite a lot, do you have any opinion regarding AQ-mode? I'm still quite unsure if the new default of mode 2 is the best general option for film.
I wrote to Ma a fem months back regarding ctu 64 being overkill for reslutions under 2160p, and asking why this as the default value. And he responded with numbers that showed a loss in compression with using lower ctu values, even for lower res material. The difference is probably pretty negligible though, but i dont see any reason to lower it IF youre not trying to saturate more threads (were it can do wonders). I only lower it to 32 for 1080p if i’m at 16 threads or more.

Last edited by excellentswordfight; 29th April 2019 at 17:42.
excellentswordfight is offline   Reply With Quote
Old 30th April 2019, 14:11   #6787  |  Link
Barough
Registered User
 
Barough's Avatar
 
Join Date: Feb 2007
Location: Sweden
Posts: 480
x265 v3.0_Au+22-feec4bdf9866 (32 & 64-bit 8/10/12bit Multilib Windows Binaries) (32bit-GCC v7.4.0 / 64bit-GCC v8.3.0)

Code:
https://bitbucket.org/multicoreware/x265/commits/branch/default
Barough is offline   Reply With Quote
Old 30th April 2019, 21:44   #6788  |  Link
Blue_MiSfit
Derek Prestegard IRL
 
Blue_MiSfit's Avatar
 
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,988
The difference isn't huge, but it does help.

IMO aq-mode 2 is definitely better in most cases. I set this always unless doing extremely low bitrate when I'll use mode 3.
Blue_MiSfit is offline   Reply With Quote
Old 1st May 2019, 21:03   #6789  |  Link
~ VEGETA ~
The cult of personality
 
~ VEGETA ~'s Avatar
 
Join Date: May 2013
Location: Planet Vegeta
Posts: 155
I wanted to encode a 10-bit H.264 MKV video (lossless, 30 gb) to HEVC 10-bit... I piped it via ffmpeg but I put this "-pix_fmt yuv420p" instead of this "yuv420p10le"... I suspect this is wrong, but it is the only way it worked. Can you kindly guide me here?

I encoded .vpy file to lossless 10-bit H.264 since my device is slow with the filters I used... so now how to encode the 10-bit .mkv using x265? << I assumed using -pix_fmt yuv420p is wrong.



Plus, next time when I want to encode like this (lossless 10-bit H.264 then 10-bit HEVC), should I encode to .yuv/.h264 in x264 (--muxer yuv\raw)? So I can input it directly to x265 without piping??



Here is my command line:

Code:
"C:\.....\ffmpeg.exe" -i "C:\....\lossless_H264_video.mkv" -pix_fmt yuv420p -f yuv4mpegpipe - | C:\....\x265_64bit_10bit.exe --y4m --preset slower --crf 17 --ref 6 --rd 6 --psy-rd 1 --ctu 64 --aq-strength 0.8 --output-depth 10 --input-res 1920x1080 - --output "C:\....\ep07_1080p_video_HEVC.hevc"

Last edited by ~ VEGETA ~; 1st May 2019 at 21:05.
~ VEGETA ~ is offline   Reply With Quote
Old 1st May 2019, 21:05   #6790  |  Link
sneaker_ger
Registered User
 
Join Date: Dec 2002
Posts: 5,565
Post your complete command line and ffmpeg log.

(Yes, if you want 10 bit output setting -pix_fmt yuv420p is wrong. Make sure your ffmpeg build is compiled with 10 bit libx265.)
sneaker_ger is offline   Reply With Quote
Old 1st May 2019, 21:07   #6791  |  Link
~ VEGETA ~
The cult of personality
 
~ VEGETA ~'s Avatar
 
Join Date: May 2013
Location: Planet Vegeta
Posts: 155
Quote:
Originally Posted by sneaker_ger View Post
Post your complete command line and ffmpeg log.

(Yes, if you want 10 bit output setting -pix_fmt yuv420p is wrong. Make sure your ffmpeg build is compiled with 10 bit libx265.)
I posted my command line as you see now.

Where can I find such ffmpeg build for windows?

I thought -pix_fmt is to get the input as 10-bit...
~ VEGETA ~ is offline   Reply With Quote
Old 1st May 2019, 21:09   #6792  |  Link
sneaker_ger
Registered User
 
Join Date: Dec 2002
Posts: 5,565
I thought you wanted to use libx265 from within ffmpeg. Ignore my last post.

Use:
Code:
ffmpeg -i "INPUT" -f yuv4mpegpipe -strict -1 -
Then make sure the log says it's "yuv420p10le".
sneaker_ger is offline   Reply With Quote
Old 1st May 2019, 21:09   #6793  |  Link
excellentswordfight
Lost my old account :(
 
Join Date: Jul 2017
Posts: 322
Quote:
Originally Posted by ~ VEGETA ~ View Post
I wanted to encode a 10-bit H.264 MKV video (lossless, 30 gb) to HEVC 10-bit... I piped it via ffmpeg but I put this "-pix_fmt yuv420p" instead of this "yuv420p10le"... I suspect this is wrong, but it is the only way it worked. Can you kindly guide me here?

I encoded .vpy file to lossless 10-bit H.264 since my device is slow with the filters I used... so now how to encode the 10-bit .mkv using x265? << I assumed using -pix_fmt yuv420p is wrong.



Plus, next time when I want to encode like this (lossless 10-bit H.264 then 10-bit HEVC), should I encode to .yuv/.h264 in x264 (--muxer yuv\raw)? So I can input it directly to x265 without piping??



Here is my command line:

Code:
"C:\.....\ffmpeg.exe" -i "C:\....\lossless_H264_video.mkv" -pix_fmt yuv420p -f yuv4mpegpipe - | C:\....\x265_64bit_10bit.exe --y4m --preset slower --crf 17 --ref 6 --rd 6 --psy-rd 1 --ctu 64 --aq-strength 0.8 --output-depth 10 --input-res 1920x1080 - --output "C:\....\ep07_1080p_video_HEVC.hevc"
I never had issues with the following:
Code:
"ffmpeg.exe" -i "input.mkv" -pix_fmt yuv420p10le -f yuv4mpegpipe -strict -1 - | "x265.exe" --y4m
So maybe you need to ad -strict -1
excellentswordfight is offline   Reply With Quote
Old 1st May 2019, 21:14   #6794  |  Link
~ VEGETA ~
The cult of personality
 
~ VEGETA ~'s Avatar
 
Join Date: May 2013
Location: Planet Vegeta
Posts: 155
Quote:
Originally Posted by sneaker_ger View Post
I thought you wanted to use libx265 from within ffmpeg. Ignore my last post.

Use:
Code:
ffmpeg -i "INPUT" -f yuv4mpegpipe -strict -1 -
Then make sure the log says it's "yuv420p10le".
I see:

video: wrapped_avframe, yuv420p10le in cmd so I guess this is ok.

what about my other question about better way to encode next time?
~ VEGETA ~ is offline   Reply With Quote
Old 1st May 2019, 21:36   #6795  |  Link
sneaker_ger
Registered User
 
Join Date: Dec 2002
Posts: 5,565
Quote:
Originally Posted by ~ VEGETA ~ View Post
I encoded .vpy file to lossless 10-bit H.264 since my device is slow with the filters I used

Plus, next time when I want to encode like this (lossless 10-bit H.264 then 10-bit HEVC), should I encode to .yuv/.h264 in x264 (--muxer yuv\raw)? So I can input it directly to x265 without piping??
I don't understand the need for a temporary file at all in this case. Usually this is to speed up encoding when we do slow filtering + multiple passes of video encoding and time of
filtering + 1st pass + filtering + 2nd pass + .. > filtering + encode to lossless + 1st pass from lossless + 2nd pass from lossless ..

But you are encoding CRF, e.g. now instead of filtering + x265 crf you do filtering + x264 lossless + x265 crf. That's slower?
sneaker_ger is offline   Reply With Quote
Old 1st May 2019, 21:46   #6796  |  Link
~ VEGETA ~
The cult of personality
 
~ VEGETA ~'s Avatar
 
Join Date: May 2013
Location: Planet Vegeta
Posts: 155
Quote:
Originally Posted by sneaker_ger View Post
I don't understand the need for a temporary file at all in this case. Usually this is to speed up encoding when we do slow filtering + multiple passes of video encoding and time of
filtering + 1st pass + filtering + 2nd pass + .. > filtering + encode to lossless + 1st pass from lossless + 2nd pass from lossless ..

But you are encoding CRF, e.g. now instead of filtering + x265 crf you do filtering + x264 lossless + x265 crf. That's slower?
I have very low fps doing my filters, so I put everything to lossless x264 --preset ultrafast... then I can encode directly using x264 with slow settings in both 1080p and 720p. While If I want to encode in 1080 and 720 from vpy in both times I guess it will be slower since now I've gotta encode the filtering twice.

while if I wanna encode 720p only (from horriblesubs tv shows), then yes I go vpy directly without lossless.


what do you do for encoding BDs to 1080 and 720?
~ VEGETA ~ is offline   Reply With Quote
Old 1st May 2019, 22:18   #6797  |  Link
sneaker_ger
Registered User
 
Join Date: Dec 2002
Posts: 5,565
So you encode to multiple resolutions? Ok, then it may make sense to create an intermediate to only do (slow) filtering once.

I think you may save a little bit of time/CPU by saving to uncompressed y4m/yuv instead of encoding x264 lossless as intermediate. But with slow filtering + slow x265 it will probably not make too much of a difference either way. And the size of the file increases so the wear on the HDD/SSD increases. P.S.: ffmpeg allows multiple outputs but I don't know how it affects speed and it may result in HDD fragmentation.


Quote:
Originally Posted by ~ VEGETA ~ View Post
what do you do for encoding BDs to 1080 and 720?
I don't.

Last edited by sneaker_ger; 1st May 2019 at 22:20.
sneaker_ger is offline   Reply With Quote
Old 1st May 2019, 22:28   #6798  |  Link
~ VEGETA ~
The cult of personality
 
~ VEGETA ~'s Avatar
 
Join Date: May 2013
Location: Planet Vegeta
Posts: 155
Quote:
Originally Posted by sneaker_ger View Post
So you encode to multiple resolutions? Ok, then it may make sense to create an intermediate to only do (slow) filtering once.

I think you may save a little bit of time/CPU by saving to uncompressed y4m/yuv instead of encoding x264 lossless as intermediate. But with slow filtering + slow x265 it will probably not make too much of a difference either way. And the size of the file increases so the wear on the HDD/SSD increases. P.S.: ffmpeg allows multiple outputs but I don't know how it affects speed and it may result in HDD fragmentation.

save to uncompressed video?? like using virtualdub2 to do that? not bad but I never tried it. I just hope it won't be more size than lossless h.264, at least not too much. plus, it has to be 10-bit.

I encode on a dedicated server so no problem about HDD xD, or so I think.
~ VEGETA ~ is offline   Reply With Quote
Old 1st May 2019, 22:35   #6799  |  Link
sneaker_ger
Registered User
 
Join Date: Dec 2002
Posts: 5,565
You don't need VirtualDub. vspipe can create y4m files. But yes, unsurprisingly, uncompressed files are bigger than compressed ones...

But again: if you are doing slow filtering and slow x265 encoding the difference between uncompressed or lossless x264 --preset ultrafast intermediate is probably not worth worrying about. If you want to know exactly you have to benchmark it.

Last edited by sneaker_ger; 1st May 2019 at 22:38.
sneaker_ger is offline   Reply With Quote
Old 1st May 2019, 22:41   #6800  |  Link
~ VEGETA ~
The cult of personality
 
~ VEGETA ~'s Avatar
 
Join Date: May 2013
Location: Planet Vegeta
Posts: 155
Quote:
Originally Posted by sneaker_ger View Post
You don't need VirtualDub. vspipe can create y4m files. But yes, unsurprisingly, uncompressed files are bigger than compressed ones...

But again: if you are doing slow filtering and slow x265 encoding the difference between uncompressed or lossless x264 --preset ultrafast intermediate is probably not worth worrying about. If you want to know exactly you have to benchmark it.
I thought of doing this:

Code:
ffmpeg -i inputfile.vpy -c:v rawvideo outputfile.yuv
so you suggested vspipe, so maybe this:

Code:
vspipe script.vpy output.raw
or add --y4m

I will try next time to see how much time will it take. Lossless x264 ultrafast 1080p is about 8 hours.
~ VEGETA ~ is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 22:08.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.