Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Announcements and Chat > General Discussion

Reply
 
Thread Tools Search this Thread Display Modes
Old 8th December 2018, 10:35   #41  |  Link
richardpl
Registered User
 
Join Date: Jan 2012
Posts: 272
Quote:
Originally Posted by dipje View Post
From my Vapoursynth experience it's not zscale that is the speed limiting factor, but I'm only guessing.
ffmpeg isn't very good at multi-threading at all, and I think the tonemap filter is just made as a proof of concept as a single-threaded demo. Guess you will have to ask on the ffmpeg-user list.
Have proof for such harsh claims?
richardpl is offline   Reply With Quote
Old 8th December 2018, 10:38   #42  |  Link
tebasuna51
Moderator
 
tebasuna51's Avatar
 
Join Date: Feb 2005
Location: Spain
Posts: 6,890
Quote:
Originally Posted by jhughy2010 View Post
... I'm getting the following error: "at least one output file must be specified"...
Work for me using a unique line like this:
Quote:
ffmpeg.exe -i D:\Videos\Media\Temp\video.mkv -vf zscale=t=linear:npl=100,format=gbrpf32le,zscale=p=bt709,tonemap=tonemap=hable:desat=0,zscale=t=bt709:m=bt709:r=tv,format=yuv420p -c:v libx265 -crf 18 -preset slower D:\Videos\Media\Temp\output.mkv
Copy/Paste or make a .bat file with a unique line.
__________________
BeHappy, AviSynth audio transcoder.
tebasuna51 is offline   Reply With Quote
Old 10th December 2018, 05:13   #43  |  Link
jhughy2010
Registered User
 
Join Date: Aug 2015
Posts: 9
Quote:
Originally Posted by tebasuna51 View Post
Work for me using a unique line like this:


Copy/Paste or make a .bat file with a unique line.
Thank you. I appreciate your help... it appears it's working; however, I'm getting approximately 0.5 fps for the encode. At this rate I figured it would take 96 hours to complete the encode. A few questions: what do I have to do in order to speed up the encode? I'm using an i7-4790k overclocked to 4.6GHz. It is a bit old... I'm willing to upgrade to speed the process up.

Perhaps there is a command that'll speed it up as well? This is my first experience with ffmpeg and/or CLI encoding. Thanks!
jhughy2010 is offline   Reply With Quote
Old 10th December 2018, 10:25   #44  |  Link
tebasuna51
Moderator
 
tebasuna51's Avatar
 
Join Date: Feb 2005
Location: Spain
Posts: 6,890
Quote:
Originally Posted by jhughy2010 View Post
I'm getting approximately 0.5 fps for the encode. At this rate I figured it would take 96 hours to complete the encode...
Is a normal speed for encode 4K HEVC with your CPU.
Is more or less the same if you use x265 to encode instead ffmpeg.
You can use other -preset between: ultrafast superfast veryfast faster fast medium slow slower veryslow placebo
to improve the speed obtaining less quality or more size.

Only if you can use some GPU to encode HEVC you can obtain much more speed.
__________________
BeHappy, AviSynth audio transcoder.

Last edited by tebasuna51; 10th December 2018 at 10:29.
tebasuna51 is offline   Reply With Quote
Old 14th December 2018, 02:17   #45  |  Link
EmKayVe
Registered User
 
EmKayVe's Avatar
 
Join Date: Dec 2018
Posts: 21
Quote:
Originally Posted by tebasuna51 View Post
Work for me using a unique line like this:


Copy/Paste or make a .bat file with a unique line.
Quote:
Originally Posted by tebasuna51 View Post
Is a normal speed for encode 4K HEVC with your CPU.
Is more or less the same if you use x265 to encode instead ffmpeg.
You can use other -preset between: ultrafast superfast veryfast faster fast medium slow slower veryslow placebo
to improve the speed obtaining less quality or more size.

Only if you can use some GPU to encode HEVC you can obtain much more speed.
This worked wonders for me. I changed the preset to "medium" and crf to 20. I was getting around 3 to 4 fps for the encode.

One thing I would like to know with ffmpeg is now to set the audio to encode to AAC 320kbps 5.1 channels. Also, how do you set the target resolution to 1920x1080. I found that in order to make those two adjusments, I am having to utilize handbrake after ffmpeg since I don't know the CLI.
EmKayVe is offline   Reply With Quote
Old 14th December 2018, 11:43   #46  |  Link
tebasuna51
Moderator
 
tebasuna51's Avatar
 
Join Date: Feb 2005
Location: Spain
Posts: 6,890
Quote:
Originally Posted by EmKayVe View Post
One thing I would like to know with ffmpeg is now to set the audio to encode to AAC 320kbps 5.1 channels. Also, how do you set the target resolution to 1920x1080. I found that in order to make those two adjusments, I am having to utilize handbrake after ffmpeg since I don't know the CLI.
A full recode with ffmpeg is out of scope of this thread (HDR to SDR) and need a answer from a ffmpeg expert (I'm not).

BTW if you want a 1920x1080 SDR output better use a BluRay source.
__________________
BeHappy, AviSynth audio transcoder.
tebasuna51 is offline   Reply With Quote
Old 17th December 2018, 08:11   #47  |  Link
EmKayVe
Registered User
 
EmKayVe's Avatar
 
Join Date: Dec 2018
Posts: 21
Quote:
Originally Posted by tebasuna51 View Post
BTW if you want a 1920x1080 SDR output better use a BluRay source.
Can you be more specific? What do you mean... use a BD rip?
EmKayVe is offline   Reply With Quote
Old 17th December 2018, 10:40   #48  |  Link
tebasuna51
Moderator
 
tebasuna51's Avatar
 
Join Date: Feb 2005
Location: Spain
Posts: 6,890
Of course.
A 4K HDR BT-2020 is better than a BD by resolution and colour space, if you reduce resolution to 1920x1080 and colour space to BT-709 you obtain the same quality (or less) than the BD.

BTW to reduce resolution you have the answer in previous post by SpasV:

add at end of video filter: ...,format=yuv420p,zscale=s=1920x1080
__________________
BeHappy, AviSynth audio transcoder.

Last edited by tebasuna51; 17th December 2018 at 10:44. Reason: add info
tebasuna51 is offline   Reply With Quote
Old 20th December 2018, 01:12   #49  |  Link
EmKayVe
Registered User
 
EmKayVe's Avatar
 
Join Date: Dec 2018
Posts: 21
Quote:
Originally Posted by tebasuna51 View Post
Of course.
A 4K HDR BT-2020 is better than a BD by resolution and colour space, if you reduce resolution to 1920x1080 and colour space to BT-709 you obtain the same quality (or less) than the BD.

BTW to reduce resolution you have the answer in previous post by SpasV:

add at end of video filter: ...,format=yuv420p,zscale=s=1920x1080
Ok thank you for your help!

I tried the following and I am getting an error:

ffmpeg.exe -i D:\Videos\Media\Temp\video.mkv -vf zscale=t=linear:npl=100,format=gbrpf32le,zscale=p=bt709,tonemap=tonemap=hable:desat=0,zscale=t=bt709:m=bt709:r=tv,format=yuv420p, zscale=s=1920x1080 -c:v libx265 -crf 20 -preset medium D:\Videos\Media\Temp\output.mkv

The error is "unable to find a suitable output format for zscale=s=1920x1080: Invalid argument.

Any suggestions?
EmKayVe is offline   Reply With Quote
Old 20th December 2018, 10:44   #50  |  Link
Monarc
Registered User
 
Join Date: Jun 2002
Posts: 35
Quote:
Originally Posted by EmKayVe View Post
-vf zscale=t=linear:npl=100,format=gbrpf32le,zscale=p=bt709,tonemap=tonemap=hable:desat=0,zscale=t=bt709:m=bt709:r=tv,format=yuv420p, zscale=s=1920x1080
remove the space before zscale=s=1920x1080
Monarc is offline   Reply With Quote
Old 28th December 2018, 02:36   #51  |  Link
EmKayVe
Registered User
 
EmKayVe's Avatar
 
Join Date: Dec 2018
Posts: 21
Quote:
Originally Posted by Monarc View Post
remove the space before zscale=s=1920x1080
Awesome! Thanks that worked.

I'd love to learn more about ffmpeg configuration. Can anyone point me in the right direction for a thread that can help me with this - like audio tracks and such?

So I feel like the HDR to SDR conversion taking place with the configuration in the ffmpeg encode is acceptable. Is there a way to improve upon it or is that pretty much as good as it gets.
EmKayVe is offline   Reply With Quote
Old 28th December 2018, 11:03   #52  |  Link
tebasuna51
Moderator
 
tebasuna51's Avatar
 
Join Date: Feb 2005
Location: Spain
Posts: 6,890
https://www.ffmpeg.org/documentation.html

All about command line decoders/encoders/filters...
__________________
BeHappy, AviSynth audio transcoder.
tebasuna51 is offline   Reply With Quote
Old 28th December 2018, 17:39   #53  |  Link
EmKayVe
Registered User
 
EmKayVe's Avatar
 
Join Date: Dec 2018
Posts: 21
Ok thank you. I am now only looking for a line to pass through the audio. Then I'll take the resulting file and finish up in Handbrake. I'll just use ffmpeg to tonemap HDR to SDR.
EmKayVe is offline   Reply With Quote
Old 22nd May 2019, 19:29   #54  |  Link
Beam
Registered User
 
Join Date: May 2019
Posts: 1
What am i doing wrong?
Quote:
ffmpeg -ss 00:01:00.12 -i Sony_4K_HDR_Camp.mp4 -frames 1 -vf zscale=t=linear,format=gbrpf32le,zscale=p=bt709,tonemap=tonemap=hable,zscale=t=bt709:m=bt709:r=tv Sony_4K_HDR_Camp_HABLE.png
Quote:
ffmpeg version 4.1.3 Copyright (c) 2000-2019 the FFmpeg developers
built with FreeBSD clang version 6.0.1 (tags/RELEASE_601/final 335540) (based on LLVM 6.0.1)
configuration: --prefix=/usr/local --mandir=/usr/local/man --datadir=/usr/local/share/ffmpeg --pkgconfigdir=/usr/local/libdata/pkgconfig --enable-shared --enable-pic --enable-gpl --enable-postproc --enable-avfilter --enable-avresample --enable-pthreads --cc=cc --disable-alsa --disable-libopencore-amrnb --disable-libopencore-amrwb --disable-libaom --disable-libass --disable-libbs2b --disable-libcaca --disable-libcdio --disable-libcelt --disable-libcodec2 --disable-libdav1d --disable-libdavs2 --disable-libdc1394 --disable-debug --disable-htmlpages --disable-libdrm --enable-libfdk-aac --disable-libflite --disable-fontconfig --disable-libfreetype --disable-frei0r --disable-libfribidi --disable-gcrypt --disable-libgme --enable-gmp --enable-gnutls --enable-version3 --disable-libgsm --enable-iconv --disable-libilbc --disable-libjack --disable-libklvanc --disable-libkvazaar --disable-ladspa --enable-libmp3lame --disable-liblensfun --enable-libbluray --disable-librsvg --disable-librtmp --disable-libxml2 --disable-lv2 --disable-mbedtls --enable-mmx --disable-libmodplug --disable-libmysofa --enable-nonfree --disable-openal --disable-opencl --disable-libopencv --disable-opengl --disable-libopenh264 --disable-libopenjpeg --disable-libopenmpt --disable-openssl --enable-optimizations --disable-libopus --disable-libpulse --enable-runtime-cpudetect --disable-librubberband --disable-sdl2 --disable-libsmbclient --disable-libsnappy --disable-sndio --disable-libsoxr --disable-libspeex --disable-libsrt --enable-sse --disable-libssh --disable-libtensorflow --disable-libtesseract --disable-libtheora --disable-libtwolame --disable-libv4l2 --disable-indev=v4l2 --disable-outdev=v4l2 --disable-vaapi --disable-vapoursynth --disable-vdpau --disable-libvidstab --disable-libvorbis --disable-libvo-amrwbenc --disable-libvpx --disable-libwavpack --disable-libwebp --enable-libx264 --enable-libx265 --disable-libxavs2 --disable-libxcb --enable-libxvid --disable-outdev=xv --enable-libzimg --disable-libzmq --disable-libzvbi
libavutil 56. 22.100 / 56. 22.100
libavcodec 58. 35.100 / 58. 35.100
libavformat 58. 20.100 / 58. 20.100
libavdevice 58. 5.100 / 58. 5.100
libavfilter 7. 40.101 / 7. 40.101
libavresample 4. 0. 0 / 4. 0. 0
libswscale 5. 3.100 / 5. 3.100
libswresample 3. 3.100 / 3. 3.100
libpostproc 55. 3.100 / 55. 3.100
Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'Sony_4K_HDR_Camp.mp4':
Metadata:
major_brand : isom
minor_version : 1
compatible_brands: isom
creation_time : 2016-02-03T08:01:30.000000Z
Duration: 00:02:07.15, start: 0.000000, bitrate: 75806 kb/s
Stream #0:0(und): Video: hevc (Main 10) (hvc1 / 0x31637668), yuv420p10le(tv, bt2020nc/bt2020/smpte2084), 3840x2160 [SAR 1:1 DAR 16:9], 75620 kb/s, 59.94 fps, 59.94 tbr, 60k tbn, 59.94 tbc (default)
Metadata:
creation_time : 2016-02-03T07:59:49.000000Z
handler_name : Video Media Handler
encoder : HEVC Coding
Stream #0:1(eng): Audio: aac (LC) (mp4a / 0x6134706D), 48000 Hz, stereo, fltp, 192 kb/s (default)
Metadata:
creation_time : 2016-02-03T07:59:49.000000Z
handler_name : Sound Media Handler
File 'Sony_4K_HDR_Camp_HABLE.png' already exists. Overwrite ? [y/N] y
Stream mapping:
Stream #0:0 -> #0:0 (hevc (native) -> png (native))
Press [q] to stop, [?] for help
[hevc @ 0x805186600] First slice in a frame missing.
Last message repeated 6 times
[hevc @ 0x8059b6000] First slice in a frame missing.
Last message repeated 6 times
Output #0, image2, to 'Sony_4K_HDR_Camp_HABLE.png':22.77 bitrate=N/A speed=N/A
Metadata:
major_brand : isom
minor_version : 1
compatible_brands: isom
encoder : Lavf58.20.100
Stream #0:0(und): Video: png, rgb48be, 3840x2160 [SAR 1:1 DAR 16:9], q=2-31, 200 kb/s, 59.94 fps, 59.94 tbn, 59.94 tbc (default)
Metadata:
creation_time : 2016-02-03T07:59:49.000000Z
handler_name : Video Media Handler
encoder : Lavc58.35.100 png
frame= 1 fps=0.3 q=-0.0 Lsize=N/A time=00:00:00.01 bitrate=N/A speed=0.00537x
video:808kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
And this is result:

Beam is offline   Reply With Quote
Old 30th May 2019, 22:39   #55  |  Link
Gravitator
Registered User
 
Join Date: May 2014
Posts: 292
Quote:
Originally Posted by Beam View Post
What am i doing wrong?
Tried it?

ffmpeg.exe -ss 00:01:00.12 -i Sony_4K_HDR_Camp.mp4 -frames 1 -vf zscale=transfer=linear,tonemap=clip,zscale=transfer=bt709,format=yuv420p c:\users\dave\desktop\ffmpeg.png
__________________
Win10x64, Xeon E5450, GTX 750 2GB, DDR3 8GB.
Gravitator is offline   Reply With Quote
Old 4th February 2020, 19:50   #56  |  Link
manolito
Registered User
 
manolito's Avatar
 
Join Date: Sep 2003
Location: Berlin, Germany
Posts: 3,078
Quote:
Originally Posted by Beam View Post
What am i doing wrong?
I believe I figured it out...

Have a look here:
https://forum.doom9.org/showthread.p...20#post1898420

My HDR to SDR readme for using FFmpeg:
Code:
Using FFmpeg to convert HDR sources to SDR:
======================================================================

This video filter call will do it (thanks to dipje and Atak_Snajpera):

zscale=s=704x396,zscale=tin=smpte2084:min=bt2020nc:pin=bt2020:rin=tv:t=smpte2084:m=bt2020nc:p=bt2020:r=tv,zscale=t=linear:npl=100,format=gbrpf32le,zscale=p=bt709,tonemap=tonemap=hable:desat=0,zscale=t=bt709:m=bt709:r=tv,format=yuv420p,colormatrix=bt709:bt601

These parameters are easy to modify for HD output. Edit the "zscale" 
parameter for the desired size and remove the "colormatrix" parameter.

This FFmpeg parameter line is meant to be inserted in the "Picture 
Settings -> More" field of dmMediaConverter. If you want to use them 
in an existing FFmpeg commandline you need to add them as a video 
filter and enclose the parameters in double quotes. 
Like:
-vf "parameter list"


Note:
If you use the source file directly as the FFmpeg input you may get 
borked colors in the output depending on the source format. The safest 
way to correct this is to use an AVS script with just the source 
filter as the input for FFmpeg (of course only if AviSynth is installed). 
Otherwise you have to pre-convert the source to a lossless intermediate 
file keeping all the source properties. Then use this lossless file as 
the input for FFmpeg and apply the HDR to SDR filter.

Cheers
manolito
manolito is offline   Reply With Quote
Old 30th March 2020, 11:56   #57  |  Link
PatchWorKs
Registered User
 
PatchWorKs's Avatar
 
Join Date: Aug 2002
Location: Italy
Posts: 303
Hi everyone, I honestly don't know if these are overlapping color correction (or equalization ?) problems, but here's another interesting 3ad - I'm partecipating in - about them:

https://forum.videohelp.com/threads/...eg-Color-Range

Last edited by PatchWorKs; 30th March 2020 at 12:43.
PatchWorKs is offline   Reply With Quote
Old 8th December 2021, 06:13   #58  |  Link
damian101
Registered User
 
Join Date: Feb 2021
Location: Germany
Posts: 17
Quote:
Originally Posted by dipje View Post
Think I got the desaturation a bit figured out. You / we never change the color primaries and/or the colormatrix. That means the primaries are still in bt2020. We tonemap the brightness but not much else.

There might be a simpler method, but what I do:
zscale=t=linear,format=gbrpf32le,zscale=p=bt709,tonemap=tonemap=hable,zscale=t=bt709:m=bt709:r=tv

(after the whole 'setting the correct input format' thing).

The first zscale=t=linear turns the YUV format into linear-space-RGB. (Linear space is only a thing in RGB). But the colorprimaries are still in bt2020. (Note, not the colormatrix. The colormatrix is a YUV only thing. But the colorprimarie is basically your colorspace).
Then comes the format=gbrpf32le to turn it all into floating-point RGB. _Then_ do we tell zscale to change the primaries to bt709. According to the zimg documentation changing primaries need to be done in linear space anyway, and this way we make sure the data is linear first. Because we're changing from bt2020 to bt709, we actually start to clip the data at very saturated colors. This is why we first convert to floating point, so this data isn't lost, it's just out-of-gamut.
Then we do the tonemapping, and because it now has data that falls outside of the 0.0 - 1.0 range, it can do it's thing and start desaturating the colors where it thinks it's needed.
Don't do this. Primarily because tonemap desat will clip out-of-gamut pixels to black, which produces ugly artifacts. I also can't quite follow your reasoning, even without desat I can only think of negative possible side effects of this. If you want to prevent desaturation, just set desat to 0. Which is something I don't recommend btw. You always want at least some desaturation for clipped pixels, otherwise some pixels will clip to white way too late, or even never, when one of the three primary colors is completely missing, regardless of brightness This can create unrealistic brightness differences, which is way more distracting than desaturation, especially without reference. Smaller desat value means more desaturation. Don't go below 1 though, at least at 0.2 and lower desat I encountered very visible desaturation on unclipped pixels as well. The default of 2.0 is fine, I normally go lower to catch some edge cases (did some encodes with 1.3 and 1.2, with higher npl lower desat values tend to be a better idea), I rarely mind the desaturation in movies, which might be because I tend to use very high npl (400-700) with mobius, so only extremely bright pixels are desaturated at all. If you want to prevent desaturation at the cost of less accurate brightness differences I recommend a higher desat value, 4 for example. Wouldn't really recommend going higher than that, way too many cases where brighter light sources clip later than less bright light sources because of color differences.
damian101 is offline   Reply With Quote
Old 8th December 2021, 06:36   #59  |  Link
damian101
Registered User
 
Join Date: Feb 2021
Location: Germany
Posts: 17
Quote:
Originally Posted by Atak_Snajpera View Post
desat destroys image. Desat must be disabled. Period!
Well, that's probably because you applied BT.709 primaries before tonemapping.
damian101 is offline   Reply With Quote
Old 8th December 2021, 06:41   #60  |  Link
damian101
Registered User
 
Join Date: Feb 2021
Location: Germany
Posts: 17
Quote:
Originally Posted by Atak_Snajpera View Post
Single threaded tonemapping is creating huge bottleneck on my 8C/16T CPU. Is there any way to speed up zscale by using multi-threading?


zscale colorspace conversion seems indeed to be the bottleneck. However, it's not much slower than tonemapping in my testing, so not a huge bottleneck.

Some benchmarks:

Code:
-vf zscale=t=linear:npl=300,format=gbrpf32le
-> 19 fps

Code:
-vf zscale=t=linear:npl=300,format=gbrpf32le,zscale=t=bt709:p=bt709:m=bt709
-> 10 fps

Code:
-vf zscale=t=linear:npl=300,format=gbrpf32le,tonemap=tonemap=mobius
-> 13 fps

Code:
-vf zscale=t=linear:npl=300,format=gbrpf32le,tonemap=tonemap=mobius,zscale=t=bt709
-> 9.5 fps

Code:
-vf zscale=t=linear:npl=300,format=gbrpf32le,tonemap=tonemap=mobius,zscale=t=bt709:p=bt709:m=bt709
-> 8.3 fps

Code:
-vf zscale=t=linear:npl=300,format=gbrpf32le,tonemap=tonemap=mobius,zscale=t=bt709:p=bt709:m=bt709,format=420p
-> 7.2 fps

Code:
-vf zscale=t=linear:npl=300,format=gbrpf32le,tonemap=tonemap=mobius,zscale=t=bt709:p=bt709:m=bt709:d=ordered,format=yuv420p
-> 7.2 fps

Code:
-vf zscale=t=linear:npl=300,format=gbrpf32le,tonemap=tonemap=mobius,zscale=t=bt709:p=bt709:m=bt709:d=error_diffusion,format=yuv420p
-> 6.3 fps

Code:
-vf zscale=t=linear:npl=300,format=gbrpf32le,tonemap=tonemap=mobius,zscale=t=bt709:p=bt709:m=bt709 -pix_fmt yuv420p
-> 4.3 fps

Code:
-vf zscale=t=linear:npl=300,format=gbrpf32le,zscale,format=yuv420p
-> 12 fps

Code:
-vf zscale=t=linear:npl=300,format=gbrpf32le -pix_fmt yuv420p
-> 6.3 fps

You can also see that swscale format conversion is the slowest operation of them all, so make sure you do it with zscale instead.

mobius and hable are only insignificantly slower than clip

Last edited by damian101; 9th December 2021 at 16:25.
damian101 is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 09:17.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.