Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 11th July 2019, 21:37   #6901  |  Link
Barough
Registered User
 
Barough's Avatar
 
Join Date: Feb 2007
Location: Sweden
Posts: 323
V3.1+7 is from the Default channel and includes what the v3.1.1 have and some more

Skickat från min SM-G975F via Tapatalk
Barough is offline   Reply With Quote
Old 12th July 2019, 00:34   #6902  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 1,042
Quote:
Originally Posted by Stereodude View Post
Why hasn't direct AVIsynth input been added to the windows builds of x265 like x264 has? I'm rather sick of having to pipe video into it.
Use x265-Yuuki or ffmpeg.
qyot27 is offline   Reply With Quote
Old 12th July 2019, 00:38   #6903  |  Link
stax76
Registered User
 
Join Date: Jun 2002
Location: Germany
Posts: 5,565
Quote:
Originally Posted by qyot27 View Post
Use x265-Yuuki or ffmpeg.
Or Wolfberry
stax76 is offline   Reply With Quote
Old 12th July 2019, 01:24   #6904  |  Link
Stereodude
Registered User
 
Join Date: Dec 2002
Location: Region 0
Posts: 1,084
Quote:
Originally Posted by qyot27 View Post
Use x265-Yuuki or ffmpeg.
Is there some trick to making x265-Yuuki work with avisynth+?

Code:
C:\HDTV Tools\x265_Yuuki>x265-gcc-multilib-full.exe -f 1500 --crf 16.0 -p placebo --no-sao --aq-strength 1.15 --vbv-maxrate 25000 --vbv-bufsize 25000 --level 5.0 
--keyint 120 --open-gop -D 10 --colorprim "bt709" --transfer "bt709" --colormatrix "bt709" --sar 1:1 -o HZLIP_p_1.15_16.0_nosao_D10_dith_p1_t_10in.265 
"E:\TempBD\HZLIP\HZLIP_comb_1.avs"
[avs2 @ 00000000037d8300] Format avs2 detected only with low score of 1, misdetection possible!
[avs2 @ 00000000037d8300] Could not find codec parameters for stream 0 (Video: avs2, none): unspecified size
Consider increasing the value for the 'analyzeduration' and 'probesize' options
[NULL @ 0000000004e14700] No codec provided to avcodec_open2()
[NULL @ 0000000004e14700] No codec provided to avcodec_open2()
lavf [error]: could not find decoder for video stream
x265 [error]: unable to open input file <E:\TempBD\HZLIP\HZLIP_comb_1.avs>
Edit: Wolfberry works with the same command line (changing only the name of the executable).

Last edited by Stereodude; 12th July 2019 at 02:02.
Stereodude is offline   Reply With Quote
Old 12th July 2019, 03:46   #6905  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 1,042
Yuuki is the patchset/branch (the patch itself is just generic LAVF input); whether the particular build was linked against a libavformat that itself was built with --enable-avisynth is up to the one building it. 'avs2' is not AviSynth (it's the Chinese AVS2 format), and the only reason libavformat would jump to that when given an AviSynth script is that FFmpeg hadn't been built with --enable-avisynth.
qyot27 is offline   Reply With Quote
Old 12th July 2019, 11:43   #6906  |  Link
Stereodude
Registered User
 
Join Date: Dec 2002
Location: Region 0
Posts: 1,084
So why doesn't the Wolfberry build of x265 honor affinity restrictions? Using
Code:
START "Enc #1" /NORMAL /NODE 0 /AFFINITY 00FF
in front of the x265 command line still results in the same number of threads being spawned as no affinity restrictions or /AFFINITY 000F. It always sees the full processor which is the same problem I have with piping.
Stereodude is offline   Reply With Quote
Old 13th July 2019, 01:20   #6907  |  Link
Stereodude
Registered User
 
Join Date: Dec 2002
Location: Region 0
Posts: 1,084
--seek doesn't work correctly with the Wolfberry build with an avisynth+ input either.

Code:
x265.exe" --pools 4 -F 1 --crf 16.0 -p veryslow --no-sao --aq-mode 1 --aq-strength 1.15 
--vbv-maxrate 40000 --vbv-bufsize 40000 --level 5.1 --keyint 120 --open-gop -D 10 
--colorprim "bt709" --transfer "bt709" --colormatrix "bt709" --sar 1:1 --seek 16609 
--frames 16678 -o out_2.265 "source.avs"
I get this output in the console: (which says all the right things)
Code:
lavf [info]:
 Format    : avisynth
 Codec     : rawvideo ( raw video )
 PixFmt    : yuv420p10le
 Framerate : 24/1
 Timebase  : 1/24
 Duration  : 2:18:31
lavf [info]: 1920x1080 fps 24/1 i420p10 sar 1:1 frames 16609 - 33286 of 199487
raw  [info]: output file: out_2.265
x265 [info]: HEVC encoder version 3.1.1+1-04b37fdfd2dc
x265 [info]: build info [Windows][GCC 9.1.1][64 bit] 10bit
x265 [info]: using cpu capabilities: MMX2 SSE2Fast SSSE3 SSE4.2 AVX
x265 [info]: Main 10 profile, Level-5.1 (Main tier)
x265 [info]: Thread pool created using 4 threads
x265 [info]: Slices                              : 1
x265 [info]: frame threads / pool features       : 1 / wpp(17 rows)
x265 [info]: Coding QT: max CU size, min CU size : 64 / 8
x265 [info]: Residual QT: max TU size, max depth : 32 / 3 inter / 3 intra
x265 [info]: ME / range / subpel / merge         : star / 57 / 4 / 5
x265 [info]: Keyframe min / max / scenecut / bias: 12 / 120 / 40 / 5.00
x265 [info]: Lookahead / bframes / badapt        : 40 / 8 / 2
x265 [info]: b-pyramid / weightp / weightb       : 1 / 1 / 1
x265 [info]: References / ref-limit  cu / depth  : 5 / off / off
x265 [info]: AQ: mode / str / qg-size / cu-tree  : 1 / 1.1 / 32 / 1
x265 [info]: Rate Control / qCompress            : CRF-16.0 / 0.60
x265 [info]: VBV/HRD buffer / max-rate / init    : 40000 / 40000 / 0.900
x265 [info]: tools: rect amp rd=6 psy-rd=2.00 rdoq=2 psy-rdoq=1.00 rskip
x265 [info]: tools: signhide tmvp b-intra strong-intra-smoothing deblock
[0.3%] 54/16678 frames, 0.16 fps, 6617.67 kb/s, 1.77 MB, eta 29:46:47, est.size 548.21 MB
But the actual out_2.265 output file starts from frame 0 in the source, not frame 16609.
Stereodude is offline   Reply With Quote
Old 13th July 2019, 01:56   #6908  |  Link
MeteorRain
結城有紀
 
Join Date: Dec 2003
Location: NJ; OR; Shanghai
Posts: 528
AviSynth support came from LAVF input filter.
LAVF input filter was ported from x264.
x264 didn't support seeking.

So this input filter never had seeking feature.

As a side note, LAVF was never designed to take AVS script (and that's exactly the reason I did not include AVS support in my build).

I'd hope one day someone can add a proper AVS native input. But on the other hand my scripts are on 32-bit so I have no choice but to use piper anyway.
MeteorRain is offline   Reply With Quote
Old 13th July 2019, 03:36   #6909  |  Link
Stereodude
Registered User
 
Join Date: Dec 2002
Location: Region 0
Posts: 1,084
AFAIK, piping can't seek either. The workaround is a giant pain. I have to make a unique .avs for every segment the outputs the right frames using trim, and then I have to make a unique .chp for every segment/x265 call that's shifted by the correct amount.
Stereodude is offline   Reply With Quote
Old 13th July 2019, 05:21   #6910  |  Link
alex1399
Registered User
 
Join Date: Jun 2018
Posts: 50
How about multiple piping? Just pipe the avs + lossless x264 encode ultrafast before the x265 ones
alex1399 is offline   Reply With Quote
Old 13th July 2019, 13:37   #6911  |  Link
Stereodude
Registered User
 
Join Date: Dec 2002
Location: Region 0
Posts: 1,084
Quote:
Originally Posted by alex1399 View Post
How about multiple piping? Just pipe the avs + lossless x264 encode ultrafast before the x265 ones
Can you elaborate a bit more? I'm not sure I'm following you.
Stereodude is offline   Reply With Quote
Old 13th July 2019, 17:29   #6912  |  Link
alex1399
Registered User
 
Join Date: Jun 2018
Posts: 50
Quote:
Originally Posted by Stereodude View Post
Why hasn't direct AVIsynth input been added to the windows builds of x265 like x264 has? I'm rather sick of having to pipe video into it.
Never mind. Doesn't saw this previous post before.
alex1399 is offline   Reply With Quote
Old 14th July 2019, 04:27   #6913  |  Link
MeteorRain
結城有紀
 
Join Date: Dec 2003
Location: NJ; OR; Shanghai
Posts: 528
Quote:
Originally Posted by Stereodude View Post
AFAIK, piping can't seek either. The workaround is a giant pain. I have to make a unique .avs for every segment the outputs the right frames using trim, and then I have to make a unique .chp for every segment/x265 call that's shifted by the correct amount.
Can you shed more light on what you are trying to accomplish?

Why would a chp file be affected by seeking support? Suppose it's a chapter file, and x265 does not take chapter files as input.

Talking about my experience, if I want to backup a movie, usually I want to encode it in multiple segments. I have scripts which can produce "import().trim()" type of AVS files, and I encode them in different batch, then concatenate them together, attach the chapter, and call it a day.
MeteorRain is offline   Reply With Quote
Old 14th July 2019, 09:20   #6914  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 4,542
3-level Hierarchial Motion Estimation

I see there are two recent commits related to hierachial motion estimation. I couldn't find anything specific as to what the use cases would be, does anyone have any insights? Or is this once again one of those things that just appear in the encoder out of the blue (probably requested by some paying client) and the devs leave it at that.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 14th July 2019, 09:34   #6915  |  Link
froggy1
ffx264/ffhevc author
 
froggy1's Avatar
 
Join Date: May 2007
Location: Belgium
Posts: 1,461
Quote:
Originally Posted by Boulder View Post
I see there are two recent commits related to hierachial motion estimation. I couldn't find anything specific as to what the use cases would be, does anyone have any insights? Or is this once again one of those things that just appear in the encoder out of the blue (probably requested by some paying client) and the devs leave it at that.
Here is a rather technical explanation http://homepages.inf.ed.ac.uk/rbf/CV...stimation.html
__________________
ffx264--ffhevc--ffxvid
froggy1 is offline   Reply With Quote
Old 14th July 2019, 11:58   #6916  |  Link
Stereodude
Registered User
 
Join Date: Dec 2002
Location: Region 0
Posts: 1,084
Quote:
Originally Posted by MeteorRain View Post
Can you shed more light on what you are trying to accomplish?

Why would a chp file be affected by seeking support? Suppose it's a chapter file, and x265 does not take chapter files as input.

Talking about my experience, if I want to backup a movie, usually I want to encode it in multiple segments. I have scripts which can produce "import().trim()" type of AVS files, and I encode them in different batch, then concatenate them together, attach the chapter, and call it a day.
I'm encoding something in multiple segments to improve quality (less threads) and speed (multiple encodes at once). I'm talking about having to shift frames around in the qpfile. Each unique .avs needs a unique qpfile.

Seek support would simply this since the same .avs and qpfile could be used and only the seek point and frame number on the command line would need to change.

This would also be more simple if affinity restrictions from the START command in Windows 10 worked with x265. It seems that somewhere in the Windows 10 feature update chain MS changed the behavior. Affinity restrictions on a .bat file that called an executable used to result in the same affinity restrictions being placed on the executable started by the .bat file. Now it doesn't. Affinity restrictions placed directly on the x265 command line don't even work. (They do work on the x264 command line.)

Ideally I would run a batch file with these four lines and be done.
Code:
START "Enc #1" /NORMAL /NODE 0 /AFFINITY F000 "C:\HDTV Tools\x265\x265.exe" -F 1 --crf 16.0 -p veryslow ...
START "Enc #2" /NORMAL /NODE 0 /AFFINITY 0F00 "C:\HDTV Tools\x265\x265.exe" -F 1 --crf 16.0 -p veryslow ...
START "Enc #3" /NORMAL /NODE 0 /AFFINITY 00F0 "C:\HDTV Tools\x265\x265.exe" -F 1 --crf 16.0 -p veryslow ...
START "Enc #4" /NORMAL /NODE 0 /AFFINITY 000F "C:\HDTV Tools\x265\x265.exe" -F 1 --crf 16.0 -p veryslow ...
Each x265 instance would see only 4 logical cores and would be locked to those 4 cores. But instead each x265 instance sees all 16 logical cores and I have to add --pools 4 to the command line and then manually modify the affinity of each x265 process in the Task Manager once they're all started.
Stereodude is offline   Reply With Quote
Old 14th July 2019, 15:06   #6917  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Hollola, Finland
Posts: 4,542
Quote:
Originally Posted by froggy1 View Post
Here is a rather technical explanation http://homepages.inf.ed.ac.uk/rbf/CV...stimation.html
Thanks - yes, quite technical. To me it looks somehow similar to MRecalculate in MVTools, but maybe someone can explain it in layman's terms or simply come up with some POC they must have performed at some point. Looks interesting anyway.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 14th July 2019, 17:34   #6918  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 6,856
Quote:
Affinity restrictions placed directly on the x265 command line don't even work. (They do work on the x264 command line.)
I can confirm this behaviour. Looks like that x265 unlike x264 resets affinity at launch. I would also prefer that x265 didn't change initial affinity set by user.
Atak_Snajpera is offline   Reply With Quote
Old 15th July 2019, 12:01   #6919  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 5,851
I believe it may be possible to use the --pools parameter to control the affinity of x265 instances if it was just documented verbosely enough...

Quote:
x265 creates one or more thread pools per encoder, one pool per NUMA node (typically a CPU socket). --pools specifies the number of pools and the number of threads per pool the encoder will allocate. By default x265 allocates one thread per (hyperthreaded) CPU core on each NUMA node.
This may be not sufficient without examples. Right away you will notice that the online documentation does not even mention some parameters in the help output:

Code:
   --pools <integer,...>         Comma separated thread count per thread pool (pool per NUMA node)
                                 '-' implies no threads on node, '+' implies one thread per core on node
I would guess that calls with a similar result as the above mentioned "start /AFFINITY" might look similar to:

Code:
x265 --pools 4,"+---"
x265 --pools 4,"-+--"
x265 --pools 4,"--+-"
x265 --pools 4,"---+"
I hope Ma can enlighten us.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 15th July 2019, 12:12   #6920  |  Link
Stereodude
Registered User
 
Join Date: Dec 2002
Location: Region 0
Posts: 1,084
Quote:
Originally Posted by LigH View Post
I believe it may be possible to use the --pools parameter to control the affinity of x265 instances if it was just documented verbosely enough...



This may be not sufficient without examples. Right away you will notice that the online documentation does not even mention some parameters in the help output:

Code:
   --pools <integer,...>         Comma separated thread count per thread pool (pool per NUMA node)
                                 '-' implies no threads on node, '+' implies one thread per core on node
I would guess that calls with a similar result as the above mentioned "start /AFFINITY" might look similar to:

Code:
x265 --pools 4,"+---"
x265 --pools 4,"-+--"
x265 --pools 4,"--+-"
x265 --pools 4,"---+"
I hope Ma can enlighten us.
Your example doesn't work. And, it contradicts what's in this: https://x265.readthedocs.io/en/default/cli.html

It has these examples:
Code:
""        - default, unspecified, all numa nodes are used for thread pools
"*"       - same as default
"none"    - no thread pools are created, only frame parallelism possible
"-"       - same as "none"
"10"      - allocate one pool, using up to 10 cores on all available nodes
"-,+"     - allocate one pool, using all cores on node 1
"+,-,+"   - allocate one pool, using only cores on nodes 0 and 2
"+,-,+,-" - allocate one pool, using only cores on nodes 0 and 2
"-,*"     - allocate one pool, using all cores on nodes 1, 2 and 3
"8,8,8,8" - allocate four pools with up to 8 threads in each pool
"8,+,+,+" - allocate two pools, the first with 8 threads on node 0, and the second with all cores on node 1,2,3
There's considerably more verbage on it too.

I see no references to core restrictions, only node restrictions in the documentation.

Last edited by Stereodude; 15th July 2019 at 12:16.
Stereodude is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 02:16.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2019, vBulletin Solutions Inc.