E and P cores in new laptop and video encoding

[hellgauss]

Even if related to video encoding, I post here because it seems an hardware/OS issue.

I usually encodes my video on a small Windows 10 home-server with "standard" cores, using e.g. this command line via .bat:

start /low /b /wait cmd /c "ffmpeg -hide_banner -nostdin -y -bitexact -i "Temp\input.mp4" -map 0 -vf "removegrain=1:1:1,scale=1280:720:flags=bicublinaram0=0.3125aram1=0.34375" -sws_flags accurate_rnd -c:v libx264 -preset veryslow -tune animation -profile:v high -level 40 -me_method tesa -me_range 56 -bf 16 -threads 1 -crf 19.3 -rc-lookahead 99 -deblock 1:1 -psy-rd 0.5:0 -aq-mode 3 -aq-strength 0.6 -chromaoffset 1 -x264-params b-adapt=1:scenecut=42:qcomp=0.58:vbv-maxrate=8000:vbv-bufsize=12000: -bitexact "output.mp4" 2>&1 | tee "log.txt""

The rationale on the start + cmd + "ffmpeg | tee" is that

- I prefer to set the priority to run in background using /low, hence i need "start"
- cmd is needed because I've not been able to simply "start" a complex command with redirection and pipe, and I need both log and stdout.

That command use single threaded encoding for quality, and I usually run automatically up to 3 of that command in parallel (I usually encode series). The system is stable, I can do other tasks on the PC and everything is ok.

Just bought a new performant laptop with Windows 11, with 4 E-cores and 6 P-cores. It seems that the command above only run ffmpeg in E-cores, which have very low performance. I can move ffmpeg to P-core by manually rising the priority via task manager to "abovenormal", gaining x4 performance. Furthermore even if i use the /abovenormal switch, ffmpeg is run on normal priority, probably because of "cmd".

The best solution to the problem is to find a way to run on low priority and choose the core to use. Or at least to be able to run ffmpeg in abovenormal priority without manually use task manager. I already tuned the OS and BIOS on high performance, but seems not to work.

Any suggestions?

Thanks,
HG

[StainlessS]

Maybe try Prio – Process Priority Saver (Homepage):- https://www.prnwatch.com/prio/
Softonic:- https://prio.en.softonic.com/
CNET:- https://download.cnet.com/prio-64-bi...?ex=WLS-2202.1

Free for personal use, else $20 commercial.

Developer’s Description
By O&K Software
Monitor and record the priority of every process listed in the Task Manager.
Prio (64-bit) is a utility for saving the priority of applications and interface enhancements for the standard Task Manager. The main purpose of Prio is Saving the priority of each process. Detecting the full path to the executable file and providing detailed information about it from the name of the process in the list of running processes in Windows Task Manager. Viewing services hosted by processes. Quick access to system services and controlling them.Analyzing the current TCP/IP connections of the computer. Checking a digital signature of an executable file for each process.

32 bit also available.

Last update seems to be Sept 2013 (Worked fine back then [EDIT: maybe about 2018 W10], assume still does)
Can (permanent option for specific executable) set process priority, processor affinity + other stuff.

EDIT:
It creates an extra TAB for TaskManager [ CTRL/SHIFT ESC ]
Also, on Performance TAB. right click on CPU graph, and "Change Graph to Logical Processors" is nice.

Prio - Alternative to:- https://alternativeto.net/software/p...riority-saver/
Softonic::Top Alternatives to Prio for Windows (EDIT: Seems to be a Bum Steer, dont bother) :- https://prio.en.softonic.com/windows/alternatives

IIRC, "Process Lasso" is supposed to be similar and good alternative.

Dont forget to post whether or not you have success or not, will be of interest to many methinks.

EDIT: Related(YouTube How to Permanently set High Priority Process in Windows (No 3rd Party Programs) Guide) :- https://www.youtube.com/watch?v=eson2ywtC0A

EDIT: Process Lasso [Real-Time CPU Optimization and Automation] Free and Pro versions:- https://bitsum.com/
See ALL on the Products TAB. <some for AMD only>

EDIT: Also, see how to set processor affinity via start command <likely require further investigation>

Starts a separate window to run a specified program or command.

START ["title"] [/D path] [/I] [/MIN] [/MAX] [/SEPARATE | /SHARED]
      [/LOW | /NORMAL | /HIGH | /REALTIME | /ABOVENORMAL | /BELOWNORMAL]
      [/NODE <NUMA node>] [/AFFINITY <hex affinity mask>] [/WAIT] [/B]
      [command/program] [parameters]

    "title"     Title to display in window title bar.
    path        Starting directory.
    B           Start application without creating a new window. The
                application has ^C handling ignored. Unless the application
                enables ^C processing, ^Break is the only way to interrupt
                the application.
    I           The new environment will be the original environment passed
                to the cmd.exe and not the current environment.
    MIN         Start window minimized.
    MAX         Start window maximized.
    SEPARATE    Start 16-bit Windows program in separate memory space.
    SHARED      Start 16-bit Windows program in shared memory space.
    LOW         Start application in the IDLE priority class.
    NORMAL      Start application in the NORMAL priority class.
    HIGH        Start application in the HIGH priority class.
    REALTIME    Start application in the REALTIME priority class.
    ABOVENORMAL Start application in the ABOVENORMAL priority class.
    BELOWNORMAL Start application in the BELOWNORMAL priority class.
    NODE        Specifies the preferred Non-Uniform Memory Architecture (NUMA)
                node as a decimal integer.
    AFFINITY    Specifies the processor affinity mask as a hexadecimal number.
                The process is restricted to running on these processors.

                The affinity mask is interpreted differently when /AFFINITY and
                /NODE are combined.  Specify the affinity mask as if the NUMA
                node's processor mask is right shifted to begin at bit zero.
                The process is restricted to running on those processors in
                common between the specified affinity mask and the NUMA node.
                If no processors are in common, the process is restricted to
                running on the specified NUMA node.
    WAIT        Start application and wait for it to terminate.
    command/program
                If it is an internal cmd command or a batch file then
                the command processor is run with the /K switch to cmd.exe.
                This means that the window will remain after the command
                has been run.

                If it is not an internal cmd command or batch file then
                it is a program and will run as either a windowed application
                or a console application.

    parameters  These are the parameters passed to the command/program.

NOTE: The SEPARATE and SHARED options are not supported on 64-bit platforms.

Specifying /NODE allows processes to be created in a way that leverages memory
locality on NUMA systems.  For example, two processes that communicate with
each other heavily through shared memory can be created to share the same
preferred NUMA node in order to minimize memory latencies.  They allocate
memory from the same NUMA node when possible, and they are free to run on
processors outside the specified node.

    start /NODE 1 application1.exe
    start /NODE 1 application2.exe

These two processes can be further constrained to run on specific processors
within the same NUMA node.  In the following example, application1 runs on the
low-order two processors of the node, while application2 runs on the next two
processors of the node.  This example assumes the specified node has at least
four logical processors.  Note that the node number can be changed to any valid
node number for that computer without having to change the affinity mask.

    start /NODE 1 /AFFINITY 0x3 application1.exe
    start /NODE 1 /AFFINITY 0xc application2.exe

If Command Extensions are enabled, external command invocation
through the command line or the START command changes as follows:

non-executable files may be invoked through their file association just
    by typing the name of the file as a command.  (e.g.  WORD.DOC would
    launch the application associated with the .DOC file extension).
    See the ASSOC and FTYPE commands for how to create these
    associations from within a command script.

When executing an application that is a 32-bit GUI application, CMD.EXE
    does not wait for the application to terminate before returning to
    the command prompt.  This new behavior does NOT occur if executing
    within a command script.

When executing a command line whose first token is the string "CMD "
    without an extension or path qualifier, then "CMD" is replaced with
    the value of the COMSPEC variable.  This prevents picking up CMD.EXE
    from the current directory.

When executing a command line whose first token does NOT contain an
    extension, then CMD.EXE uses the value of the PATHEXT
    environment variable to determine which extensions to look for
    and in what order.  The default value for the PATHEXT variable
    is:

        .COM;.EXE;.BAT;.CMD

    Notice the syntax is the same as the PATH variable, with
    semicolons separating the different elements.

When searching for an executable, if there is no match on any extension,
then looks to see if the name matches a directory name.  If it does, the
START command launches the Explorer on that path.  If done from the
command line, it is the equivalent to doing a CD /D to that path.

[Z2697]

I've heard Process Lasso is a good processes performance tuning software, though I haven't try it myself.

[StainlessS]

Originally Posted by Z2697

I've heard Process Lasso is a good processes performance tuning software, though I haven't try it myself.

Yes, and links from my previous post.

Originally Posted by StainlessS

IIRC, "Process Lasso" is supposed to be similar and good alternative.

EDIT: Process Lasso [Real-Time CPU Optimization and Automation] Free and Pro versions:- https://bitsum.com/
See ALL on the Products TAB. <some for AMD only>

EDIT: "Fix Your FPS When CPU-Bound With This Tool!.. | Process Lasso CPU-Specific Affinity Guide For Frames"
(For gamers, may not be best guide for encoding)
https://www.youtube.com/watch?v=RLw31O_9Nqo

Another YT clip, less Games oriented.
How to set up process lasso for increased system performance
https://www.youtube.com/watch?v=i1eK7xIdXIs

Youtube Search, "set CPU affinity process Lasso"
https://www.youtube.com/results?sear...rocess%20Lasso

EDIT:
I aint done any encoding for about 6 months, last time that I did on a
Dell Optiplex 7000 Desktop Micro (little 1 Litre i7-12700T machine), I just disabled the E cores in the BIOS.
(as MeGUI x264/x265 encode insisted on using only the 4 E-Cores, leaving all Performance ie 8 Physical and 8 Logical cores unused. [W10])

Optimise your CPU with Process Lasso - Intel 12th, 13th and 14th Gen CPU Optimisation.
https://www.youtube.com/watch?v=m7Ki5DYME9M

[hellgauss]

Thanks for replies. I will check.

However I think that the correct solution is to write my own command scheduler instead of using start and cmd, e.g. using CreateProcessA() in cpp. Even if the E core are slow, they are still there and worth to be used together with P. And maybe in some occasion I would like to run ffmpeg completely in background. I need a solution which works in a bat file. start + cmd + "redirect/pipe" is somewhat an hack and need to be handled more efficiently. It also creates lots of useless intermediate processes in which priority is not correctly inherited.

[StainlessS]

OK, whatever you come up with will likely interest others, incl me.

Also, on Performance TAB. right click on CPU graph, and "Change Graph to Logical Processors" is nice.

Above nothing to do with Prio/Process_Lasso, TaskManager/Performance TAB, allows to see individual CORE graphs.

Good Luck, No reply necessary

EDIT: W11 is supposed to be better at handling P/E cores.

[VoodooFX]

So, 12th gen is even shittier than 11th gen?

[Z2697]

Originally Posted by StainlessS

EDIT: W11 is supposed to be better at handling P/E cores.

But, put background tasks in E cores only is a feature. So W11 isn't gonna help much.

[hellgauss]

So, 12th gen is even shittier than 11th gen?

I think it is a Windows problem, and its philosophy "I'll choose (wrong) for you", together with the "green" mantra for low consumption.

To enable the "High performance" preset I had to regedit.

[StainlessS]

So, 12th gen is even shittier than 11th gen?

I only have a Lenovo 11th Gen i5-11400T (400 = slower version i5) in the 1 Litre format 35W CPU m/c so have not really compared with 35W 12th Gen i7.
(Although I got a 65W 11th Gen i7-11700 m/c, cheapish ACER Aspire TC-1660 a few months back, not really done anything with it except move to W10 from W11).
I think MeGUI is normally set to "Below Normal" priority for encodes (x264/x265), and that is at least part reason for the exclusive use of E cores.
(I was though actually really quite surprised at how good/fast 35W 12th gen i7 was using only E cores, but a bit sickening seeing all of them there P cores doing nuttin').
At the time I did a little 12th Gen encoding, I did not remember the name of the Prio/Process_Lasso apps, so did not try them.

But, put background tasks in E cores only is a feature. So W11 isn't gonna help much.

Life's a bitch, and then you die.

EDIT: Windows only has a handful of priority levels, even the Sinclair QL (~1985/1986, Motorola MC 68008) has/had 128 priority levels for multi-tasking.

[GeoffreyA]

Originally Posted by VoodooFX

So, 12th gen is even shittier than 11th gen?

Well, not talking from experience, but I think the Thread Director and mixing of P- and E-cores were a mess, using a roundabout solution to tackle the key problem at the time: efficiency. Nonetheless, the Gracemont cores, in themselves, were well designed and, if I remember rightly, on par with Skylake. (For comparison, Zen 1 was slightly behind the latter.) The recent Skymont has also proved potent.

[hellgauss]

Solved with uninstalling Windows Terminal, installed by default in Win11, and use standard shell.

Settings-->Apps-->Installed Apps-->Search "Terminal"-->Uninstall Windows Terminal

I still cannot set priority abovenormal, but if there is nothing else to do ffmpeg runs in P even on low priority.
In normal priority it seems to prefer P even if the shell is minimized, but under heavy loads the task goes in E.

Also, I gained in stability, since WT have issue with the graphic driver and sometimes crashes all of my encoding.

[Z2697]

From my observation the priority is inherited from parent process only when it's lower, so you need to find all child processes and set their priority individually. (which is very hard if even possible to do in cmd)

[hellgauss]

@Z2697 Indeed priority higher than normal cannot be inherited.

**Important update**

Great improvements on P-Core usage by doing this:
Control Panel --> Power Options --> Change plan settings --> Change Advanced Power Settings --> Change settings that are currently unavailable (*) --> Processor power management

On "Heterogeneous policy in effect" select "Policy 2" ( see e.g. https://community.bitsum.com/forum/i...?topic=11811.0 )
On "Heterogeneous thread scheduling policy" select "Prefer performant processor" (I think it is the same as above)

At this moment I have 7 running encoding started with normal priority and minimized, and they are all on P while I'm focusing on this forum in firefox. Tomorrow I will check what happens on low priority.

(*) This options maybe is not available by default, I do not remembere what I did to enable, you should activate it, see e.g. https://answers.microsoft.com/en-us/...3-b176db12d75f , or use google.

[StainlessS]

Mine (W10 home on main machine, HP Omen 880) shows option to change in percentage, not specifically MHz, in Control Panel/ Power/ Advanced settings.
But anyway, here is a page on related.
How to Change Maximum Processor Frequency in Windows
https://www.ninjaone.com/blog/change...cy-in-windows/

I dont have this key in my registry
HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Power\User\PowerSettings\

Anyway, I personally dont have any immediate need for process control, but maybe in some not too distant future I will.

EDIT:
On the above linked page, shows

powercfg -attributes SUB_PROCESSOR CPMAXFREQ -ATTRIB_HIDE

to enable in control panel/power,

BUT, on my system, that command dont seem to be legal (-attributes).

C:\Users\steve>powercfg /?

POWERCFG /COMMAND [ARGUMENTS]

Description:
  Enables users to control power settings on a local system.

  For detailed command and option information, run "POWERCFG /? <COMMAND>"

Command List:
  /LIST, /L          Lists all power schemes.

  /QUERY, /Q         Displays the contents of a power scheme.

  /CHANGE, /X        Modifies a setting value in the current power scheme.

  /CHANGENAME        Modifies the name and description of a power scheme.

  /DUPLICATESCHEME   Duplicates a power scheme.

  /DELETE, /D        Deletes a power scheme.

  /DELETESETTING     Deletes a power setting.

  /SETACTIVE, /S     Makes a power scheme active on the system.

  /GETACTIVESCHEME   Retrieves the currently active power scheme.

  /SETACVALUEINDEX   Sets the value associated with a power setting
                     while the system is powered by AC power.

  /SETDCVALUEINDEX   Sets the value associated with a power setting
                     while the system is powered by DC power.

  /IMPORT            Imports all power settings from a file.

  /EXPORT            Exports a power scheme to a file.

  /ALIASES           Displays all aliases and their corresponding GUIDs.

  /GETSECURITYDESCRIPTOR
                     Gets a security descriptor associated with a specified
                     power setting, power scheme, or action.

  /SETSECURITYDESCRIPTOR
                     Sets a security descriptor associated with a
                     power setting, power scheme, or action.

  /HIBERNATE, /H     Enables and disables the hibernate feature.

  /AVAILABLESLEEPSTATES, /A
                     Reports the sleep states available on the system.

  /DEVICEQUERY       Returns a list of devices that meet specified criteria.

  /DEVICEENABLEWAKE  Enables a device to wake the system from a sleep state.

  /DEVICEDISABLEWAKE Disables a device from waking the system from a sleep
                     state.

  /LASTWAKE          Reports information about what woke the system from the
                     last sleep transition.

  /WAKETIMERS        Enumerates active wake timers.

  /REQUESTS          Enumerates application and driver Power Requests.

  /REQUESTSOVERRIDE  Sets a Power Request override for a particular Process,
                     Service, or Driver.

  /ENERGY            Analyzes the system for common energy-efficiency and
                     battery life problems.

  /BATTERYREPORT     Generates a report of battery usage.

  /SLEEPSTUDY        Generates a diagnostic system power transition report.

  /SRUMUTIL          Dumps Energy Estimation data from System Resource Usage
                     Monitor (SRUM).

  /SYSTEMSLEEPDIAGNOSTICS
                     Generates a diagnostic report of system sleep transitions.

  /SYSTEMPOWERREPORT Generates a diagnostic system power transition report.

  /POWERTHROTTLING   Control power throttling for an application.

[hellgauss]

I found it. Run cmd as admin and

For /f %K in ('Reg.exe query HKLM\SYSTEM\CurrentControlSet\Control\Power\PowerSettings /s /v "Attributes"^|FindStr HKEY_') do Reg.exe add %K /v Attributes /t REG_DWORD /d 0 /f

Reference: https://www.reddit.com/r/Amd/comment..._power_scheme/

[Z2697]

Heterogeneous cores is kinda cool from the look but...
Alas, I guess it doesn't have a good outcome when you are not desperately trying to save power, and are outside of the benchmarks...

Or, say that QAT in Xeon is good heterogeneous computing, this PE core is not, there's a difference.

[hellgauss]

It is not cool, it is a mess. I spent +150$ to get a more powerful CPU, also to play with video editing and night encoding. If I wanted power saving I would have bought a low performance CPU, like I did with my previous home-server. Or set a low power profile option. It is absurd that I have to unlock such hidden options only to run tasks efficiently, with the risk to make mess.

BTW: it seems that "Heterogeneous policy in effect" should be set to 0, it is not the same as "Heterogeneous thread scheduling policy". If you suspect to make mess with options, just click "restore default" (remember to turn off sleep properly afterwards). I'll update previous post with correct info and references...

And fortunately I did not install any game. I imagine that with the GPU it is the same...

[microchip8]

Poor Windows users...

[StainlessS]

Just thought that I'de point this out while I remember it (currently doing it).
I've just been making images of my W10 machines with final updates, and for my 1 Litre Dell Optiplex 7000 Micro, CPU i7-12700T (35W TDP),
and the speed of image writing using Macrium Reflect USB bootable tool was ~238Mb/sec (rather slow), and so I re-did with E-Cores switched off and
an currently getting 1.5->1.8Gb/sec write speed (@ highest compression to nvme SSD).
So, when using any kind of simple DOS/USB type tool, looks like it will be using pretty much only the E-Cores.

EDIT: I may be using an older ver$ of Macrium Reflect USB tool. [EDIT: v8.0, build 6979]