Hybrid: Input -> x264/x265/Xvid/VP8/VP9

[Havokdan]

Originally Posted by Selur

Hmm, looking at what tags Hybrid supports (for the main containers):

• Container tags:
  • Name
  • Genre
  • Comments
  • Copyright
  • Cover
• Stream tags:
  • Title
  • Language

and the tags Kodi allows (https://kodi.wiki/view/NFO_files/Movies#nfo_Tags)
I don't see anything for Copyright in the movie info and the Cover info would probably best match to the 'thumb' info, but to be useful Hybrid would need to download stuff (not likely to happen).

I see the following mapping:
Container->Name <=> movie->title
Container->Genre <=> movie->genre
Container->Comments <=> movie->plot
and
for audio streams:
Stream->Title <=> movie->fileinfo->streamdetails->audio->language
There is no language for video streams.

Since normally at least the audio language tags are usually properly flagged in the files itself, the only real gain from this would be to import the container tags.
(Importing the audio language could also be done, but probably not worth the trouble.)

-> don't really think that it importing the container tags is worth the effort, but if you find a few other users which think this is a nice feature I can probably write some code for it.

Cu Selur

Ps.: for mkv output adding a 'import for nfo' under 'Config->Output->MKV Tags' might be more useful, since there would be a bigger overlap of the kodi-NFO data and the mkv tags.

Ok, thanks

[Selur]

Hybrid rev 2020.01.12.1:
*fixed*

• decode: crash when ffmpeg was not compiled with 'zscale'

-> downloads: http://www.selur.de/downloads

Cu Selur

[Gravitator]

Хай!
Hybrid v2020.01.12.1

I use a two-second sample ProRes for encoding.
The culprit of the memory consumption of Deinterlacer: QTGMC (ffmpeg not eating).

[Selur]

No surprise, QTGMC will use quite a bit of memory and cpu power depending on the resolution and settings,...
Not sure why you mention ffmpeg, you are using VSPipe and x265, ffmpeg isn't used.

Cu Selur

[Gravitator]

Originally Posted by Selur

No surprise, QTGMC will use quite a bit of memory and cpu power depending on the resolution and settings,...

Deinterlacer: QTGMC - eats 4GB at any encoding settings (Overcoming the memory limit leads to the reset of information on the virtual memory HDD - fatal system throttling / only reboot).

Originally Posted by Selur

Not sure why you mention ffmpeg, you are using VSPipe and x265, ffmpeg isn't used.

Deinterlacer: FFMpeg - eats 80MB.
This only happens when using ProRes (Drop sample?).

[Gravitator]

Checking the new release leads to the collapse of Hybrid.

[Gravitator]

AOM
10bit mod & Profile: Main, High, Professional - оn any settings, disabling the "64-pt transform" leads to an instant encoding finish (Empty *.mkv or nothing at the output);
8bit mod & Profile: Main - ok, High - no, Professional - no.

[Gravitator]

RAV1E
Add new features: --threads, --first/second-pass (Instead of --pass 1/2), --rdo-lookahead-frames.

[Selur]

Deinterlacer: QTGMC - eats 4GB at any encoding settings

You can:
a. limit the thread count
b. limit the cache size
in Hybrid which will allow you to reduce the memory footprint.

Deinterlacer: FFMpeg - eats 80MB.

So ? Using a simple Yadif (or most others simpler deinterlacing methods) instead of QTGMC will use less memory.

Checking the new release leads to the collapse of Hybrid.

fixed locally.

AOM
10bit mod & Profile: Main, High, Professional - оn any settings, disabling the "64-pt transform" leads to an instant encoding finish (Empty *.mkv or nothing at the output);
8bit mod & Profile: Main - ok, High - no, Professional - no.

may be the command line options changed

RAV1E
Add new features: --threads, --first/second-pass (Instead of --pass 1/2), --rdo-lookahead-frames.

will think about it

[Gravitator]

It’s not normal that for 2 second (60 frames) 1080p Prores video, it consumes 4GB or more (There is no control of memory consumption).
-----
This only happens with ProRes in the *.mkv container! Then comes the call to VSPipe (*.mov moderate memory consumption and without VSPipe).

[Selur]

It’s not normal that for 2 second (60 frames) 1080p Prores video, it consumes 4GB or more (There is no control of memory consumption).

You are right QTGMC should not be used at all for 1080p content unless you explicitly tell Hybrid to do so.

Using:

# Imports
import os
import sys
import ctypes
# Loading Support Files
Dllref = ctypes.windll.LoadLibrary("I:/Hybrid/64bit/vsfilters/Support/libfftw3f-3.dll")
import vapoursynth as vs
core = vs.get_core()
# Import scripts folder
scriptPath = 'I:/Hybrid/64bit/vsscripts'
sys.path.append(os.path.abspath(scriptPath))
# Loading Plugins
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/GrainFilter/AddGrain/AddGrain.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/DenoiseFilter/FFT3DFilter/fft3dfilter.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/DenoiseFilter/DFTTest/DFTTest.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/EEDI3.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/ResizeFilter/nnedi3/vsznedi3.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/libmvtools.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/temporalsoften.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/scenechange.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/Support/fmtconv.dll")
core.std.LoadPlugin(path="I:/Hybrid/64bit/vsfilters/SourceFilter/LSmashSource/vslsmashsource.dll")
# Import scripts
import havsfunc
# Loading F:\TestClips&Co\files\interlaceAndTelecineSamples\interlaced\avc - interlaced.m2ts using LWLibavSource
clip = core.lsmas.LWLibavSource(source="F:/TESTCL~1/files/INTERL~1/INTERL~1/AVC-IN~1.M2T", format="YUV420P8", cache=0, prefer_hw=0)
# making sure input color matrix is set as 709
clip = core.resize.Point(clip, matrix_in_s="709",range_s="limited")
# making sure frame rate is set to 29.970
clip = core.std.AssumeFPS(clip, fpsnum=30000, fpsden=1001)
# Setting color range to TV (limited) range.
clip = core.std.SetFrameProp(clip=clip, prop="_ColorRange", intval=1)
# setting field order to what QTGMC should assume
clip = core.std.SetFrameProp(clip=clip, prop="_FieldBased", intval=2)
# Deinterlacing using QTGMC
clip = havsfunc.QTGMC(Input=clip, Preset="Fast", TFF=True)
# make sure content is preceived as frame based
clip = core.std.SetFieldBased(clip, 0)
clip = clip[::2]
# Output
clip.set_output()

Vapoursynth uses ~4.2GB, using preset 'placebo' it uses an additional ~300MB here.
Settin "Filtering->Vapoursynth->Misc->Limit Cache MB" to '1024', less than 1500MB get used.

Checking the new release leads to the collapse of Hybrid.

the option is removed now; too much trouble fixing it properly

Cu Selur

[Selur]

AOM
10bit mod & Profile: Main, High, Professional - оn any settings, disabling the "64-pt transform" leads to an instant encoding finish (Empty *.mkv or nothing at the output);
8bit mod & Profile: Main - ok, High - no, Professional - no.

calling it in a command line I got a few messages:
TX64 can only be disabled on I420 images.
8bit + profile 1: Profile 1 requires 4:4:4 color format
8bit + profile 2: Profile 2 bit-depth < 10 requires 4:2:2 color format

The main surprise here is the color space restriction since this is more restrictive than https://en.wikipedia.org/wiki/AV1#Profiles which doesn't allow High profile with 4:2:0,...
-> created an issue entry over at: https://github.com/mozilla/aom/issues/10

[Selur]

Hybrid rev 2020.02.16.1:
*fixed*

• Video: QSVEnc encoding mode mapping.
• Vapoursynth: custom script mismapping
• Vapoursynth: SelectRangeEvery and Tweak
• Vapoursynth: Fixed HDR to SDR DG call
• Vapoursynth: HDRToSDR
• Vapoursynth: LWLivavSource use track instead of streamindex
• Avisynth: removed 8bit enforcement for Tweak and Levels
• Video: aomenc - tx64 handling
• Profile: error message regarding 'input' when loading a GlobalDataModel
• Muxing: mp4 video&audio tagging
• Video: NVEnc, arib-std-b67 vs. arib-srd-b67
• Muxing: h264 avi muxing
• Analysis: mediainfo video bitrate indication
• Avisynth: handling high bitdepth content
• Decoder: FFmpeg decoder call

*changed*

• Avisynth: force LibavSource for AV1 content
• Avisynth: make sure BalanceBorders isn't used when Top/Bottom/Left/Right isn't mod 2

*added*

• Vapoursynth: option to save/load custom script content
• Video: rav1e added "--threads X", "--switch-frame-interval X", "--reservoir-frame-delay X" and "--rdo-lookahead-frames X"
• Video: NVEncC added support for '--psnr', '--ssim'

*removed*

• check for update
• Synth: removed FFms2k (it's changes were incorporated in ffms2)

-> downloads: http://www.selur.de/downloads

Cu Selur

[Gravitator]

Hybrid v2020.02.16.1

RAV1E
Unable to make the first pass, as in the TEMP folder is not created file *.ivf

[Selur]

I see the problem, will fix it when I find the time.
-> fixed locally, will be fine during next release

[Gravitator]

1st and 2nd pass works, but there is a problem with complex coding 2pass (The output is an empty container).

[Selur]

no clue what's causing this,... also I can't access the files attached there unless a moderator approves them.
-> got it

[Selur]

Hybrid rev 2020.02.17.1:
*fixed*

• video: rav1e 2pass calls

*changed*

• video: x265, adjusted to 3.3 branch

-> downloads: http://www.selur.de/downloads

Cu Selur

[Selur]

Hybrid rev 2020.02.19.1:
*fixed*

• Video: x265, messed up during 3.3. branch adjustment

*changed*

• Internals: removed some abandoned code which causes unneeded slow downs after files with lot of meta data were analysed

-> downloads: http://www.selur.de/downloads

Cu Selur

[Gravitator]

Hybrid v2020.02.19.1

AOM
Noise reduction and AQ are not available for: abr, cq, cqr encoding mode?
2-pass > abr, cq, cqr, lossless > 2-pass - noise reduction is blocked.