Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Closed Thread
 
Thread Tools Search this Thread Display Modes
Old 1st June 2016, 14:24   #1681  |  Link
Groucho2004
 
Join Date: Mar 2006
Location: Barcelona
Posts: 5,054
Quote:
Originally Posted by pinterf View Post
No need for v140_xp toolset, nor the extra /Zc:threadSafeInit- parameter. Why?
My wild guess would be that only certain Win32 APIs/functions require these switches in order for the code to run on XP. For others, the object code created by the compiler would be the same with or without the switches.
Groucho2004 is offline  
Old 1st June 2016, 14:56   #1682  |  Link
Reel.Deel
Registered User
 
Join Date: Mar 2012
Location: Texas
Posts: 1,191
Quote:
Originally Posted by Groucho2004 View Post
Building the installer is very easy and takes one minute. If pinterf is willing to supply an installer - I have uploaded my installer build environment here.
  1. Install InnoSetup (innosetup-5.5.9-unicode.exe)
  2. Update the binaries in "Bin_x86" and "Bin_x64"
  3. Open "avs_plus.iss" with InnoSetup
  4. Set "Version" (first line) to the correct revision
  5. Build (Ctrl-F9)
Done.
Wow, that was super easy!

---

@qyot27
I know a while back you were working on the documentation. I'm not sure if it was finished or not but from what I've seen it looks really good. How would I go about helping out?

Reel.Deel is offline  
Old 1st June 2016, 15:19   #1683  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 1,268
Quote:
Originally Posted by Reel.Deel View Post
@qyot27
I know a while back you were working on the documentation. I'm not sure if it was finished or not but from what I've seen it looks really good. How would I go about helping out?

Yes, the RST/Sphinx conversion was finished, at least to the point that the documentation was on par with the docs from classic AviSynth at the time (2.6 RC1, and RC2-Final if the rc2integrate branch is considered).

The rc2integrate branch also adds a note into README.md about how to build the Sphinx documentation.

Actually editing the docs mostly only requires basic text editing, since the RST syntax is meant to be human-readable. Updating the docs against classic AviSynth's changes is pretty trivial, just do an dump of the page with lynx before and after the commit that changes a particular file(s) (so you don't have to deal with all the HTML), compare the changes in something like Meld, and adjust the corresponding *.rst files in AviSynth+'s docs to match the text content, adjusting for RST or Sphinx-specific syntax.

EDIT: After looking at the commits since 2.6 Final, there were no docs changes, so the docs in rc2integrate are still as up-to-date as classic's docs. Also, generally, some of those 2.6.1 changes in classic's CVS make me want to scream in agony.

Last edited by qyot27; 1st June 2016 at 18:59.
qyot27 is offline  
Old 2nd June 2016, 00:57   #1684  |  Link
bilditup1
Registered User
 
bilditup1's Avatar
 
Join Date: Feb 2004
Location: NYC
Posts: 124
1080i, ColorMatrix, Expected FPS.

The latest mt_modes list says:

Code:
#note1: Can't use mode 1 or 2 on any filters if encoding 1080i files(480i works perfectly fine).
The alternative mt_modes list by tp7 further says:

Code:
#As far as I know, the only workaround seems to be running a deinterlacer right after the video source and running it as "MT_SERIALIZED",
#than running any desired filters after. Filters after the deinterlacer, seem to be able use any mode.
So:
  • Is this information still accurate? I am trying to use the 32-bit version of pinterf's latest mod with the following plugins:with A.SONY/real.finder's QTGMC mod.
  • What behavior should we expect with a 1080i source, if modes 1 or 2 are used normally? A crash? Artifacts? Very slow processing?
  • Since QTGMC uses several filters, which of them should be set to MT_SERIALIZED in order to avoid problems? NNEDI3 is one, I suppose, but others are used as well, right?
    • Specifically I'm curious about the filters used with the Medium and Fast presets of QTGMC.

Separately:
  • Is this note about ColorMatrix still accurate?
    Code:
    #note2: tried multiple files, seems to corrupt video even when it is the only filter in a script.
    #tried mode 1, 2, and 3, none worked. however it works fine if MT isn't enabled.
    #should ColorMatrix still be in the list? as a notice? or should it be removed?
  • If so, is there any alternative for doing colorspace conversions that works with AVS+ MT?

Without making any modifications to the MT modes file, on a 4770K@4.5Ghz, I seem to be getting 5-6.5 fps with MT enabled on a 1080i source being downscaled to 360p. By contrast, mainline AVS 2.6.1 with avs_tp (and ColorMatrix enabled) yields 6-7.5 fps, and AVS MT yields 12 fps but always crashes about 50-66% of the way through (again with ColorMatrix enabled). Is this within expected norms?

Also, this line in the MT modes files is incorrect:
Code:
SetFilterMTMode("DGDecIM",             MT_SERIALIZED)     #DGDecIM        beta 50   2015/10/10
The correct line with the proper function name is:
Code:
SetFilterMTMode("DGSourceIM",          MT_SERIALIZED)     #DGDecIM        beta 50   2015/10/10

Last edited by bilditup1; 2nd June 2016 at 01:10. Reason: Added currents speeds.
bilditup1 is offline  
Old 2nd June 2016, 01:58   #1685  |  Link
Reel.Deel
Registered User
 
Join Date: Mar 2012
Location: Texas
Posts: 1,191
@bilditup1

tp7's list is just an older revision of the latest MTModes list. He put it up on GitHub a while back as a backup because the original list was tampered with. Unfortunately it hasn't been updated since in over a year.

Regarding the notes, I'm not sure who wrote that or how or when it was tested. It was probably before this issue was fixed: https://github.com/AviSynth/AviSynthPlus/issues/37
It's probably best to test and report back if you find any problems.

All filters used by QTGMC work with MT mode 1 or 2 and they're already included in the MT Modes list (except for some filters from MVTools). I've successfully used QTGMC with:

Code:
SetFilterMTMode("WhateverSource" 3)
SetFilterMTMode("DEFAULT_MT_MODE", 2)

WhateverSource("somevideo.file")
QTGMC()

Prefetch(x)
or

Code:
Import("MTModes.avsi")
SetFilterMTMode("WhateverSource" 3)
SetFilterMTMode("DEFAULT_MT_MODE", 2)

WhateverSource("somevideo.file")
QTGMC()

Prefetch(x)
I do not use ColorMatrix so I don't if it crashes with MT or not. I use dither tools to change colorimetry.

Thanks for the DGSourceIM report. I'll correct it right now.
Reel.Deel is offline  
Old 2nd June 2016, 02:19   #1686  |  Link
bilditup1
Registered User
 
bilditup1's Avatar
 
Join Date: Feb 2004
Location: NYC
Posts: 124
Quote:
Originally Posted by Reel.Deel View Post
@bilditup1

tp7's list is just an older revision of the latest MTModes list. He put it up on GitHub a while back as a backup because the original list was tampered with. Unfortunately it hasn't been updated since in over a year.

Regarding the notes, I'm not sure who wrote that or how or when it was tested. It was probably before this issue was fixed: https://github.com/AviSynth/AviSynthPlus/issues/37
It's probably best to test and report back if you find any problems.
Thanks for all that; will do. I am testing with QTGMC and without ColorMatrix now and will do another one with CM later.

Quote:
Originally Posted by Reel.Deel View Post
All filters used by QTGMC work with MT mode 1 or 2 and they're already included in the MT Modes list (except for some filters from MVTools). I've successfully used QTGMC with:
OK. I am using QTGMC atm without issue. Let's see if it holds. Hopefully we'll be able to remove these disclaimers and prevent this kind of confusion

Quote:
Originally Posted by Reel.Deel View Post
I do not use ColorMatrix so I don't if it crashes with MT or not. I use dither tools to change colorimetry.
Off-topic I know, but how does this work? I found this snippet at the wiki:

Code:
Dither_convert_8_to_16 ()
Dither_resize16 (1280, 720)
Dither_convert_yuv_to_rgb (matrix="601", output="rgb48y", lsb_in=true)
r = SelectEvery (3, 0)
g = SelectEvery (3, 1)
b = SelectEvery (3, 2)
Dither_convert_rgb_to_yuv (r, g, b, matrix="709", lsb=false, mode=0)
For HD to SD conversion, should I just get rid of the resize, and switch the matrix parameters in the third and seventh lines? I don't understand what I'm looking at, specifically the SelectEvery() lines.
Quote:
Originally Posted by Reel.Deel View Post
Thanks for the DGSourceIM report. I'll correct it right now.
OK great! No problem.
bilditup1 is offline  
Old 2nd June 2016, 02:59   #1687  |  Link
Reel.Deel
Registered User
 
Join Date: Mar 2012
Location: Texas
Posts: 1,191
Quote:
Originally Posted by bilditup1 View Post
OK. I am using QTGMC atm without issue. Let's see if it holds. Hopefully we'll be able to remove these disclaimers and prevent this kind of confusion
Indeed, hopefully more people will help out in testing and validating filters. This list was meant to be community-driven but so far only a few people have actually contributed.
Quote:
Originally Posted by bilditup1 View Post
Off-topic I know, but how does this work? I found this snippet at the wiki:

Code:
Dither_convert_8_to_16 ()
Dither_resize16 (1280, 720)
Dither_convert_yuv_to_rgb (matrix="601", output="rgb48y", lsb_in=true)
r = SelectEvery (3, 0)
g = SelectEvery (3, 1)
b = SelectEvery (3, 2)
Dither_convert_rgb_to_yuv (r, g, b, matrix="709", lsb=false, mode=0)
For HD to SD conversion, should I just get rid of the resize, and switch the matrix parameters in the third and seventh lines? I don't understand what I'm looking at, specifically the SelectEvery() lines.
That script is on the wiki upscales SD to HD content, you want to downscale so use this instead:

Code:
Dither_convert_8_to_16 ()
Dither_resize16 (640, 480, csp="YV24") # change to the desired dimensions
Dither_convert_yuv_to_rgb (matrix="709", output="rgb48y", lsb_in=true) # convert to RGB using the 709 color matrix commonly used in HD content
r = SelectEvery (3, 0)
g = SelectEvery (3, 1)
b = SelectEvery (3, 2)
Dither_convert_rgb_to_yuv (r, g, b, matrix="601", lsb=false, mode=0, output="YV12") # convert to YUV using the 601 color matrix commonly used in SD content
First the YUV clip is scaled from HD to SD. Then it gets converted to RGB and finally back to YUV with the appropriate matrix for SD. The SelectEvery() is used to process 48-bit RGB clips, from the docs:
Quote:
48-bit RGB. The components R, G and B are conveyed on three YV12 or Y8 (if supported) stack16 clips interleaved on a frame basis.
I added csp="YV24" to Dither_resize16 to avoid any unnecessary resizing on the chroma planes. Finally, the output will be 8-bit YV12 if you want 16-bit just set lsb=true, if you want another colorspace just change the output parameter.
Reel.Deel is offline  
Old 2nd June 2016, 03:52   #1688  |  Link
bilditup1
Registered User
 
bilditup1's Avatar
 
Join Date: Feb 2004
Location: NYC
Posts: 124
Quote:
Originally Posted by Reel.Deel View Post
Indeed, hopefully more people will help out in testing and validating filters. This list was meant to be community-driven but so far only a few people have actually contributed.
Yeah they must not have if that's stayed in there for two years. That's rough. Whelp, I'll be sure to report anything I find personally, at least

Quote:
Originally Posted by Reel.Deel View Post
That script is on the wiki upscales SD to HD content, you want to downscale so use this instead [...]
Thanks for this, quite helpful. I know I'm really hijacking the thread here, but have a couple more questions.
Code:
Dither_resize16 (640, 480, csp="YV24") # change to the desired dimensions
So it is best to do the resize now, with Dither's resize, as opposed to just putting a Spline36Resize() at the end?
Code:
Dither_convert_yuv_to_rgb (matrix="709", output="rgb48y", lsb_in=true) # convert to RGB using the 709 color matrix commonly used in HD content
r = SelectEvery (3, 0)
g = SelectEvery (3, 1)
b = SelectEvery (3, 2)
Dither_convert_rgb_to_yuv (r, g, b, matrix="601", lsb=false, mode=0, output="YV12") # convert to YUV using the 601 color matrix commonly used in SD content
OK so basically, you just switched the matrix parameters between the first and last lines here. Cool.
Quote:
The SelectEvery() is used to process 48-bit RGB clips, from the docs:
Quote:
48-bit RGB. The components R, G and B are conveyed on three YV12 or Y8 (if supported) stack16 clips interleaved on a frame basis.
But then I'm wondering why we're using 48-bit RGB - I don't really understand the advantages or necessity either way. Is this because of the Dither_convert_8_to_16 () before?
bilditup1 is offline  
Old 2nd June 2016, 03:53   #1689  |  Link
Reel.Deel
Registered User
 
Join Date: Mar 2012
Location: Texas
Posts: 1,191
Quote:
Originally Posted by qyot27 View Post
Yes, the RST/Sphinx conversion was finished, at least to the point that the documentation was on par with the docs from classic AviSynth at the time (2.6 RC1, and RC2-Final if the rc2integrate branch is considered).

The rc2integrate branch also adds a note into README.md about how to build the Sphinx documentation.

Actually editing the docs mostly only requires basic text editing, since the RST syntax is meant to be human-readable. Updating the docs against classic AviSynth's changes is pretty trivial, just do an dump of the page with lynx before and after the commit that changes a particular file(s) (so you don't have to deal with all the HTML), compare the changes in something like Meld, and adjust the corresponding *.rst files in AviSynth+'s docs to match the text content, adjusting for RST or Sphinx-specific syntax.

EDIT: After looking at the commits since 2.6 Final, there were no docs changes, so the docs in rc2integrate are still as up-to-date as classic's docs. Also, generally, some of those 2.6.1 changes in classic's CVS make me want to scream in agony.
Thanks for the information. Hopefully it won't be too hard.

I would like too add some updates from the wiki, mainly the internal filters which raffriff42 has worked on (sadly the official documentation does not include these changes, see here). Also add a few things from the AviSynth+ wiki page, and maybe start documenting MT.
Reel.Deel is offline  
Old 2nd June 2016, 04:07   #1690  |  Link
bilditup1
Registered User
 
bilditup1's Avatar
 
Join Date: Feb 2004
Location: NYC
Posts: 124
Using AVSMeter 2.2.8 with r1858, it appears to work most of the way through...and then just sort of hangs with zero CPU activity.

Click here for a screenshot of where it stopped and the script I used. Source is still 1080i60 MPEG2, OS is Win10x64.

Last edited by bilditup1; 2nd June 2016 at 04:24.
bilditup1 is offline  
Old 2nd June 2016, 04:08   #1691  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,190
Quote:
Originally Posted by bilditup1 View Post
Is there any alternative for doing colorspace conversions that works with AVS+ MT?
The problem with ColorMatrix is that it can cause banding due to rounding errors as it processes with integers. Processing with 16-bit-depth avoids this problem.

In addition to DitherTools, you can try AviSynthShader. You can convert color matrix and resize your image through the GPU. Might be faster than DitherTools, not sure.
MysteryX is offline  
Old 2nd June 2016, 04:17   #1692  |  Link
Reel.Deel
Registered User
 
Join Date: Mar 2012
Location: Texas
Posts: 1,191
Quote:
Originally Posted by bilditup1 View Post
Using AVSMeter 2.2.8 with r1858, it appears to work most of the way through...and then just sort of hangs with zero CPU activity. Attached is a screenshot of where it stopped and the script I used. Source is still 1080i60 MPEG2, OS is Win10x64.
Can share attachments elsewhere? Sometimes it takes a while to get approved.

Regarding your other questions, if you post them in appropriate thread I'll gladly help where I can.
Reel.Deel is offline  
Old 2nd June 2016, 04:20   #1693  |  Link
bilditup1
Registered User
 
bilditup1's Avatar
 
Join Date: Feb 2004
Location: NYC
Posts: 124
Quote:
Originally Posted by MysteryX View Post
The problem with ColorMatrix is that it can cause banding due to rounding errors as it processes with integers. Processing with 16-bit-depth avoids this problem.

In addition to DitherTools, you can try AviSynthShader. You can convert color matrix and resize your image through the GPU. Might be faster than DitherTools, not sure.
I didn't know that ColorMatrix was basically obsolete, will keep that mind. Thanks for the info.

I'm not sure who is in charge of the AviSynth wiki, but maybe this can be pointed out somewhere? Like maybe here, where neither DitherTools nor AviSynthShader are mentioned at all?

ED: I just realized that anyone can sign up there, and just did. But I don't think I'm really qualified to begin mucking around there just yet.

Last edited by bilditup1; 2nd June 2016 at 04:33.
bilditup1 is offline  
Old 2nd June 2016, 04:25   #1694  |  Link
bilditup1
Registered User
 
bilditup1's Avatar
 
Join Date: Feb 2004
Location: NYC
Posts: 124
Quote:
Originally Posted by Reel.Deel View Post
Can share attachments elsewhere? Sometimes it takes a while to get approved.
Hmph, I hadn't noticed. I've uploaded them to mega.nz and placed the link in the original post above. Here it is again.

Quote:
Regarding your other questions, if you post them in appropriate thread I'll gladly help where I can.
Aight. But I think this:
Quote:
Originally Posted by MysteryX View Post
Processing with 16-bit-depth avoids this problem.
basically answers one of them already. I'll ask there too though, as I can't be the only one curious about this.

Last edited by bilditup1; 2nd June 2016 at 04:33.
bilditup1 is offline  
Old 2nd June 2016, 04:38   #1695  |  Link
bilditup1
Registered User
 
bilditup1's Avatar
 
Join Date: Feb 2004
Location: NYC
Posts: 124
Quote:
Originally Posted by bilditup1 View Post
Balls, I never modified this line to what it should have properly been:
Code:
SetFilterMTMode("WhateverSource", 3)
The MT mode for my source filter was setup properly in the mt_modes file I imported at the beginning, so presumably this line just did nothing. Still...sloppy of me, eh
bilditup1 is offline  
Old 2nd June 2016, 05:39   #1696  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 1,268
Quote:
Originally Posted by Reel.Deel View Post
Thanks for the information. Hopefully it won't be too hard.

I would like too add some updates from the wiki, mainly the internal filters which raffriff42 has worked on (sadly the official documentation does not include these changes, see here). Also add a few things from the AviSynth+ wiki page, and maybe start documenting MT.
It made it a lot easier for me to proof the changes between old (HTML) and new (Sphinx) versions by using the Tile View extension in Firefox, so I could see both side-by-side instead of needing to switch between tabs, with Notepad2 (or Leafpad, since I was under Linux while doing this) off to the side to edit the actual RST files.

It's really as simple as 'make html' in the distrib/docs/english directory, so long as you've installed Python and Sphinx. Sphinx will tell you if there are syntax errors in the files, and if the built HTML version is open in your browser, you can see the errors and work them down until it complies. And it'll also only rebuild the files that have been touched, rather than needing to rebuild the entire doc tree.
qyot27 is offline  
Old 2nd June 2016, 08:42   #1697  |  Link
bilditup1
Registered User
 
bilditup1's Avatar
 
Join Date: Feb 2004
Location: NYC
Posts: 124
Quote:
Originally Posted by bilditup1 View Post
Using AVSMeter 2.2.8 with r1858, it appears to work most of the way through...and then just sort of hangs with zero CPU activity.

Click here for a screenshot of where it stopped and the script I used. Source is still 1080i60 MPEG2, OS is Win10x64.
I ran this again, but this time using MeGUI and encoding to x264. The first time, MeGUI appears to have totally crashed. The second time, it completed without issue, at 10.17fps. The latter is pretty good. I will continue doing tests and reporting back, especially with respect to how well 1080i works and regarding matrix operations.
bilditup1 is offline  
Old 2nd June 2016, 08:43   #1698  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 1,852
Quote:
Originally Posted by bilditup1 View Post
Using AVSMeter 2.2.8 with r1858, it appears to work most of the way through...and then just sort of hangs with zero CPU activity.

Click here for a screenshot of where it stopped and the script I used. Source is still 1080i60 MPEG2, OS is Win10x64.
What happens when you trim the video right after the source filter and process only around the part when the freeze occurs. If the position of the freeze also moves back, the problem is source dependent. Maybe the problem is not with the source itself, but filters can behave content dependent e.g. at a scene change.
pinterf is offline  
Old 2nd June 2016, 09:22   #1699  |  Link
bilditup1
Registered User
 
bilditup1's Avatar
 
Join Date: Feb 2004
Location: NYC
Posts: 124
Quote:
Originally Posted by pinterf View Post
What happens when you trim the video right after the source filter and process only around the part when the freeze occurs.
In other words, everything after frame #34406, per the screenshot?

Quote:
Originally Posted by pinterf View Post
If the position of the freeze also moves back, the problem is source dependent.
Not sure what this means, sorry?

Quote:
Originally Posted by pinterf View Post
Maybe the problem is not with the source itself, but filters can behave content dependent e.g. at a scene change
OK, so that could be a problem with this artificial test of mine, where I'm taking 1,000 frames out of every 5,000, to avoid having to do a full encode that I don't even need? That should cause pretty abrupt scene changes, in theory.
bilditup1 is offline  
Old 2nd June 2016, 09:57   #1700  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 1,852
Quote:
Originally Posted by bilditup1 View Post
In other words, everything after frame #34406, per the screenshot?
Some month ago I fixed a problem when the memory consumption in avisynth suddenly increased after the 33000th frame, then at around 39000th, etc.. When I trimmed the source to begin at the 32000th frame, the same thing happened 32000 frames earlier, at the 1000th, 7000th, etc. frames. The problem was source dependent. And it turned out that there had been heavy camera shake at that problematic position. One of the filters detected a new scene (too much difference between neighbouring frames) and other parts of that filter were involved in processing and caused memory leak.

Such a basic script must not cause freeze should it be artificial or not.
pinterf is offline  
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 08:29.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, vBulletin Solutions Inc.