GradCurve VapourSynth plugin port of virtualdub Gradation Curves plugin

[xekon]

After doing some more testing, I started noticing that artifacts were being introduced. On some frames it appears to be working perfectly, but on other frames it is clearly introducing artifacts:



Well that sure is interesting, the artifacts are only introduced on the linux version, the one I installed to windows does not do it. (on windows both debug & release dll work without artifacts)



I just did an encode, to make sure it could be reproduced without VSedit, using the below script:

import vapoursynth as vs
core = vs.get_core()
src = core.d2v.Source(input=r'/media/sf_moon/enc/032t.d2v', rff=False)
src = core.resize.Bicubic(clip=src, format=vs.RGBS)
src = core.grad.Curve(src,'/media/sf_moon/enc/032t.amp',7,1)
src = core.resize.Bicubic(clip=src, matrix=1, format=vs.YUV420P16)
src.set_output()

vspipe --y4m 032t.vpy - | x264 --demuxer y4m - --output 032t.mkv

I still get the artifacts on linux with the encoded file, but not under windows. Does anybody know where or what I should look at to determine the reason for the artifacts?

could this be a vapoursynth bug? I cannot understand why it looks good in windows and makes artifacts in linux.

[jackoneill]

Originally Posted by xekon

I just did an encode, to make sure it could be reproduced without VSedit, using the below script:

import vapoursynth as vs
core = vs.get_core()
src = core.d2v.Source(input=r'/media/sf_moon/enc/032t.d2v', rff=False)
src = core.resize.Bicubic(clip=src, format=vs.RGBS)
src = core.grad.Curve(src,'/media/sf_moon/enc/032t.amp',7,1)
src = core.resize.Bicubic(clip=src, matrix=1, format=vs.YUV420P16)
src.set_output()

vspipe --y4m 032t.vpy - | x264 --demuxer y4m - --output 032t.mkv

I still get the artifacts on linux with the encoded file, but not under windows. Does anybody know where or what I should look at to determine the reason for the artifacts?

could this be a vapoursynth bug? I cannot understand why it looks good in windows and makes artifacts in linux.

1. Try to remove -ffast-math from configure. It enables potentially unsafe optimisations for floating point math.
2. Try to remove all instances of the type "long" and use (u)int64_t if you want 64 bit integers. The size of long is apparently 32 bits in Windows and 64 bits in 64 bit Linux.

To see if it's a bug in VapourSynth, try your script with the grad.Curve filter commented out.

[xekon]

thank you for the reply Jackoneill, i will try -02, and thanks for telling me how to debug it in linux.

thanks for the reply Are_, very interesting you dont have the bug, that leads me to believe its a problem with the compiler i have on my linux machine, i will try GCC-5.4.0 if thats not what im currently using.

Thank you for the reply Jackoneill. I removed -ffast-math from configure and replace all instances of long with int64_t.
Also if I comment out the gradcurve filter then it does not introduce the artifacts.

After making the changes to the project, the windows one still works, and the linux one still has the artifacts.
could it be anything else in configure that is causing it?

I also tried removing -mfpmath=sse, but it had no effect either.

PS: Also its worth mentioning, the Artifacts, are always in the exact same spot, it is only affecting certain pixels, im gonna try and step through the program to see if anything obvious jumps out at me.
but since I am debugging on windows(where there are no artifacts) I am not so sure that anything will be obvious to me.

[jackoneill]

Originally Posted by xekon

Thank you for the reply Jackoneill. I removed -ffast-math from configure and replace all instances of long with int64_t.
Also if I comment out the gradcurve filter then it does not introduce the artifacts.

After making the changes to the project, the windows one still works, and the linux one still has the artifacts.
could it be anything else in configure that is causing it?

I also tried removing -mfpmath=sse, but it had no effect either.

PS: Also its worth mentioning, the Artifacts, are always in the exact same spot, it is only affecting certain pixels, im gonna try and step through the program to see if anything obvious jumps out at me.
but since I am debugging on windows(where there are no artifacts) I am not so sure that anything will be obvious to me.

Maybe -O2 instead of -O3, but I doubt that helps.

You can debug it in Linux too. You need to add -ggdb3 to CXXFLAGS and make sure the library doesn't get stripped. Then you can invoke gdb:

gdb --args vspipe script.py /dev/null

To set a breakpoint at line 42 (answer yes):

(gdb) break file.cpp:42

To run the program:

(gdb) run

To step into functions:

(gdb) step

To go to the next line without stepping into functions:

(gdb) next

To print the value of variable "asdf":

(gdb) print asdf

[Are_]

I tied your test clip here but I could not reproduce your bug. GCC-5.4.0

[xekon]

Are_ was kind enough to share their compiled libgradcurve.so which for them did not show the bug, I tried their compiled libgradcurve.so and the bug was still present for me. We compared VSedit and VSpipe versions and theirs was newer, so I thought that maybe that was the issue.

I have reinstalled kubuntu as well as freshly installed vapoursynth and vsedit and all of their dependencies, I am now using the same version of VSedit and VSpipe as Are_ and I STILL have the bug I show in screenshots above.

Can different OSes cause the issue? I am using Kubuntu and they are not, I am considering using another linux distro just to test what could be causing this bug.

I think i may know the problem, the math used to determine the indexes in the color lookup tables may be returning different result on different OSes, (which is a problem in itself.) I am going to think about a better way to do that:

I am referring to these lines:

				oldr = srcpR[x]*255;
				oldg = srcpG[x]*255;
				oldb = srcpB[x]*255;

is there a way I can convert srcpR[x]/srcpG[x]/srcpB[x] to their respective 0-255 values? other than taking that float value and multiplying it by 255?

I am going to find some worse scenes that show the bug more, and try debugging the way that jackoneill pointed out, but I am thinking those 3 line are the most likely culprit....

[jackoneill]

Originally Posted by xekon

Are_ was kind enough to share their compiled libgradcurve.so which for them did not show the bug, I tried their compiled libgradcurve.so and the bug was still present for me. We compared VSedit and VSpipe versions and theirs was newer, so I thought that maybe that was the issue.

I have reinstalled kubuntu as well as freshly installed vapoursynth and vsedit and all of their dependencies, I am now using the same version of VSedit and VSpipe as Are_ and I STILL have the bug I show in screenshots above.

Can different OSes cause the issue? I am using Kubuntu and they are not, I am considering using another linux distro just to test what could be causing this bug.

I think i may know the problem, the math used to determine the indexes in the color lookup tables may be returning different result on different OSes, (which is a problem in itself.) I am going to think about a better way to do that:

I am referring to these lines:

				oldr = srcpR[x]*255;
				oldg = srcpG[x]*255;
				oldb = srcpB[x]*255;

is there a way I can convert srcpR[x]/srcpG[x]/srcpB[x] to their respective 0-255 values? other than taking that float value and multiplying it by 255?

I am going to find some worse scenes that show the bug more, and try debugging the way that jackoneill pointed out, but I am thinking those 3 line are the most likely culprit....

Make your filter accept (only?) integer clips and give it the video straight from d2vsource, without the conversion to RGBS. Then you can directly use the pointers returned by getReadPtr (they are unsigned char*).

This is a really weird problem.

[xekon]

So far it only happens to very dark colored pixels, mostly on lines and edges, I am hesitant to change it to integer clips, when I may possibly be discovering an underlying bug someplace.
I am still scanning through footage looking for one that displays the bug predominantly, then I will upload another sample and try doing some debugging under linux.

hmm, not sure what I'm doing wrong. It seems to be not honoring the breakpoint. I configured with --enable-debug which is supposed to not strip and keep the debug symbols....

enc@enc-vbox:~/.installs/VapourSynthPlugins/GradCurve$ ./configure --enable-debug
CXXFLAGS/LDFLAGS checking...
checking for vapoursynth headers...

settings...
CXX          = g++
LD           = g++
STRIP        = strip
CXXFLAGS     = -O2 -std=c++11 -Wall -Wno-strict-aliasing -I. -I./include -g  -msse2  -fPIC -ggdb3 -mfpmath=sse -I/usr/local/include/vapoursynth -I/usr/local/include
LDFLAGS      = -shared -fPIC  
LIBS         =  
LIBNAME      = libgradcurve.so
install path = /usr/local/lib/vapoursynth
configure finished.

enc@enc-vbox:~/.installs/VapourSynthPlugins/GradCurve$ make
g++ -O2 -std=c++11 -Wall -Wno-strict-aliasing -I. -I./include -g  -msse2  -fPIC -ggdb3 -mfpmath=sse -I/usr/local/include/vapoursynth -I/usr/local/include -c GradCurve/GradCurve.cpp -o GradCurve/GradCurve.o
GradCurve/GradCurve.cpp: In function ‘void ImportCurve(GradCurveData*)’:
GradCurve/GradCurve.cpp:374:35: warning: ignoring return value of ‘int fscanf(FILE*, const char*, ...)’, declared with attribute warn_unused_result [-Wunused-result]
     fscanf(pFile, "%d", & stor[i]);
                                   ^
g++ -o libgradcurve.so -shared -fPIC   GradCurve/GradCurve.o 

enc@enc-vbox:~/.installs/VapourSynthPlugins/GradCurve$ sudo make install
install -d /usr/local/lib/vapoursynth
install -m 755 libgradcurve.so /usr/local/lib/vapoursynth

enc@enc-vbox:~/.installs/VapourSynthPlugins/GradCurve$ gdb --args vspipe /media/sf_moon/enc/003a/003a.vpy /dev/null
GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.04) 7.11.1
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from vspipe...(no debugging symbols found)...done.
(gdb) break GradCurve.cpp:604
No symbol table is loaded.  Use the "file" command.
Make breakpoint pending on future shared library load? (y or [n]) y
Breakpoint 1 (GradCurve.cpp:604) pending.
(gdb) run
Starting program: /usr/local/bin/vspipe /media/sf_moon/enc/003a/003a.vpy /dev/null
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
[New Thread 0x7fffd20e1700 (LWP 2130)]
[New Thread 0x7fffcc428700 (LWP 2131)]
Output 9 frames in 0.09 seconds (103.69 fps)
[Thread 0x7fffcc428700 (LWP 2131) exited]
[Thread 0x7fffd20e1700 (LWP 2130) exited]
[Inferior 1 (process 2126) exited normally]
(gdb)

I never get the chance to use step, next, or print, because the breakpoints are not firing.

Did I miss something, is there a setting I should double check? or maybe I should compile differently? more manually?


I am thinking I should setup an IDE on kubuntu for compiling and debugging c++, maybe I will try eclipse, or possibly netbeans?

after a bit of reading it seems many of the IDEs available for linux, dont allow you to view a variables value breakpoint/watchpoint but gdb does, so I really should figure out what it is im doing wrong.

gonna atleast give Code::Blocks a try.

EDIT: code::blocks is not stopping at the breakpoints either, I removed the original GradCurve library to make sure it was not being used, and set my script to load the debug library that code::blocks generatd. I will keep plugging away at it, trying different thing, surely i'll figure it out eventually. So frustrating when try something intuitively and it does not work the way you expected it to.

[jackoneill]

Originally Posted by xekon

hmm, not sure what I'm doing wrong. It seems to be not honoring the breakpoint. I configured with --enable-debug which is supposed to not strip and keep the debug symbols....

Now you know why this set of GNUmakefile and configure sucks. Even though lots of people adopted them. It does strip the library. It may work better if you add this under line 110 in configure:

STRIP=""

(Consider it a temporary workaround, not a proper solution.)

[xekon]

Thank you for the reply jackoneill, gonna give that a try today!

very interesting, I thought only linux was displaying this bug, however my windows machine does to, just to a much lesser degree, and less frequently, here is a scene that does it in both windows and linux

here is the sample: http://www.mediafire.com/file/5gsfbg3a7ubahnh/003b.7z

Linux Kubuntu 16.10_64:


win7_64:


going to finish debugging this, and then probably change it to use integer clips instead

[xekon]

woa, if you go back and forth between the windows and linux ones.... the blacks are WAAAY darker on the linux one (atleast they are on my linux install)
I commented out my plugin, the linux one is still waaaay darker, so its not my plugin doing it, why is there a difference between the linux one and the windows one?

edit: the three lines that I thought were the problem, indeed were, I made the following change:

				oldr = srcpR[x]*255;
				if (oldr < 0) { oldr = 0; }
				if (oldr > 255) { oldr = 255; }
				oldg = srcpG[x]*255;
				if (oldg < 0) { oldg = 0; }
				if (oldg > 255) { oldg = 255; }
				oldb = srcpB[x]*255;
				if (oldb < 0) { oldb = 0; }
				if (oldb > 255) { oldb = 255; }

that way they would not be out of range of the index, it works, but it seems messy, I think I should still modify this to only use integer clips, but feedback on this is welcome.

edit: updated the github to include this bugfix: https://github.com/jieiku/GradCurve

EDIT:
I added the greater than 255 checking for good measure, but that is not what was setting off the bug.

the bug was happening when a value was less than 0, therefore beyond the index, but the math is to take the source pointer(getReadPtr) and multiply it by 255.... the only way I could see it getting a value less than 0 is if one of those source pointers were returning a negative value? is that supposed to ever happen in vapoursynth?

[TheFluff]

Floating point chroma is typically in the range -0.5 to 0.5, but I don't know if you're accepting YUV input? You're not getting integer wraparound, are you?

Also, clamping a value x to a range is usually written in an ideomatic fashion using min and max, like so:

min(max(0, x), 255)

[jackoneill]

You can have float pixels with values outside the "legal" ranges in VapourSynth. However, I'm not sure the resizer is supposed to create such pixels in this case.

The difference between the black levels is due to d2vsource. The latest Windows DLLs have a bug that was fixed in git. They incorrectly mark the video as full range, and so the resizer treats it as such.

PS. The range -0.5..0.5 applies to U and V channels. RGB channels all use the range 0.0..1.0.

[feisty2]

Originally Posted by jackoneill

You can have float pixels with values outside the "legal" ranges in VapourSynth. However, I'm not sure the resizer is supposed to create such pixels in this case.

it's supposed to keep out of range pixels like that with floating point samples.
the only exception would be a floating point mask clip, which should be clamped to [0.0, 1.0]

[xekon]

Originally Posted by TheFluff

Floating point chroma is typically in the range -0.5 to 0.5, but I don't know if you're accepting YUV input? You're not getting integer wraparound, are you?

Also, clamping a value x to a range is usually written in an ideomatic fashion using min and max, like so:

min(max(0, x), 255)

I am using RGBS input vis this line before going to my plugin, not sure how to check if its integer wraparound, i will need to read up on that, can't say i've ran into it before.
src = core.resize.Bicubic(clip=src, format=vs.RGBS)

jackoneill I didn't realize that about the d2vsource, I will get the appropriate version.

so did I find a bug? or is this normal behavior and I should just use integer input only for my plugin to solve this?

[jackoneill]

Originally Posted by xekon

so it sounds like the out of range pixels will only be an issue using floating point?

Not quite. If you have a 9..15 bit clip, filters can produce pixels with values greater than the bit depth's maximum, up to 65535. I think this can be considered a bug, though, i.e. someone else's problem.

[feisty2]

Originally Posted by xekon

I am using RGBS input vis this line before going to my plugin
src = core.resize.Bicubic(clip=src, format=vs.RGBS)

so it sounds like the out of range pixels will only be an issue using floating point? if so then I am probably best off just converting this plugin to only accept RGB integer input.

the out of range pixels will only NOT be an issue using floating point.

[xekon]

oh.... hmmm that's a conundrum.... so should I just clamp it like im doing for now and call it lovely?

as of my last update to github, the plugin appears to be working correctly to my eyes, thats not to say i'm not overlooking something.

I really want to make this filter function properly, would suck to encode a bunch of stuff only to discover that this plugin introduced a subtle problem into the video that I didn't realize lol.

[xekon]

oh, I got an idea, at the very least I could compare the output results of this plugin with the original gradation curves plugin... I think there are differences in colorspace conversions though, so I dont know if i'de be able to test for an exact match or not.

[shekh]

The original gradation curves was 8-bit and you still have 8-bit lookup tables. You should just stay at 8-bit input.