Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Closed Thread
 
Thread Tools Search this Thread Display Modes
Old 18th August 2016, 23:21   #2301  |  Link
Sparktank
47.952fps@71.928Hz
 
Sparktank's Avatar
 
Join Date: Mar 2011
Posts: 940
Phew. So many updates for a lot of stuff.

I love the progress. A lot. Everything looks so promising.

Great work, guys. The world is changing.
__________________
Win10 (x64) build 19041
NVIDIA GeForce GTX 1060 3GB (GP106) 3071MB/GDDR5 | (r435_95-4)
NTSC | DVD: R1 | BD: A
AMD Ryzen 5 2600 @3.4GHz (6c/12th, I'm on AVX2 now!)
Sparktank is offline  
Old 18th August 2016, 23:33   #2302  |  Link
ultim
AVS+ Dev
 
ultim's Avatar
 
Join Date: Aug 2013
Posts: 359
Quote:
Originally Posted by MasterNobody View Post
ultim
Also imho "#ifdef MSVC" change in capi.h is wrong because apps compiled with MinGW using this header (avisynth_c.h i.e. C inteface) wouldn't be compatible with standard (MSVS) AviSynth+ because they will try to call functions with __cdecl instead of __stdcall calling convention.
You're right. Unwillingly but I keep forgetting about the C-interface. I undefined __stdcall for MinGW because I thought it is only for future Linux-support anyway, and anybody else will have to be using MSVC on Windows. But this doesn't hold for the C interface. I'll patch this up too.
__________________
AviSynth+
ultim is offline  
Old 19th August 2016, 04:45   #2303  |  Link
MysteryX
Soul Architect
 
MysteryX's Avatar
 
Join Date: Apr 2014
Posts: 2,559
Quote:
Originally Posted by ultim View Post
On the contrary, it is counterintuitive for me how memory grows only so little when you increase the number of engines from 4 to 8, since the load on the caches should increase by the same amount as if you used the core's MT. One possible explanation is I have a wrong understanding of what your "engines" do and how they work in parallel.
An Engine is an instance of a DirectX device that processes a chain of commands. Such device can only be used by a thread at once. Because it holds all the graphic card textures, it tends to take a lot of memory (twice more in x64 for an unknown reason).

What was giving me best performance was running Avisynth with 4 or 8 threads and splitting out the work between 2 DX engines (which saturates the GPU). Something is now different in the way memory behaves, but I can't pin-point what's different.
MysteryX is offline  
Old 19th August 2016, 13:06   #2304  |  Link
Groucho2004
 
Join Date: Mar 2006
Location: Barcelona
Posts: 5,034
Quote:
Originally Posted by rean View Post
Access Violation using LoadPlugin nnedi.dll
Wow, I didn't think that the original nnedi was still being used.

Anyway, this bug report is retarded.
- No mentioning of the Avisynth+ revision used
- No script
- No client with which the error occurred specified
- Access violation in which module?

Since you consider yourself a programmer, you should know better.
__________________
Groucho's Avisynth Stuff
Groucho2004 is offline  
Old 19th August 2016, 14:27   #2305  |  Link
ultim
AVS+ Dev
 
ultim's Avatar
 
Join Date: Aug 2013
Posts: 359
Quote:
Originally Posted by rean View Post
Access Violation using LoadPlugin nnedi.dll
Hi rean, please send me:
- your script
- the Avs+ revision that was used
- your nnedi plugin dll
__________________
AviSynth+
ultim is offline  
Old 19th August 2016, 21:02   #2306  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 2,309
I have a question about running external plugin. When is the destructor called ? If i have several external filters (from several dll files), is it possible that a destructor of a filter A is called when frames of another filter B are still do be processed (but all frames of A are finished), or are the destructors begin to be called ONLY when ALL the frames of ALL the filters are being processed ? (When everything is finished). This last seems the more logical, but i can't take it for granted.
jpsdr is offline  
Old 19th August 2016, 21:40   #2307  |  Link
MasterNobody
Registered User
 
Join Date: Jul 2007
Posts: 552
Can someone explain me use cases for 10/12/14 bit pixel types? imho for processing it easier to use either 8-bit or 16-bit if higher precision is needed (there is no point to make processing at >8 and <16 bits as it would have same speed as 16-bit and will need special casing algorithms). Or this pixel types are really 16-bit and this is only metadata which doesn't really change position of most significant bit (i.e. 10-bit have real bits at 6..15 and zeroes at bits 0..5 and not real bits at 0..9 and zeroes at 10..15)? If this is not metadata than I don't understand why ConvertTo16bits() and ConvertTo16bits(bits=10) results in same byte output. Also why ConvertToYUV420/YUV422/YUV444 from 10-bit pixel type results in 16-bit pixel type.
MasterNobody is offline  
Old 19th August 2016, 22:13   #2308  |  Link
ultim
AVS+ Dev
 
ultim's Avatar
 
Join Date: Aug 2013
Posts: 359
Quote:
Originally Posted by MasterNobody View Post
Can someone explain me use cases for 10/12/14 bit pixel types? imho for processing it easier to use either 8-bit or 16-bit if higher precision is needed (there is no point to make processing at >8 and <16 bits as it would have same speed as 16-bit and will need special casing algorithms). Or this pixel types are really 16-bit and this is only metadata which doesn't really change position of most significant bit (i.e. 10-bit have real bits at 6..15 and zeroes at bits 0..5 and not real bits at 0..9 and zeroes at 10..15)? If this is not metadata than I don't understand why ConvertTo16bits() and ConvertTo16bits(bits=10) results in same byte output. Also why ConvertToYUV420/YUV422/YUV444 from 10-bit pixel type results in 16-bit pixel type.

10/12/14 formats follow Microsofts VUV description for higher bit-depths, and store valid bits in the MSBs of the two bytes for each pixel.

These formats are not meant for processing in Avisynth, they just exist so that they can be output to (and be read from). So basically only conversion functions will have support for them.

If you mean that based on your testing, some 16bit and 10bit conversion functions have the same output, I think that is because the implementation is not yet finished and code is still missing to handle all cases. But pinterf can provide more insight in this matter than me. He's still in the middle of implementation, he only seems inactive recently because he's on holidays with his family.
__________________
AviSynth+

Last edited by ultim; 19th August 2016 at 22:15.
ultim is offline  
Old 19th August 2016, 22:13   #2309  |  Link
Wilbert
Moderator
 
Join Date: Nov 2001
Location: Netherlands
Posts: 6,364
Quote:
Originally Posted by jpsdr View Post
I have a question about running external plugin. When is the destructor called ? If i have several external filters (from several dll files), is it possible that a destructor of a filter A is called when frames of another filter B are still do be processed (but all frames of A are finished), or are the destructors begin to be called ONLY when ALL the frames of ALL the filters are being processed ? (When everything is finished). This last seems the more logical, but i can't take it for granted.
Neither. Otherwise non-linear access wouldn't be possible, so there is not really a last frame. The deconstructor of a filter is called, when the filter is destroyed. This happens when ScriptEnvironment is destroyed and that happens when your script is closed.
Wilbert is offline  
Old 19th August 2016, 22:43   #2310  |  Link
real.finder
Registered User
 
Join Date: Jan 2012
Location: Mesopotamia
Posts: 2,587
Quote:
Originally Posted by ultim View Post
10/12/14 formats follow Microsofts VUV description for higher bit-depths, and store valid bits in the MSBs of the two bytes for each pixel.

These formats are not meant for processing in Avisynth, they just exist so that they can be output to (and be read from). So basically only conversion functions will have support for them.

If you mean that based on your testing, some 16bit and 10bit conversion functions have the same output, I think that is because the implementation is not yet finished and code is still missing to handle all cases. But pinterf can provide more insight in this matter than me. He's still in the middle of implementation, he only seems inactive recently because he's on holidays with his family.
so if we has a 10 bit source and work on it without convert it to 16 bit will get error message? or it will convert to 16 bit silently and get process and then convert to 10 bit silently again?
__________________
See My Avisynth Stuff
real.finder is offline  
Old 19th August 2016, 22:52   #2311  |  Link
MasterNobody
Registered User
 
Join Date: Jul 2007
Posts: 552
Quote:
Originally Posted by ultim View Post
10/12/14 formats follow Microsofts VUV description for higher bit-depths, and store valid bits in the MSBs of the two bytes for each pixel.
OK. So this formats differ from memory layout used for yuv 10-bit layout used by x264, ffmpeg and avs2yuv (yuv4mpeg extension). And so this apps should really ignore it and use them as 16-bit or refuse to accept it? Then I would need to change avs2yuv and x264 patch.

btw. what the point of this formats i.e. why not always mark them as 16-bit.

Last edited by MasterNobody; 19th August 2016 at 22:57.
MasterNobody is offline  
Old 19th August 2016, 23:17   #2312  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 1,419
The individual 10/12/14 pix_fmt constants (AVS_CS_YUV422P10, etc.) are there mostly to ease auto-selection with applications like FFmpeg so the user doesn't have to second-guess swscale/avutil. For example, AVS_CS_YUV422P10 is directly matched to AV_PIX_FMT_YUV422P10 in the libavformat AviSynth demuxer, and there's no need for the user to invoke -pix_fmt and override anything. That's what the intention was for those constants.

Also, because existing implementations (f3kdb*, Dither**) are capable of outputting those depths directly, so the reconstituted output is reported as the same depth, given to FFmpeg as the same depth, and processed there in the same depth.

*output_depth parameter
*Dither_quantize

Last edited by qyot27; 19th August 2016 at 23:26.
qyot27 is offline  
Old 19th August 2016, 23:33   #2313  |  Link
MasterNobody
Registered User
 
Join Date: Jul 2007
Posts: 552
Quote:
Originally Posted by qyot27 View Post
The individual 10/12/14 pix_fmt constants (AVS_CS_YUV422P10, etc.) are there mostly to ease auto-selection with applications like FFmpeg so the user doesn't have to second-guess swscale/avutil. For example, AVS_CS_YUV422P10 is directly matched to AV_PIX_FMT_YUV422P10 in the libavformat AviSynth demuxer, and there's no need for the user to invoke -pix_fmt and override anything. That's what the intention was for those constants.
Which doesn't conform to suggestion of Microsofts VUV description for higher bit-depths because this are different formats that can't be directly mapped to current AV_PIX_FMT_*s.
MasterNobody is offline  
Old 20th August 2016, 00:09   #2314  |  Link
Wilbert
Moderator
 
Join Date: Nov 2001
Location: Netherlands
Posts: 6,364
Quote:
Originally Posted by MasterNobody View Post
Which doesn't conform to suggestion of Microsofts VUV description for higher bit-depths because this are different formats that can't be directly mapped to current AV_PIX_FMT_*s.
Looks the same to me: https://github.com/FFmpeg/FFmpeg/blo...vutil/pixfmt.h

Code:
    /**
     * The following 12 formats have the disadvantage of needing 1 format for each bit depth.
     * Notice that each 9/10 bits sample is stored in 16 bits with extra padding.
     * If you want to support multiple bit depths, then using AV_PIX_FMT_YUV420P16* with the bpp stored separately is better.
     */
    ...
    AV_PIX_FMT_YUV420P10BE,///< planar YUV 4:2:0, 15bpp, (1 Cr & Cb sample per 2x2 Y samples), big-endian
AV_PIX_FMT_YUV420P10LE,///< planar YUV 4:2:0, 15bpp, (1 Cr & Cb sample per 2x2 Y samples), little-endian
Stored as 16 bit with extra padding means stored as MSB.
Wilbert is offline  
Old 20th August 2016, 10:06   #2315  |  Link
MasterNobody
Registered User
 
Join Date: Jul 2007
Posts: 552
Quote:
Originally Posted by Wilbert View Post
Looks the same to me: https://github.com/FFmpeg/FFmpeg/blo...vutil/pixfmt.h

Code:
    /**
     * The following 12 formats have the disadvantage of needing 1 format for each bit depth.
     * Notice that each 9/10 bits sample is stored in 16 bits with extra padding.
     * If you want to support multiple bit depths, then using AV_PIX_FMT_YUV420P16* with the bpp stored separately is better.
     */
    ...
    AV_PIX_FMT_YUV420P10BE,///< planar YUV 4:2:0, 15bpp, (1 Cr & Cb sample per 2x2 Y samples), big-endian
AV_PIX_FMT_YUV420P10LE,///< planar YUV 4:2:0, 15bpp, (1 Cr & Cb sample per 2x2 Y samples), little-endian
Stored as 16 bit with extra padding means stored as MSB.
This comment doesn't say if padding is in MSB or LSB. You should look at AVComponentDescriptor definition (especially at "shift" field which is 4-th field in struct):
Code:
    /**
     * Number of least significant bits that must be shifted away
     * to get the value.
     */
    int shift;
and than at real descrioption of AV_PIX_FMT_YUV420P10LE/AV_PIX_FMT_YUV420P10BE formats:
Code:
    [AV_PIX_FMT_YUV420P10LE] = {
        .name = "yuv420p10le",
        .nb_components = 3,
        .log2_chroma_w = 1,
        .log2_chroma_h = 1,
        .comp = {
            { 0, 2, 0, 0, 10, 1, 9, 1 },        /* Y */
            { 1, 2, 0, 0, 10, 1, 9, 1 },        /* U */
            { 2, 2, 0, 0, 10, 1, 9, 1 },        /* V */
        },
        .flags = AV_PIX_FMT_FLAG_PLANAR,
    },
    [AV_PIX_FMT_YUV420P10BE] = {
        .name = "yuv420p10be",
        .nb_components = 3,
        .log2_chroma_w = 1,
        .log2_chroma_h = 1,
        .comp = {
            { 0, 2, 0, 0, 10, 1, 9, 1 },        /* Y */
            { 1, 2, 0, 0, 10, 1, 9, 1 },        /* U */
            { 2, 2, 0, 0, 10, 1, 9, 1 },        /* V */
        },
        .flags = AV_PIX_FMT_FLAG_BE | AV_PIX_FMT_FLAG_PLANAR,
},
as you see all "shift" fields are 0 i.e. it means that value is in LSB and zero padding is in MSB.

Last edited by MasterNobody; 20th August 2016 at 10:10.
MasterNobody is offline  
Old 20th August 2016, 14:19   #2316  |  Link
Wilbert
Moderator
 
Join Date: Nov 2001
Location: Netherlands
Posts: 6,364
You are right: plane = 0/1/2, step = 2 (2 bytes for a pixel), offset = 0, shift = 0, depth = 10 (and the last three members are deprecated).

That's confusing. So the layout is not conform the s274m standard where it is stored in the MSB:
Quote:
7.7 Digital R', G', B', Y' components shall
be computed as follows:
L'd = int(219DC'+16D+0.5); D = 2^(n-8)
where L' is the component value in abstract terms
from zero to unity, n takes the value 8 or 10
corresponding to the number of bits to be represented,
and L'd is the resulting digital code.

7.8 Digital C'B and C'R components of the Y'C'BC'R
set shall be computed as follows:
C'd = int(224DC'+128D+0.5); D = 2^(n-8)
where C' is the component value in abstract terms
from -0.5 to +0.5 and C'd is the resulting digital code.

Last edited by Wilbert; 20th August 2016 at 14:26.
Wilbert is offline  
Old 20th August 2016, 14:33   #2317  |  Link
shekh
Registered User
 
Join Date: Mar 2015
Posts: 775
Quote:
Originally Posted by Wilbert View Post
That's confusing. So the layout is not conform the s274m standard where it is stored in the MSB:
Is this manual relevant to image memory layout at all?

"NOTE -- This scaling places the extrema of R′, G′, B′, and
Y′components at codewords 64 and 940 in a ten-bit
representation"
shekh is offline  
Old 20th August 2016, 19:11   #2318  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 1,419
Considering the fact that the FFmpeg patch produces a correct image when the AV_PIX_FMT_* and corresponding AVS_CS_* constants are tethered together, either the formats are not different and there's a breakdown in the description(s) somewhere, or any differences between them are trivial and ultimately do not matter. The conversion routines inside AviSynth+ haven't been completely fleshed out*, but using the external plugins that can resample and scale bitdepth, the content can be reconstructed into a proper image.

*this is what I mean (all seen through opening the script in ffplay):
Version().ConvertToYV12().ConvertTo16bit() = correct image
Version().ConvertToYV12().ConvertTo16bit(bits=10) = hot pink and white incorrect image
Version().ConvertToYV12().f3kdb(output_depth=10,output_mode=2).ConvertFromDoubleWidth(10) = correct image
Version().ConvertToYV12().Dither_convert_8_to_16().Dither_quantize(10,reducerange=true).Dither_out().ConvertFromDoubleWidth(10)>test.avs = correct image

Also consistent with these results, is that using a value in ConvertFromDoubleWidth that does not match what the clip was scaled to in f3kdb or Dither results in an incorrect image.

Last edited by qyot27; 20th August 2016 at 19:14.
qyot27 is offline  
Old 20th August 2016, 19:22   #2319  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 1,419
What's probably happening is that ConvertTo16bit() is just simply outputting the wrong depth/byte size/memory location for anything other than 16 - in other words, that particular filter has a bug/is still WIP and needs to be fixed for 10/12/14. The vi.BitsPerPixel values that the libavformat demuxer relies on are correctly sized for X bit depth and do report the right sizes.
qyot27 is offline  
Old 21st August 2016, 13:00   #2320  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 2,309
Quote:
Originally Posted by qyot27 View Post
What's probably happening is that ConvertTo16bit() is just simply outputting the wrong depth/byte size/memory location for anything other than 16 - in other words, that particular filter has a bug/is still WIP and needs to be fixed for 10/12/14. The vi.BitsPerPixel values that the libavformat demuxer relies on are correctly sized for X bit depth and do report the right sizes.
Work in progress. 10-12-14 bit conversions only change the pixel format flag, there is a yet-not-implemented parameter that converts the range to real 10-12-14 bits. I focused only on implementing the existance of the new formats before I left for holiday two weeks ago. Next week I'm going to continue the work.
pinterf is offline  
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 00:41.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.