Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Closed Thread
 
Thread Tools Search this Thread Display Modes
Old 10th January 2017, 17:12   #2821  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 2,309
Quote:
Originally Posted by shekh View Post
If you promise to port some useful filters for rgb planar mode, I can implement it.
Otherwise just having planar source "supported" but immediately converted to packed - is not useful.
No, so indeed, don't bother.

Quote:
Originally Posted by shekh View Post
Since original VirtualDub was never hosted publicly, I have no idea about last code base.
If you have it, please share.
Can get it here. That's the last version that has been made avaible a loong time ago already (you can get it only in the forum, now unfortunately closed).
Thanks for your work on this upgraded version.

I've tested only the modes avaibles on the Decode format menu, that's why i've tested 16 bits 4:2:0.

Thanks for the links with all VDubMod version.

Last edited by jpsdr; 10th January 2017 at 17:25.
jpsdr is offline  
Old 10th January 2017, 17:19   #2822  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 2,309
New test build
Avisynth plus r2372-dev

20170110 r2372dev (vdubmod14 VfW test)
  • New: SSE2/SSE4 for 10-16 bit <-> 10-16 bit Planar RGB (and Alpha plane) full scale conversions
    (needed for automatic planar RGB -> packed RGB VfW conversions)
  • VfW:
  • Fixed: Y3[10][10],Y3[10][16] fourcc's byte order
  • New: Planar RGB(A) (MagicYUV)
    10,12,14,16 bits: G3[0][10], G4[0][10], G3[0][12], G4[0][12], G3[0][14], G4[0][14], G3[0][16], G4[0][16]
  • New: YUV444P16 to fourcc Y416
  • New: Automatic conversion of 12, 14 and float YUV formats to 16 bit for 4:2:0 and 4:2:2
    Note: OPT_Enable_Y3_10_16 is still valid as if format was originally 16 bits
  • New: Automatic conversion of 10, 12, 14 and float YUV formats to 16 bit for 4:4:4
  • New: Conversion of 10, 12, 14 and float planar RGB formats to RGB64
    when global Avisynth variable Enable_PlanarToPackedRGB is true
  • New: Conversion of 8 bit planar RGB formats to RGB24
    when global Avisynth variable Enable_PlanarToPackedRGB is true
  • New: Conversion of 8 bit planar RGBA formats to RGB32
    when global Avisynth variable Enable_PlanarToPackedRGB is true

    Note: OPT_VDubPlanarHack=true may still be needed for Virtualdub for YUV planar outputs other than YV12 or else U and V planes are exchanged
    Edit: this must affect only 8 bit videos. Fixed in a following release

  • Supported formats:
    BRA[64],b64a,BGR[48],P010,P016,P210,P216,Y3[10][10],Y3[10][16],v210,Y416
    G3[0][10], G4[0][10], G3[0][12], G4[0][12], G3[0][14], G4[0][14], G3[0][16], G4[0][16]
  • Default format FourCCs:
    RGB64: BRA[64]
    RGB48: BGR[48]
    YUV420P10: P010
    YUV420P16: P016
    YUV422P10: P210
    YUV422P16: P216
    YUV444P16: Y416
    Planar RGB 10-16 bit: G3[0][10], G3[0][12], G3[0][14], G3[0][16]
    Planar RGBA 10-16 bit: G4[0][10], G4[0][12], G4[0][14], G4[0][16]
  • Global variables to override default formats:
    OPT_Enable_V210 = true --> v210 for YUV422P10
    OPT_Enable_Y3_10_10 = true --> Y3[10][10] for YUV422P10
    OPT_Enable_Y3_10_16 = true --> Y3[10][16] for YUV422P16
    OPT_Enable_b64a = true --> b64a for RGB64
    Enable_PlanarToPackedRGB = true --> RGBP8->RGB24, RGBAP8->RGB32, all other bit depths to RGB64

Last edited by pinterf; 10th January 2017 at 22:29. Reason: VDubHack
pinterf is offline  
Old 10th January 2017, 17:43   #2823  |  Link
shekh
Registered User
 
Join Date: Mar 2015
Posts: 775
Quote:
Originally Posted by pinterf View Post
> New: YUV444P16 to fourcc Y416

video data is too short (w*h*6 bytes, should be w*h*8 bytes)

> Note: OPT_VDubPlanarHack=true may still be needed for Virtualdub for YUV planar outputs other than YV12 or else U and V planes are exchanged

I dont know, who is guilty? The Y3[10][10] output works with magic, but requires hack with avs+
__________________
VirtualDub2
shekh is offline  
Old 10th January 2017, 17:45   #2824  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 2,309
It's history.
Comment in avisynth.h:
// Hack YV16 and YV24 chroma plane order for old VDub's
pinterf is offline  
Old 10th January 2017, 17:50   #2825  |  Link
shekh
Registered User
 
Join Date: Mar 2015
Posts: 775
Quote:
Originally Posted by pinterf View Post
It's history.
Comment in avisynth.h:
// Hack YV16 and YV24 chroma plane order for old VDub's
Does it mean there is old (bad) VD, and there is new one (which?), where the hack is not needed anymore?
But I have to apply it now, is this expected?
__________________
VirtualDub2
shekh is offline  
Old 10th January 2017, 20:50   #2826  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 2,309
1.) OK, Y416 is now really done but cannot test, vdub says: XYUV64 output is not implemented. Using vdubmod 38494.

2.) (plane hack)
It looks something like this in the avs source

Code:
// Old VDub wants YUV for YV24 and YV16 and YVU for YV12.
  if (parent->VDubPlanarHack && !vi.IsYV12()) {
    plane1 = PLANAR_U;
    plane2 = PLANAR_V;
  }
  else {
    // Set default VFW output plane order.
    plane1 = PLANAR_V;
    plane2 = PLANAR_U;
  }
pinterf is offline  
Old 10th January 2017, 21:39   #2827  |  Link
shekh
Registered User
 
Join Date: Mar 2015
Posts: 775
"XYUV64 output is not implemented" is message from output (dubbing) part, why you see it?
Has nothing to do with raw/avs input.

I think there is misunderstanding, Y3[10][16] is not 16-bit YV16, it has YUV plane order while YV16 has YVU plane order.

Quote:
ConvertToYV16()
works (YV16)

Quote:
OPT_Enable_Y3_10_16 = true
ConvertToYV16().ConvertBits(16)
swapped

Quote:
OPT_VDubPlanarHack=true
OPT_Enable_Y3_10_16 = true
ConvertToYV16().ConvertBits(16)
correct

There is no reason for Y3[10][16] to be affected by VDubPlanarHack since it is brand new fourcc.
__________________
VirtualDub2
shekh is offline  
Old 10th January 2017, 21:50   #2828  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 2,309
Quote:
Originally Posted by shekh View Post
"XYUV64 output is not implemented" is message from output (dubbing) part, why you see it?
Has nothing to do with raw/avs input.
I just had a look at the log window after clicking on play.
pinterf is offline  
Old 10th January 2017, 21:56   #2829  |  Link
shekh
Registered User
 
Join Date: Mar 2015
Posts: 775
Quote:
Originally Posted by pinterf View Post
I just had a look at the log window after clicking on play.
Thanks, I too often forget about play button.
__________________
VirtualDub2
shekh is offline  
Old 10th January 2017, 22:04   #2830  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 2,309
Quote:
Originally Posted by shekh View Post
I think there is misunderstanding, Y3[10][16] is not 16-bit YV16, it has YUV plane order while YV16 has YVU plane order.

There is no reason for Y3[10][16] to be affected by VDubPlanarHack since it is brand new fourcc.
O.K. I will then use (vi.IsYV16() || vi.IsYV24())
instead of !vi.IsYV12()
In the 8 bit world the latter was enough.
What about YV411? (Y41B)
pinterf is offline  
Old 10th January 2017, 23:42   #2831  |  Link
ajp_anton
Registered User
 
ajp_anton's Avatar
 
Join Date: Aug 2006
Location: Stockholm/Helsinki
Posts: 805
Question: why are there two versions of RGB, planar and packed? Does the user really have to know/care which one is used?
ajp_anton is offline  
Old 11th January 2017, 01:07   #2832  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 2,309
Something came to my mind, i've checked in avisynth.h and didn't see any information about it.
Standard/normal YUV mode is 16..235 for Y and 16..240 for UV. You can have a "full range" mode, but it's "unusual" or not the standard.
VDub has in its color mode a difference for the YUV mode between full range or limited range. I didn't see anything like this in avisynth. Have i missed it (in that case where is it), or is it just that there is not a such thing ?
If there is not something to chose between full or limited range, can we assume that the behavior in this case is the standard/common behavior, meaning the limited range ?
What are the specifications for the extended bit range ? Are they also limited, or are they just only full range by default ?
If limited, how is the range ? Is it proportional (16-240 becomes 4096-61440 in 16 bits) or fixed (16-240 becomes 16-65520 in 16 bits) ?

That make me realise that since the begining, at least the resample and nnedi3 filters are prone to produce incorrect results, as they clip only to 0-255, meaning they can produce out of range values in case of YUV data (but not in case of RGB data, of course).

So, what is exactly the situation about these points ?
jpsdr is offline  
Old 11th January 2017, 01:54   #2833  |  Link
TheFluff
Excessively jovial fellow
 
Join Date: Jun 2004
Location: rude
Posts: 1,100
Avisynth has functions to convert between full and limited range (just pointing that out, because it does seem like you might have somehow missed this) but does not have metadata flagging to keep track of which range a given clip has.

YUV limited range ("TV range") numerical limits for bitdepths above 8 are quite well-defined (see f.ex. the H.264 standard). For any bitdepth, the luma offset is
Code:
16 * (2^(bitdepth - 8))
while the range is
Code:
219 * (2^(bitdepth - 8))
For chroma, the offset is the same, but substitute 224 for 219 to get the range.

Full range YUV on the other hand isn't well defined at all. It's been discussed extensively in this thread before, see here and a number of posts forward.

Last edited by TheFluff; 11th January 2017 at 02:09.
TheFluff is offline  
Old 11th January 2017, 09:42   #2834  |  Link
jpsdr
Registered User
 
Join Date: Oct 2002
Location: France
Posts: 2,309
Quote:
Originally Posted by TheFluff View Post
but does not have metadata flagging to keep track of which range a given clip has.
Argh.... That is very troublesome, it's a shame something like this have been missed when it should have been present since the begining...
Thanks for the others informations.

Last edited by jpsdr; 11th January 2017 at 11:32.
jpsdr is offline  
Old 11th January 2017, 13:35   #2835  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 2,309
Quote:
Originally Posted by tuanden0 View Post
Can someone help me the filter fft3dGPU crashed if I use SetFilterMTMode("FFT3dGPU", 3)
Here is my script:
Reproduced, but cannot help with it.

I tried with this simplified script:
Code:
Colorbars(pixel_type="YV12")
SetFilterMTMode("FFT3dGPU", 3)
FFT3DGPU(sigma=4,bt=3)
Prefetch(6)
Debug log shows, that sometimes this filter requests negative frames.
This was already mentioned, related link:
https://forum.doom9.org/showthread.p...25#post1671525

Debug log with Prefetch(6) until the sad finish, which occured in fft3dgpu.dll when frame number 7 was requested from it.
Code:
ColorBars::GetFrame 0
ColorBars::GetFrame 3
ColorBars::GetFrame 2
ColorBars::GetFrame 1
ColorBars::GetFrame 1
ColorBars::GetFrame -1
ColorBars::GetFrame 5
ColorBars::GetFrame 4
ColorBars::GetFrame 4
ColorBars::GetFrame 4
ColorBars::GetFrame 2
ColorBars::GetFrame 1
ColorBars::GetFrame 0
ColorBars::GetFrame 6
ColorBars::GetFrame 4
ColorBars::GetFrame 6
ColorBars::GetFrame 5
ColorBars::GetFrame 7
ColorBars::GetFrame 1
ColorBars::GetFrame 0
ColorBars::GetFrame 7
ColorBars::GetFrame 8
ColorBars::GetFrame 3
ColorBars::GetFrame 7
Cache::GetFrame LRU_LOOKUP_NO_CACHE: [FFT3DGPU] n=     2 child=007302F8 frame=0A4BF9E8 vfb=0900AEF8 videoCacheSize=0 SeekTime            :0.068393
Cache::GetFrame LRU_LOOKUP_NO_CACHE: [FFT3DGPU] n=     1 child=007302F8 frame=0A4BF8B0 vfb=006B6D20 videoCacheSize=0 SeekTime            :0.069542
ColorBars::GetFrame 2
ColorBars::GetFrame 6
ColorBars::GetFrame 13
ColorBars::GetFrame 1
Cache::GetFrame LRU_LOOKUP_NO_CACHE: [FFT3DGPU] n=     6 child=007302F8 frame=0A4BFAB8 vfb=0900AEB8 videoCacheSize=0 SeekTime            :0.075657
ColorBars::GetFrame 8
Cache::GetFrame LRU_LOOKUP_NO_CACHE: [FFT3DGPU] n=     0 child=007302F8 frame=0A4BFA50 vfb=0900C178 videoCacheSize=0 SeekTime            :0.077987
Cache::GetFrame LRU_LOOKUP_NO_CACHE: [FFT3DGPU] n=     4 child=007302F8 frame=0A4BF918 vfb=0900A4F8 videoCacheSize=0 SeekTime            :0.078408
Cache::GetFrame LRU_LOOKUP_NO_CACHE: [FFT3DGPU] n=     3 child=007302F8 frame=0A4BF980 vfb=0900C1B8 videoCacheSize=0 SeekTime            :0.078792
Cache::GetFrame LRU_LOOKUP_NO_CACHE: [FFT3DGPU] n=     5 child=007302F8 frame=0A4BFB20 vfb=0900ADF8 videoCacheSize=0 SeekTime            :0.078573
ColorBars::GetFrame 8
Exception thrown at 0x02DF1FDA (FFT3dGPU.dll) in AVSMeter.exe: 0xC0000005: Access violation reading location 0x00000000.
Although this filter is set for serialized mode in MT, the frame requests are not sequential in a multithreaded environment. Requested frame numbers can jump randomly back and forward. Maybe this situation is not handled too well in the code, but seeing no source for it I cannot make any further assumption.

This post got a bit long, but interesting to see what happens in the background:

Without Prefetch (non-MT):
Code:
ColorBars::GetFrame -1
ColorBars::GetFrame 0
ColorBars::GetFrame 1
ColorBars::GetFrame 0
ColorBars::GetFrame 2
Cache::GetFrame LRU_LOOKUP_NO_CACHE: [FFT3DGPU] n=     0 child=090A34B0 frame=005CF580 vfb=08E3F838 videoCacheSize=0 SeekTime            :0.025495
ScriptEnvironment::GetNewFrame NEW METHOD EXACT hit! VideoFrameListSize=      1 GotSize= 460831 FrReg.Size=     1 vfb=08E3F838 frame=005CF580 SeekTime:0.000010
                                          frame 005CF580 RowSize=640 Height=480 Pitch=640 Offset=24
ScriptEnvironment::GetNewFrame returning frame. VideoFrameListSize was 1
ColorBars::GetFrame 1
ColorBars::GetFrame 3
Cache::GetFrame LRU_LOOKUP_NO_CACHE: [FFT3DGPU] n=     1 child=090A34B0 frame=005CF580 vfb=08E3F838 videoCacheSize=0 SeekTime            :0.006794
ScriptEnvironment::GetNewFrame NEW METHOD EXACT hit! VideoFrameListSize=      1 GotSize= 460831 FrReg.Size=     1 vfb=08E3F838 frame=005CF580 SeekTime:0.000006
                                          frame 005CF580 RowSize=640 Height=480 Pitch=640 Offset=24
ScriptEnvironment::GetNewFrame returning frame. VideoFrameListSize was 1
ColorBars::GetFrame 2
ColorBars::GetFrame 4
Cache::GetFrame LRU_LOOKUP_NO_CACHE: [FFT3DGPU] n=     2 child=090A34B0 frame=005CF580 vfb=08E3F838 videoCacheSize=0 SeekTime            :0.006903
ScriptEnvironment::GetNewFrame NEW METHOD EXACT hit! VideoFrameListSize=      1 GotSize= 460831 FrReg.Size=     1 vfb=08E3F838 frame=005CF580 SeekTime:0.000016
                                          frame 005CF580 RowSize=640 Height=480 Pitch=640 Offset=24
ScriptEnvironment::GetNewFrame returning frame. VideoFrameListSize was 1
ColorBars::GetFrame 3
ColorBars::GetFrame 5
Cache::GetFrame LRU_LOOKUP_NO_CACHE: [FFT3DGPU] n=     3 child=090A34B0 frame=005CF580 vfb=08E3F838 videoCacheSize=0 SeekTime            :0.006695
ScriptEnvironment::GetNewFrame NEW METHOD EXACT hit! VideoFrameListSize=      1 GotSize= 460831 FrReg.Size=     1 vfb=08E3F838 frame=005CF580 SeekTime:0.000005
                                          frame 005CF580 RowSize=640 Height=480 Pitch=640 Offset=24
ScriptEnvironment::GetNewFrame returning frame. VideoFrameListSize was 1
ColorBars::GetFrame 4
ColorBars::GetFrame 6
Cache::GetFrame LRU_LOOKUP_NO_CACHE: [FFT3DGPU] n=     4 child=090A34B0 frame=005CF580 vfb=08E3F838 videoCacheSize=0 SeekTime            :0.005836
ScriptEnvironment::GetNewFrame NEW METHOD EXACT hit! VideoFrameListSize=      1 GotSize= 460831 FrReg.Size=     1 vfb=08E3F838 frame=005CF580 SeekTime:0.000005
                                          frame 005CF580 RowSize=640 Height=480 Pitch=640 Offset=24
ScriptEnvironment::GetNewFrame returning frame. VideoFrameListSize was 1
ColorBars::GetFrame 5
ColorBars::GetFrame 7
Cache::GetFrame LRU_LOOKUP_NO_CACHE: [FFT3DGPU] n=     5 child=090A34B0 frame=005CF580 vfb=08E3F838 videoCacheSize=0 SeekTime            :0.006020
ScriptEnvironment::GetNewFrame NEW METHOD EXACT hit! VideoFrameListSize=      1 GotSize= 460831 FrReg.Size=     1 vfb=08E3F838 frame=005CF580 SeekTime:0.000005
                                          frame 005CF580 RowSize=640 Height=480 Pitch=640 Offset=24
ScriptEnvironment::GetNewFrame returning frame. VideoFrameListSize was 1
ColorBars::GetFrame 6
ColorBars::GetFrame 8
pinterf is offline  
Old 11th January 2017, 15:12   #2836  |  Link
pinterf
Registered User
 
Join Date: Jan 2014
Posts: 2,309
New dev build
Avisynth Plus r2380-dev

20170111 r2380
  • Overlay: Modes "Blend", "Luma" and "Chroma" now support all bit depths and 444 conversionless (use444=false) mode
  • Overlay: fix SSE2 Blend for mask+opacity for 10-16 bits
  • VfW: (vdubmod14 VfW test)
    Fix: YUV444P16 or YUVA444P16 to fourcc Y416
    if alpha channel is present, it will we copied, else filled with FFFF
    Fix: VDubPlanarHack is checked only for 8 bit YUV planar sources

Now practically all color formats can be shown in vdubmod. If a specific bitdepth does not exist, it will be converted on-the-fly, and some alternative output formats can be hinted with OPT_xxx variables.
For details see readme.

(I'd like to separate github releases from these 'daily' builds. Albeit this version is quite usable, there were huge refactorizations and earthquake e.g. in Overlay, so I'd like to wait a bit more)
pinterf is offline  
Old 11th January 2017, 18:50   #2837  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,753
Quote:
Originally Posted by ajp_anton View Post
Question: why are there two versions of RGB, planar and packed? Does the user really have to know/care which one is used?
The "user" may not care. But a programmer has to. Planar formats have an important advantage to process video filters fast, but it is completely different to the way the video memory stores pixels displayed on screen.

Planar RGB (components can be kept in separate memory blocks, the same pixel will have the same offset in all components, needs to be calculated only once per pixel):
Code:
RRRR...RRRR

GGGG...GGGG

BBBB...BBBB
Packed RGB24 (please notice that memory addresses aligned to 32-bit structures = DWORD are addressed most efficiently in 32-bit systems):
Code:
(R,G,B)(R
G,B)(R,G
B)(R,G,B)
...
Packed RGB32 (to align pixels with 32-bit addresses, a fourth byte is inserted which may contain an alpha value – or be ignored, which wastes RAM):
Code:
(R,G,B,A)
(R,G,B,A)
(R,G,B,A)
...
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid

Last edited by LigH; 11th January 2017 at 19:01.
LigH is offline  
Old 11th January 2017, 19:13   #2838  |  Link
TheFluff
Excessively jovial fellow
 
Join Date: Jun 2004
Location: rude
Posts: 1,100
VS only supports packed RGB for backwards compatibility with Avisynth and for output. Internally in a filtering framework there is no reason whatsoever to ever use packed. Just unpack on input and pack on output if you have to - my gut feeling says that for any but the most trivial filters that's going to be insignificantly slower than filtering packed (and you can drop a dumb extra code path too). There are most likely no particularly useful or interesting RGB only filters anyway, so breaking things shouldn't be a big deal.
TheFluff is offline  
Old 11th January 2017, 22:45   #2839  |  Link
ajp_anton
Registered User
 
ajp_anton's Avatar
 
Join Date: Aug 2006
Location: Stockholm/Helsinki
Posts: 805
But what is the point then in using packed in Avisynth, which is about processing video, not displaying it? If the input serves packed, then immediately "unpack" it. And pack it again for the output if needed.
Is there any reason for the user to ever convert to packed RGB? And if there are rare cases, why not just have one type of planar "ConvertToRGB24" (/32/48/64) function, and require "packed=true" for packed? It's easier to remember from previous experience that RGB48 is 16-bit per component RGB without alpha, than remembering whatever the name of the planar versions are (tried to google them for this post, but didn't find them fast enough). Having everything the same type (planar?) makes things easier for the user.

Last edited by ajp_anton; 11th January 2017 at 22:48.
ajp_anton is offline  
Old 11th January 2017, 23:09   #2840  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,753
Many VfW compatible tools don't support planar RGB. They can't handle it. It was not supported by Microsoft in a blank installation of most Windows versions (I am not even sure if there are any supporting VfW codecs as of today). I would even believe, if the semi-pro home users did not insist in it, hardly any professional software company would have cared about it... So, for ancient compatibility's sake, AviSynth still must be able to output packed RGB. The most compatible way of displaying images in Windows, DIB = "Device Independent Bitmaps", was invented with packed-pixel RGB and palette formats only, AFAIK. Planar RGB is a rather recent invention, I believe...

YUV based formats are useful for real-life content, whereas RGB based formats are common for computer generated content. Just imagine screen recordings of a game or an application's user interface. Recording that in a YUV based format, especially with the most common 4:2:0 chroma subsampling and TV range, would limit the color saturation and chrominance resolution, blurring and fading 1-pixel wide colored lines.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline  
Closed Thread

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 21:06.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.