Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > VapourSynth

Reply
 
Thread Tools Search this Thread Display Modes
Old 3rd February 2021, 19:57   #4261  |  Link
Jukus
Registered User
 
Join Date: Jul 2019
Location: Russia
Posts: 65
How should such a video be indexed?
Code:
Format                                   : AVI
Format/Info                              : Audio Video Interleave
Commercial name                          : DV
Format profile                           : OpenDML
Overall bit rate mode                    : Constant
Overall bit rate                         : 30 Mb/s

Video
ID                                       : 0
Format                                   : DV
Codec ID                                 : dvsd
Codec ID/Hint                            : Sony
Bit rate mode                            : Constant
Width                                    : 720 pixels
Height                                   : 480 pixels
Display aspect ratio                     : 16:9
Frame rate mode                          : Constant
Frame rate                               : 29.970 (30000/1001) FPS
Original frame rate                      : 29.970 (29970/1000) FPS
Standard                                 : NTSC
Color space                              : YUV
Chroma subsampling                       : 4:1:1
Bit depth                                : 8 bits
Scan type                                : Interlaced
Scan order                               : Bottom Field First
Compression mode                         : Lossy
Jukus is offline   Reply With Quote
Old 3rd February 2021, 20:04   #4262  |  Link
stax76
Registered User
 
stax76's Avatar
 
Join Date: Jun 2002
Location: On thin ice
Posts: 6,706
You can open that with ffms2 or L-Smash-Source.
stax76 is offline   Reply With Quote
Old 3rd February 2021, 21:39   #4263  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,383
Disadvantage: Because it is a "keyframe only" video format, the index will be huge and indexing will take a lot of time.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 3rd February 2021, 21:47   #4264  |  Link
poisondeathray
Registered User
 
Join Date: Sep 2007
Posts: 4,551
If you're on windows, you can open it up with core.avisource.AVISource without indexing. Disadvantage - requires system installed DV codec and none that I know of return the original NTSC 4:1:1 .
poisondeathray is offline   Reply With Quote
Old 4th February 2021, 13:40   #4265  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Providence, RI
Posts: 2,525
does anyone actually need half precision (fp16) facilities and does any plugin actually provide support for it? I don't really wanna have it in my project and I have very good reasons:

1) there's no native support for fp16 in standard C/C++ and most CPUs provide very little or no support for it, meaning you will have to create your homegrown software-emulated fp16 type. it's not portable and it's very likely that it will be way slower than fp32 which comes with native hardware support.

2) unlike fp32 and fp64 which are universally well defined (literally every implementation out there adopts the representation defined by IEEE 754), the representation of fp16 varies on different platforms. there's the IEEE fp16, and BFloat16 (Google TPU), and who knows what else! applying arithmetic operations defined for one fp16 representation on another fp16 representation leads to garbage result. The user (or maybe even the developer) usually has absolutely no clue which fp16 representation is being used and all sorts of weird shit can happen. It might be better to avoid all this by simply not supporting fp16.
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 4th February 2021, 13:44   #4266  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 2,289
Quote:
Originally Posted by feisty2 View Post
does anyone actually need half precision (fp16) facilities and does any plugin actually provide support for it? I don't really wanna have it in my project and I have very good reasons:

1) there's no native support for fp16 in standard C/C++ and most CPUs provide very little or no support for it, meaning you will have to create your homegrown software-emulated fp16 type. it's not portable and it's very likely that it will be way slower than fp32 which comes with native hardware support.

2) unlike fp32 and fp64 which are universally well defined (literally every implementation out there adopts the representation defined by IEEE 754), the representation of fp16 varies on different platforms. there's the IEEE fp16, and BFloat16 (Google TPU), and who knows what else! applying arithmetic operations defined for one fp16 representation on another fp16 representation leads to garbage result. The user (or maybe even the developer) usually has absolutely no clue which fp16 representation is being used and all sorts of weird shit can happen. It might be better to avoid all this by simply not supporting fp16.
1. That's not how you do it. You use the unpack/pack instructions for fp16 for load and store and then feed that to the normal fp32 path. It's the only way to make it perform well.

2. Obviously it's whatever the cpu native fp16 format is, just like with other floats. So IEEE all the way on x86 (and just about all sane cpus).

Support is mostly pointless.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 4th February 2021, 17:46   #4267  |  Link
Jukus
Registered User
 
Join Date: Jul 2019
Location: Russia
Posts: 65
Quote:
Originally Posted by LigH View Post
Disadvantage: Because it is a "keyframe only" video format, the index will be huge and indexing will take a lot of time.
The file weighs over 30 gb
Duration over 02:30:00
The size of the ffms index is 50+ kb
Jukus is offline   Reply With Quote
Old 5th February 2021, 11:16   #4268  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,435
What function do I use to retrieve warning message from the api? If I run a script with clip = core.get_core() in PyCharm it'll give me a nice deprecated warning, but in vseditor I couldn't find a function that can do that. the getError() only return on fatal error like frame failed to load.
lansing is offline   Reply With Quote
Old 5th February 2021, 11:26   #4269  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Providence, RI
Posts: 2,525
https://github.com/vapoursynth/vapou...ynth.pyx#L2337
it has nothing to do with VS per se. you just have to find a way to utilize python's warnings module in your application.
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 5th February 2021, 23:12   #4270  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,383
@Jukus: OK, maybe it's worse with uncompressed audio. My memories may be not completely correct in this case.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 6th February 2021, 01:36   #4271  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,435
Quote:
Originally Posted by feisty2 View Post
https://github.com/vapoursynth/vapou...ynth.pyx#L2337
it has nothing to do with VS per se. you just have to find a way to utilize python's warnings module in your application.
I don't really know what to do. Maybe like embedding Python into my vseditor project and then have it evaluate the script so I can catch the warning?
lansing is offline   Reply With Quote
Old 6th February 2021, 08:20   #4272  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Providence, RI
Posts: 2,525
you can use vaporsynth's FFI facilities to execute arbitrary python code in your C++ programs. you need to:
1) define a python function that does what you want, in this case, the function should collect all warning messages from warnings and return them as a list of strings.
2) pass the function to a special filter and execute it, in this case, you can instantly execute the acquired function in your filter's constructor, then call Core.Alert() to send all warning messages to console.
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 6th February 2021, 17:18   #4273  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,435
Quote:
Originally Posted by feisty2 View Post
you can use vaporsynth's FFI facilities to execute arbitrary python code in your C++ programs. you need to:
1) define a python function that does what you want, in this case, the function should collect all warning messages from warnings and return them as a list of strings.
2) pass the function to a special filter and execute it, in this case, you can instantly execute the acquired function in your filter's constructor, then call Core.Alert() to send all warning messages to console.
What is vaporsynth's FFI facilities?
lansing is offline   Reply With Quote
Old 6th February 2021, 17:40   #4274  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Providence, RI
Posts: 2,525
Quote:
Originally Posted by lansing View Post
What is vaporsynth's FFI facilities?
https://github.com/IFeelBloated/vsFi...e/Function.vxx

and use it to run python code thru a special filter
Code:
struct PythonEval {
	static constexpr auto Name = "PythonEval";
	static constexpr auto Signature = "code:func;";

	PythonEval(auto Arguments, auto Core) {
		for (auto PythonCode = static_cast<Function>(Arguments["code"]); auto&& WarningMessage : PythonCode())
			Core.Alert(static_cast<std::string>(WarningMessage));
	}
};
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 7th February 2021, 07:51   #4275  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,435
This is what I found to catch the warnings:

Code:
import warnings

with warnings.catch_warnings(record=True) as warning_list:
    core = vs.get_core()
    clip = core.dgdecodenv.DGSource(r'hello.dgi')
    clip.set_output()

for warning in warning_list:
    print(warning.message)
But I need to wrap the whole script inside this warnings.catch_warnings in order to catch them.
lansing is offline   Reply With Quote
Old 7th February 2021, 08:32   #4276  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Providence, RI
Posts: 2,525
Quote:
Originally Posted by lansing View Post

But I need to wrap the whole script inside this warnings.catch_warnings in order to catch them.
no need.

Code:
import logging

class MessageAbsorber(logging.Handler):
    def __init__(self, MessageContainer):
        logging.Handler.__init__(self)
        self.MessageContainer = MessageContainer
    def emit(self, MessageRecord):
        self.MessageContainer += [MessageRecord.getMessage()]

WarningMessages = []
logging.captureWarnings(True)
logging.getLogger('py.warnings').addHandler(MessageAbsorber(WarningMessages))

def PassWarningMessagesToCxxPrograms():
    return WarningMessages

# user script goes here

core.vsedit.PythonEval(PassWarningMessagesToCxxPrograms)
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 8th February 2021, 06:34   #4277  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,435
Quote:
Originally Posted by feisty2 View Post
no need.

Code:
import logging

class MessageAbsorber(logging.Handler):
    def __init__(self, MessageContainer):
        logging.Handler.__init__(self)
        self.MessageContainer = MessageContainer
    def emit(self, MessageRecord):
        self.MessageContainer += [MessageRecord.getMessage()]

WarningMessages = []
logging.captureWarnings(True)
logging.getLogger('py.warnings').addHandler(MessageAbsorber(WarningMessages))

def PassWarningMessagesToCxxPrograms():
    return WarningMessages

# user script goes here

core.vsedit.PythonEval(PassWarningMessagesToCxxPrograms)
I came to understand the logging part after some reading, but I still don't get the last line, what is this PythonEval() and how does it talk to C++?

The script will be pass to vsscript_evaluateScript(), which doesn't return warning messages. Where do this PythonEval() play in the process?
lansing is offline   Reply With Quote
Old 8th February 2021, 07:29   #4278  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Providence, RI
Posts: 2,525
PythonEval() is a special C++ filter that takes a python function as its argument and returns nothing. it's somewhat similar to std.LoadPlugin() or std.SetMaxCPU() in terms of functionality (an infrastructure function rather than an actual filter that applies some processing to a video clip). its implementation should be something similar to #4274, basically it retrieves the warning messages by calling the foreign python function and sends all retrieved messages to stderr thru vsapi->logMessage()
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Old 8th February 2021, 09:49   #4279  |  Link
lansing
Registered User
 
Join Date: Sep 2006
Posts: 1,435
Quote:
Originally Posted by feisty2 View Post
PythonEval() is a special C++ filter that takes a python function as its argument and returns nothing. it's somewhat similar to std.LoadPlugin() or std.SetMaxCPU() in terms of functionality (an infrastructure function rather than an actual filter that applies some processing to a video clip). its implementation should be something similar to #4274, basically it retrieves the warning messages by calling the foreign python function and sends all retrieved messages to stderr thru vsapi->logMessage()
If I get this right, you're saying to create a dummy vs filter to sends the collected messages through stdder, and then in the C++ side, retrieve the messages through stdder and then output them with vsapi->logMessage()?
lansing is offline   Reply With Quote
Old 8th February 2021, 10:02   #4280  |  Link
feisty2
I'm Siri
 
feisty2's Avatar
 
Join Date: Oct 2012
Location: Providence, RI
Posts: 2,525
no, it sends the collected messages TO stderr BY USING vsapi->logMessage(), read the documentation of vaporsynth's C API. any string passed to logMessage() already automatically appears in vsedit, you don't need any extra stuff.
__________________
If I got new ideas, will post here: https://github.com/IFeelBloated
feisty2 is offline   Reply With Quote
Reply

Tags
speed, vaporware, vapoursynth

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 18:09.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, vBulletin Solutions Inc.