Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > Avisynth Development

Reply
 
Thread Tools Search this Thread Display Modes
Old 19th May 2015, 08:59   #2041  |  Link
speedyrazor
Registered User
 
Join Date: Mar 2003
Posts: 194
I would love to use this, but I open very large HD Quicktime movies (120GB) and indexing kills, taking way too long to even start working on the file. Is there a way to disable the indexing and cut the the chase?
speedyrazor is offline   Reply With Quote
Old 19th May 2015, 10:14   #2042  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 2,079
Quote:
Originally Posted by speedyrazor View Post
I would love to use this, but I open very large HD Quicktime movies (120GB) and indexing kills, taking way too long to even start working on the file. Is there a way to disable the indexing and cut the the chase?
Sure, just use something else to open files instead.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 19th May 2015, 10:23   #2043  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,004
Indexing will be necessary to provide random access in an AviSynth script, using closest previous GOP starts before a seek point. This is quite important if you consider multithreading, where each thread may request a different frame number, and sequential access will be improbable.

FFMS2 will not rely on the possible existence of a keyframe index chunk because not all supported media formats will contain one, so it prefers to create an own index, but only once, it will be reused each time you access the same source file. You may disable writing the cache file to disk, but FFMS2 will still need to gather positions of decoding starts, so if you don't let it write a cache file, it will need to build an index in RAM each time you process the script.

If you want to rely on an index already present in a media file, rather use a different source filter which does so (in this case, maybe QTSource, which will require QuickTime to be installed).

Furthermore, QuickTime movies should belong to the "ISO Media" container family, so the LSMASHVideoSource() filter of L-SMASH Works may be recommendable as another alternative; it will usually write an own index cache file as well, though, but might do that faster.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 19th May 2015, 11:17   #2044  |  Link
RTW47
Rome Total War
 
RTW47's Avatar
 
Join Date: Mar 2013
Location: C:\Python33
Posts: 39
Quote:
Originally Posted by LigH View Post
LSMASHVideoSource() filter of L-SMASH Works may be recommendable as another alternative; it will usually write an own index cache file as well, though, but might do that faster.
For lsmashvideosource() will the index always be stored inside ram, just like ffms2 with cache=false?
__________________
GOTO:EOF

Last edited by RTW47; 19th May 2015 at 11:35.
RTW47 is offline   Reply With Quote
Old 19th May 2015, 11:34   #2045  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,004
It may surprise you ... but L-SMASH Works contains a README file.

According to this one, there seems to be no cache file for the functions using the L-SMASH demultiplexer:

Code:
            LSMASHVideoSource(string source, int track = 0, int threads = 0, int seek_mode = 0, int seek_threshold = 10,
                              bool dr = false, int fpsnum = 0, int fpsden = 1, bool stacked = false, string format = "")
In contrast to the functions using libavformat as demultiplexer:

Code:
            LWLibavVideoSource(string source, int stream_index = -1, int threads = 0, bool cache = true,
                               int seek_mode = 0, int seek_threshold = 10, bool dr = false,
                               int fpsnum = 0, int fpsden = 1,
                               bool repeat = false, int dominance = 0, bool stacked = false, string format = "")
So it seems quite probable that LSMASHVideoSource() relies on media file internal index chunks. It will support only a few specific media formats, though: MP4, MOV, 3GPP
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 19th May 2015, 12:20   #2046  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,452
It would be kind of nice if ffms could index on-demand for those huge files -- index up to the requested frame as frames are requested -- or in a background thread, but that means getting a developer interested, and so far no one seems to be. It would sure save time in those massive source situations.
__________________
There are four boxes to be used in defense of liberty: soap, ballot, jury, and ammo. Please use in that order.
foxyshadis is offline   Reply With Quote
Old 19th May 2015, 12:25   #2047  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 2,079
Quote:
Originally Posted by foxyshadis View Post
It would be kind of nice if ffms could index on-demand for those huge files -- index up to the requested frame as frames are requested -- or in a background thread, but that means getting a developer interested, and so far no one seems to be. It would sure save time in those massive source situations.
There's a huge problem with that idea. You don't know the framecount until it's too late to tell avisynth about it. That's why I never implemented it.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 20th May 2015, 17:55   #2048  |  Link
mawen1250
Registered User
 
Join Date: Aug 2011
Posts: 104
I've got wrong color when reading RGB24 image files (tested png and bmp) with ffms2.Source on VapourSynth.
Looks like the channels are swapped: BGR->GRB
The AviSynth's plugin works fine.

Last edited by mawen1250; 20th May 2015 at 18:02.
mawen1250 is offline   Reply With Quote
Old 20th May 2015, 18:12   #2049  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 2,079
Quote:
Originally Posted by mawen1250 View Post
I've got wrong color when reading RGB24 image files (tested png and bmp) with ffms2.Source on VapourSynth.
Looks like the channels are swapped: BGR->GRB
The AviSynth's plugin works fine.
Doh, I forgot abour that detail. Use shuffleplanes to work around it until a fixed version is available.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 20th May 2015, 18:12   #2050  |  Link
RTW47
Rome Total War
 
RTW47's Avatar
 
Join Date: Mar 2013
Location: C:\Python33
Posts: 39
you can use ShufflePlanes() to fix output
__________________
GOTO:EOF

Last edited by RTW47; 20th May 2015 at 18:40. Reason: too slow!.. ><;
RTW47 is offline   Reply With Quote
Old 20th May 2015, 23:11   #2051  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 2,079
Now it's fixed in git. I guess there'll be another release in a week or two.
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 22nd May 2015, 19:30   #2052  |  Link
Myrsloik
Professional Code Monkey
 
Myrsloik's Avatar
 
Join Date: Jun 2003
Location: Ikea Chair
Posts: 2,079
I'm back with what I guess is kinda 2.22 rc1.

So what needs to be tested is that the functions FFmpegSource2, FFImageSource and FFCopyrightInfringement still work as expected. They're now integrated into the plugin for convenience reasons (normal users will no logner need to fidget with ffms2.avsi). Don't forget to replace ffms2.avsi or you may get unexpected results.

If someone has the patience to test all the arguments for FFmpegSource2 and FFImageSource that'd be great. I may have made a typo somewhere.

- 2.22
- avisynth: Make FFMS2 a shorter alias for FFmpegSource2 to save some typing (Myrsloik)
- avisynth: Moved all source functions in ffms2.avsi into the actual plugin (Myrsloik)
- avisynth: Use 2.6 RC API and add output support for all new colorspaces (Myrsloik)
- vapoursource: Fix swapped RGB channels bug introduced in 2.21 (Myrsloik)
__________________
VapourSynth - proving that scripting languages and video processing isn't dead yet
Myrsloik is offline   Reply With Quote
Old 23rd May 2015, 10:07   #2053  |  Link
stax76
Registered User
 
stax76's Avatar
 
Join Date: Jun 2002
Location: 36 Chambers
Posts: 5,928
Thanks for the update Myrsloik, I tested various things with 3 files, with one sample there is a async problem, it is a DVB capture muxed to mkv with dsmux, some info in case you have time and interest:

http://www.mediafire.com/watch/y4biq...g3vm/async.mkv

source filter which name I forgot: freeze, FAILURE
FFVideoSource: async
LWLibavVideoSource: OK
DSS: async

I posted the original TS file lately to the AviSynth+ thread:

http://forum.doom9.org/showthread.ph...64#post1723364

here is some technical info:

http://forum.doom9.org/showthread.ph...82#post1723382

It was explained there are different ways to handle things, a possible solution to achieve compatibility between tools and workflows might be agreeing to the same approach.
stax76 is offline   Reply With Quote
Old 23rd May 2015, 13:12   #2054  |  Link
videoh
Registered User
 
Join Date: Jul 2014
Posts: 1,104
Quote:
Originally Posted by stax76 View Post
FFVideoSource: async
LWLibavVideoSource: OK
Have you, or can you, compare the video output of these two frame-by-frame to determine if FFVideoSource is including or excluding extra frames at the start compared to LWLibavSource? And is the async constant or does it grow as the stream plays?
videoh is offline   Reply With Quote
Old 23rd May 2015, 20:52   #2055  |  Link
manolito
Registered User
 
manolito's Avatar
 
Join Date: Sep 2003
Location: Berlin, Germany
Posts: 2,724
@stax76

FWIW I downloaded your async.mkv (it was packed into an FLV container), I also repacked your original .TS file into an MKV container using MKVMerge, and for all these source files the result was consistent:

Use DSS2Mod together with LAVFilters, ignore audio delays reported by MediaInfo if they are > 100ms, and get a result without any audio sync issues.

I tested it using AVStoDVD for MPEG2 output and also the old StaxRip version 1.1.9.0 for XviD and AVC output.


Cheers
manolito
manolito is offline   Reply With Quote
Old 23rd May 2015, 21:12   #2056  |  Link
stax76
Registered User
 
stax76's Avatar
 
Join Date: Jun 2002
Location: 36 Chambers
Posts: 5,928
Quote:
Originally Posted by manolito View Post
@stax76

FWIW I downloaded your async.mkv (it was packed into an FLV container), I also repacked your original .TS file into an MKV container using MKVMerge, and for all these source files the result was consistent:

Use DSS2Mod together with LAVFilters, ignore audio delays reported by MediaInfo if they are > 100ms, and get a result without any audio sync issues.

I tested it using AVStoDVD for MPEG2 output and also the old StaxRip version 1.1.9.0 for XviD and AVC output.


Cheers
manolito
Don't forget that DVB often has ads and errors, there could be various cut/trim points to remove ads. I had av sync problems with mkvmerge and DSS. I need something 100% reliable with cutting with possibility to retain original audio format.
stax76 is offline   Reply With Quote
Old 23rd May 2015, 22:04   #2057  |  Link
videoh
Registered User
 
Join Date: Jul 2014
Posts: 1,104
Any chance of an answer to my query, stax76?
videoh is offline   Reply With Quote
Old 23rd May 2015, 22:37   #2058  |  Link
stax76
Registered User
 
stax76's Avatar
 
Join Date: Jun 2002
Location: 36 Chambers
Posts: 5,928
Quote:
Originally Posted by videoh View Post
Any chance of an answer to my query, stax76?
Comparing lsmash against ffms2 frame 77 is the same in the StaxRip preview and the frame count is also the same in the StaxRip preview, when I encode it's very surprising that lsmash is fine and ffms2 is not, this could be the issue a StaxRip user reported recently, I have no idea what's going on!

edit: 2.20 is fine

Last edited by stax76; 23rd May 2015 at 22:44.
stax76 is offline   Reply With Quote
Old 23rd May 2015, 22:49   #2059  |  Link
stax76
Registered User
 
stax76's Avatar
 
Join Date: Jun 2002
Location: 36 Chambers
Posts: 5,928
2.20 is sync, 2.21 is async, this must be the issue two StaxRip users reported:

Quote:
Originally Posted by jkilez View Post
Using the new 1.3.1.3 beta, I am getting 7 frames (300ms) of blank video added to the beginning of encoded videos causing audio sync issues. Actually the framecount stays correct (it cuts 300ms at the end), so the problem is more that the video is offset by 300ms. So far I have only had a chance to check this with an AVC source. The problem does not appear when previewing, only in the final encodes and happens regardless of output (1-pass/2-pass, Xvid/x264, AVI/MKV, AC3/MP3/None).

The problem appears to be with the ffms2 indexer that was updated. If I replace the /Apps/Plugins/ffms2 directory with the version that came with 1.3.1.0, there is no delay.

ETA: Just compared encodes from an MPEG-2 (DVD) source:
1.1.9.0 = 0 extra frames
1.3.1.0 = 0 extra frames
1.3.1.3 = 2 extra frames

Last edited by stax76; 23rd May 2015 at 22:52.
stax76 is offline   Reply With Quote
Old 24th May 2015, 03:14   #2060  |  Link
videoh
Registered User
 
Join Date: Jul 2014
Posts: 1,104
Quote:
Originally Posted by stax76 View Post
Comparing lsmash against ffms2 frame 77 is the same in the StaxRip preview and the frame count is also the same in the StaxRip preview, when I encode it's very surprising that lsmash is fine and ffms2 is not, this could be the issue a StaxRip user reported recently, I have no idea what's going on!

edit: 2.20 is fine
Thanks. It sure seems that ffms2 has a regression of some sort. Surely Myrsloik will look into it and we just need a little patience. Or you could diff the source code for the two versions to see what might have caused it.

Last edited by videoh; 24th May 2015 at 03:18.
videoh is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 06:09.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.