L-SMASH Source

[hello_hello]

Originally Posted by Are_

I know I already bit the bait but, shouldn't we ignore the obvious troll? He doesn't use the plugin but he's just wasting others time with ridiculous questions nobody cares just because he's probably bored and all. In the last post he even retorted to mild insulting, sad view.

FFS, that'll teach me to post about something that doesn't effect me at the moment simply to help someone else understand a problem they were having. I do use the plugin, but I don't use the current version and I made the reason for that clear. I'm sorry you didn't understand... and all... but in case I need to state the obvious, I'd prefer to eventually use a more recent version, only I'd rather it retained the ability to open index files.

For the record, it's not the 1990's and the only trolls in forums are the ones who troll by insulting others with accusations of trolling. Did you forget to discuss the topic or were you just bored?

[videoh]

Suppose you want to load a series of M2TS files which are scattered in a directory and not in sort order (just about any bluray/UHD disk). Another case is multiple VOBs from a DVD. Loading a DGIndex(NV) project file will conveniently and automatically load them all in order, because the file list is stored in the index file. Without that you'd have to navigate to and tediously select all the files again every time you want to work on the project again. Is this something applicable here? If smash whatever can only load one source file then it would be irrelevant. I don't know enough about the smash stuff to know if this is relevant, but it sure is a big advantage for DGIndex(NV) to have that capability.

[videoh]

Originally Posted by LigH

I'm not fond of religious wars in technical forums.

LigH arrogantly thinks that what he is fond of is an important consideration. Newsflash, nobody cares what you are fond of.

[MeteorRain]

Sorry for not keeping an eye on this thread for a few days. Let me add some point from my view.

The fundamental difference between dgindex / dgnv and lsmash / ffms is that dg index is generated using a standalone tool while lsmash / ffms index is (usually) generated by the plugin itself.

That's why I was so confused by the behavior of MeGUI. You have to use the source file as the parameter passed to lsmash in order to create an index. Why would it change that parameter (completely unnecessarily) to the index then?

For dgindex and dgnv it's pretty natural to use index because you create the index first. Besides, as videoh said, dg family supports loading multiple files in a single shot, which makes more sense for it to load an index.

[hello_hello]

Originally Posted by videoh

Suppose you want to load a series of M2TS files which are scattered in a directory and not in sort order (just about any bluray/UHD disk). Another case is multiple VOBs from a DVD. Loading a DGIndex(NV) project file will conveniently and automatically load them all in order, because the file list is stored in the index file. Without that you'd have to navigate to and tediously select all the files again every time you want to work on the project again. Is this something applicable here? If smash whatever can only load one source file then it would be irrelevant. I don't know enough about the smash stuff to know if this is relevant, but it sure is a big advantage for DGIndex(NV) to have that capability.

I don't actually know if lsmash can open split vob files as a single file as I've always used DGIndex for that, but it's the same as ffms2... by default it ignores repeat flags and outputs the average frame rate. It's probably fine if the ultimate goal is a variable frame rate encode, but I've seen a few threads at VideoHelp where someone has ripped a DVD with MakeMKV and indexed with lsmash and posted to ask why the audio sync is all over the place.

As far as I know there's no way to load multiple sources as a single index file or project.

[hello_hello]

Originally Posted by MeteorRain

That's why I was so confused by the behavior of MeGUI. You have to use the source file as the parameter passed to lsmash in order to create an index. Why would it change that parameter (completely unnecessarily) to the index then?

I can only repeat what I've repeated several times. The main advantage of being able to open an index file directly is when it's not in the source folder. That's why MeGUI works the way it does. It can index the source for you Monday, move the index file to the working directory, and Wednesday you can open it to create a script. It couldn't work that way for ffms2, but Lsmash had the ability to open index files and the person maintaining MeGUI was clever enough to make use of it. It baffles me why anyone would think that's a bad thing.

If lsmash wrote the path to the index file while indexing, but ignored it unless an index file is used as the source, it seems to me it'd be the best of both worlds. I honestly don't understand why there's so much resistance to the idea. Why break functionality unnecessarily? Even if the response was "can't be bothered", or "don't care" it'd make more sense to me than denying there's ever a reason to open index files. I often do because I rarely keep them in the source folder, so it saves having to specify each source location plus the cachefile argument for the index file in a new script, and we already know one of the most popular GUIs opens index files and it'll break.

[kedautinh12]

Originally Posted by HolyWu

So how did you right click on a nonexistent index file in the explorer when the index wasn't created yet for new media files? Didn't you still need a script and point the media file as source to let LWLibavVideoSource create the index file first? Then you either modify the existing script or create another new script and point the index file as source for LWLibavVideoSource just because you want to directly open the index file so deadly.

I see you hardwork with l-smash source, can you continute with ffmpegsource??

[hello_hello]

Originally Posted by HolyWu

Of course it was. That's what the users did before, not only because of the file/folder being moved/renamed for some reasons, but also the file being opened from another computer using network path. But why must the users bother to edit the index file when they can avoid this unnecessary step once and for all?

I didn't say they should have to. Not that I recall complaints from users who move/rename source files after indexing, so I'm not sure how it was determined they found it a problem or troublesome.

Originally Posted by HolyWu

So how did you right click on a nonexistent index file in the explorer when the index wasn't created yet for new media files? Didn't you still need a script and point to the media file as source to let LWLibavVideoSource create the index file first? Then you either modify the existing script or create another new script and point to the index file as source for LWLibavVideoSource just because you want to directly open the index file so deadly.

After explaining numerous times?
At the risk of bringing MeGUI into it again, I often add a batch of indexing jobs to the job queue, let MeGUI run them while moving the index files to the specified location(s), and later I create the scripts from scratch. There's no scripts to copy or modify. MeGUI does the indexing "behind the scenes" via a command prompt... or something. You can't do the same thing with ffms2 unless the index files are in the source folder, which is often a determining factor for me when choosing which indexer to use.
I often use Avisynthesizer to create scripts using the index file as the source, and sometimes I do that rather than copy or modify an existing script because it's more convenient, or sometimes I get MeGUI to run a batch of indexing jobs and create the scripts myself etc.

And to go around in circles again, MeGUI is designed so the user isn't required to have any knowledge of Avisynth including how to open a source in a script, and I can't think of a GUI of that type that doesn't require the user to open a source file. I did ask for examples of the GUIs you claimed to be better choices with no response, but in MeGUI's case, the Script Creator can add the cachefile argument as long as it's configured to automatically open after indexing. To open a source that's already been indexed, the index file must be in the source folder or of a type the Script Creator can open directly, otherwise MeGUI can't know it exists. Are there psychic GUIs I'm unaware of, or is a bad thing MeGUI allows you to specify a working directory while still avoiding re-indexing whenever possible?

I've offered several suggestions as to how I think it'd be possible to have the best of both worlds, but they've been ignored in preference to insisting everybody does things the same way, or labelling a GUI as badly designed for having the audacity to use existing functionality, and that's something I find a little hard to understand.

[dREV]

MeGUI has a field section called "Working" which is where index files goes. It doesn't have to be set to target to a specific folder as it'll create a folder with the indexed file where ever the m2ts file is without moving it which is its default. However, even with leaving it on its default using the latest lsmash it'll still give the errors I've pointed at #1085. Image here https://i.ibb.co/4jSWfZh/vampi.jpg

I also agree that it's not cool to call MeGUI out of date when it's been using something that was included in lsmash. Somehow the author knew about it but was taken out cuz the updaters didn't know why it was there or something. So if the MeGUI author wants to resort to having to update lsmash I'll be fine using MeteorRain's working one till then despite not being able to use that 1 frame fix.

Originally Posted by videoh

Suppose you want to load a series of M2TS files which are scattered in a directory and not in sort order (just about any bluray/UHD disk). Another case is multiple VOBs from a DVD. Loading a DGIndex(NV) project file will conveniently and automatically load them all in order, because the file list is stored in the index file. Without that you'd have to navigate to and tediously select all the files again every time you want to work on the project again. Is this something applicable here? If smash whatever can only load one source file then it would be irrelevant. I don't know enough about the smash stuff to know if this is relevant, but it sure is a big advantage for DGIndex(NV) to have that capability.

Originally Posted by hello_hello

I don't actually know if lsmash can open split vob files as a single file as I've always used DGIndex for that, but it's the same as ffms2... by default it ignores repeat flags and outputs the average frame rate. It's probably fine if the ultimate goal is a variable frame rate encode, but I've seen a few threads at VideoHelp where someone has ripped a DVD with MakeMKV and indexed with lsmash and posted to ask why the audio sync is all over the place.

As far as I know there's no way to load multiple sources as a single index file or project.

Would that even work when using trim? When I find banding sections I have to do multi-uses of the same m2ts file (banding here, sharpening here, anti-aliasing here, etc) and that would be really helpful cuz it takes a loooong time (well the 2017 lsmash MeGUI was using but even with MeteorRain's working one still takes time) but I guess it won't if I use trim? I don't really know what the index does. Does it index the entire m2ts file or is it frame based indexing?

I have my MeGUI setup to use lsmash via One Click > Config > Other tab > Indexer / Opener Priority and seems to ignore those and goes directly to DGI. To nitpick on this I dunno why it varies between DVD's with regards to how it does VOBs because I've had some DVD's where it only did each single episode while in others it did the entire DVD episodes into one huge container.

[hello_hello]

Originally Posted by dREV

MeGUI has a field section called "Working" which is where index files goes. It doesn't have to be set to target to a specific folder as it'll create a folder with the indexed file where ever the m2ts file is without moving it which is its default. However, even with leaving it on its default using the latest lsmash it'll still give the errors I've pointed at #1085. Image here https://i.ibb.co/4jSWfZh/vampi.jpg

I don't use OneClick, but I assume MeGUI still uses the index file as the source for lsmash in the script even when it's in the source folder because there's previously been no reason not to.

Originally Posted by dREV

Would that even work when using trim? When I find banding sections I have to do multi-uses of the same m2ts file (banding here, sharpening here, anti-aliasing here, etc) and that would be really helpful cuz it takes a loooong time (well the 2017 lsmash MeGUI was using but even with MeteorRain's working one still takes time) but I guess it won't if I use trim? I don't really know what the index does. Does it index the entire m2ts file or is it frame based indexing?

It'd be better to discuss this elsewhere if need be, but I'm not sure exactly what you mean anyway. I'll send you a PM shorty so we can do it that way if you like.

Originally Posted by dREV

I have my MeGUI setup to use lsmash via One Click > Config > Other tab > Indexer / Opener Priority and seems to ignore those and goes directly to DGI. To nitpick on this I dunno why it varies between DVD's with regards to how it does VOBs because I've had some DVD's where it only did each single episode while in others it did the entire DVD episodes into one huge container.

I'm guessing, but it probably ignores the preferred indexer and uses DGIndex for vob files due to the problem I mentioned earlier.
Vob files are generally indexed as a single file when they're sequentially numbered. There's often a title for each episode, but sometimes there's just a single title with the episodes divided by chapters and that'd be why sometimes all the episodes are encoded together. It depends how the DVD is authored. I'll explain how I re-author DVDs before encoding to solve that problem in the PM.

Edit: I tried to send you a PM but you have them disabled.

[dREV]

Originally Posted by hello_hello

It'd be better to discuss this elsewhere if need be, but I'm not sure exactly what you mean anyway. I'll send you a PM shorty so we can do it that way if you like.

Sorry for not writing my words more properly I hope this will do better. The index is not based on what's in the AviSynth script such as if trim were to be used, correct?

For example, if I have a banding scene from frames (200, 500) and I get lsmash to index and encode these scenes it's not really doing it for those scenes but for the m2ts file itself? I'll then have to run the lsmash index 2 more times to do frames scenes (0, 199) and frames (501, 33000) to finish up the episode.

It'll be very helpful if I can just index once and get straight to encoding especially when I use trim as it could take up to 2 to 50 different frames for just 1 single episode which could take a very long time to do. Especially on the original MeGUI lsmash which I've timed it to be over well an 1 hour or two and movies are worse as the numbers shoot even higher.

Originally Posted by hello_hello

I'm guessing, but it probably ignores the preferred indexer and uses DGIndex for vob files due to the problem I mentioned earlier. Vob files are generally indexed as a single file when they're sequentially numbered. There's often a title for each episode, but sometimes there's just a single title with the episodes divided by chapters and that'd be why sometimes all the episodes are encoded together. It depends how the DVD is authored. I'll explain how I re-author DVDs before encoding to solve that problem in the PM.

Edit: I tried to send you a PM but you have them disabled.

Sorry I never expected to receive any messages. Sorry if I wasted your time if you wrote stuff via PM. I'm not looking to re-author DVD's as I just do backups in mkv. Thanks for your offer

[poisondeathray]

Originally Posted by dREV

Sorry for not writing my words more properly I hope this will do better. The index is not based on what's in the AviSynth script such as if trim were to be used, correct?

Yes, index is for the main video

For example, if I have a banding scene from frames (200, 500) and I get lsmash to index and encode these scenes it's not really doing it for those scenes but for the m2ts file itself? I'll then have to run the lsmash index 2 more times to do frames scenes (0, 199) and frames (501, 33000) to finish up the episode.

No, you only need to index once, the whole video

[hello_hello]

Originally Posted by dREV

For example, if I have a banding scene from frames (200, 500) and I get lsmash to index and encode these scenes it's not really doing it for those scenes but for the m2ts file itself? I'll then have to run the lsmash index 2 more times to do frames scenes (0, 199) and frames (501, 33000) to finish up the episode.

While sometimes you can't know there's problems with an encode till after it's done, that's why I don't use the OneClick encoder, as normally I'd preview the Avisynth output before encoding and try to do everything in a single script, but even if you encode the video in sections you should be able to re-use the index file. If your workflow is causing MeGUI to create a new script and re-index the source for each section you encode you should change it if you can. OneClick is really designed to be an automatic process. If you're creating complex scripts and/or fixing problems it's probably not the best choice.

LWLibavVideoSource("D:\Video.m2ts.lwi")
MaybeSomeFilterHere()
MaybeAnotherOne()
A = last
B = A.SomeDebanding()
A.Trim(0, 199)\
++B.Trim(200, 500)\
++A.Trim(501, 0).MaybeAnotherFilter().OrTwo()
MaybeSomethingElse()

By the way, I had a better look at your previous screenshot and it seems even when you don't specify a working directory, OnceClick creates a subfolder with a random name in the source directory, so it'd put the index file in that folder and open it directly. OneClick probably always creates subfolders regardless of the output location, but as I said, I don't use it.

Originally Posted by dREV

Especially on the original MeGUI lsmash which I've timed it to be over well an 1 hour or two and movies are worse as the numbers shoot even higher.

That seems an extremely long time even for a 1080p source with several audio streams. I just indexed a 1080p movie with the old XP compatible lsmash and it took 70 seconds. It was only 6GB but it was the largest MKV I had handy. Sometimes indexing a 1080p Bluray movie can take a while, but I don't think I've indexed a movie that came close to an hour to complete.

Originally Posted by dREV

Sorry I never expected to receive any messages. Sorry if I wasted your time if you wrote stuff via PM. I'm not looking to re-author DVD's as I just do backups in mkv. Thanks for your offer

The idea of re-authoring them first is so the episodes can be encoded individually. It's easy to do and doesn't take long.
To keep discussing MeGUI itself though, it'd probably be better to do so elsewhere rather than sidetrack this thread too much.

[Atak_Snajpera]

@HolyWu
Could you help me to decrypt those values?

Index=0,POS=9566774,PTS=-9223372036854775808,DTS=35085050,EDI=0
Key=1,Pic=1,POC=2,Repeat=1,Field=0

My guesses
Index -> stream index
POS -> position in file
PTS -> ?
DTS -> ?
EDI -> ?
Key -> keyframe
Pic -> ?
Repeat -> ?
Field -> Interlaced field

Also What is purpose of these entries?

<StreamDuration=0,0>-9223372036854775808</StreamDuration>
<StreamIndexEntries=0,0,16061>
POS=0,TS=0,Flags=1,Size=0,Distance=0
POS=12116,TS=500500,Flags=1,Size=0,Distance=0
POS=26960,TS=1101100,Flags=1,Size=0,Distance=0
POS=95503,TS=1601600,Flags=1,Size=0,Distance=0
POS=154509,TS=2002000,Flags=1,Size=0,Distance=0
POS=215660,TS=2452450,Flags=1,Size=0,Distance=0
POS=314434,TS=2752750,Flags=1,Size=0,Distance=0
POS=476442,TS=3253250,Flags=1,Size=0,Distance=0
POS=658539,TS=3753750,Flags=1,Size=0,Distance=0
POS=763885,TS=4104100,Flags=1,Size=0,Distance=0
POS=925780,TS=4654650,Flags=1,Size=0,Distance=0

[Atak_Snajpera]

They seem to be used for format which doesn't support seeking natively.

For example raw stream unmuxed in container (.264 , .265 , ivf) ?

[LigH]

Probably also MPEG Transport Stream (and Program Stream?): No internal index chunk, originally designed to be "played-a-live" in a broadcast; seeking is only possible in a kind of interval bisection method: test seek with good guess assuming a rather constant average bitrate, read next found GOP timestamps, calculate new difference, new test seek with a better guess...

[Atak_Snajpera]

Originally Posted by LigH

Probably also MPEG Transport Stream (and Program Stream?): No internal index chunk, originally designed to be "played-a-live" in a broadcast; seeking is only possible in a kind of interval bisection method: test seek with good guess assuming a rather constant average bitrate, read next found GOP timestamps, calculate new difference, new test seek with a better guess...

So another question would be Why do we need those entries for mkv,mp4 and others?

[LigH]

LwLibav*Source generally does not rely on internal index chunks of any container format. It always re-indexes content streams as they are demultiplexed by the libavformats splitters. Only a fresh index guarantees frame-exact seeking; internal index chunks chould have been damaged, missing, or been built by "incompetent" software...

LSMASH*Source relies on index chunks of ISO Base Media container formats only.

[Atak_Snajpera]

Originally Posted by LigH

LwLibav*Source generally does not rely on internal index chunks of any format. It always re-indexes content streams as they are demultiplexed by the libavformats splitters. Only a fresh index guarantees frame-exact seeking; internal index chunks chould have been damaged, missing, or been built by "incompetent" software...

I meant why do we need those extra entries in index if we already have this above

<LSMASHWorksIndexVersion=0.0.2.0>
<LibavReaderIndexFile=16>
<FileSize=2465531781>
<FileHash=0x94ac5e38>
<LibavReaderIndex=0x00000100,1,mpegvideo>
<ActiveVideoStreamIndex>+0000000000</ActiveVideoStreamIndex>
<ActiveAudioStreamIndex>-0000000002</ActiveAudioStreamIndex>
<StreamInfo=0,0>
Codec=2,TimeBase=1/1200000,Width=720,Height=480,Format=yuv420p,ColorSpace=6
</StreamInfo>
Index=0,POS=0,PTS=-9223372036854775808,DTS=0,EDI=0
Key=1,Pic=1,POC=0,Repeat=1,Field=0
Index=0,POS=7988,PTS=-9223372036854775808,DTS=50050,EDI=0
Key=0,Pic=2,POC=1,Repeat=1,Field=0
Index=0,POS=8576,PTS=-9223372036854775808,DTS=100100,EDI=0
Key=0,Pic=2,POC=3,Repeat=1,Field=0
Index=0,POS=9128,PTS=150150,DTS=150150,EDI=0
Key=0,Pic=3,POC=2,Repeat=1,Field=0
Index=0,POS=9446,PTS=-9223372036854775808,DTS=200200,EDI=0
Key=0,Pic=2,POC=5,Repeat=1,Field=0
Index=0,POS=10002,PTS=250250,DTS=250250,EDI=0
Key=0,Pic=3,POC=4,Repeat=1,Field=0
Index=0,POS=10320,PTS=-9223372036854775808,DTS=300300,EDI=0
Key=0,Pic=2,POC=6,Repeat=1,Field=0
Index=0,POS=10884,PTS=-9223372036854775808,DTS=350350,EDI=0

[l33tmeatwad]

Was just trying out the latest updates from HolyWu, apparently every version after 20190910 either crashes or gives an access violation on x86 with AviSynth+ 3.4.0; the x64 for the latest appears to work fine.