Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Capturing and Editing Video > New and alternative a/v containers

Reply
 
Thread Tools Search this Thread Display Modes
Old 9th April 2019, 20:45   #23421  |  Link
ashlar42
Registered User
 
Join Date: Jun 2007
Posts: 655
Quote:
Originally Posted by nevcairiel View Post
I don't have any plans for such a feature, sorry. Someone could write a file source filter that sits before LAV that does that, if one really wanted to.
Thanks for your reply nevcariel. Not being a developer (absolutely far from it), I can't judge how tough of a job this would be.
From the top of my absolute ignorance, it doesn't sound too hard but, as mentioned, I have no clue.

Quote:
Originally Posted by lvqcl View Post
It's not clear for me what you want.

Do you want an option to read a file from a local HDD and cache it in RAM? Or an option to read a stream from network and cache it in a local HDD?
The second one. Cache it all on a local HDD from a network source, downloading it as fast as the network allows (throttling options might be useful for somebody but I personally don't care too much and I'd be fine with an "all or nothing" approach).

Quote:
Originally Posted by el Filou View Post
Apart from Blu-rays I don't see any media that wouldn't fit in 32 GB of RAM and even then, using a reasonable buffer of 24 GB you could buffer from 1/4 to 1/2 of the movie. Are you saying your network is so bad that 24 GBs of buffer still would cause drops?
Nope, my network is gigabit fiber and it really works at gigabit speeds. The reason I want to cache as much as possible is to avoid too many "pings" on the source, as those create problems of a different kind than playback performance (ie. API limits).
Quote:
el Filou;1871400]The settings are not opaque at all as they were described in detail the last time we discussed it:
https://forum.doom9.org/showthread.p...30#post1865530
Even just the tooltips in LAV config are pretty self-explanatory.
Basically: set Maximum Queue Memory to the max RAM you want to use for buffering, then use the highest bitrate (and feature-length playtime) file you have to test while increasing Maximum Queue Packets until the player process uses up that amount of RAM.
I just did a test again: with a 25 Mbps UHD mkv, you need a ~20.000 paquet queue for a 2 GB buffer (which already holds 10 minutes of material), and it takes just 30 seconds of playback for MPC-HC to fill it up (from local HDD). Something like a 250.000 paquet queue should let you fill up a 24 GB buffer (leaving a bit of RAM for the other apps ).
Ok, I concede... it's not settings that are opaque. My brain more likely is. Thanks for the in depth reply. I'll test this (although on my HTPC I have "only" 16GB), this is much appreciated.
Edit: although... from a local source (thus speed not being a problem), setting 16,000 MB and 250,000 packets has MPC-HC x64 stopping at about 4,2 GB of RAM used. Maybe there's a limit in LAV Filters? Maybe the format of the variable doesn't allow for such a big number?

Last edited by ashlar42; 9th April 2019 at 21:07.
ashlar42 is offline   Reply With Quote
Old 9th April 2019, 21:16   #23422  |  Link
nevcairiel
Registered Developer
 
Join Date: Mar 2010
Location: Hamburg/Germany
Posts: 10,346
Quote:
Originally Posted by ashlar42 View Post
Edit: although... from a local source (thus speed not being a problem), setting 16,000 MB and 250,000 packets has MPC-HC x64 stopping at about 4,2 GB of RAM used. Maybe there's a limit in LAV Filters? Maybe the format of the variable doesn't allow for such a big number?
There was actually a bug with such huge values, I've fixed it for the next nightly, and it successfully buffers a full BD file at 20GB for me now.
One thing to note however is that the buffer is not "smart", if you seek it'll drop the entire buffer, do the seek in the actual file, and then re-fill the buffer from scratch. Due to that behavior, seeking is actually really slow, since freeing tens of thousands of small memory blocks is actually quite an intensive task to do. Hence why the default buffer sizes are only as big as I dare make them, since it can slow stuff down.
__________________
LAV Filters - open source ffmpeg based media splitter and decoders

Last edited by nevcairiel; 9th April 2019 at 21:20.
nevcairiel is offline   Reply With Quote
Old 9th April 2019, 21:30   #23423  |  Link
ashlar42
Registered User
 
Join Date: Jun 2007
Posts: 655
Quote:
Originally Posted by nevcairiel View Post
There was actually a bug with such huge values, I've fixed it for the next nightly, and it successfully buffers a full BD file at 20GB for me now.
One thing to note however is that the buffer is not "smart", if you seek it'll drop the entire buffer, do the seek in the actual file, and then re-fill the buffer from scratch. Due to that behavior, seeking is actually really slow, since freeing tens of thousands of small memory blocks is actually quite an intensive task to do. Hence why the default buffer sizes are only as big as I dare make them, since it can slow stuff down.
Thanks nevcariel. I'm not surprised that this bug wasn't "caught" before. It surely is an "extreme" usage. And you are right, I remember you mentioning this before: it's probably not the correct way to go having such a huge buffer, for the reasons you describe.

This is why I was inquiring about disk caching (to which you've already answered).
ashlar42 is offline   Reply With Quote
Old 9th April 2019, 21:57   #23424  |  Link
VipZ
Registered User
 
Join Date: Jun 2009
Location: London
Posts: 378
Hi nevcairiel

I have switched to VS 2019 to test it out and found it no longer has the Win 8.1 SDK as an option.

I had then tested using the Win 10.0.17763.0 SDK and all seems to work for both LAV and libbluray.

While all seems to work thought I would ask if you think there could be some issue by changing the SDK like this.

MPC-BE allows one to set the SDK via environment variable, could this approach be useful for LAV as well if its fully expected to work?

Thanks
__________________
Windows 11 Pro, CPU: Ryzen 9 5900X, GPU: GeForce RTX 3090 /w 512.95, Audio: HDMI to Yamaha RX-A3060 (5.1)
VipZ is offline   Reply With Quote
Old 9th April 2019, 22:14   #23425  |  Link
nevcairiel
Registered Developer
 
Join Date: Mar 2010
Location: Hamburg/Germany
Posts: 10,346
I don't really expect problems from using the Windows 10 SDK, but I have not validated it yet to make sure it still runs on all old platforms as expected.
I did however just make a change to always select the latest Windows 10 SDK when running with VS2019, since indeed the 8.1 SDK is no longer present.
__________________
LAV Filters - open source ffmpeg based media splitter and decoders
nevcairiel is offline   Reply With Quote
Old 9th April 2019, 22:38   #23426  |  Link
VipZ
Registered User
 
Join Date: Jun 2009
Location: London
Posts: 378
Thanks for the quick update.
__________________
Windows 11 Pro, CPU: Ryzen 9 5900X, GPU: GeForce RTX 3090 /w 512.95, Audio: HDMI to Yamaha RX-A3060 (5.1)
VipZ is offline   Reply With Quote
Old 12th April 2019, 06:23   #23427  |  Link
juffi
Registered User
 
Join Date: Jul 2007
Posts: 16
LAV Filters do not work with 3D MVC. MPC-BE with internal filters work but with LAV Filters not, otherwise same setup.

MPC-HC / MPC-BE (v1.5.3, build 4455), LAV Filters (0.73.1.30-git), MadVR. Intel NUC7i3BNK, Intel i3-7100U, Intel graphics driver 25.20.100.6618. Windows 10 Pro 1809.
juffi is offline   Reply With Quote
Old 12th April 2019, 08:17   #23428  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,781
And you installed the non-default MVC pack?

__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 12th April 2019, 08:19   #23429  |  Link
nevcairiel
Registered Developer
 
Join Date: Mar 2010
Location: Hamburg/Germany
Posts: 10,346
You're on a old version of LAV Filters, 0.74.1 has since been released. Additionally, ensure that you actually have the MVC decoder component installed inside the LAV folder, otherwise it might not find it.
Otherwise, all I can say is that it works for me.
__________________
LAV Filters - open source ffmpeg based media splitter and decoders
nevcairiel is offline   Reply With Quote
Old 12th April 2019, 09:39   #23430  |  Link
Manni
Registered User
 
Join Date: Jul 2014
Posts: 942
Not only for you, it's been working for most for a while now, but as you say the optional component has to be downloaded during install
__________________
Win11 Pro x64 b23H2
Ryzen 5950X@4.5Ghz 32Gb@3600 Zotac 3090 24Gb 551.33
madVR/LAV/jRiver/MyMovies/CMC
Denon X8500HA>HD Fury VRRoom>TCL 55C805K
Manni is offline   Reply With Quote
Old 12th April 2019, 11:24   #23431  |  Link
RealSnoopyDog
Registered User
 
Join Date: May 2011
Posts: 76
Quote:
Originally Posted by ashlar42 View Post
nevcariel, is there a possibility in the future for LAV Filters to offer an option to cache completely a file on a local hard drive for playback?
What sense would it make to download every file first before you can start playing it back? This may cause many problems
  • Copyright violations
  • Waiting for hours before playback starts, depending on the file size.
  • For internet streams, there isn't always a "file" to download, it is just a stream.

It is not the job of the splitter to handle such things. If you are using software like DVBViewer which handles things transparent, you can store an internet stream as a normal DVB channel so you can also record it on your local disk.

Last edited by RealSnoopyDog; 12th April 2019 at 11:28.
RealSnoopyDog is offline   Reply With Quote
Old 12th April 2019, 12:17   #23432  |  Link
huhn
Registered User
 
Join Date: Oct 2012
Posts: 7,922
that's not how a buffer would work it doesn't delay playback it just doesn't stop buffering until the stream ends. this works for live streaming too it just downloads everything that's available at the given time.
huhn is offline   Reply With Quote
Old 12th April 2019, 13:31   #23433  |  Link
ashlar42
Registered User
 
Join Date: Jun 2007
Posts: 655
Quote:
Originally Posted by RealSnoopyDog View Post
What sense would it make to download every file first before you can start playing it back? This may cause many problems
  • Copyright violations
  • Waiting for hours before playback starts, depending on the file size.
  • For internet streams, there isn't always a "file" to download, it is just a stream.

It is not the job of the splitter to handle such things. If you are using software like DVBViewer which handles things transparent, you can store an internet stream as a normal DVB channel so you can also record it on your local disk.
I'm talking about my videos, in cloud storage. Don't know what you meant with "copyright violations" but I'm talking about personal files, not Neflix/Prime Video/YouTube.
Playback starts as soon as you start playback, as huhn explained the disk buffer would fill up as playback goes on.
As explained above, my use case would not be with internet "streams" in the way you thought it would.
ashlar42 is offline   Reply With Quote
Old 12th April 2019, 15:34   #23434  |  Link
RealSnoopyDog
Registered User
 
Join Date: May 2011
Posts: 76
Ah, ok. But even this is not a thing that a splitter should do. This is something that the playback software can handle.

If you have (e.g.) a copy of a 1920x1080 movie encoded in h.264, then 20-30 GB of data needs to be copied over the network to a local storage on the PC when playback starts. When playback is finished regulary, the the local copy needs to be deleted and so on. And what happens if the disc is full, because the files didn't get deleted?

Quote:
Playback starts as soon as you start playback, as huhn explained the disk buffer would fill up as playback goes on.
But then this option wouldn't make any sense, it only produces more useless overhead and IO accesses.

When you start playback from a "cloud" storage and the network speed is fast enough (the "player" or "consumer" reads the data slower than it gets delivered from the storage) then playback will start immediately.
If the network speed is too slow ("player" or "consumer" reads faster than the data gets transmitted), then you need to wait until a buffer got filled. Playback can not start immediately in this case even if the buffer gets filled in the background because the player will consume the data in the buffer faster than it gets filled.

@huhn: he said that he wants to have an option for a buffer that holds a complete copy of a file on the local disk and gets filled when playback starts. This doesn't make any sense. And he said that his scenario wouldn't violate any copyright things but what you wrote would definitely (possibly) violate copyright.

Last edited by RealSnoopyDog; 12th April 2019 at 15:41.
RealSnoopyDog is offline   Reply With Quote
Old 12th April 2019, 17:00   #23435  |  Link
huhn
Registered User
 
Join Date: Oct 2012
Posts: 7,922
you are aware that lavfilter is already able todo all these things. the difference is it will use the ram not the disc first. so it will fill your ram to the maximum and the system will then switch to the harddisk to buffer the rest which my result in playback issues which would not be the case if the buffer would use directly the harddisk.

so does this violate copy right whatever thingy somewhere on earth maybe i don't care.
there is no word to use it on slow internet he just wants the connection to do the whole act in one transfer as fast as possible. normal access may start and stop transfer all the time.
huhn is offline   Reply With Quote
Old 12th April 2019, 17:50   #23436  |  Link
juffi
Registered User
 
Join Date: Jul 2007
Posts: 16
Quote:
Originally Posted by Manni View Post
Not only for you, it's been working for most for a while now, but as you say the optional component has to be downloaded during install
Sure I have loaded that. LAV Filters used to work a few weeks ago, but not anymore. I installed the new LAV Filters version but it did not help. For some reason the internal filters stopped working as well. Today I once managed to get perfect playback with LAV Video decoder and other filters internal. Problem could be in the audio filter and Intel drivers.

Kodi 3D works perfectly.
juffi is offline   Reply With Quote
Old 12th April 2019, 21:39   #23437  |  Link
Grimsdyke
Registered User
 
Join Date: Nov 2013
Location: Hannover, Germany
Posts: 292
Just wanted to watch the Arrow-US-BluRay of Romero's THE CRAZIES and there is a problem that I only have with one other Blu-Ray (so far).
- MPC-BE and LAV filters => one audio track falsely identified as an mp3-file !!! Trying to play the film using this audio freezes MPC-BE !!
- MPC-BE and LAV Splitter blocked => audio correctly identified as PCM. Movie plays fine.

Only if I directly load the movie file located in the stream folder on the disc playback is fine with MPC-BE and LAV so I think there must be something in the authoring
of the disc that throws LAV off-track since BE's internal filter seem to have no problems. Does anyone maybe knows what's causing this and ideally how to fix it ??
Grimsdyke is offline   Reply With Quote
Old 15th April 2019, 11:04   #23438  |  Link
foxyshadis
Angel of Night
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Tangled in the silks
Posts: 9,559
Quote:
Originally Posted by RealSnoopyDog View Post
What sense would it make to download every file first before you can start playing it back? This may cause many problems
  • Copyright violations
  • Waiting for hours before playback starts, depending on the file size.
  • For internet streams, there isn't always a "file" to download, it is just a stream.

It is not the job of the splitter to handle such things. If you are using software like DVBViewer which handles things transparent, you can store an internet stream as a normal DVB channel so you can also record it on your local disk.
Turn on mobile data/tethering -> download -> turn off mobile data/tethering. Ten years ago, it would have been spin up the disk, read it all, spin down.

Only the most naive filters would ever have fully buffered before releasing the buffer. Start the download, start playing when requested, these operations aren't connected. Isn't this how many streams work today? What's important is that it keeps downloading constantly until finished, not
foxyshadis is offline   Reply With Quote
Old 15th April 2019, 12:43   #23439  |  Link
RealSnoopyDog
Registered User
 
Join Date: May 2011
Posts: 76
  1. My PC doesn't use mobile data or tethering and I did not install LAV on my mobile phone
  2. You didn't follow the discussion. The person who requested this wrote that he would like to have a feature which FULLY downloads FILES before the software should start playing back.
RealSnoopyDog is offline   Reply With Quote
Old 15th April 2019, 14:06   #23440  |  Link
huhn
Registered User
 
Join Date: Oct 2012
Posts: 7,922
no he just wants an full file disk buffer.
https://forum.doom9.org/showpost.php...ostcount=23476
huhn is offline   Reply With Quote
Reply

Tags
decoders, directshow, filters, splitter

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 02:52.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.