Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 Encoder GUIs

Reply
 
Thread Tools Search this Thread Display Modes
Old 9th August 2024, 16:00   #20821  |  Link
rlev11
Registered User
 
Join Date: Aug 2020
Location: Pennsylvania
Posts: 124
The only issue I have found with an autocrop, and this only affects it when using SMdegrain (never an issue with just Mdegrain) is that occasionally the autocrop will come up with an aspect ratio that is just a bit off of a standard aspect ratio and when it goes to actually start the encode there will be an error saying something like info.txt could not be found. In this case usually just adjusting the aspect ratio by removing a few bits on top and bottom (or left and right) will clear that up. Over the last couple of years I have had maybe a half dozen encodes that whatever I did SMdegrain just would not work. In those few cases, i just ran them using regular old Mdegrain and called it a day.

In fact Strange New worlds is like this. When I do an autocrop, it detects 278 lines on the top and bottom and shows an aspect ratio of 2.394:1. This will fail the encode. I add 2 lines to the top and bottom, so it is 280 on both using the up arrows, and it will run just fine with SMdegrain. I know in the past it looks like if it comes up as 2.391 or 2.4 it is always good, but 2.394 might fail "sometimes". Just one of the quirks of using SMdegrain.... It happens too infrequently to be bothered by it

Last edited by rlev11; 9th August 2024 at 16:08.
rlev11 is offline   Reply With Quote
Old 9th August 2024, 17:42   #20822  |  Link
mikehbkwm
Registered User
 
Join Date: Apr 2020
Posts: 8
I updated server 2 with Ripbot to the latest firmware. The same thing happened once again with Plex but I've encoded 5ish movies since and it hasn't done that again. Plex is showing Direct Play on the server end no matter what, technically nothing should be transcoding and I'm not out of HDD space. Only thing I can think of, just a random issue with Ripbot...I don't know.
mikehbkwm is offline   Reply With Quote
Old 9th August 2024, 18:21   #20823  |  Link
rlev11
Registered User
 
Join Date: Aug 2020
Location: Pennsylvania
Posts: 124
Quote:
Originally Posted by mikehbkwm View Post
I updated server 2 with Ripbot to the latest firmware. The same thing happened once again with Plex but I've encoded 5ish movies since and it hasn't done that again. Plex is showing Direct Play on the server end no matter what, technically nothing should be transcoding and I'm not out of HDD space. Only thing I can think of, just a random issue with Ripbot...I don't know.
So first off, what are the sizes of the videos you are playing? Also what are you using to do the conversion to 4k, something like Topaz?? If you are just up converting in ripbot, I doubt you are really accomplishing anything. Using a program like Topaz AI to clean the video before running it through ripbot to do the HEVC conversion is a different thing altogether and can really clean up DVD sources. Although with Topaz, just from a time standpoint, you would be better off upscaling to 1080p, convert to HEVC in ripbot, then let you TV do the upconversion from 1080p to the panel native 4k

Lots of results on that space error on google for plex. Most results center around setting your transcoding directory in the settings menu. Are all the ones failing have subtitles? There was/is a bug in plex that caused this error due to having to transcode the subtitles for proper viewing
rlev11 is offline   Reply With Quote
Old 13th August 2024, 18:34   #20824  |  Link
Juha
Registered User
 
Join Date: Aug 2014
Posts: 17
Hi, how will Ripbot work with 3D Blu-rays? Can (or will) 3D content be converted into 2D?
Juha is offline   Reply With Quote
Old 14th August 2024, 01:14   #20825  |  Link
Guest
Guest
 
Posts: n/a
Quote:
Originally Posted by Juha View Post
Hi, how will Ripbot work with 3D Blu-rays? Can (or will) 3D content be converted into 2D?
If I recall correctly, side by side & over/under 3D movies will retain the 3D effect, BUT the MVC types WILL lose the 3D.

However, you can demux those with tsmuxer to extract the mvp track, and then run the "normal" track thru RB if you need to do some filtering (but not much), then mux everything back together again with tsmuxer to create the 3D in .m2ts.

And have to be encoded with x264 !!!

Last edited by FTLOY; 14th August 2024 at 15:39.
  Reply With Quote
Old 14th August 2024, 14:45   #20826  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 7,117
MVC (Multi-View Coding) requires support for two parallel video streams. Right now x265 is being tested to support it, but that broke ffmpeg integration for now, and there are crashes in different circumstances.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 15th August 2024, 12:06   #20827  |  Link
Guest
Guest
 
Posts: n/a
Quote:
Originally Posted by Atak_Snajpera View Post
can't wait to see the same tests with latest Zen5 cpus. I wonder if larger L3 cache in TR would help in this case?
Well, the Zen 5's are out, and pretty much EVERY reviewer has said the same thing....disappointed, not worth the money to upgrade, etc, etc.

Also a complicated procedure to make the 9950X behave properly.

So, I guess we'll have to wait another couple of years to the next generation, 10950X, or whatever.

OR, "risk" a 14th Gen Intel, but maybe a recent microcode may "fix" that issue.

OR, blow the budget completely, and go for a TR. (a 7960X would be good enough)
  Reply With Quote
Old 15th August 2024, 12:18   #20828  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,887
I think it is safe to say that this threading issues is most likely caused by avisynth code. Something wrong with caching? Too small memory allocation?
Atak_Snajpera is offline   Reply With Quote
Old 15th August 2024, 12:24   #20829  |  Link
Guest
Guest
 
Posts: n/a
Quote:
Originally Posted by Atak_Snajpera View Post
I think it is safe to say that this threading issues is most likely caused by avisynth code. Something wrong with caching? Too small memory allocation?
What is this reply in response to ??

What about Vapoursynth ??
  Reply With Quote
Old 15th August 2024, 12:29   #20830  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,887
Quote:
Originally Posted by FTLOY View Post
What is this reply in response to ??

What about Vapoursynth ??
Regarding number of prefetch threads which has to be reduced for 4k content to 12.
Atak_Snajpera is offline   Reply With Quote
Old 15th August 2024, 15:43   #20831  |  Link
Guest
Guest
 
Posts: n/a
Quote:
Originally Posted by Atak_Snajpera View Post
Regarding number of prefetch threads which has to be reduced for 4k content to 12.
I just thought of a test that I don't think I've ever done before, and that would be to run a 5 minute sample 4K using DE with one server, and a 5 minute chunk setting, with those Avisynth server settings, and then doing the same sample on single mode where those server settings are probably not in play.

Will they take the same time to complete ???
  Reply With Quote
Old 15th August 2024, 23:31   #20832  |  Link
rlev11
Registered User
 
Join Date: Aug 2020
Location: Pennsylvania
Posts: 124
Quote:
Originally Posted by FTLOY View Post
I just thought of a test that I don't think I've ever done before, and that would be to run a 5 minute sample 4K using DE with one server, and a 5 minute chunk setting, with those Avisynth server settings, and then doing the same sample on single mode where those server settings are probably not in play.

Will they take the same time to complete ???

With my 7950x
So it looks like single mode uses the setting "use multiple processing threads" in the main settings window. Running GOT with Mdegrain3-200 in single mode with that set to it's default of 0 which I would take as All. After a couple minutes that settled into about 2.5 fps. If I change that setting from 0 to 12, bumped that up to 6fps after a couple minutes.

The fact that it scales up where it should be when no avisynth filters are being thrown at it, I agree it is definitely something coming from Avisynth code with some buffer being overloaded especially with full frame 16x9 4k. If you are just doing a x265 encode everything runs as expected speed wise in comparison
rlev11 is offline   Reply With Quote
Old 16th August 2024, 02:02   #20833  |  Link
Guest
Guest
 
Posts: n/a
Quote:
Originally Posted by rlev11 View Post
With my 7950x
So it looks like single mode uses the setting "use multiple processing threads" in the main settings window. Running GOT with Mdegrain3-200 in single mode with that set to it's default of 0 which I would take as All. After a couple minutes that settled into about 2.5 fps. If I change that setting from 0 to 12, bumped that up to 6fps after a couple minutes.

The fact that it scales up where it should be when no avisynth filters are being thrown at it, I agree it is definitely something coming from Avisynth code with some buffer being overloaded especially with full frame 16x9 4k. If you are just doing a x265 encode everything runs as expected speed wise in comparison
Interesting, I think I will do some other tests, (when I can), but if it's an Avisynth thing, this needs to be addressed on the appropriate thread of the forum.

It would be interesting to test a basic file with StaxRip, using similar filters, with Avisynth & Vapoursynth.

Unfortunately, AFAIK, Atak hasn't got access to a high core count system, so he can't check the problems we have
  Reply With Quote
Old 16th August 2024, 05:49   #20834  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,803
In any multithreading related issues, I'd start the analysis by restricting the amount of prefetched frames. With 4K sources, the frame sizes are getting so big that the default 2*threads can easily cause Avisynth cache trashing if you have not raised the size of the cache. The Avisynth default cache amount has not been changed in a long time, and it's very low.

I have these on top of my mtmodes.avsi to make sure there's enough cache memory for all situations (I have 64GB of RAM). Also mode 1 works better on my use cases.
SetMemoryMax(32000)
SetCacheMode(1)
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 16th August 2024, 06:07   #20835  |  Link
Guest
Guest
 
Posts: n/a
Quote:
Originally Posted by Boulder View Post
In any multithreading related issues, I'd start the analysis by restricting the amount of prefetched frames. With 4K sources, the frame sizes are getting so big that the default 2*threads can easily cause Avisynth cache trashing if you have not raised the size of the cache. The Avisynth default cache amount has not been changed in a long time, and it's very low.

I have these on top of my mtmodes.avsi to make sure there's enough cache memory for all situations (I have 64GB of RAM). Also mode 1 works better on my use cases.
SetMemoryMax(32000)
SetCacheMode(1)
Thankyou Boulder,

I have had these settings in my scripts, but now that I also have 64Gb, I might up it to what you've suggested

Code:
SetCacheMode()
SetMemoryMax(20480)
Wouldn't (32768) be more accurate ???
  Reply With Quote
Old 16th August 2024, 06:12   #20836  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,803
Quote:
Originally Posted by FTLOY View Post
Wouldn't (32768) be more accurate ???
It really doesn't matter, the highest amount of memory consumed has been around 23-24 gigabytes. Avisynth+ only uses the memory it needs for caching (based on what and how the plugins themselves request cache). It's good to take a look at Task Manager and see the memory consumption of the corresponding processes.

For Prefetch, I've used Prefetch(threads=32, frames=12) on my 5950X. Usually there's not much difference between frames 4-12.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 16th August 2024, 06:35   #20837  |  Link
Guest
Guest
 
Posts: n/a
Quote:
Originally Posted by Boulder View Post
It really doesn't matter, the highest amount of memory consumed has been around 23-24 gigabytes. Avisynth+ only uses the memory it needs for caching (based on what and how the plugins themselves request cache). It's good to take a look at Task Manager and see the memory consumption of the corresponding processes.

For Prefetch, I've used Prefetch(threads=32, frames=12) on my 5950X. Usually there's not much difference between frames 4-12.
In RipBot prefetch is set elsewhere in the app, and is added to the final script.

But there are options to customise certain settings, but when using the Distributed Encoding function you can customise each encoding server like so:-

Code:
Server1CommandLine=/port 1000 /priority normal /restart-if-no-progress /avisynth-prefetch-threads 12 /x264-threads 32 /x265-threads 32
Changing the "avisynth-prefetch-threads 12", has the most impact on encoding speed, on 16 core CPU's using x265, 12 - 14 seems to be the sweet spot, Ryzen 5950X's, and especially the 7950X are very particular with this setting.

With x264 & x265 threads, not sure if those numbers make any significant difference.
  Reply With Quote
Old 16th August 2024, 06:42   #20838  |  Link
Boulder
Pig on the wing
 
Boulder's Avatar
 
Join Date: Mar 2002
Location: Finland
Posts: 5,803
What you would need to do is somehow restrict the amount of prefetched frames. With threads 12, Avisynth is prefetching 24 frames which is probably far from optimal and will consume a lot of memory.
__________________
And if the band you're in starts playing different tunes
I'll see you on the dark side of the Moon...
Boulder is offline   Reply With Quote
Old 16th August 2024, 06:52   #20839  |  Link
Guest
Guest
 
Posts: n/a
Quote:
Originally Posted by Boulder View Post
What you would need to do is somehow restrict the amount of prefetched frames. With threads 12, Avisynth is prefetching 24 frames which is probably far from optimal and will consume a lot of memory.
I'm pretty sure I've tried the frames "call", but from what I remember, I couldn't detect any difference, and tbh, not sure where it would need to be added to work with RipBot

This is the only reference in the avs script to prefetch:-

Code:
#MT
#PREFETCH_LIMIT=12
And this is determined by a "check box" on the settings page, the only other place I can think it could be put is in a custom script.

Last edited by FTLOY; 16th August 2024 at 06:56.
  Reply With Quote
Old 16th August 2024, 20:45   #20840  |  Link
rlev11
Registered User
 
Join Date: Aug 2020
Location: Pennsylvania
Posts: 124
Just not sure where to set frames into the ripbot script or the proper syntax to use in the job script. Don't see a "/avisynth-prefetch-frames" setting in the help for the encoding server command line like threads. If it actually the frames and not threads that's killing it, once we figure out where and how to set the frames, we could set threads to 16 (or total physical cores), and just play with the threads to see where things go south and see if that same number works across different platforms. It may just end up that 1 proper frames count may be optimized for BOTH 4k and 1080p which would be perfect, or at the minimum very little difference between them. Set the frames once and be done with it.

Ideal test would be to set threads to logical cpu's, then find happy frame count doing a full frame 4k. The run 1080p and start increasing the frames and see if there is any appreciable performance gain.

A buffer size increase might work as well, but could cause issues using distributed if you have servers with varying memory sizes. I would think the buffer size would get sent out to all the servers and if you have one with say 16gig instead of a setting optimized for 32 or 64, things could go crazy real fast.


Just a quick initial test, On my 1080p test with Mdegrain3-200, I set avisynth prefetch threads to 32 on my 7950x, so 64 prefetch frames, and I saw about a 9% speed increase just on that alone.

Last edited by rlev11; 16th August 2024 at 22:16. Reason: add quick test
rlev11 is offline   Reply With Quote
Reply

Tags
264, 265, appletv, avchd, bluray, gui, iphone, ipod, ps3, psp, ripbot264, x264 2-pass, x264 gui, x264_64, x265, xbox360

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 08:57.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.