Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 Encoder GUIs

Reply
 
Thread Tools Search this Thread Display Modes
Old 18th December 2018, 11:57   #16481  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,806
Quote:
Originally Posted by burt123 View Post
Of course you are...

So how many movie length x265 encodes to you do ??

And I'm sure as hell happy I don't have to pay your power bill !!!!

Just imagine how much time (and money) you would save it RB didn't spend so much time gathering information & indexing & demuxing & remuxing, it would be nearly an hour per job.

For example, if I had a Job Queue that had multiple x265 jobs, and I set it going, it would take approx 30 minute before it would even think about encoding, then a day later (if you're lucky), it would then take approx another 30 minutes to mux it all back together, then on the next job....same again, so over several jobs, hours have been wasted doing nothing

Like I said in a previous post, MKVToolNix does it's job in minutes, admittedly there's no DE, or filtering, but if the movie quality is good, (and most 4K movie's are), why does RB have to pull it apart, and put it back together. That's what takes all the time (other than the encoding), and why that has to take so long, when the file is already "up to scratch".
The problem with you is that you do not see difference between simple remuxing (copying streams from one container to another) and re-encoding! (processing frames and encoding to reduce file size). You also do not understand simple things like why encoding 3840x2160 frames take 4 times more time to process/encode than 1920x1080. Next. You also use x265 which uses a lot more complex algorithms in order to increase compression efficiency than x264. If MKVToolnix offers so amazing "compression and speed" then what are you even doing here? Seriously! Your "frustration" comes mainly from lack of knowledge in this particular topic.
Atak_Snajpera is offline   Reply With Quote
Old 18th December 2018, 13:31   #16482  |  Link
Ryushin
Registered User
 
Ryushin's Avatar
 
Join Date: Mar 2011
Posts: 431
Quote:
Originally Posted by burt123 View Post
Of course you are...

So how many movie length x265 encodes to you do ??

And I'm sure as hell happy I don't have to pay your power bill !!!!

Like I said in a previous post, MKVToolNix does it's job in minutes, admittedly there's no DE, or filtering, but if the movie quality is good, (and most 4K movie's are), why does RB have to pull it apart, and put it back together. That's what takes all the time (other than the encoding), and why that has to take so long, when the file is already "up to scratch".
I have about 160 4K discs. Only about 2/3 done with my encoding for 4K right now.

Power bill is heavy. Wife complains.

So as Atak just posted, I understand what you're thinking and trying to do. There is a big difference between remuxing and re-encoding. If you just want to get the 4K movie off the disc and onto your computer, then use MakeMKV and it will take the film and place it into a MKV file at it's original quality and size from the 4K disc. Average size of your file will be between 45-100GB in size. Should take about an hour or so per disc to do and then you are done. All the original subtitles will be there as well.

Now if you want to make changes to said video, making it smaller, degraining, etc. then you will need to re-encode. This is what RB is for. It takes time to extract each component to set it up for how RB processes it (Handbrake skips this step, but it also can not do DE). After that, its encoding time. So for me it is 1-3 movies per day, depending on if I'm using MDegrain or not. If the file size is not right (for me) then I tweak its settings (MDegrain) and back it goes for encoding. I like to shoot for the movie to be 1/3 to 1/4 of it's original size. So about 15-25GB depending on length. Depending on grain, I might have to redo the encode 2-3 times. So a single movie, might take 1-4 days to get done.

Since time seems really important to you, just use MakeMKV and keep the original movie. Get a couple of 10GB drives and store your movies natively.
Ryushin is offline   Reply With Quote
Old 18th December 2018, 13:56   #16483  |  Link
burt123
Registered User
 
burt123's Avatar
 
Join Date: Jun 2010
Location: NSW, Australia.
Posts: 366
Quote:
Originally Posted by Atak_Snajpera View Post
The problem with you is that you do not see difference between simple remuxing (copying streams from one container to another) and re-encoding! (processing frames and encoding to reduce file size). You also do not understand simple things like why encoding 3840x2160 frames take 4 times more time to process/encode than 1920x1080. Next. You also use x265 which uses a lot more complex algorithms in order to increase compression efficiency than x264. If MKVToolnix offers so amazing "compression and speed" then what are you even doing here? Seriously! Your "frustration" comes mainly from lack of knowledge in this particular topic.
One last comment/reply...

I have helped you with a lot of problem solving with your program, over the past couple of years, to help get it better, and I'm happy that I was able to help.

I have told you many times that this IS way over my head, and sometimes you expect "us" to be able to read your mind, when it comes to feedback of problems, and where & what to look for.

I have had NO IT training at all.

Having said that, in my experience, RB does an awesome job with anything up and including 1080, x264.

And it's probably clearly the fastest x265 encoder around, mainly due to the DE function.

My suggestions of building a simple remuxing tool, that also offers some filtering, denoising, & DE, would be very welcome, as most 4K movies probably don't require encoding, and I'm not too interested in reducing file size.

So there's an opportunity for you to create another great program.

You're right, I probably shouldn't be here, so I will disappear.

I won't stop using RB.
burt123 is offline   Reply With Quote
Old 18th December 2018, 14:01   #16484  |  Link
ReinerSchweinlin
Registered User
 
Join Date: Oct 2001
Posts: 454
Exactly ... Storage with traditional HDs is so cheap these days..... If remiuxing is faster on a SSD, one could take this intermediate step and pull everything in one peace over to the HD.. Much cheaper on the bill and a lot quicker than re-encoding...

If Size/Quality Ratio is not that big of a concern - one could easily buy a recent GPU oder CPU with hardware encoding and be very quick with very good results. My 50 Euro Pentium (Kaby Lake) was able to do 1080p in H265 10 Bit in "faster than realtime" with no sweat... and Q20 gave results for a normal Movie (with not too much grain in it) at max 6mbit or so.... All my quicksync encodes look very good with about 50% to 100% more Bitrate thrown at them than x265 - and are done much faster..

Of course, If one wants to squeeze out the best quality per bitrate possible - you have to filter, use X265 instead of hardware, take slow settings, wait, twak, CU-losless your brains out, etc.. I really enjoy optmizing things, but thats more of a question of "whats important to me - time? Quality? Having a Hobby? Learning the secrets of encdoing? go to avisynth college...??

But it all comes down to the basics - understanding the difference between re-muxing and re-encoding - I agree on that
ReinerSchweinlin is offline   Reply With Quote
Old 18th December 2018, 14:48   #16485  |  Link
ReinerSchweinlin
Registered User
 
Join Date: Oct 2001
Posts: 454
Quote:
Originally Posted by burt123 View Post

And it's probably clearly the fastest x265 encoder around, mainly due to the DE function.
RB uses x264 and x265 - which are used by almost every other software for encoding... Speed of encoding mainly relies on the x265 part... Given the same parameters while encoding, most Software bundles (Staxrip, RB, handbrake, Mediacoder, etc... thousands probably) are of the same speed.

RB has this one wonderfull feature of Distributed encoding - which is probably unique... (Iīve seen Mediaencoder having some network settings.. never got it to work).

This way your file can be split up into many small parts - and every PC in your network can contribute encoding..

Thatīs the speed advantage....

Of course, RB does some muxing, indexing, etc... This is helpful and necessary to keep everything together and in sync... Handbrake (for exmaple) can deal with a videofile without that many "pre-steps", because it simply starts at the beginnig (but take a look at staxrip, wich demuxes also). Handbrake does subtitle scanning which can take a while, too..

But all this time (preparing before encoding) is "next to nothing" compared to the encoding time itself when using x265 in 4k and high quality settings (if you filter, it gets even worse)...

Quote:
My suggestions of building a simple remuxing tool, that also offers some filtering, denoising, & DE, would be very welcome, as most 4K movies probably don't require encoding, and I'm not too interested in reducing file size.
And here probably lies the biggest misunderstanding....
Filtering, Denoising (which is filtering).... requires re-encoding.. There is NO WAY to simply remux and have a filter applied....

So your suggestion is impossible (not because no one wants to try, it really is impossible!).
ReinerSchweinlin is offline   Reply With Quote
Old 18th December 2018, 15:21   #16486  |  Link
slalom
Registered User
 
slalom's Avatar
 
Join Date: Jan 2010
Posts: 456
Quote:
Originally Posted by burt123 View Post
Having said that, in my experience, RB does an awesome job with anything up and including 1080, x264.

And it's probably clearly the fastest x265 encoder around, mainly due to the DE function.
It's not the program man, It's your Hardware

Quote:
Originally Posted by burt123 View Post
My suggestions of building a simple remuxing tool, that also offers some filtering, denoising, & DE, would be very welcome, as most 4K movies probably don't require encoding, and I'm not too interested in reducing file size.
That requires re-encoding

remuxing & re-encoding are two different things
__________________
E5 2697 v2 @ 3.0GHz on P9X79 Deluxe 24GB
Xeon E5-2680 v2 @ 3.1GHz 16GB
Sony Vaio VPC-F13Z1E/B
slalom is offline   Reply With Quote
Old 19th December 2018, 03:16   #16487  |  Link
FuzzyNutz
Registered User
 
Join Date: Jun 2016
Location: Canada
Posts: 131
Removed jobs should move to recycle bin

Clicking "remove" permanently deletes jobs. Removed jobs should move to the recycle bin.

Last edited by FuzzyNutz; 19th December 2018 at 08:07.
FuzzyNutz is offline   Reply With Quote
Old 19th December 2018, 08:59   #16488  |  Link
Wishbringer
Silent Reader
 
Wishbringer's Avatar
 
Join Date: Dec 2003
Location: Germany
Posts: 295
Only a remark how good or bad is --opencl in x264:

I am encoding in DE mode.
Two Ryzen 2700X with RX550 and DDR4-3200

using placebo preset...

With opencl enabled for RX550 I get around 13 fps alltogether, but cpu-utilization is around 45%,
when opencl is disabled cpu-utilization increases to 100%, but encoding fps only increases to 19 fps.
(this means higher wattage for CPU but lower demands for GPU)

With opencl enabled and using two encodingservers per Client cpu-utilization stays at around 45%, no increase of encoding fps.
(so opencl at RX550 seems to be the bottleneck, I am curious that opencl at RX550 seems to be slower than without on Ryzen 2700X)

Is there a possible option to enable x264 opencl for the first encodingserver per client and disable for the second one?

Last edited by Wishbringer; 19th December 2018 at 09:34.
Wishbringer is offline   Reply With Quote
Old 19th December 2018, 11:04   #16489  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,806
Quote:
(so opencl at RX550 seems to be the bottleneck, I am curious that opencl at RX550 seems to be slower than without on Ryzen 2700X)
Have you checked GPU usage in Task Manager? This would clearly show if GPU is a bottleneck here.

Quote:
Is there a possible option to enable x264 opencl for the first encodingserver per client and disable for the second one?
Not yet but I will add this to my short to do list.

UPDATE: Have tried adding incorrect device id to disable opencl acceleration?
/x264-opencl-device-id 9

Last edited by Atak_Snajpera; 19th December 2018 at 11:34.
Atak_Snajpera is offline   Reply With Quote
Old 19th December 2018, 11:06   #16490  |  Link
ReinerSchweinlin
Registered User
 
Join Date: Oct 2001
Posts: 454
AFAIR Remember, Win10 Task Manager not always is 100% reliable to show GPU workload. I am using GPU-Z to cross-check.
ReinerSchweinlin is offline   Reply With Quote
Old 19th December 2018, 14:04   #16491  |  Link
Ryushin
Registered User
 
Ryushin's Avatar
 
Join Date: Mar 2011
Posts: 431
Quote:
Originally Posted by burt123 View Post
One last comment/reply...

You're right, I probably shouldn't be here, so I will disappear.
You don't have to go away. Atak has done and is doing an amazing job evolving RB. He spent a lot of time trying to solve your problem, but doing without being able to duplicate the issue in his development environment, makes it REALLY hard to debug and troubleshoot. At some point, he cannot solve it from his end.

You also have to understand, Atak may be a bit short sometimes. He may feel sometimes that some are ungrateful with all of his hard work. (I'm very grateful for RB!) And he just gets exasperated. He has a life too outside of RB and who knows how much stress is involved in that. And on top of all of this, he develops RB for FREE and on his own time. We have to be more grateful for everything he has done.

So in the end, we can post bugs that we find, and hope that Atak can fix them. But if he can't, and a work around exists, then we should just live with it. Just look at how much more RB does year after year. 4K support just got added around this time last year. Amazing stuff.

Thank you Atak for such an amazing problem and your awesome support.
Ryushin is offline   Reply With Quote
Old 19th December 2018, 18:06   #16492  |  Link
byteshare
ByteShare
 
byteshare's Avatar
 
Join Date: Sep 2014
Location: On the Internet
Posts: 560
Quote:
Originally Posted by FuzzyNutz View Post
Clicking "remove" permanently deletes jobs. Removed jobs should move to the recycle bin.
You can just not have a job get removed until you're ready to have the temp files deleted.
There is an option in the settings for this: Keep jobs after conversion
byteshare is offline   Reply With Quote
Old 19th December 2018, 18:41   #16493  |  Link
FuzzyNutz
Registered User
 
Join Date: Jun 2016
Location: Canada
Posts: 131
Quote:
Originally Posted by byteshare View Post
You can just not have a job get removed until you're ready to have the temp files deleted.
There is an option in the settings for this: Keep jobs after conversion
Not applicable when clicking "remove" and doesn't address non-use of recycle bin.

Last edited by FuzzyNutz; 19th December 2018 at 22:43.
FuzzyNutz is offline   Reply With Quote
Old 19th December 2018, 21:47   #16494  |  Link
Wishbringer
Silent Reader
 
Wishbringer's Avatar
 
Join Date: Dec 2003
Location: Germany
Posts: 295
Quote:
Originally Posted by Atak_Snajpera View Post
UPDATE: Have tried adding incorrect device id to disable opencl acceleration?
/x264-opencl-device-id 9
with "/x264-opencl-device-id 9" at second encodingserver now I got 100% CPU usage.
So your suggestion works. That x264 task from 2nd encodingserver has 0% GPU usage.
Thanks!


now the weird ones:
with one encodingserver using opencl and one encodingserver not using opencl I got per client 100% CPU usage an 2% GPU usage (GPU usage is unexpected low!)
with both encodingservers using opencl per client I got again around 45% CPU usage and 40-60% GPU usage (again unexpected low, expected 100%)
But the difference in GPU usage between one and two servers is remarkable, and curious.

HDD usage can't be the reason: using 1TB 970 Evo M.2 NVMe with around 3.5 GB/Sek reading and 2 GB/Sek writing.
RAM is 64GB DDR4-3200-16-16-18-38
Network is 1GBit per client over HPE 1920S-48 switch (at moment only 3 pcs on it)

the only result I got like expected :
Both encodingservers without opencl gave 100% CPU and 0% GPU usage.

Last edited by Wishbringer; 20th December 2018 at 14:33.
Wishbringer is offline   Reply With Quote
Old 20th December 2018, 15:12   #16495  |  Link
Ryushin
Registered User
 
Ryushin's Avatar
 
Join Date: Mar 2011
Posts: 431
Updates Not Working

Atak, not sure if you were aware that updates are not working right now:

2018-12-20 06:46:42 : =========================[UPDATER ACTIVATED]=========================
2018-12-20 06:46:42 : Looking for correct UUID link in http://atak-snajpera.5v.pl/
2018-12-20 06:46:43 : [SUCCESS] http://atak-snajpera.5v.pl/ has correct UUID link 6c966b28-e0dd-48f6-b1c7-a56e8a275ec0
2018-12-20 06:46:43 : Downloading update file http://atak-snajpera.5v.pl/ripbot264update/update.zip
2018-12-20 06:46:43 : [ERROR] Unable to download update file http://atak-snajpera.5v.pl/ripbot264update/update.zip
2018-12-20 06:46:43 : Searching for UUID link with DuckDuckGo...
2018-12-20 06:46:44 : Search results...
2018-12-20 06:46:44 : http://atak-snajpera.5v.pl
2018-12-20 06:46:44 : Looking for correct UUID link in http://atak-snajpera.5v.pl
2018-12-20 06:46:44 : [SUCCESS] http://atak-snajpera.5v.pl has correct UUID link 6c966b28-e0dd-48f6-b1c7-a56e8a275ec0
2018-12-20 06:46:44 : [ERROR] Unable to download update file http://atak-snajpera.5v.pl/ripbot264update/update.zip
2018-12-20 06:46:44 : No results.

I was hoping for an update to fix the OpenCLInfo hang problem.
Ryushin is offline   Reply With Quote
Old 20th December 2018, 15:28   #16496  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,806
Quote:
Originally Posted by Wishbringer View Post
with "/x264-opencl-device-id 9" at second encodingserver now I got 100% CPU usage.
So your suggestion works. That x264 task from 2nd encodingserver has 0% GPU usage.
Thanks!


now the weird ones:
with one encodingserver using opencl and one encodingserver not using opencl I got per client 100% CPU usage an 2% GPU usage (GPU usage is unexpected low!)
with both encodingservers using opencl per client I got again around 45% CPU usage and 40-60% GPU usage (again unexpected low, expected 100%)
But the difference in GPU usage between one and two servers is remarkable, and curious.

HDD usage can't be the reason: using 1TB 970 Evo M.2 NVMe with around 3.5 GB/Sek reading and 2 GB/Sek writing.
RAM is 64GB DDR4-3200-16-16-18-38
Network is 1GBit per client over HPE 1920S-48 switch (at moment only 3 pcs on it)

the only result I got like expected :
Both encodingservers without opencl gave 100% CPU and 0% GPU usage.
What happens with CPU and GPU usage if you run single encodingserver with opencl ON and then during conversion activate extra server with Opencl OFF?
Atak_Snajpera is offline   Reply With Quote
Old 21st December 2018, 12:00   #16497  |  Link
gryph
Registered User
 
Join Date: Nov 2018
Posts: 7
Quote:
Originally Posted by gryph View Post
Finally had some free time to make some samples.
You can get them here: https://mega.nz/#F!8Tom3YjL!3QWwKc6cwiwQf-C3WxCZKw
There are 4 files in total.



I always use batch convert... so I did the same for this sample.
These are the settings used for the sample encode.



And as you'll see, the "bug" only happens when I use .ass subtitles (which is all I do use )
However, once the job is created and you go to the job-folder, the subtitle is fine - all lines are there.
Somehow the last line goes missing doing the encode.

Did a few quick test with older versions of RB
RipBot264v1.21.0 = Same result. Last line is missing.
RipBot264v1.20.0 = Works! All lines are encoded.
RipBot264v1.19.6 = Works! All lines are encoded.
RipBot264v1.19.4 = Works! All lines are encoded.

So it seems something changed from v1.20.0 to v1.21.0
Now... I need coffee! Please let me know if there's anything else I can do to help sort this issue.

Gryph
Noone have any ideas to why this is happening?
Or to what have changed from those two versions?
Saw a post about v1.20.0 was the last 32-bit version and v1.21.0 beeing 64-bit or something, but can't find that post nomore.
Dunno if this would change anything(?)
gryph is offline   Reply With Quote
Old 21st December 2018, 18:28   #16498  |  Link
Wishbringer
Silent Reader
 
Wishbringer's Avatar
 
Join Date: Dec 2003
Location: Germany
Posts: 295
Quote:
Originally Posted by Atak_Snajpera View Post
What happens with CPU and GPU usage if you run single encodingserver with opencl ON and then during conversion activate extra server with Opencl OFF?
Not possible at moment, am now at vacation. Around 02.01.19 back.
Wishbringer is offline   Reply With Quote
Old 21st December 2018, 19:21   #16499  |  Link
gryph
Registered User
 
Join Date: Nov 2018
Posts: 7
Update on my own issue:
Decided to downgrade to 1.20.0 and disable the updater. This turned out to be a great idea!
The bug with the subs is gone and even better is, I now get 40 FPS encodes up form 13.5 - 15 FPS.
Whatever the diffrences is on the versions above 1.20.0 there seems to be some huge changes.
As I don't care for H265/4K stuff I don't know if there's a boost to this encode form - which some have been complaining about taking too long.
Try downgrading and see if that changes things for you guys aswell.
Anyways... Good luck!

Gryph
gryph is offline   Reply With Quote
Old 22nd December 2018, 21:08   #16500  |  Link
ReinerSchweinlin
Registered User
 
Join Date: Oct 2001
Posts: 454
Hey Gryph,
do you have some OPNECL Filters enabled?
ReinerSchweinlin is offline   Reply With Quote
Reply

Tags
264, 265, appletv, avchd, bluray, gui, iphone, ipod, ps3, psp, ripbot264, x264 2-pass, x264 gui, x264_64, x265, xbox360

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 13:29.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.