Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 Encoder GUIs

Reply
 
Thread Tools Search this Thread Display Modes
Old 5th November 2019, 00:30   #17661  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,425
Quote:
Originally Posted by Ryushin View Post
Atak,

Going to buy a temporary CPU from Ebay. Wondering if it's more important to have hertz or cores for Ripbot. Looking at a 6 core 2.6Ghz or a 8 core 2.0Ghz.

Found out my current CPU goes for $6.00 now on Ebay. LOL
2.6 * 6 = ~16
2.0 * 8 = 16
In this case i would choose higher clock .
Atak_Snajpera is offline   Reply With Quote
Old 5th November 2019, 00:45   #17662  |  Link
mparade
Registered User
 
Join Date: Nov 2013
Posts: 576
Quote:
Originally Posted by Atak_Snajpera View Post
By the way. Are you using VPN built in windows or you use other custom software? What are other ips detected in client IP combobox? What does Adapter description show for each extra IP?
Admin installed ZyWALL IPSec VPN Client on my home-pc.
10.10.10.10 the other ip detected in client IP combobox.
Adapter description includes two options regardless of ip chosen in client IP combobox:

1.Intel(R) I211 Gigabit Network Connection
2. TheGreenBow Virtual Miniport Adapter
mparade is offline   Reply With Quote
Old 5th November 2019, 01:42   #17663  |  Link
Ryushin
Registered User
 
Join Date: Mar 2011
Posts: 279
Quote:
Originally Posted by mparade View Post
Admin installed ZyWALL IPSec VPN Client on my home-pc.
10.10.10.10 the other ip detected in client IP combobox.
Adapter description includes two options regardless of ip chosen in client IP combobox:

1.Intel(R) I211 Gigabit Network Connection
2. TheGreenBow Virtual Miniport Adapter
Just to let you know, I've seen some VPN's tunnel ALL traffic down the VPN. Local traffic (such as network printers) could no longer be accessed. I don't like these types of VPNs at all and I set up firewalls and VPNs for a living. I understand their usage for extreme security, but not having access to your local network, c'mon.
Ryushin is offline   Reply With Quote
Old 5th November 2019, 03:29   #17664  |  Link
Pauly Dunne
Grumpy Old Man.
 
Pauly Dunne's Avatar
 
Join Date: Jul 2019
Location: Out There....
Posts: 308
4K demuxing time test's (if anyone's interested)

So I have noticed a few posts about very long times to start a 4K job, so I thought I'd do a couple of tests for myself.

I chose Harry Potter #1, which is a remux 4K, 60.7Mb/s, 84.5Gb mkv.

I also ran it thru MKVToolnix to rip out EVERYTHING but the video, which took 4:30 mins (Raid-0 to nvme)

Also wanted to see what difference there was between LSmash & FFMS2.

So, chose the untouched mkv to start with, and it took 4:30 mins to demux, and a further 4:20 mins until the job creation was ready, with LSmash.

Using FFMS2 it took 4:30 mins to demux, and further 3:50 mins 'til job was ready.

Then the stripped mkv took 0 mins to demux, and a total of 3:25 mins with LSmash.

Same file using FFMS2 took 0 mins to demux, and a total of 3:12 mins.

Once they were queued up, it was time to see how long it took to copy the file, get the chunks created, etc, etc.

The untouched job took 12:00 mins 'til the chunks were all ready to go.

With the stripped mkv, it took 8:45 mins...not forgetting the it took 4:30 mins to strip out the file in the first place, so it's close'ish.

All these figures are having the original file on a RAID-0 of 2 x Seagate 3Tb 7,200 rpm spinners, copying to an ADATA NVM'e, using a single Xeon 12c cpu.

Now, I cannot provide any final muxing, etc info at this time, as that will take a couple of days for me to get around to do that, but I WILL post my results.

So regardless of how long that's going to take, there WILL be a reasonable amount of time that all DE servers aren't doing anything.
__________________
Not poorly done, just doin' it my way !!!
Pauly Dunne is offline   Reply With Quote
Old 5th November 2019, 10:05   #17665  |  Link
howzz
Registered User
 
Join Date: Jun 2008
Posts: 41
@Atak_Snajpera

i noticed that in the latest update, you included an option for user to select decoding method. although Lmash is slow, on my NvMe it's still fast for demuxing, indexing etc.
but my question is, is there any real speed difference in using GPU for "decoding" method?

btw, i see a lot of folks are discussing about NvMe here. i was an early adapter of NvMe for my 4K workflow, trust me, invest in a couple good NvMe drives. it's worth it. and do yourself a favor, don't go below 1TB. my current 4K workflow, nothing takes longer than 4 mins for initial demux, indexing, gathering information altogether. most 4K UHD sources take 3~4 mins total to do all that. if you have the source sits on the same NvMe drive, it'll only take 3~4 mins.

the trick with shopping for NvMe drives is look for TLC drives and avoid QLC drives at all cost. QLC's raw writing speed is less than 100 MB/s. i've tried them all and went through them all and experience all of it the hard way. once the SLC cache run out, QLC's raw writing speed will fall below 100 MB/s sometimes 70 MB/s, that's slower than most high end 7200rpm spindle. and smaller the drive, smaller the SLC casche will be. on a 1TB QLC drive, your SLC cache is probably gonna be about less than 20GB when it's filled at 50%. the intel 660P 2TB drive, theoretically, if you over provision it and treat it as a 1.5TB drive, theoretically you'll always have more than 50GB of SLC caching. but just do yourself a favor, avoid all the QLC drives out there. go for a fast TLC drive from Corsair, Sillicon power, Samsung, and you'll be a lot happier. most TLC's raw writing speed, provided if the onboard processor is of recent version, will be around 600~800 MB/s even after it runs out of its SLC caching. so when doing 4K UHD workflow working with large 60~70 GB files, you're going to be counting on the raw writing speed. when shopping around, look for reviews that do tests on large 25~50 GB file "write" tests, and the chart will show you the speed difference once those SLC cache runs out.

if money is of no object, Samsung is the way to go. but otherwise get one of those Corsair MP510, or silicon power TLC drives. 1~2TB will be most ideal. and make sure that your mobo supports PCIe 3.0 x4 M.2, even though realistically, ripbot workflow will never really exceed 1500 MB/s throuput, just under the max bandwidth of PCIe 2.0 x4

Last edited by howzz; 5th November 2019 at 10:07.
howzz is offline   Reply With Quote
Old 5th November 2019, 10:21   #17666  |  Link
duffbeer
Registered User
 
Join Date: Mar 2019
Posts: 39
Quote:
Originally Posted by Pauly Dunne View Post
So I have noticed a few posts about very long times to start a 4K job, so I thought I'd do a couple of tests for myself.

I chose Harry Potter #1, which is a remux 4K, 60.7Mb/s, 84.5Gb mkv.

I also ran it thru MKVToolnix to rip out EVERYTHING but the video, which took 4:30 mins (Raid-0 to nvme)

Also wanted to see what difference there was between LSmash & FFMS2.

So, chose the untouched mkv to start with, and it took 4:30 mins to demux, and a further 4:20 mins until the job creation was ready, with LSmash.

Using FFMS2 it took 4:30 mins to demux, and further 3:50 mins 'til job was ready.

Then the stripped mkv took 0 mins to demux, and a total of 3:25 mins with LSmash.

Same file using FFMS2 took 0 mins to demux, and a total of 3:12 mins.

Once they were queued up, it was time to see how long it took to copy the file, get the chunks created, etc, etc.

The untouched job took 12:00 mins 'til the chunks were all ready to go.

With the stripped mkv, it took 8:45 mins...not forgetting the it took 4:30 mins to strip out the file in the first place, so it's close'ish.

All these figures are having the original file on a RAID-0 of 2 x Seagate 3Tb 7,200 rpm spinners, copying to an ADATA NVM'e, using a single Xeon 12c cpu.

Now, I cannot provide any final muxing, etc info at this time, as that will take a couple of days for me to get around to do that, but I WILL post my results.

So regardless of how long that's going to take, there WILL be a reasonable amount of time that all DE servers aren't doing anything.
Now try it with full 4K UHD disc as the source. You will see how much longer it takes.
duffbeer is offline   Reply With Quote
Old 5th November 2019, 10:33   #17667  |  Link
Pauly Dunne
Grumpy Old Man.
 
Pauly Dunne's Avatar
 
Join Date: Jul 2019
Location: Out There....
Posts: 308
Quote:
Originally Posted by duffbeer View Post
Now try it with full 4K UHD disc as the source. You will see how much longer it takes.
Haven't got access to any "full" disk's, unless I acquire a UHD rip as an .iso.

But why would there be a big difference ?? Is it the ripping process that takes so long ??

Do you rip it first, or let RB do that (if it can) ??

Ripping Bluray disk's was a PITA, I'm guessin' 4K are SO much worse.
__________________
Not poorly done, just doin' it my way !!!

Last edited by Pauly Dunne; 5th November 2019 at 10:36.
Pauly Dunne is offline   Reply With Quote
Old 5th November 2019, 12:22   #17668  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,425
Quote:
but my question is, is there any real speed difference in using GPU for "decoding" method?
IT is complicated..






Quote:
most TLC's raw writing speed, provided if the onboard processor is of recent version, will be around 600~800 MB/s even after it runs out of its SLC caching.
TLC is still pathetic in comparison to old good MLC. Write speed on Samsung 970 PRO is ~2.5 GiB/s (No SLC cache BS)!

Last edited by Atak_Snajpera; 5th November 2019 at 15:18.
Atak_Snajpera is offline   Reply With Quote
Old 5th November 2019, 14:24   #17669  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,425
Quote:
Originally Posted by Ryushin View Post
Just to let you know, I've seen some VPN's tunnel ALL traffic down the VPN. Local traffic (such as network printers) could no longer be accessed. I don't like these types of VPNs at all and I set up firewalls and VPNs for a living. I understand their usage for extreme security, but not having access to your local network, c'mon.
Since you have been doing this VPN stuff for living I would like you to ask something.

When you connect to remote LAN via VPN then shouldn't home-pc get the same three sections of IP?





My common sense tells me that after connecting to remote LAN Client IP should be starting from 192.168.2.xxx
Atak_Snajpera is offline   Reply With Quote
Old 5th November 2019, 16:52   #17670  |  Link
byteshare
ByteShare
 
byteshare's Avatar
 
Join Date: Sep 2014
Location: On the Internet
Posts: 560
Quote:
Originally Posted by Ryushin View Post
Just to let you know, I've seen some VPN's tunnel ALL traffic down the VPN. Local traffic (such as network printers) could no longer be accessed. I don't like these types of VPNs at all and I set up firewalls and VPNs for a living. I understand their usage for extreme security, but not having access to your local network, c'mon.
Which VPN solution would you recommend for setting up RipBot to have remote servers?
byteshare is offline   Reply With Quote
Old 5th November 2019, 17:33   #17671  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,425
Quote:
Originally Posted by byteshare View Post
Which VPN solution would you recommend for setting up RipBot to have remote servers?
Just use builtin in windows.
Atak_Snajpera is offline   Reply With Quote
Old 5th November 2019, 19:38   #17672  |  Link
NasaRacer
Registered User
 
Join Date: Apr 2019
Posts: 2
Not sure if this is a known issue but on a few of my recent 4k UHD Mkvs, the file Demuxing and indexing are normal. Then during "gathering information" it fails with the message "FFAudioSource: No audio track found
(C:\Temp\RipBot264temp\job1\getinfo.avs, line 4)" So using MKVToolNix I made a shorter version of the same file 1min of the file and then another with 10min of the file. The 1min version works but the 10min version fails with the same error. Then I made a full version of the file with the video only and no audio track and that is working okay as well. I used XMedia Recode to encode just the audio and that worked correctly. So I am not sure what the issue is.
The audio is Dolby TrueHD 7.1 both movies in question are from Disney.
NasaRacer is offline   Reply With Quote
Old 5th November 2019, 19:52   #17673  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,425
Quote:
Originally Posted by NasaRacer View Post
Not sure if this is a known issue but on a few of my recent 4k UHD Mkvs, the file Demuxing and indexing are normal. Then during "gathering information" it fails with the message "FFAudioSource: No audio track found
(C:\Temp\RipBot264temp\job1\getinfo.avs, line 4)" So using MKVToolNix I made a shorter version of the same file 1min of the file and then another with 10min of the file. The 1min version works but the 10min version fails with the same error. Then I made a full version of the file with the video only and no audio track and that is working okay as well. I used XMedia Recode to encode just the audio and that worked correctly. So I am not sure what the issue is.
The audio is Dolby TrueHD 7.1 both movies in question are from Disney.
Can you send me that not working 10 minute version?
Atak_Snajpera is offline   Reply With Quote
Old 5th November 2019, 20:01   #17674  |  Link
Ryushin
Registered User
 
Join Date: Mar 2011
Posts: 279
Quote:
Originally Posted by Atak_Snajpera View Post
Since you have been doing this VPN stuff for living I would like you to ask something.

When you connect to remote LAN via VPN then shouldn't home-pc get the same three sections of IP?

My common sense tells me that after connecting to remote LAN Client IP should be starting from 192.168.2.xxx
A lot of VPN software/gateways will use a IP subnet to connect to, then the VPN gateway will route the traffic to the appropriate network.

For example, I use OpenVPN for and Fortigate for my VPNs. The IP block that you use for the VPN users should be almost a random block because you do not want your VPN users to have the same block as you are running on your networks. In fact, to go further, any networks that are behind your VPNs, you will not want them to be commonly used for home networks.

I try and stay away from the 192.168.*.* blocks for business use. Most likely the home users are using 192.168.0.* and 192.168.1.*, but not always.

An example set up simple corporate network.
Internal Networks: 10.20.10.0/24
DMZ: 10.20.30.0/24
Secured Development: 10.20.40.0/24
VPN Network: 172.27.99.0/24

A road warrior or home user would connect to the VPN. They will get a 172.27.99.* for their VPN address. If I have configured a split VPN, then only the networks behind the VPN gateway will have routes added to the VPN client. The VPN client will still be able to access their local traffic and get access to the Internet without going through the VPN. If I configured Tunnel Mode VPN, and can force ALL traffic to go through the VPN or just allow the client to have access to their local network, but any traffic outside of their local network gets tunneled down the VPN.

There are some VPNs that will bridge traffic and make the VPN client seem like their are on one of the local work networks, but these are not done much any more.

Does RipBot assume everything is flat network (Layer2) or does it allow routing to take place (Layer 3) and jump networks to appropriate routed destination.

Does RipBot bind to single IP address or does it bind to all the local addresses on the machine. I wondering if that could be causing issues if it is binding to a single address as this will cause issues with VPNs.

Did I understand the question right?

As for what VPN I recommend, that would be OpenVPN. I would look at using a Raspberry Pi and a firewall distro, or a wireless router that comes bundled with a OpenVPN gui. Learning OpenVPN, especially, with everything it can do, is not for the faint of heart.
Ryushin is offline   Reply With Quote
Old 5th November 2019, 20:57   #17675  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,425
Quote:
Does RipBot bind to single IP address or does it bind to all the local addresses on the machine. I wondering if that could be causing issues if it is binding to a single address as this will cause issues with VPNs.
It binds to one specified IP for encodingclient (selected in Client IP combobox) and encodingserver (selected via command line).

According to mparade post here https://forum.doom9.org/showthread.p...85#post1889585
other ip is 10.10.10.10 - TheGreenBow Virtual Miniport Adapter. Does not look like anything useful...
Atak_Snajpera is offline   Reply With Quote
Old 5th November 2019, 22:10   #17676  |  Link
howzz
Registered User
 
Join Date: Jun 2008
Posts: 41
Quote:
Originally Posted by Atak_Snajpera View Post
IT is complicated..







TLC is still pathetic in comparison to old good MLC. Write speed on Samsung 970 PRO is ~2.5 GiB/s (No SLC cache BS)!
are these numbers in FPS i assume?

my suggestion for the TLC was because realistically, for cost reasons. But if money is no object, the Samsung Pro really is the way to go. but for those building large capacity 1TB/2TB drives for NvMe, TLC is a pretty fast compromise imo.

edit: are there even any 2TB samsung MLC NvMes? at least i can't find any.

Last edited by howzz; 5th November 2019 at 22:15.
howzz is offline   Reply With Quote
Old 5th November 2019, 22:44   #17677  |  Link
howzz
Registered User
 
Join Date: Jun 2008
Posts: 41
and secondly, i forgot to ask. is there any quality lost if choosing GPU as the decoder as oppose to Lmash.
i mean decoding is decoding right? quality is affected during the x265 "encoding" calculations.
howzz is offline   Reply With Quote
Old 5th November 2019, 22:57   #17678  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,425
Quote:
Originally Posted by howzz View Post
and secondly, i forgot to ask. is there any quality lost if choosing GPU as the decoder as oppose to Lmash.
i mean decoding is decoding right? quality is affected during the x265 "encoding" calculations.
No quality penalty. GPU just provides frames to encoder.
Atak_Snajpera is offline   Reply With Quote
Old 6th November 2019, 04:01   #17679  |  Link
Ryushin
Registered User
 
Join Date: Mar 2011
Posts: 279
Quote:
Originally Posted by Atak_Snajpera View Post
It binds to one specified IP for encodingclient (selected in Client IP combobox) and encodingserver (selected via command line).

According to mparade post here https://forum.doom9.org/showthread.p...85#post1889585
other ip is 10.10.10.10 - TheGreenBow Virtual Miniport Adapter. Does not look like anything useful...
Can you change Ripbot so there is an option to bind to all IPs on the local machine which I think should be the default? I also run IPv6, so it will be nice to just put a DNS name in instead of a IP. Also, me being me, to be able to bind to certain IPs is also very beneficial and I do that all the time for machines that are connected on multiple networks, especially if one of them is Internet facing and I want to only provide a service to internal clients.
Ryushin is offline   Reply With Quote
Old 6th November 2019, 05:50   #17680  |  Link
Pauly Dunne
Grumpy Old Man.
 
Pauly Dunne's Avatar
 
Join Date: Jul 2019
Location: Out There....
Posts: 308
Greatly improved muxing times :)

So today I was able to test a reasonably sized 4K movie..

I chose to do "Frozen", which I stripped out the audio, as I don't need/want TrueHD (turned out the thd track was corrupted anyway).

So the file ended up being 42.5Gb @ 56.8Mb/s, NO audio, or subs !!

After the encoding if was 35.25Gb @ 47.1Mb/s, using CRF 12.

After the initial loading of the job, it took a further 3 hrs:37 m to complete the encode.

The combining took 1:45 mins, and the muxing took 2:55 mins for a total of only 4:40 mins.

A lot better than this :-

..the combining of the chunks took 10 minutes 30, then the muxing process took a further 15 minutes 30 .. (from post #17533)

which was admittedly a larger file with all it's tracks, but on slow SSD's.

Now I would think that "Frozen" would have taken a little longer if it had to do EVERYTHING, but I did the final "mux" with MKVToolnix, to add audio & subs.
__________________
Not poorly done, just doin' it my way !!!
Pauly Dunne is offline   Reply With Quote
Reply

Tags
264, 265, appletv, avchd, bluray, gui, iphone, ipod, ps3, psp, ripbot264, x264 2-pass, x264 gui, x264_64, x265, xbox360

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 00:38.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.