Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion. Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules. |
19th January 2010, 03:20 | #1 | Link |
Registered User
Join Date: Aug 2009
Posts: 6
|
Looking for multi-stream SD H.264 encoding hardware
Hello,
I am working on software component for realtime encoding of a bunch of synthesized video streams. Full requirements on target system component are as follows: Input: 50+ uncompressed (YUV or RGB) video streams of SD quality (640x480, 25fps). May be represented as memory buffers, socket streams or whatever. Output: AVC-encoded (preferably main profile) video streams of low (like x264 ultrafast preset) or medium (like x264 fast preset) quality. Target bandwidth is about 512 - 1024 kbps. Low latency (< 500 ms) is preferred but may be considered as a trade-off for other requirements. Output representation may vary but RTP streaming seems to be the most convenient way. I have developed a reference implementation spawning slave VLC process for each stream with x264 as encoder. Quality and other requirements are roughly fulfilled so the main concern is an overall system cost and reliability. I haven't done all necessary testing yet but it seems we need something like one or more multi-processor Xeon PCs (roughly estimating 2 low quality streams per 1 core). Reliability of VLC is also questionable. I am wondering if there is a hardware solution meeting my specification which may reduce cost or improve other properties of VLC-based solution. The best solution I can imagine is a bunch of PCI/PCIE cards capable of multi-stream SD encoding. I have found feature-rich encoding servers from HaiVision, Hivionics, ATEME, AdvancedDigital, TANDBERG, Elecard etc capable of multi-stream encoding but all of them seems heavy-weight and far to expensive. Can somebody give me a hint on searching cost-effective solution for my problem? Thanks. |
19th January 2010, 04:04 | #2 | Link |
x264 developer
Join Date: Sep 2005
Posts: 8,666
|
x264 can easily do 10-20 SD streams on a single Core i7. 2-4 Core i7s are far, far cheaper than any decent hardware solution.
Remember the rule of hardware; there are four types: 1. Cheap, featureless, terrible compression. 2. Expensive, mediocre features, slightly less terrible compression. 3. Really expensive, good features, mediocre compression. 4. Really expensive, good features, pretty good compression. Do also remember that most hardware solutions have much worse compression than x264, even on "veryfast", let alone "fast" or "medium". Furthermore, low latency is often a feature that cheaper systems won't have, while x264 has a large set of low latency-streaming features.
__________________
Follow x264 development progress | akupenguin quotes | x264 git status ffmpeg and x264-related consulting/coding contracts | Doom10 Last edited by Dark Shikari; 19th January 2010 at 04:07. |
20th January 2010, 16:37 | #3 | Link |
Registered User
Join Date: Aug 2009
Posts: 6
|
Thank you for your encouraging answer. I am going to continue evaluation of x264-based solution on dedicated i7-950 machie and if it is capable of handling of more then 10 simultaneous streams it would be the most promising solution.
|
20th January 2010, 20:17 | #4 | Link |
Derek Prestegard IRL
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,989
|
Instead of an i7 950, consider a dual Xeon 5520 box. This is basically the cheapest way to get 2 Core i7 CPUs for 8 physical and 16 logical cores. Plus, you can but hordes of RAM in it so you can run 20 streams in real-time without worrying about memory usage! A 64 bit OS is obviously mandatory, in any case You can get a box like this for under 2 grand...
~MiSfit
__________________
These are all my personal statements, not those of my employer :) |
20th January 2010, 21:01 | #5 | Link |
Registered User
Join Date: Dec 2008
Posts: 589
|
299.99$ x 1 http://www.newegg.com/Product/Produc...82E16813151085
199.99$ x 4 http://www.newegg.com/Product/Produc...82E16819105235 29.99$ x 4 http://www.newegg.com/Product/Produc...82E16835114078 109.99$ x 8 http://www.newegg.com/Product/Produc...82E16820134936 279.99$ x 1 http://www.newegg.com/Product/Produc...82E16811165196 154.99$ x 1 http://www.newegg.com/Product/Produc...82E16817815003 ------ 2535 $ for 4 x Quad Core 2.1 Ghz, 16 GB of memory. Add hard drives. So you have 16 cores that can probably do 3-4 simultaneous 640x480 SD encodings per core for a total of about 40 and the motherboard gives you 3 gigabit network cards so you could stream uncompressed videos directly. Will probably beat any hardware solution and dedicated hardware equipment. ps. this particular model of motherboard has an extension module that you can use to add 4 additional processors for a total of 8, so you'd have 8 x quad core processors. you'll need 2 x 850w power supplies though, or at least one 1700w power supply according to the reviews on the product page. ps/disclaimer. I did not check if the motherboard is compatible with the processors and the memory I selected. What I wrote is just a suggestion, it's your duty to check the motherboard manual and see what's supported or not. Last edited by mariush; 20th January 2010 at 21:11. |
21st January 2010, 16:45 | #6 | Link | |
Registered User
Join Date: Jan 2002
Posts: 9
|
Quote:
You need 8300 series Opteron for 4 x socket Mobos, but those processors are highly priced. Advising to stick on 8 (2 x CPU) core Xeon or 12 core (2 x CPU) Opteron |
|
21st January 2010, 17:20 | #7 | Link |
Registered User
Join Date: Dec 2008
Posts: 589
|
hmmm... that's why i added the disclaimer there, I knew the prices were a bit cheap... I remember when I bought a xeon processor for a dual cpu motherboard and was about 600$.... the processors in the 8xxx series are about 1500$ and up: http://ncix.com/products/index.php?m...minorcatid=172
So i guess you could use for example this motherboard: http://ncix.com/products/index.php?s...ure=SuperMicro http://www.supermicro.com/Aplus/moth...55/H8DA3-2.cfm at around 560 Canadian$ with two of these http://ncix.com/products/index.php?s...anufacture=AMD at 339 Canadian $ each again disclaimer/nitpick/whatever: i'm picking these because they're both available at a well known online store and I assume the motherboard supports the cpu because there's no cpu support list on the supermicro website, though it says the MB supports all 2000 series quad cores and six cores (with a bios update also, i don't have any preference for amd vs intel, it's just that i started with amd in my previous post so i keep going with it later edit.. ok, let's add intel alternative.. 384.99$ x 1 http://www.newegg.com/Product/Produc...82E16813131373 384.99$ x 2 http://www.newegg.com/Product/Produc...82E16819117185 (yes, they're the same price. of course, you have to hunt for coolers as the processors don't come with it) Last edited by mariush; 21st January 2010 at 17:32. Reason: bad intel mb url, was same as cpu |
22nd January 2010, 03:16 | #8 | Link |
Derek Prestegard IRL
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,989
|
My advice for this kind of solution would be to buy something from Dell / HP. They are priced very competitively, are well supported, and "just work" out of the box. Building a server with off the shelf components requires extremely careful scrutinizing of the components to ensure perfect compatibility etc..
I'd say stick with Intel CPUs. The i7 is a real x264 monster. ~MiSfit
__________________
These are all my personal statements, not those of my employer :) |
22nd January 2010, 14:35 | #9 | Link | |
Registered User
Join Date: Nov 2004
Location: Poland
Posts: 2,843
|
Quote:
Andrew |
|
22nd January 2010, 22:04 | #11 | Link |
Derek Prestegard IRL
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,989
|
Supermicro is indeed an excellent alternate vendor. They scale into 4 (and more) socket systems well also, but that gets pricey
~MiSfit
__________________
These are all my personal statements, not those of my employer :) |
24th January 2010, 19:02 | #12 | Link |
Registered User
Join Date: Feb 2003
Location: Palmcoast of Norway
Posts: 363
|
Im looking at something similar at work where i have 1 HD-SDI stream input and i wanna try x264 as a RT encoder for 1080i. My biggest consern is how to do this with audio in synch and output the lot as a Transportstream in mapping it to a UDP/Multicast stream. Anyone have a suggestion on how to design this?
br TE Last edited by TEB; 24th January 2010 at 19:05. |
24th January 2010, 21:32 | #13 | Link |
Derek Prestegard IRL
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,989
|
I Imagine the difficult part will be getting the HD-SDI feed somehow into ffmpeg or a similar tool - which I know nothing of
Once you get it into there (probably as a stream of uncompressed audio and video), you should easily be able to specify encoding settings, and store the output in a transport stream, or pipe the output to some kind of streaming software. ~MiSfit
__________________
These are all my personal statements, not those of my employer :) |
25th January 2010, 15:51 | #17 | Link |
Registered User
Join Date: Aug 2009
Posts: 6
|
Thank you everybody for your advices.
We will closely consider multi-processor Xeon 5520 systems and similar solutions. For now I have done some naive tests to estimate multi-stream capabilities of Core i7-950 in comparison with my Athlon X2 7850. I haven't manually configured x264 much but I assume that this is enough for rough estimation: Code:
profile main preset ultrafast/veryfast/faster/fast bframes 1 keyint 25 keyint_min 5 Code:
preset/CPU Core i7-950 Athlon X2 7850 Speed gain ultrafast (22, 40) (8, 11 ) (2.8, 3.6) veryfast (18, 30) (5, 8 ) (3.6, 3.8) faster (9, 15) (2.5, 4.5) (3.6, 3.3) fast (6, 10) (1.5, 3.5) (4, 2.9) I have also tried CUDA encoders with slightly surprising results: Code:
encoder/GPU GeForce GTX295 GeForce GTX260 NVCuVEnc (16, 20 ) (6.5, 8.5) Badaboom (4.5, 6.5) (4.5, 6.5) |
25th January 2010, 19:53 | #18 | Link | |
Registered User
Join Date: Mar 2006
Posts: 272
|
Quote:
how Does this generic HD-SDI stream output ?, some form of V4Lv2 !, then perhaps someone should take the time as a side project way to get into, and understand the main x264 code, and add that V4Lv2 and related updated RFC's as another input for x264, given its perhaps soon ready for a mass of industial uses too sooner or later. http://www.spinics.net/lists/linux-media/msg09797.html http://www.linuxtv.org/downloads/vid...ngle/v4l2.html http://www.linuxtv.org/hg/v4l-dvb http://x264dev.multimedia.cx/?p=270 "01/17/2010 (10:40 pm) What’s coming next in x264 development Filed under: development, low latency, ratecontrol, speed, streaming, x264 :: As seen in the previous post, a whole category of x264 improvements has now been completed and committed. So, many have asked–what’s next? Many major features have, from the perspective of users, seemingly come out of nowhere, so hopefully this post will give a better idea as to what’s coming next. Big thanks to everyone who’s been helping with a lot of the changes outside of the core encoder. This, by the way, is one of the easiest ways to get involved in x264 development; while learning how the encoder internals work is not necessarily that difficult, understanding enough to contribute useful changes often takes a lot of effort, especially since by and large the existing developers have already eliminated most of the low-hanging fruit. By comparison, one can work on x264cli or parts of the libx264 outside the analysis sections with significantly less domain knowledge. This doesn’t mean the work is any less difficult–only that it has a lower barrier to entry. .... more snip " Last edited by popper; 25th January 2010 at 20:24. |
|
25th January 2010, 20:57 | #19 | Link |
Derek Prestegard IRL
Join Date: Nov 2003
Location: Los Angeles
Posts: 5,989
|
CUDA quality similar to --preset veryfast???? Last I heard, CUDA encoders were beyond terrible. I would call performance like this "not so terrible"!
I think the CUDA stuff is useful for quick transcodes for portable devices etc, but not much else. I'd be interested to see some test encodes, if you can post them ~MiSfit
__________________
These are all my personal statements, not those of my employer :) |
25th January 2010, 21:49 | #20 | Link |
Registered User
Join Date: Feb 2003
Location: Palmcoast of Norway
Posts: 363
|
Hi. Evedently more is happening in x264 that i even could dream of..
If someone as u say are interested in implementing DVB-TS support internally, AAC and Mpeg1-L2 support and would be the start on making the worlds first opensource RT H264 encoder. If we could implement some kind of v4l input support and UDP multicast read support and multicast output support that would be even better |
Tags |
avc, encoder, h.264, hardware, multi-stream |
Thread Tools | Search this Thread |
Display Modes | |
|
|