Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 AVC / H.264

Reply
 
Thread Tools Search this Thread Display Modes
Old 14th October 2014, 20:16   #1  |  Link
fredrum
Registered User
 
Join Date: Oct 2014
Posts: 5
Noob Searching for better interactive stream settings

Hi All,
this is my very first post on this forum and I am also a noob at looking into the nitty gritty internals of h264 video stream making/decoding. So sorry in advance for my ignorance, I am trying to read up on articles and are doing a lot of searches on this forum and elsewhere.

I am looking for any advice into improving a 'realtime' streaming program I am trying to put together. I am basing it off this article here that seems widely referenced, http://x264dev.multimedia.cx/archives/249 and I started by using pieces of the code here https://github.com/oau/streamer.


My absolute priority is Latency. (Then of course, image quality)
Source materials might vary from screen sharing to films. (probably have to have different sets of settings?)

That article and also the code is now a few years old, are they still good guides to how to do this? Or are there new methods or additions I should try out?

My current code is achieving about 67ms from screengrab acquisition through encode, send to localhost client for decode and finally display.
A good start I feel but the article cites that 10ms should be possible and that was some years ago.

The image quality I am having at the moment is not great a lot of the time. Blocky, mushy text and wide compression bands going both horizontally and vertically across the screen now and then.
Sometimes it seems to get 'stuck' and not refresh the image much for a second or two with mushy blocky edges.

Doubtlessly I have to keep learning to understand better but I was wondering if anyone had any general advice?

I would be much grateful for any such.

Cheers
Fred
fredrum is offline   Reply With Quote
Old 15th October 2014, 18:02   #2  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,720
You are measuring latency from what to what? 67 ms is pretty awesome if you're talking about from screen on capture system to screen on another device connected via the public internet! If it's from bitmap being handed to the video codec to getting that frame's encoded representation back, you can probably do better.

There are a whole lot of places where latency can come in (video capture, video codec, audio codec, packaging, upload, transmission, download, decode, playback pipeline, and buffers buffers everywhere). Being able to instrument the latency of different stages is critical. Can you do that?

As for quality, low latency encoding makes lots of quality compromises versus a nice offline file-to-file encode where you can use B-frames, lookahead, more VBV variability, etcetera. So plan to use a substantially higher bitrate than you'd need for an offline encode. To improve quality, your options include: reduce frame size, reduce frame rate, increase encoder complexity (MIPS/pixel). Getting a nice fast multicore processor like a Core i7 Extreme 5930K can offer a lot higher quality and higher height*width*fps than a high end CPU from even a couple of years ago.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 15th October 2014, 19:24   #3  |  Link
fredrum
Registered User
 
Join Date: Oct 2014
Posts: 5
Hi and thanks for the reply!

No I haven't got as far as to try it out over the actual internet yet. I'm just running this on my own home computer. It is a server program and a client program, but just doing it locally for now.

The 67ms is also just a 640x480 stream and my machine is an i7 4(+4 virtual) cores @ 3.4ghz. The reported cpu usage is still pretty low.
I am aiming for 720p at 60fps with no more latency than 1frame over 30, we'll see.

Like you also suggest I yesterday started putting in timers and trying to figure out how much time the different parts are taking.

The numbers seem to vary pretty much and I don't yet know why, I have to look into that. See if upping the priority of the programs/processes might help with that?

But looking at the results it seems like I should be able to do better, as you say. For example the BitBlt() screengrab I am doing is a main offender. Also the inherited source code is doing some timer catchup/sync thing I don't yet fully understand and that has some high numbers.
I am optimistic I should be able to improve the speed/latency a bit further with a bit of work.


But the image quality needs to improve so thanks so much for your suggestions. I actually need to raise resolution and hope to even raise the fps so that seems to leave increasing encoder complexity (MIPS/pixel).

I will look into that now.
Thanks again.


Some example images:



Last edited by fredrum; 15th October 2014 at 21:06.
fredrum is offline   Reply With Quote
Old 16th October 2014, 21:52   #4  |  Link
fredrum
Registered User
 
Join Date: Oct 2014
Posts: 5
Don't know if I should post this as a separate thread or not but will just put it here for now.

I have two questions that I feel would increase my chances of getting a better stream.


1. Now and then (but quite often) the image gets locked in a 'garbled' state. This often happens with big visual changes to the content.
This then gets more 'cleared up' by the sideways scan that I believe is the Periodic Intra Refresh, could that be right?

This process seems very slow, first I often wait a few seconds for it to visually 'kick in'. Secondly it takes it a long time (1-2sec) to scan across the image left to right.

What decides the speed of this and how can I make it faster?


2. The compression seems a lot better in B/W high contrast areas such as black text on white. In my orange text on black more pixels seems to get affected/mushied in a visually bad way. (see pic)

Is there some kind of setting or operation that can help picking out details where color/luminance contrast in not that high?








Last edited by fredrum; 16th October 2014 at 21:54.
fredrum is offline   Reply With Quote
Old 21st October 2014, 20:37   #5  |  Link
Asmodian
Registered User
 
Join Date: Feb 2002
Location: San Jose, California
Posts: 4,197
You need to use 4:4:4 instead of 4:2:0 if you want red text to look reasonable.
Asmodian is offline   Reply With Quote
Reply

Tags
latency, realtime, streaming

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 23:17.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2021, vBulletin Solutions Inc.