Intel QuickSync Decoder - HW accelerated FFDShow decoder with video processing

[egur]

Originally Posted by nevcairiel

I've been thinking about this thing today, and i've been wondering - what exactly does the Media SDK offer over a DXVA2 decoder (assuming you copy the frame back into system ram as well) ?
I'm only interested in actual user visible advantages, i realize coding might be simpler with the SDK, but then DXVA2 works with more GPUs.

PS:
Its not the same as CUDA decoding, CUDA is handled quite differently. As i understand it, the MSDK is just a "wrapper" around DXVA2, hence my question.

As far as I know it should be a more user friendly wrapper. Maybe cleanup stream errors, etc. You know that using DXVA naively doesn't work well.

DXVA is the implementation underneath. Maybe some day, DXVA will be replaced or Media SDK will be enabled on other platforms so using a wrapper speeds porting as well as writing an application. There's also the chance that DXVA will be too limited compared to the HW capabilities and media SDK will wrap another API (I'm guessing here).

There's very little programmable code that runs in the Intel GPU, most of the decoding/emcoding/VPP is done by ASIC (fixed function HW). That's why its so fast even when compared to a 250W GPU.

[CruNcher]

There's very little programmable code that runs in the Intel GPU, most of the decoding/emcoding/VPP is done by ASIC (fixed function HW). That's why its so fast even when compared to a 250W GPU.

Hmm though according to another Intel Engineer (Francois Piednoel , Senior Performance analyst at Intel Corp Santa Clara) there should have been a possibility to use these functions (execute on them) outside in your own Encoder Code for example (not using the whole Intel Quicksync Encoder @ all)
X264 could have benefited from that (direct acceleration on the ASIC) but it never happened to bee sadly.

This "Intel guy" disappeared after Dark Shikari asked him about low level QuickSync API.

Since the original Intel failure, I have learned quite a bit more about the lower-level details, and I'd quite love to explain more, but unfortunately I am now deep into NDA territory. If this means people are going to blame x264 for QuickSync's failings, well, unfortunately there's not much I can legally do about it anymore.

does that mean you are now technically able to allow some parts of x264 encoding to be done by quicksync? If so is this support going to be added?

Maybe yes, probably not. There are some pretty devastating technical limitations.

This could have been a big hit for Intel now AMD seems to be more open and taking the chance of giving full support which is not really surprising seeing they have nothing like Quicksync yet (which @ least can reach x264 superfast quality) (or not confirmed) and only their GPU Encoder which wouldn't be up against it


Egur here are the VC-1 samples:

http://www.mediafire.com/download.php?4m9cb10oms48bv1 <--Frame Interlaced/Progressive VC-1
http://www.mediafire.com/download.php?1uc5b42u55ue280 <-- Field Interlaced VC-1

Both sync problems (MPC-HC Splitter) the Field Interlaced also shows decoding issues. (was before the silent updates rechecking with 0.12)

Nice evil trees.ts plays wonderful smooth even without manually correcting it and with auto interlaced flags send (perfectly telecined) from ffdshow on EVR Custom (so other streams get properly double framerate deinterlaced).
Thats something no current DXVA Decoder can do on EVR Custom (not without losing double framerate deinterlacing)
With Lav Splitter it fails on EVR custom though works only with MPC-HC Splitter for now

[egur]

New and improved version. Zip files contain documentation, please read.

Download version 0.12 alpha:
32 bit http://www.multiupload.com/5L5NL03997
64 bit http://www.multiupload.com/41UGJ3TQMI

Revision highlights:
v1.12:
* 64bit version is working.
* Optimized CPU usage (faster copying from GPU to CPU)
* More stable with LAV splitter. Previous version crashed on several MPEG2 transport with AVC1 (H264) video.
* Added time stamp stabilizing (transport stream issues).
* Added inverse telecine when stream has the right flags.

[CruNcher]

@egur

the MC.ts is still problematic (wrong decoded)

the CD.ts is fine also sync wise

http://www.mediafire.com/?37kyc94d6n22tkf <- stops @ start


also i have a sample (very bad condition one) where i don't understand why Deinterlacing doesn't work on EVR Custom with ffdshow-quicksync but works fine on EVR (as if something adaptive would work on EVR (no flags needed) that's not being used on EVR Custom)

Upload of that one in progress (playback btw is fine for how corrupted this is only the Deinterlacing EVR Custom failing is what makes me wonder, several others shows this behavior too, though bitstream wise all are correct flagged though still sometimes EVR Custom Deinterlacing works sometimes it fails, when i look directly @ it and compare it seems for Mpeg-2 it always works but for H.264 it seems to fail interesting)

Does it mean Adaptive Deinterlacing works only on EVR and is it maybe possible to make it usable (from within Intels Drivers to work for other Renderer like EVR Custom as well ????)

Ahh seems the Problem is in ffdshow-quicksyncs MBAFF handling

Yep normal Interlaced Streams get correctly Deinterlaced on EVR Custom (Interlaced(PAFF))
MBAFF streams fail and only get correctly Deinterlaced on EVR

(hmm not sure yet but it seems the telecine fails on smooth cuts (fades) doesn't feel right on EVR Custom @ least)

Yep again EVR results are much better


ffdshow-quicksync telecine Mpeg-2 EVR:

http://img855.imageshack.us/img855/7...ynctelecin.png

ffdshow-quicksync Telecine Mpeg-2 EVR Custom (fail):

http://img97.imageshack.us/img97/707...ynctelecin.png

I guess that will be interesting to compare vs Nvidia

[egur]

CrunNcher:
I have a small bug in the memcpy function on 32 bit. You'll see a corruption on the right side (less than 128 pixel wide stripe on the rightest side).
I also improved the speed a little bit so I'll release again soon.
I'll check the new clips.
I saw that one of them was heavility corrupted (MC.ts). One played fine (CD.ts).

Do you have any info on MBAFF that can help me?

Aslo regarding invserse telecine, what do you mean by slow fades? I don't perform image analysis, just look at the flags.
If a few frames pass and there's no "repeat field" flag, than I drop out of IVT back to hte original frame rate.
I think I'm missing some code that deals with format change during playback. I'll add that too.

[tetsuo55]

Hello Eric,

First of all i want to say i am happy to see a release of a more stable dxva decoder for sandybridge and higher gpu's.
Also i cannot wait for the source to be released so this code can be integrated in arguably superior codecs.

I am a little bit confused though. Historically Intel has been working with Casimir of MPC-HC for integration of its DXVA codecs, has anything changed in this regard?

[nevcairiel]

LAV Video will get support for decoding through Intels Media SDK sooner or later, be it with Erics help or without.
Like Eric said, he didn't use any magic, he just implemented a decoder based on the publicly available SDK.

[egur]

Originally Posted by tetsuo55

Hello Eric,

First of all i want to say i am happy to see a release of a more stable dxva decoder for sandybridge and higher gpu's.
Also i cannot wait for the source to be released so this code can be integrated in arguably superior codecs.

I am a little bit confused though. Historically Intel has been working with Casimir of MPC-HC for integration of its DXVA codecs, has anything changed in this regard?

This work is my own initiative and it's aligned with Intel's interests as well as the users.
BTW, 100,000 people work at Intel and I don't know most them (or Casimir)...
Several groups support companies and open source projects, I'm awareof only a handful of people. I myself work in OEM support for the CPU but this is irrelevant to my little project here.

Source code will be sent if requested. It's meant for everyone to see, modify and use for free in either open or closed source projects.
I think it's a little early to send the source cose as it changes quite a bit due to feedback. But if you want it, I can send it to you.

[tetsuo55]

Thanks for the reply's.

I'm not in a rush to see the sourcecode, but as an open-source project manager i would like to see your work on a source management platform and licenced with GPL as soon as possible.

You can choose any site that you like; github, google code, etc...

Personally i tend to follow the commitlog for projects more than forum chat, as it is more condensed and to the point, plus i get to review the actual code changes. This will also give you the possibility to have people open tickets and attach small samples, etc...

Thanks in advance and good luck with this project!

Do you think this code will at some later point in any way help older DXVA implementations of pre-sandy-bridge hardware?

[squid_80]

Since ffdshow is licensed under the GPL and you are distributing modified builds of it, I think you are required to make the source available regardless of anyone requesting it or not. At least that's the point of view a certain moderator on this forum took with my work in the past.

[Superb]

I believe it's not the moderator's view... It's the license's view...

[egur]

Originally Posted by squid_80

Since ffdshow is licensed under the GPL and you are distributing modified builds of it, I think you are required to make the source available regardless of anyone requesting it or not. At least that's the point of view a certain moderator on this forum took with my work in the past.

I didn't contact CLSID (ffdshow's admin) about when and if I can integrate my ffdshow changes into the ffdshow source control in SourceForge.
BTW, the changes to ffdshow itself are small and trivial - just add a new decoder and assign it H264/MPEG2/VC1.
The majority of my code (a decoder DLL) should be on its own and I'm not sure what's the best way to post it's code. The most generic solution would be to create a separate project in SourceForge or another depository and have ffdhsow use it as an external lib (like it uses other decoders).
I'd like the decoder DLL to be LGPL not GPL so it can be used in any project (open or closed source).

ATM, it's much easier for me to have a single VC2010 solution and develop on it. The entire ffdshow source code is ~60MB zip and it's a little big to post, so I don't post it. My DLL's source code is ~130K and I have no problems sending it to anyone.

I'm not an expert in these matters and I'd be happy to get suggestions on the matter - both technical and legal.

[tetsuo55]

Originally Posted by egur

I didn't contact CLSID (ffdshow's admin) about when and if I can integrate my ffdshow changes into the ffdshow source control in SourceForge.
BTW, the changes to ffdshow itself are small and trivial - just add a new decoder and assign it H264/MPEG2/VC1.
The majority of my code (a decoder DLL) should be on its own and I'm not sure what's the best way to post it's code. The most generic solution would be to create a separate project in SourceForge or another depository and have ffdhsow use it as an external lib (like it uses other decoders).
I'd like the decoder DLL to be LGPL not GPL so it can be used in any project (open or closed source).

ATM, it's much easier for me to have a single VC2010 solution and develop on it. The entire ffdshow source code is ~60MB zip and it's a little big to post, so I don't post it. My DLL's source code is ~130K and I have no problems sending it to anyone.

I'm not an expert in these matters and I'd be happy to get suggestions on the matter - both technical and legal.

The best thing you can do right now is open a new project, you can use GIT trickery to link to ffdshow at compile time and then patch from your own tree (that way you do not need to copy all of ffdshow).

Here is some extra info http://www.joelonsoftware.com/articl...000000043.html

(P.S. Make sure to keep the LGPL and GPL code in seperate sub-projects)

[BetaBoy]

Originally Posted by egur

I'd like the decoder DLL to be LGPL not GPL so it can be used in any project (open or closed source)

LGPL is not great for closed source as it would raise more concerns then what I think you're trying to accomplish. I would propose a BSD based license.

Alternatively make it licensed under all 3: GPL, LGPL, BSD to satisfy everyone.

[Blight]

I second it, BSD is better than LGPL for closed source applications.

[egur]

Then BSD license it is.
Next build will Tomorrow (Monday).

[egur]

Originally Posted by tetsuo55

Thanks for the reply's.
Do you think this code will at some later point in any way help older DXVA implementations of pre-sandy-bridge hardware?

Technically (MSDK docs), it should work on Core 2 Dou with Intel graphics. But no one reported success yet. And the real world is not aligned with MSDK docs
I'll try to make it work on a Penryn laptop and report back

[egur]

New and improved version. Zip files contain documentation, please read.

Download version 0.13 alpha:
32 bit http://www.multiupload.com/Z284JFR06X
64 bit http://www.multiupload.com/1YFGXD786C
Source Code http://www.multiupload.com/ZVMCN124A0

Revision highlights:
v1.13:
* Optimized memory copy even further. Memory copy has 2-6% overhead (out of process CPU usage).
* Fixed bug in memory copy when frame width wasn't mod128.
* VC1 playback is more stable. Still corruption on some clips.
* Fixed some small memory leaks.
* Compatibility with 2509 driver.
* Bug fixes & cleanup.
* Tested with driver versions 2509 and 2372.

[CruNcher]

Egur for Performance and behviour testing also of the MSDK parts you should really take WAC,WPR,WPA from the new ADK into your Dev chain (though i guess Intel is using it already longer time then we are aware now of it since Build, and this massive usability changes of it) i can really recommend it to use it's damn powerful down to the stack and much easier know then Xperf was when it started with NT 6, this is for every Windows Developer a must use in Application Development/Assessment
Like Valgrind is for *nix Devs



Argh

* Tested with driver versions 2509 and 2372.

I wasn't aware of a new version unfortunately the news about new Intel Drivers aren't spreading as fast as for example Nvidia or AMD driver releases

also you have inside knowledge coud you please explain the release cycle difference and numbering sheme difference between the Platform Drivers and the Mainboard Driver release and the difference between them ?

So the difference currently between the 2 branches (also in terms of MSDK integration)

8.15.10.2476

15.22.50.64.2509


http://downloadcenter.intel.com/Deta...adType=Treiber

http://downloadcenter.intel.com/Deta...adType=Treiber

Has it just todo with testing and Certification for the correct functioning on the Intel Mainboards or for what are those 2 different in CPU driver categories ?

[egur]

Originally Posted by CruNcher

also you have inside knowledge could you please explain the release cycle difference and numbering sheme difference between the Platform Drivers and the Mainboard Driver release and the difference between them ?

So the difference currently between the 2 branches (also in terms of MSDK integration)

8.15.10.2476

15.22.50.64.2509

Has it just todo with testing and Certification for the correct functioning on the Intel Mainboards or for what are those 2 different in CPU driver categories ?

I'm as clueless as you are on driver version numbers and release dates
The last driver released to the public was in April. The new one is about a week old.
I didn't notice any positive changes about the new driver. It broke parts of my code and I have opened a thread on the MSDK forum. VC1 corruption is still there.
I also have a laptop with an engineering sample SNB processor that has an April driver. I use the two systems to produce code that works on both.
Some MSDK functions fail using one driver and succeed on the other and vice versa.

BTW, do you mind if I share the clips you're posted here to the MSDK team?