Intel QuickSync Decoder - HW accelerated FFDShow decoder with video processing

[dukey]

the point i was trying to make was EVR creates and renderers into additional swap chains, it doesn't depend on the window really, as the size of the back buffer for the window created was only 1x1.

[egur]

Originally Posted by JanWillem32

That points out the hwnd handle nicely. Creating the window handle window handle with WS_MINIMIZE|WS_POPUP and possibly WS_DISABLED should work. http://msdn.microsoft.com/en-us/libr...=VS.85%29.aspx
If creating it minimized is a problem, using CloseWindow should also do the trick of minimizing it. http://msdn.microsoft.com/en-us/libr...=VS.85%29.aspx

Doesn't work I created the hWnd like you specified, tried offscreen coordinates (9999,9999), minimized, etc. Will not work in FS only in windowed mode.

If you've verified that it works in your system (in FSE), please send the me the device creation code starting from hWnd creation up to the call to CreateDevice.

Thanks!

[vivan]

Added black borders to images with non 16 modulo width. Retaining non standard width can cause downstream filters to crash (dvobsub/vsfilter).

Is it possible to remove this "feature"?
1) There are a lot of people who are not using such... filters. For them it makes things only worse - e.g. 712x400 is 16:9, if you add 8pix border it would be 720x400 and you will have to enjoy top, right and bottom black borders on 16:9 display.
2) ffdshow already has such feature - resize & aspect filter - "expand to next multiple of 16".

[egur]

Originally Posted by vivan

Is it possible to remove this "feature"?
1) There are a lot of people who are not using such... filters. For them it makes things only worse - e.g. 712x400 is 16:9, if you add 8pix border it would be 720x400 and you will have to enjoy top, right and bottom black borders on 16:9 display.
2) ffdshow already has such feature - resize & aspect filter - "expand to next multiple of 16".

Nev has already asked for this removal and I agreed.
What I plan to do is enable/disable this feature via config. This way the DS filter can control what's going on. I hope this will make everyone happy.
I personally use vobsub and it's quite common - that's why I implemented it in the first place.

The default will be to enable mod16 width because stability overrides quality.

BTW, ffdshow will copy the image with a significant performance penalty so it's not a good option.

[CruNcher]

Egur this is really cool http://www.mediafire.com/download.php?d0bg6khk2lk8bjl due to the vsync background noise completely gone you can immediately see what the Broadcast Encoder did wrong (without needing to look @ the bitstream) it happens @ every keyframe

I tried to get the same resolution with ffdshow-quicksync but it shows strange peakings and get a lock sometimes @ 25 fps
the nicetest scene shows extreme jitter also 16ms


Peaking wrong lock issue (telecine mpeg-2):

http://www.mediafire.com/?gauc931m49hya51

Heavy jitter issue (H.264 Interlaced):

http://www.mediafire.com/?x2crc2tuoq9il29

Interesting disabling Deinterlace auto flag output in ffdshow-quicksync fixes both of these issues but obviously no Deinterlacing anymore (which though is only really problematic for the Interlaced H.264 stream)

Though there is still strange peaking periodically going on for the telecined stream and jitter changes from 0.4xxms upto 2ms very strange, ahhh it loses the lock on the next keyframe again

Lock Lost issue (telecine mpeg-2):

http://www.mediafire.com/?tii7ytnm7di6ck4

[egur]

Originally Posted by CruNcher

Egur this is really cool http://www.mediafire.com/download.php?d0bg6khk2lk8bjl due to the vsync background noise completely gone you can immediately see what the Broadcast Encoder did wrong (without needing to look @ the bitstream) it happens @ every keyframe

I tried to get the same resolution with ffdshow-quicksync but it shows strange peakings and get a lock sometimes @ 25 fps
the nicetest scene shows extreme jitter also 16ms


Peaking wrong lock issue (telecine mpeg-2):

http://www.mediafire.com/?gauc931m49hya51

Heavy jitter issue (H.264 Interlaced):

http://www.mediafire.com/?x2crc2tuoq9il29

Interesting disabling Deinterlace auto flag output in ffdshow-quicksync fixes both of these issues but obviously no Deinterlacing anymore (which though is only really problematic for the Interlaced H.264 stream)

Though there is still strange peaking periodically going on for the telecined stream and jitter changes from 0.4xxms upto 2ms very strange, ahhh it loses the lock on the next keyframe again

Lock Lost issue (telecine mpeg-2):

http://www.mediafire.com/?tii7ytnm7di6ck4

I'm on a business trip this week, so I'll take a look at it when i return. Thanks!

[JanWillem32]

Originally Posted by egur

Doesn't work I created the hWnd like you specified, tried offscreen coordinates (9999,9999), minimized, etc. Will not work in FS only in windowed mode.

If you've verified that it works in your system (in FSE), please send the me the device creation code starting from hWnd creation up to the call to CreateDevice.

Thanks!

That's too bad. Let's first try something else. What errors does the DirectX debug runtime give in a tracing debug session? (Don't forget to define D3D_DEBUG_INFO globally in the project and enable the Direct3D debug runtime in the "Microsoft DirectX SDK (June 2010)\Utilities\bin\x64\dxcpl.exe" or "Microsoft DirectX SDK (June 2010)\Utilities\bin\x86\dxcpl.exe" utility.)
I hope you'll enjoy your trip, and hear from you later on.

[skingery]

I used to use LAV for splitting, audio and video. Recently I rebuilt my HTPC with a Sandybridge processor so I thought I'd give this build a try for video.
For a renderer, what are people generally using EVR, EVR CP or madVR?

[kwlee]

Hi,
I'm a newbie here, and testing about limitation issue...

Originally Posted by egur

There should not be a (practical) limit. I can modify ffdshow to revert to libavcodec if initialization fails. Most likely that the platform will run out RAM before this happens.

I am testing about h264 & DVXA2 by ffdshow to see how many applications(graphstudio.exe) can playback at the same time.

The maximun number is 6, the 7th graphstudio will fail. I have tested on both nVidia GT430 and AMD Radeon 6900 series...

Is that the limitation of ffdshow? and will the limitation same as this intel project ?

Thanks!

[egur]

Originally Posted by kwlee

Hi,
I'm a newbie here, and testing about limitation issue...


I am testing about h264 & DVXA2 by ffdshow to see how many applications(graphstudio.exe) can playback at the same time.

The maximun number is 6, the 7th graphstudio will fail. I have tested on both nVidia GT430 and AMD Radeon 6900 series...

Is that the limitation of ffdshow? and will the limitation same as this intel project ?

Thanks!

I gave up after 12 graph studios. GPU RAM is probably the bottleneck, if you play multiple instances of an H264, full HD, with lots of reference frames, a lot of GPU RAM is used, limiting the amount of instances. BTW, if you've lowered the amount of RAM the GPU uses (BIOS setup) than the instance count will be lower.

[egur]

The first official build of ffdshow with Intel QuickSync decoder can be downloaded from the ffdshow's download page at:
http://ffdshow-tryout.sourceforge.net/download.php

Same QuickSync decoder as v0.18, ffdshow itself contains changes mostly for subtitles.

This thread will continue to supply ffdshow builds for continuous testing of my decoder's versions.

[kwlee]

Originally Posted by egur

I gave up after 12 graph studios. GPU RAM is probably the bottleneck.

Ya, If I test ffdshow by libavcodec, It can easy up to 12 graph studios with 1080P avi files.

If I test ffdshow by DXVA2, even by smaller h264 video files,
the graph studio limitation is still 6, no idea why

My purpose is for TV wall...

[kwlee]

Hi, A programming issue about progressive frame flag,

In QuickSync.cpp
mfxU32 CQuickSync::PicStructToDsFlags(mfxU32 picStruct)

if (picStruct == MFX_PICSTRUCT_PROGRESSIVE)
{
return AM_VIDEO_FLAG_WEAVE; --> Is it better to use "AM_VIDEO_FLAG_INTERLEAVED_FRAME" ?
}

[egur]

Originally Posted by kwlee

Hi, A programming issue about progressive frame flag,

In QuickSync.cpp
mfxU32 CQuickSync::PicStructToDsFlags(mfxU32 picStruct)

if (picStruct == MFX_PICSTRUCT_PROGRESSIVE)
{
return AM_VIDEO_FLAG_WEAVE; --> Is it better to use "AM_VIDEO_FLAG_INTERLEAVED_FRAME" ?
}

This is done to insure that a deinterlacer will not run on this frame. AM_VIDEO_FLAG_INTERLEAVED_FRAME means that both fields exist. Do you see any issues with this?

[nevcairiel]

AM_VIDEO_FLAG_WEAVE is the right flag for progressive. AM_VIDEO_FLAG_INTERLEAVED_FRAME actually has the value 0, and is therefor always set. Most images we deal with here always have two interleaved fields - singe fields are rather uncommon.

[kwlee]

Originally Posted by egur

This is done to insure that a deinterlacer will not run on this frame. AM_VIDEO_FLAG_INTERLEAVED_FRAME means that both fields exist. Do you see any issues with this?

During my test it's no problem(TBF, I don't have too many
test sample files)

http://msdn.microsoft.com/en-us/libr...(v=vs.85).aspx

AM_VIDEO_FLAG_WEAVE "This flag applies only when there are two fields per sample. " As MSDN explains..

Maybe someone can help to explain more..

[nevcairiel]

A progressive frame contains two "fields", you're just not supposed to handle them separately.

[egur]

a video frame can basically come from two sources - interlaced or progressive.
In the former, the 2 fields (called top/bottom or odd/even) are from different times. Each field has half the lines of the original frame. Due to the different time stamps, they should be interpolated (several methods exist) to a full frame (e.g. deinterlaced).
In a progressive frame, there's no notion of fields, the image has all the lines and no deinterlacing is needed.

A special case is what's called "film". A video that was shot in progressive and later artificially split into fields (mostly TV broadcast and DVDs). In "film" content, the fields share the same time stamps and need to processed as progressive frames. A deinterlacer and/or a decoder will usually have a film detector mechanism of some sort as a standard deinterlacer will produce horrible artifacts for some content.

There's also the matter of how CbCr is stored (4:2:0 only). If the frame is progressive, CbCr values are stored for two consecutive lines. For interlaced content, it's stored for two consecutive lines of a specific field. Simply weaving the lines will produces color artifacts even if there's no motion.

[egur]

Version 0.19 is out with the following changes:
* Added limited support for WMC full screen exclusive mode:
- Renderer must be connected to the decoder directly - no intermediate filters.
- Screen is connected to the Intel GPU (decoder shares device with renderer).
- Might only work on single monitor setups.
* Decoder has exposed its configuration options GetConfig/SetConfig - must be called before initialized.
* Padding the image to mod16 width is now off by default. Works with vobsub.
* Decoder can be tested for compatibly with media types via the TestMediaType method
* FFDShow rev4126

Download from SourceForge home page:
http://sourceforge.net/p/qsdecoder

[haruhiko_yamagata]

Thanks a lot for the update.
Can I ask several questions and requests? Please excuse me for not reading all of this thread. I read several pages. I do not have sandy bridge and can not test your decoder.
I would like to ask you to add an entry to our wiki and answer these questions. I think most of my questions are FAQ.

Originally Posted by egur

Version 0.19 is out with the following changes:
* Added limited support for WMC full screen exclusive mode:
- Renderer must be connected to the decoder directly - no intermediate filters.
- Screen is connected to the Intel GPU (decoder shares device with renderer).
- Might only work on single monitor setups.

Can ffdshow output other color spaces than NV12?
Can ffdshow resize?
If screen is not connected to the Intel GPU, and the renderer is not the WMC full screen, does your decoder work?

* Padding the image to mod16 width is now off by default. Works with vobsub.

Are the strides aligned? ffdshow exhibit its bugs and heavy loss of performance for several features if the strides are not aligned. I may able to fix the bugs, but not the performance issue.

Additional FAQ for the wiki:
How fast is this?
For what kind of video is this useful?
Which profiles of H.264 does this support?
Can this output 10/12-bit formats?