Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > MPEG-4 AVC / H.264

Reply
 
Thread Tools Search this Thread Display Modes
Old 20th October 2015, 10:25   #1  |  Link
serhannn
Registered User
 
Join Date: Sep 2015
Posts: 11
Confused about extracted motion vectors from ffmpeg

Hi, I used the example file extract_mvs.c in ffmpeg documentation and extracted the motion vectors from a H.264 encoded video. I obtained a text file with the following format:

Code:
framenum, source, blockw, blockh, srcx, srcy, dstx, dsty, flags
I'm a little bit confused about what src and dst actually means. Until now, I thought dst ist the reference block from which the motion is estimated, and src is the current block in the current frame. So, the x and y compontents of a motion vector is calculated as:

Code:
MV_x = dstx-srcx
MV_y = dsty-srcy
However, in the text file, I see lines like this:

Code:
2,-1,16, 8, 167,  51, 168,  52, 1, 1,0x0
...
...
113,-1, 8, 8, 157, 156, 156, 156,-1, 0,0x0
So, for example in frame 2, the 16x8 source block seems to be located at (167,51), and in frame 113 8x8 source blocks is located at (157,156). This doesn't make sense to me. Shouldn't the center of the current block always be located at multiples of 8 or 16 along both x and y directions? I thought the "destination" blocks (in the reference frame) would be shifted by the magnitude of the respective motion vectors; but the situation seems to be the other way around here. Do I interpret the meaning of src and dst wrong? Or is this kind of motion prediction actually allowed?

I'm a beginner to H.264 standard so I'm sorry if the question is too naive... Thanks!
serhannn is offline   Reply With Quote
Old 20th October 2015, 12:33   #2  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,558
You have it backward: src refers to the coordinates in the reference frame that the vector points to, dst refers to the top-left coordinates of the actual coded block. All of these are in the upscaled plane (hpel for MPEG1 & MPEG2, qpel for avc & hevc, either way for asp) so divide by 4 to get the actual x,y location in the output video.

You can get a lot more information by editing mpegvideo.c, though it's pretty hard to know what's useful and what's not.
foxyshadis is offline   Reply With Quote
Old 20th October 2015, 13:15   #3  |  Link
serhannn
Registered User
 
Join Date: Sep 2015
Posts: 11
Thanks for the correction, in the documentation off ffmpeg there is no information regarding that, so it was a source of confusion for me... Can you point me to a source file where I can verify this?
Except for that, there are still two unclear points:

1. Do src and dst really refer to the top-left coordinates of the respective macroblocks?
2. The function av_frame_get_side_data does not actually give coordinates in upscaled plane as you mentioned (with H.264/AVC videos). E.g. when I run extract_mvs.c in the documentation, I get an output like:

Code:
framenum,source,blockw,blockh,srcx,srcy,dstx,dsty
2,-1,16,16,   8,   8,   8,   8
2,-1,16,16,  24,   8,  24,   8
2,-1,16,16,  40,   8,  40,   8
2,-1,16,16,  56,   8,  56,   8
2,-1,16,16,  72,   8,  72,   8
                    ...
                    ...
To me it seems that srcx, srcy, dstx, and dsty give the center point for the respective blocks rather than the top-left point.

EDIT: I also went throught the code in vf_codecview.c and the function filter_frame gave me the impression that dst_{x,y] are the coordinates of the reference block since the draw_arrow function draws an arrow from src_{x,y} to dst_{x,y}. I'm still kind of confused.

Last edited by serhannn; 20th October 2015 at 13:50. Reason: New info:
serhannn is offline   Reply With Quote
Old 21st October 2015, 00:49   #4  |  Link
foxyshadis
ангел смерти
 
foxyshadis's Avatar
 
Join Date: Nov 2004
Location: Lost
Posts: 9,558
It looks like you're completely correct about the center being the point, and it not being upscaled. d'oh, it's been a while since I used it, obviously, sorry about that. Choosing to display it the direction they do is an arbitrary choice; it would make sense either way, although I think it's more intuitive to point to where it went than where it came from.
foxyshadis is offline   Reply With Quote
Old 21st October 2015, 09:33   #5  |  Link
serhannn
Registered User
 
Join Date: Sep 2015
Posts: 11
Yes, intuitively you are right. I would expect that dst would be the current block in the current frame and src would be the reference block in the reference frame since the the reference block is actually the "source" but it seems to be the other way around.

Unfortunately, I still don't get why src_x and src_y are shifted by the magnitude of the x and y components of the motion vector (MV_x and MV_y). This would be correct if the above-mentioned logic was used but in this case, I'm not sure.

EDIT: Or maybe ffmpeg points the motion vectors not from the current block to the reference block, but the other way around. Is that also possible?

Last edited by serhannn; 21st October 2015 at 10:35.
serhannn is offline   Reply With Quote
Reply

Tags
ffmpeg, motion vectors

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 16:36.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.