[Sagittaire]

HVS matrix for exemple use really serious IEEE paper based on HVS luminance study.
http://forum.doom9.org/showthread.ph...=referencedivx

There are simple rule for CQM: Human eyes are more sensible to global (low frequencies aka gradiant) than detail (high frequencies). For HVS it's better to use higher coefficient for HF and lower coefficient for LF. You obtain intra HVS matrix for MPEG4 ASP like that:

16 16 16 16 17 18 21 24
16 16 16 16 17 19 22 25
16 16 17 18 20 22 25 29
16 16 18 21 24 27 31 36
17 17 20 24 30 35 41 47
18 19 22 27 35 44 54 65
21 22 25 31 41 54 71 88
24 25 29 36 47 65 88 115


Anyway for MPEG4 AVC you have a more complex architecture. High quantisation values for Global (LF) mean blocking and it's a better HVS way to use lower relative coef in CQM in this case. But H264 can use another way against blocking: inloop deblocking. Inloop is really good against blocking but inloop is aggressive with detail level (HF). In conclusion with inloop you change this simple HVS rule.

In fact in practice and IMO for H264, flat16 matrix is really good for "low bitrate" because inloop is really agressive in this case. Have same coef for HF and LF is certainely the best HVS way here. For "high bitrate" inloop is less aggressive. HVS matrix (low coef for LF and high coef for HF) is certainely a better HVS way in this case.

Moreover with implementation like x264 use a matrix radically different than the flat16 can change the codec functioning in bad way. If you want really use CQM with x264 you must IMO use really "soft" HVS matrix like that:

INTRA4X4_LUMA =
12,12,14,16,
12,14,16,20,
14,16,20,25,
16,20,25,32

INTRA4X4_CHROMAU =
12,12,14,16,
12,14,16,20,
14,16,20,25,
16,20,25,32

INTRA4X4_CHROMAV =
12,12,14,16,
12,14,16,20,
14,16,20,25,
16,20,25,32

INTER4X4_LUMA =
14,14,15,16,
14,15,16,17,
15,16,17,18,
16,17,18,19

INTER4X4_CHROMAU =
14,14,15,16,
14,15,16,17,
15,16,17,18,
16,17,18,19

INTER4X4_CHROMAV =
14,14,15,16,
14,15,16,17,
15,16,17,18,
16,17,18,19

INTRA8X8_LUMA =
12,12,12,12,13,14,15,16,
12,12,12,13,14,15,16,17,
12,12,13,14,15,16,17,19,
12,13,14,15,16,18,20,22,
13,14,15,16,18,21,23,27,
14,15,16,18,21,24,28,37,
15,16,17,20,23,28,39,48,
16,17,19,22,27,37,48,68

INTER8X8_LUMA =
14,14,14,14,15,15,15,16,
14,14,14,14,15,15,16,17,
14,14,14,15,15,16,17,18,
14,14,15,15,16,17,18,19,
15,15,15,16,17,18,19,21,
15,15,16,17,18,20,22,26,
15,16,17,18,19,22,27,32,
16,17,18,19,21,26,32,44