Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 31st July 2017, 17:52   #5501  |  Link
x265_Project
Guest
 
Posts: n/a
Quote:
Originally Posted by zub35 View Post
In this test-example, artifacts of macroblocks 64х64. Scene №3.

raw_1080p30.y4m [534MB] https://cloud.mail.ru/public/D74t/4LPtEm2WG
files x264,x265 [34MB] https://cloud.mail.ru/public/DdcN/HGKoYV5cU

x265 --bitrate 16000/8000 --preset placebo --merange 57 --subme 7 --psy-rd 4 --psy-rdoq 10 --pass 1/2
x264 --bitrate 16000/8000 --preset veryslow --level 4.1 --slow-firstpass --pass 1/2

http://i94.fastpic.ru/big/2017/0729/...98c86b78f6.png
Try re-encoding with default placebo settings, and let us know what you find.

Compression artifacts will occur if you set the psy-rd strength and/or psy-rdoq strength too high. Psy-rd biases mode decision towards candidates that have a similar level of detail (variance) to the source block. A little psy-rd is good, but too much psy-rd strength will cause x265 to choose candidates that aren't the best match visually (blocks that have higher or matching energy levels, even though they are not the best candidate with the lowest Rate-Distortion cost). When psy-rd is too high, you'll typically see motion inaccuracy (for example, blocks that should be portraying smooth motion that instead appear to be staying in one place too long). When psy-rdoq is too high you'll just get compression artifacts.
  Reply With Quote
Old 1st August 2017, 09:57   #5502  |  Link
excellentswordfight
Lost my old account :(
 
Join Date: Jul 2017
Posts: 322
Is there any plans of full HLG support? The arib-std-b67 transfer curve is implemented, but the DVB states that this should be flagged in the "alternative_transfer_characteristics" SEI message and bt2020-10 should be flagged in the normal VUI, this way it suppose to be more backwards compatible for non HLG equipment.

http://www.etsi.org/deliver/etsi_ts/...54v020301p.pdf

Last edited by excellentswordfight; 1st August 2017 at 09:59.
excellentswordfight is offline   Reply With Quote
Old 1st August 2017, 17:25   #5503  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,883
I gotta send a sample to a TV which requires constant bitrate (20 Mbit/s) HEVC 10bit 50fps .ts 'cause they are moving from .mxf XDCAM 1080i MPEG-2 25 Mbit/s interlaced contents to 4K HEVC 50fps progressive.
I used:

Code:
@avs4x265.exe --preset medium --level 5.1 --tune fastdecode --profile main10 --bitrate 19500 --vbv-maxrate 19500 --vbv-bufsize 19500 --strict-cbr --ref 3 --deblock -1:-1 --overscan show --colormatrix bt709 --range limited --transfer bt709 --colorprim bt709 --videoformat component --no-open-gop --fps 50 -o "raw_video.hevc" "AVS Script.avs"
and also with:

Code:
@avs4x265.exe --preset ultrafast --level 5.1 --tune fastdecode --profile main10 --bitrate 19500 --vbv-maxrate 19500 --vbv-bufsize 19500 --strict-cbr --ref 3 --deblock -1:-1 --overscan show --colormatrix bt709 --range limited --transfer bt709 --colorprim bt709 --videoformat component --no-open-gop --fps 50 -o "raw_video.hevc" "AVS Script.avs"
My AVS:

Code:
ColorBars(width = 3840, height = 2160, pixel_type = "yv12")

ConvertFPS(50)
ResampleAudio(48000)

Normalize(0.89, show=false)

trim(0, 1500)
barsnote=last

BlankClip(width=3840, height=2160, pixel_type="YV12", fps=50000, fps_denominator=1000, audio_rate=48000, channels=2, sample_type="float", color=$000000, length=1500)

TextSub("Clock.ass")
clock=last

FFmpegSource2("sampleProRes.mov", fpsnum=50000, fpsden=1000, atrack=-1)

ResampleAudio(48000)

Normalize(0.89, show=false)

sample=last

barsnote++clock++sample
But I got a variable bitrate video (about 16 Mb/s going up and down). I'm using the latest version of x265 compiled by LigH (x64). I need a constant bitrate video (or at least a way to mux it pretending that the stream is CBR).

(as to the 8bit processing done by Avisynth, that's because it's just a sample/test; I'm gonna use the 10bit "hack" in FFMpegSource2 to output 10bit, do some post-processing - light denoise and debanding - and then feed x265 directly with the 10bit stream thanks to Dither Tool in the real encode)

I cannot upload publicly the encoded sample, due to copyright rights, but I might send a very little part of it via PM if it's absolutely necessary to solve this "issue", but I'm pretty sure I'm just missing some settings in my encode line.

Last edited by FranceBB; 1st August 2017 at 17:52.
FranceBB is offline   Reply With Quote
Old 1st August 2017, 17:42   #5504  |  Link
sneaker_ger
Registered User
 
Join Date: Dec 2002
Posts: 5,565
x265 log? How did you determine the result is not CBR?
sneaker_ger is offline   Reply With Quote
Old 1st August 2017, 17:50   #5505  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,883
Well, after I encoded the .hevc file with x265 and the AC3 audio file with ffmpeg, I muxed them in .ts using ffmpeg and I got my .ts file, but Media Info says:

Code:
General 
ID : 1 (0x1) 
Complete name : G:\Sample.ts 
Format : MPEG-TS 
File size : 370 MiB 
Duration : 2 min 58 s 
Overall bit rate mode : Variable  (instead of Constant)
Overall bit rate : 17.2 Mb/s  (instead of 20 Mb/s)

Video 
ID : 256 (0x100) 
Menu ID : 1 (0x1) 
Format : HEVC 
Format/Info : High Efficiency Video Coding 
Format profile : Main 10@L5.1@Main 
Codec ID : 36 
Duration : 3 min 0 s 
Bit rate : 16.0 Mb/s  (instead of 19.5 Mb/s)
Width : 3 840 pixels 
Height : 2 160 pixels 
Display aspect ratio : 16:9 
Frame rate : 50.000 FPS 
Standard : Component 
Color space : YUV 
Chroma subsampling : 4:2:0 
Bit depth : 10 bits 
Bits/(Pixel*Frame) : 0.038 
Stream size : 343 MiB (93%) 
Writing library : x265 2.5+6-d11482e5fedb:[Windows][GCC 7.1.0][64 bit] 10bit 
Encoding settings : cpuid=1050111 / frame-threads=1 / numa-pools=2 / wpp / no-pmode / no-pme / no-psnr / no-ssim / log-level=2 / input-csp=1 / input-res=3840x2160 / interlace=0 / total-frames=9000 / level-idc=51 / high-tier=1 / uhd-bd=0 / ref=1 / no-allow-non-conformance / no-repeat-headers / annexb / no-aud / no-hrd / info / hash=0 / no-temporal-layers / no-open-gop / min-keyint=25 / keyint=250 / bframes=3 / b-adapt=0 / b-pyramid / bframe-bias=0 / rc-lookahead=5 / lookahead-slices=8 / scenecut=0 / no-intra-refresh / ctu=32 / min-cu-size=16 / no-rect / no-amp / max-tu-size=32 / tu-inter-depth=1 / tu-intra-depth=1 / limit-tu=0 / rdoq-level=0 / dynamic-rd=0.00 / no-ssim-rd / no-signhide / no-tskip / nr-intra=0 / nr-inter=0 / no-constrained-intra / strong-intra-smoothing / max-merge=2 / limit-refs=0 / no-limit-modes / me=0 / subme=0 / merange=57 / temporal-mvp / no-weightp / no-weightb / no-analyze-src-pics / deblock=-1:-1 / no-sao / no-sao-non-deblock / rd=2 / early-skip / rskip / fast-intra / no-tskip-fast / no-cu-lossless / no-b-intra / rdpenalty=0 / psy-rd=2.00 / psy-rdoq=0.00 / no-rd-refine / analysis-reuse-mode=0 / no-lossless / cbqpoffs=0 / crqpoffs=0 / rc=cbr / bitrate=19500 / qcomp=0.60 / qpstep=4 / stats-write=0 / stats-read=0 / vbv-maxrate=19500 / vbv-bufsize=19500 / vbv-init=0.9 / ipratio=1.40 / pbratio=1.00 / aq-mode=1 / aq-strength=0.00 / cutree / zone-count=0 / strict-cbr / qg-size=32 / no-rc-grain / qpmax=69 / qpmin=0 / no-const-vbv / sar=0 / overscan=1 / overscan-crop=0 / videoformat=0 / range=0 / colorprim=1 / transfer=1 / colormatrix=1 / chromaloc=0 / display-window=0 / max-cll=0,0 / min-luma=0 / max-luma=1023 / log2-max-poc-lsb=8 / vui-timing-info / vui-hrd-info / slices=1 / opt-qp-pps / opt-ref-list-length-pps / no-multi-pass-opt-rps / scenecut-bias=0.05 / no-opt-cu-delta-qp / no-aq-motion / no-hdr / no-hdr-opt / no-dhdr10-opt / analysis-reuse-level=5 / scale-factor=0 / refine-intra=0 / refine-inter=0 / refine-mv=0 / no-limit-sao / ctu-info=0 
Color range : Limited 
Color primaries : BT.709 
Transfer characteristics : BT.709 
Matrix coefficients : BT.709 

Audio 
ID : 21 (0x15) 
Menu ID : 1 (0x1) 
Format : AC-3 
Format/Info : Audio Coding 3 
Format settings, Endianness : Big 
Codec ID : 129 
Duration : 3 min 0 s 
Bit rate mode : Constant 
Bit rate : 384 kb/s 
Channel(s) : 2 channels 
Channel positions : Front: L R 
Sampling rate : 48.0 kHz 
Frame rate : 31.250 FPS (1536 spf) 
Bit depth : 16 bits 
Compression mode : Lossy 
Stream size : 8.24 MiB (2%) 
Language : English 
Service kind : Complete Main 

Menu 
ID : 4096 (0x1000) 
Menu ID : 1 (0x1) 
Duration : 2 min 58 s 
List : 256 (0x100) (HEVC) / 21 (0x15) (AC-3, English) 
Language : / English 
Service name : Service01 
Service provider : FFmpeg 
Service type : digital television
FranceBB is offline   Reply With Quote
Old 1st August 2017, 21:06   #5506  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,753
But:

Code:
rc=cbr / bitrate=19500 ... vbv-maxrate=19500 / vbv-bufsize=19500
That's the point which counts ... but CBR bitrate distribution mode is no guarantee for a really really constant bitrate. If the encoder needs less bitrate to achieve optimal quality, a multiplexer would have to stuff the video stream with empty junk if a pseudo-constant bitrate is expected. And I doubt any playback device is so picky about it. There are more important limits than a bitrate. Like, reference frames, consecutive B frames, B frame pyramid, CABAC, Profile@Level marker...
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 2nd August 2017, 11:31   #5507  |  Link
littlepox
Registered User
 
Join Date: Nov 2012
Posts: 218
Quote:
Originally Posted by LigH View Post
But:

Code:
rc=cbr / bitrate=19500 ... vbv-maxrate=19500 / vbv-bufsize=19500
That's the point which counts ... but CBR bitrate distribution mode is no guarantee for a really really constant bitrate. If the encoder needs less bitrate to achieve optimal quality, a multiplexer would have to stuff the video stream with empty junk if a pseudo-constant bitrate is expected. And I doubt any playback device is so picky about it. There are more important limits than a bitrate. Like, reference frames, consecutive B frames, B frame pyramid, CABAC, Profile@Level marker...
I have seen such user so keen to use CBR for the following reason:

1. My broadcast/streaming service provider says no more than 20Mbps
2. I should encode with a maximum bitrate of 20Mbps(or to be safe, 19Mbps)
3. in order to maximize the quality under this constrain, make the bitrate constant as 19Mbps
4. Why the fxxk I cannot find a CBR mode? are you serious in telling me to use x264 instead of my favourite one-key encoder?
littlepox is offline   Reply With Quote
Old 2nd August 2017, 14:43   #5508  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,753
a) just a casual question to the x265 developers, no accusation intended: When there are no commits for several days ... are you usually hitting a complex issue all together, which takes some time to get investigated, or is it more likely that you have appointments outside the development office (e.g. fairs or customer meetings, maybe it's just vacation season, and hopefully no natural disasters)?
_

b) a question not only to the x265 developers, but mainly: what is the intended difference between "make clean" and "make clean-generated", or more specifically, in which situations should one use either of them?
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 2nd August 2017, 15:14   #5509  |  Link
excellentswordfight
Lost my old account :(
 
Join Date: Jul 2017
Posts: 322
Quote:
Originally Posted by littlepox View Post
I have seen such user so keen to use CBR for the following reason:

1. My broadcast/streaming service provider says no more than 20Mbps
2. I should encode with a maximum bitrate of 20Mbps(or to be safe, 19Mbps)
3. in order to maximize the quality under this constrain, make the bitrate constant as 19Mbps
4. Why the fxxk I cannot find a CBR mode? are you serious in telling me to use x264 instead of my favourite one-key encoder?
I have been doing something similar, were offline files aready encoded were inserted to the live ts stream so they needed to match those settings. If its the same use case here I find it kind of weird that they are concerned about the bitrate but not GOP structure, ref frames etc. So if these are not correct, it needs to be reencoded anyway.

For my case I used ffmeg/x265 with these settings and they ended up close to "true" CBR. I saw no cases with a significant drop in bitrate.

Code:
-c:v libx265 -pix_fmt yuv420p10le -preset slow -x265-params level=51:bitrate=24000:vbv-maxrate=24000:vbv-bufsize=24000:keyint=48:no-scenecut=1:b-adapt=0:bframes=8:rc-lookahead=48:strict-cbr=1:colorprim="bt709":transfer="bt709":colormatrix="bt709":range="limited"
I didnt look that closely on your script, but it looked like you inserted bars to it right? This could explain it, they could be encoded at a much lower bitrate even though a higher bitrate is specified (I have at least seen this with abr in the past). But it should be padded (muxrate) anyway when you repack it into a TS-stream so it shouldnt matter right?

Last edited by excellentswordfight; 2nd August 2017 at 15:25.
excellentswordfight is offline   Reply With Quote
Old 2nd August 2017, 15:47   #5510  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,883
@excellentswordfight... yes, I used a closed GOP 'cause open GOP it's not supported by their specs and I kept --ref 3 just to be sure that the reference frames won't be a problem. As to the bars-note, I had to insert 30sec bars-note to show them channels mapping, audio level (about -24 LUFS more or less, 'cause according to the law commercial breaks can't be louder than the program itself, that has to be normalised) and that both luma and chroma of the program are in range (bars 75%, luma 0.7) followed by the clock with the name of program, video specs, TC-In, TC-Out and the countdown, lasting for 30 sec in order to have the program starting at 00:01:00.00.

So... yes, probably it's just because of bars-note and the clock/countdown that the encoder lowered the bitrate, which shouldn't be a big deal if it's just for that. Anyway, I sent them a sample; I hope they'll accept it.
FranceBB is offline   Reply With Quote
Old 2nd August 2017, 19:28   #5511  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 4,750
Quote:
Originally Posted by FranceBB View Post
I gotta send a sample to a TV which requires constant bitrate (20 Mbit/s) HEVC 10bit 50fps .ts 'cause they are moving from .mxf XDCAM 1080i MPEG-2 25 Mbit/s interlaced contents to 4K HEVC 50fps progressive.
But I got a variable bitrate video (about 16 Mb/s going up and down). I'm using the latest version of x265 compiled by LigH (x64). I need a constant bitrate video (or at least a way to mux it pretending that the stream is CBR).
Try adding --strict-cbr.

However, if there isn't much motion in the content, there are only so many ways to spend the bits. Adding a bit of random noise and using --tune grain is a good way to boost bitrate
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 3rd August 2017, 01:08   #5512  |  Link
FranceBB
Broadcast Encoder
 
FranceBB's Avatar
 
Join Date: Nov 2013
Location: Royal Borough of Kensington & Chelsea, UK
Posts: 2,883
@benwaggoner I used that parameter (--strict-cbr) already. Anyway, excellentswordfight was right: it was just a few kbit/s at the very beginning due to the bar-note, but the program itself was constant at 19.5 Mbit/s, in fact they just called me to tell me that they accepted the sample.

Gentlemen, it's been a pleasure. XD
No, seriously, thank you, everyone.
FranceBB is offline   Reply With Quote
Old 3rd August 2017, 03:14   #5513  |  Link
qyot27
...?
 
qyot27's Avatar
 
Join Date: Nov 2005
Location: Florida
Posts: 1,419
Quote:
Originally Posted by LigH View Post
b) a question not only to the x265 developers, but mainly: what is the intended difference between "make clean" and "make clean-generated", or more specifically, in which situations should one use either of them?
My gut reaction is that it's the same as the difference between 'make clean' and 'make distclean' in autoconf and the custom build systems used by x264, FFmpeg, etc. 'make clean' scrubs the object files and other at-compile-time stuff away, while 'make distclean' cleans both that and the build process structures generated by running configure.

At least in those systems, distclean should be used whenever a change to the build system occurs, while clean is sufficient if only the actual project code changed.
qyot27 is offline   Reply With Quote
Old 5th August 2017, 09:03   #5514  |  Link
WhatZit
Registered User
 
Join Date: Aug 2016
Posts: 60
Quote:
Originally Posted by LigH View Post
When there are no commits for several days ... are you usually hitting a complex issue all together, which takes some time to get investigated, or is it more likely that you have appointments outside the development office
Remember that there is much more to MulticoreWare's development of x265 than just the visible public open source. They have their proprietary API-based UHDKit, which you definitely WON'T see commits for.

Also, it's entirely possible that MulticoreWare's "sideline projects" (I'm sure calling them that will make 'em angry) might require some collaborative poaching of x265 developers. Their recently developed LipSync technology is a perfect example of this, requiring an amalgam of both video analysis and artificial intelligence.
WhatZit is offline   Reply With Quote
Old 5th August 2017, 19:52   #5515  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,753
@ WhatZit: I wasn't aware of additional projects with similar complexity. Of course, that's a quite probable reason.

@ qyot27: My impression was the opposite, "make clean-generated" only removing CMake generated and similar auxiliary files to force rebuilding from the end of the building sequence for maybe only sub-targets with updates. But it's not easy for me to understand make files without a real clue about their structure, syntax, dependencies ...
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid

Last edited by LigH; 5th August 2017 at 19:55.
LigH is offline   Reply With Quote
Old 6th August 2017, 04:40   #5516  |  Link
x265_Project
Guest
 
Posts: n/a
Quote:
Originally Posted by LigH View Post
a) just a casual question to the x265 developers, no accusation intended: When there are no commits for several days ... are you usually hitting a complex issue all together, which takes some time to get investigated, or is it more likely that you have appointments outside the development office (e.g. fairs or customer meetings, maybe it's just vacation season, and hopefully no natural disasters)?
_

b) a question not only to the x265 developers, but mainly: what is the intended difference between "make clean" and "make clean-generated", or more specifically, in which situations should one use either of them?
Vacations, recruiting at local colleges... no disasters, natural or otherwise.
  Reply With Quote
Old 6th August 2017, 04:41   #5517  |  Link
x265_Project
Guest
 
Posts: n/a
Quote:
Originally Posted by WhatZit View Post
Remember that there is much more to MulticoreWare's development of x265 than just the visible public open source. They have their proprietary API-based UHDKit, which you definitely WON'T see commits for.

Also, it's entirely possible that MulticoreWare's "sideline projects" (I'm sure calling them that will make 'em angry) might require some collaborative poaching of x265 developers. Their recently developed LipSync technology is a perfect example of this, requiring an amalgam of both video analysis and artificial intelligence.
We have 4 business units at MulticoreWare. I run the video business. Our LipSync product was developed by our Machine Learning business unit. They didn't poach any of our video developers.

Tom
  Reply With Quote
Old 10th August 2017, 08:49   #5518  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 6,753
x265 2.5+8-eea2afb81ef2

New modes for --refine-intra and --refine-inter
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 10th August 2017, 15:03   #5519  |  Link
Atak_Snajpera
RipBot264 author
 
Atak_Snajpera's Avatar
 
Join Date: May 2006
Location: Poland
Posts: 7,806
AMD Threadripper 1950X in x265
https://cubeupload.com/im/bKv6yQ.png

Source -> https://youtu.be/TJiP1bKxLkU?t=3m43s

Last edited by Atak_Snajpera; 10th August 2017 at 15:06.
Atak_Snajpera is offline   Reply With Quote
Old 10th August 2017, 17:28   #5520  |  Link
mastrboy
Registered User
 
Join Date: Sep 2008
Posts: 365
Quote:
Originally Posted by Atak_Snajpera View Post
So AMD is still way behind Intel in FPU performance pr core, or will there be future x265 optimizations for Ryzen/Threadripper that could close the gap?
__________________
(i have a tendency to drunk post)
mastrboy is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 21:46.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2024, vBulletin Solutions Inc.