Welcome to Doom9's Forum, THE in-place to be for everyone interested in DVD conversion.

Before you start posting please read the forum rules. By posting to this forum you agree to abide by the rules.

 

Go Back   Doom9's Forum > Video Encoding > High Efficiency Video Coding (HEVC)

Reply
 
Thread Tools Search this Thread Display Modes
Old 11th March 2016, 04:32   #1  |  Link
djesteban
Registered User
 
djesteban's Avatar
 
Join Date: Aug 2008
Posts: 110
Are zones working for x265

Hi,

I tried encoding a clip using zones, but it doesn't seem to work.

I am doing a 2 pass encode, I don't know if that has anything to do with it.

Is zones working?
Thanks in advance.
djesteban is offline   Reply With Quote
Old 12th March 2016, 17:39   #2  |  Link
djesteban
Registered User
 
djesteban's Avatar
 
Join Date: Aug 2008
Posts: 110
... I tried to find anything related to this in the documentation, and it doesn't specify if zones work in nPass or only in CRF.

Is there a place where we can get that type of information?

If indeed zones is not available for nPass encoding, it would be nice to update the documentation to reflect that...
djesteban is offline   Reply With Quote
Old 12th March 2016, 20:13   #3  |  Link
x265_Project
Registered User
 
Join Date: Jul 2013
Posts: 596
Quote:
Originally Posted by djesteban View Post
... I tried to find anything related to this in the documentation, and it doesn't specify if zones work in nPass or only in CRF.

Is there a place where we can get that type of information?

If indeed zones is not available for nPass encoding, it would be nice to update the documentation to reflect that...
The sure-fire way to see if it works is to run 2 tests, generating a CSV file with frame-level logging (--csv logfilename.csv --log-level 3 --csv-log-level 1), and you will see the frame level baseline QP values. Run a first test without zones, and a second test with zones. Note that you need to use an identical command line in both passes of a 2 pass encode, so if you want to use zones you should specify zones in the first pass as well as the second.

If it doesn't work for 2 pass, you should file an issue on our Bitbucket issues tracker, complete with your test data.
x265_Project is offline   Reply With Quote
Old 14th March 2016, 00:43   #4  |  Link
djesteban
Registered User
 
djesteban's Avatar
 
Join Date: Aug 2008
Posts: 110
Quote:
Originally Posted by x265_Project View Post
The sure-fire way to see if it works is to run 2 tests, generating a CSV file with frame-level logging (--csv logfilename.csv --log-level 3 --csv-log-level 1), and you will see the frame level baseline QP values. Run a first test without zones, and a second test with zones. Note that you need to use an identical command line in both passes of a 2 pass encode, so if you want to use zones you should specify zones in the first pass as well as the second.

If it doesn't work for 2 pass, you should file an issue on our Bitbucket issues tracker, complete with your test data.
Will try generating a CSV, even if I'm pretty sure it doesn't work since according to my encode tests I have an obvious increase of bitrate in the zone I created in CRF, but don't in 2pass (yes both my passes are identical).

Will update this thread after I get another encode generating a CSV.

I also noticed that user LigH posted the following in the main x265 thread.
Quote:
I just tested the Sintel trailer in a smaller size (640x272) in 3 passes with verbose CSV log files. Plotting the average QP per frame (in encoded order only, displayed order is too hard to reconstruct in Excel) reveals a quite confusing behaviour in "--pass 2", while it looked about as expected in "--pass 3". Developers got details via mailing list.
djesteban is offline   Reply With Quote
Old 14th March 2016, 09:39   #5  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 5,997
I already sent my results to the x265 Developers' mailing list. Just to present them here again:

The Sintel trailer in a smaller version (640272, to fit inside 360p) was encoded in 3 passes each, once without zones, once with 3 zones as follows, using x265 v1.9+88:

Code:
--preset slower --bitrate 1000 --zones 0,223,b=0.3/1140,1253,q=40 --pass [1|3|2]
Here the quantizer distribution in encoding order (reconstructing the display order in Excel was too complicated for me in the brief time of preparing the charts) without zones; the image links to a PDF:



Just like the command line, "pass 3" (purple) means the middle pass and "pass 2" (red) means the final pass. As you would expect, the refined result of pass 2 is usually in between the ABR preparation (pass 1, blue) and the CRF result of pass 3.

Now the quantizer distribution with 3 zones: 30% bitrate in the beginning (frames 0..223), constant quantizer 40 in the end (frames 1140..1253), and 100% bitrate in the middle (no explicit definition).



The result of pass 3 is already not as expected, the bitrate gets lowered after the leading zone instead of during. At least the CQP zone is more or less respected. But pass 2 goes completely crazy, doesn't respect the trailing zone quantizer at all (10..12 instead of 40).

The HEVC outputs and the detailed CSV logs are available in this archive.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid

Last edited by LigH; 14th March 2016 at 09:42.
LigH is offline   Reply With Quote
Old 14th March 2016, 10:04   #6  |  Link
Jamaika
Registered User
 
Join Date: Jul 2015
Posts: 645
Quote:
Originally Posted by LigH View Post
Now the quantizer distribution with 3 zones: 30% bitrate in the beginning (frames 0..223), constant quantizer 40 in the end (frames 1140..1253), and 100% bitrate in the middle (no explicit definition).
I have such doubts. Does the quantizer always create bad value in the ranges 0-223 and 1140-∞ for any framerate?
I'm afraid not. This is a little troublesome for the 'zone' that you first have to create the video to determine misalignment ranges the quantizer.

Last edited by Jamaika; 14th March 2016 at 11:12.
Jamaika is offline   Reply With Quote
Old 14th March 2016, 10:16   #7  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 5,997
I don't understand your argument. Is it not possible to declare zones as one likes?
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 14th March 2016, 11:23   #8  |  Link
Jamaika
Registered User
 
Join Date: Jul 2015
Posts: 645
It is possible . Just how much time you lose. I know that the NUL to pass = 1 quick count and should be corrected to pass = 2.

Last edited by Jamaika; 14th March 2016 at 11:32.
Jamaika is offline   Reply With Quote
Old 14th March 2016, 11:34   #9  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 5,997
Jamaika, what are you talking about, at all?

The topic of this thread is that if one declares a zone where the bitrate distribution shall differ from the rest of the move, the bitrate shall change as desired, not randomly or even to the opposite. Nobody talked about the speed yet, and I doubt that the encoding speed of zones is generally lower than without (except for zones with smaller quantizers, producing bigger results).

At the moment, all we discuss is: If I want x265 to make a scene smaller than normal, then x265 has to make it smaller than normal, not surprisingly bigger instead.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 14th March 2016, 23:59   #10  |  Link
djesteban
Registered User
 
djesteban's Avatar
 
Join Date: Aug 2008
Posts: 110
LigH, there's already an issue logged for this issue on bitbucket: Issue #172 Bitrate control: Zones are ignored in any but the first pass in n-pass encodes

I will add my test results there, maybe you could do the same and promote the ticket so it gets resolved faster :P
djesteban is offline   Reply With Quote
Old 15th March 2016, 00:27   #11  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 5,997
Yes, in July 2015, I already reported this issue, but forgot about it since. Possibly because there was never a reply that it got fixed, don't remember...
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 16th March 2016, 13:50   #12  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 5,997
A patch attempting to fix the zones has been proposed to the mailing list. I'll build another binary when it gets committed to the source repository.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 17th March 2016, 17:09   #13  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 5,997
Zones have been fixed in x265 1.9+96-b09998b1256e: Quantizer raises in pass 3 and 2 during the first zone (as expected for a bitrate target of 30%), and gets more or less close to the target of 40 in the last zone.

__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 22nd March 2016, 03:11   #14  |  Link
djesteban
Registered User
 
djesteban's Avatar
 
Join Date: Aug 2008
Posts: 110
Excellent news! Thanks LigH for doing the QA test
djesteban is offline   Reply With Quote
Old 28th May 2016, 14:11   #15  |  Link
RainyDog
Registered User
 
Join Date: May 2009
Posts: 176
Is possible to specify CRF rates to zones in x265 please?

For example the following works in x264 but not x265.

--zones 0,32742,crf=28 --zones 90324,95154,crf=28 --zones 181728,186768,crf=28

I'm encoding a film which has a few scenes of heavy machine-noise like grain that I want to reduce the bitrate distribution to. The majority of the film is very clean though so I want a much lower CRF for the overall value.

Thanks.
RainyDog is offline   Reply With Quote
Old 28th May 2016, 22:41   #16  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 5,997
At the moment, it seems that only a bitrate percentage or a fixed quantizer are supported. But adding support for CRF zones would probably be interesting too. And I hope it won't be too complicated, not much harder than QP zones...
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 29th May 2016, 08:56   #17  |  Link
RainyDog
Registered User
 
Join Date: May 2009
Posts: 176
Thanks LigH. Hopefully we see support for CRF zones in the not too distant future then.

In the meantime, what I've decided to do is add --vbv-bufsize 10000 --vbv-maxrate 10000 to the encode. I've run a quick x264 test encode without any bitrate limitations or zones and, looking at the bitrate graph, there's barely any spikes above 10-12mbps other than the flashback scenes with the heavy artificial grain/noise added (which shoot up to 50mbps+ if uncontrolled!).

So the majority of the encode should be unaffected by the max bitrate limitation but the aforementioned grainy scenes will get capped at 10mbps.
RainyDog is offline   Reply With Quote
Old 29th May 2016, 11:16   #18  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 5,997
If the encoder implements VBV checks correctly, then you can trust the result being decodable in a device with given limits.

From my experience in a DVD authoring studio, peaks are not your worst concern. I remember DVD Video material with occasional 13 Mbps GOP peaks causing no issues, but a permanent 10 Mbps burst being rejected by the authoring tool. The development of the bitrate during several seconds is the criterion, not a single GOP peak.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Old 31st May 2016, 20:32   #19  |  Link
benwaggoner
Moderator
 
Join Date: Jan 2006
Location: Portland, OR
Posts: 3,070
Quote:
Originally Posted by LigH View Post
If the encoder implements VBV checks correctly, then you can trust the result being decodable in a device with given limits.

From my experience in a DVD authoring studio, peaks are not your worst concern. I remember DVD Video material with occasional 13 Mbps GOP peaks causing no issues, but a permanent 10 Mbps burst being rejected by the authoring tool. The development of the bitrate during several seconds is the criterion, not a single GOP peak.
And one nice feature of x265 is that, if not otherwise specified, it will pick the lowest level that accommodates the frame size and fps, and then set vbv-maxrate and vbv-bufsize to the maximum values for that level. So it's pretty hard to make a non-complaint stream, unlike x264 which requires VBV parameters to be set manually (fixed in some patches).

For DVD, and all VBV cases, remember that a "peak" is measured over a number of bytes, so even if a single GOP is at more than 13 Mbps, the peak as defined by VBV can still be <10 Mbps. DVD GOPs are around half a second. average bitrate per GOP and peak bitrate converge a lot more with longer GOPs, but you can still have things that look non-compliant but aren't. And sometimes something that looks compliant that isn't, like if two clips are concatenated with the first ending in a peak and the second starting with a peak. Both could have been compliant by themselves, but VBV gets violated at the stich point.
__________________
Ben Waggoner
Principal Video Specialist, Amazon Prime Video

My Compression Book
benwaggoner is offline   Reply With Quote
Old 31st May 2016, 22:23   #20  |  Link
LigH
German doom9/Gleitz SuMo
 
LigH's Avatar
 
Join Date: Oct 2001
Location: Germany, rural Altmark
Posts: 5,997
One reason of sudden bitrate bursts are also subtitles, especially when there are subtitle pictures in several streams at the same time. Hardly an issue in home made media, but certainly an issue in commercial DVD productions with more than half a dozen subtitle streams.
__________________

New German Gleitz board
MediaFire: x264 | x265 | VPx | AOM | Xvid
LigH is offline   Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off

Forum Jump


All times are GMT +1. The time now is 04:49.


Powered by vBulletin® Version 3.8.11
Copyright ©2000 - 2020, vBulletin Solutions Inc.