Understand the rate control mode rate control mode CBR, VBR, CRF (x264, x265, VPX)

Statement ： This paper is about Understanding Rate Control Modes (x264, x265, vpx) Translation of .

What is? “ rate control ”？ This is what the video encoder does when deciding how many bits to spend for a given frame .（ lossy ） The goal of video coding is to save as many bits as possible , Reduce the file size of the original input file , While maintaining as much quality as possible . Rate control is a key step in determining the trade-off between size and mass .

There are many forms of rate control —— You will know the terminology “1-pass” and “2-pass”、“CBR” and “VBR”, Maybe you know “VBV Encoding” or “CRF”.

Why do you need to care ？ You often see examples of video coding commands that apply the wrong type of rate control mode or the wrong bit rate . This article is a brief guide to different models ; It explains when you use which as an end user . Please note that , This has nothing to do with the details of rate distortion optimization .

Preface ： Variable bit rate and constant bit rate

Many people are more familiar with rate control in audio coders , Especially those like me from MP3 People who grow up . To rip our CD, We have been using constant bit rate (CBR) code , Then came variable bit rate (VBR) code . Variable bit rates ensure that you are within a given constraint （ from VBR Quality level setting ） Get the lowest possible file size with the highest possible quality .

In short ,VBR Let the encoder be “ Hard to encode ” Use more , And save bits for the easily compressed part of the file . What are the difficulties and ease of compression ？ for example , A lot of motion in video needs more bits to encode , Because the difference between adjacent video frames will be greater . High spatial details and complex textures are also difficult to encode .

What is your coding scheme ？

Choosing a rate control mode depends largely on your use case . Generally speaking , There are many different scenarios that will affect the way you design your coding pipeline ：

1. The archive - You want to compress the file to store it in a file , For example, in external hard disk drives or networked storage . The file should be of the best quality possible with the smallest possible file size , But you don't care about the exact size .
2. Streaming — You want to use a typical VOD (VoD) Streaming solutions （ for example HTTP Progressive download or HTTP Adaptive streaming ） adopt Internet Send a file . You need to make sure that the file does not exceed a certain bit rate , Or you need to provide different representations of the same file at different nominal bit rates （ For adaptive flow ）
3. Real time streaming - And 2. similar , But you want to finish coding as soon as possible , And you know nothing about the content in advance .
4. Equipment code —— You want to put your documents in DVD、 Blue light, wait . You need to make sure that the file eventually has a certain size .

Understanding the scenario will help you choose the rate control mode .

Rate control mode

Now? , Let's take a closer look at different models . My post will be based on popular H.264 and H.265 Encoder x264 and x265 as well as libvpx Supported in ffmpeg Modes available in .

about x265, You may have to use --enable-libx265 compile ffmpeg. Use x265, Not all parameters can be passed directly , So you have to use -x265-params Options . about libvpx,ffmpeg Need to use --enable-libvpx compile .

A word of warning ： By default , image x264 Such an encoder will not use bits unnecessarily by default “ fill ” frame . It means , If you have a scenario that is very easy to code , Your bit rate may always be lower than the bit rate you specify . Don't worry about that —— Just remember that , If it is wasteful , Then it is meaningless to achieve an accurate target bit rate .

Constant Constant QP (CQP)

Quantizing parameters (Quantization Parameter) Control the compression amount of each macroblock in the frame . A larger value means a higher quantification 、 More compression and lower quality . A lower value means the opposite . stay H.264 in QP Range from 0 To 51, You can easily use for your entire x264 and x265 The coding process sets a fixed QP. Be careful ：libvpx There is no fixed QP Pattern .

ffmpeg -i <input> -c:v libx264 -qp 23 <output>
ffmpeg -i <input> -c:v libx265 -x265-params qp=23 <output>

Unless you know what you're doing and clearly want to do it , otherwise This mode is not recommended ！ Set fixed QP It means that the generated bit rate will vary greatly according to the complexity of each scene , This will lead to the low coding efficiency of the input video . If you set it low enough QP, The quality will be very good , But with CRF（ See below ） comparison , You may waste space , Or you cannot control the actual bit rate .

fit ： Research on video coding , Or if you don't CRF Pattern
Not suitable for ： Almost anything else

Please note that ,Netflix It is recommended to use fixed QP Encode for each shot encoding optimization , To achieve the best coding for each scenario . However , This requires a lot of processing and careful assembly of each coded lens , So unless you implement the whole framework , Otherwise, it is not what you should use “ One size fits all ” Methods .

Average bit rate Average Bitrate（ABR, Also known as “ Target bit rate ”）

ad locum , We give the encoder a target bit rate , And expect it to figure out how to achieve this bit rate ：

ffmpeg -i <input> -c:v libx264 -b:v 1M <output>
ffmpeg -i <input> -c:v libx265 -b:v 1M <output>
ffmpeg -i <input> -c:v libvpx-vp9 -b:v 1M <output>

Avoid using this mode ！ A major x264 Developers themselves say You should never use it . Why? ？ Because the encoder cannot know the previous content in time and accurately , It must guess how to achieve this bit rate . This means that the rate itself will be different , Especially at the beginning of the clip , And achieve the goal at a certain point in time . Especially for HAS Type of streaming media , This will lead to great quality changes in the short clip .

this Not a constant bit rate mode ！ although ABR Technically, it is a kind of VBR Pattern , But it is not much better than specifying a constant bit rate , Because it cannot reliably provide good quality .

fit ： Fast （ And dirty ） code
Not suitable for ： Almost anything

Constant bit rate Constant Bitrate (CBR)

If this is what your use case requires , You can force the encoder to always use a certain bit rate . Even if you don't need , The bit rate will always （ roughly ） identical . You can set nal-hrd The options are x264 Enable it ：

ffmpeg -i <input> -c:v libx264 -x264-params "nal-hrd=cbr:force-cfr=1" -b:v 1M -minrate 1M -maxrate 1M -bufsize 2M <output>

The output file must be .ts(MPEG-2 TS), because MP4 I won't support it NAL fill . Please note that , If your resources are easy to code , This mode wastes bandwidth , But it can ensure that the bit rate remains constant throughout the stream . Using this pattern in some applications may make sense , But you usually want to allow the stream to use a lower bit rate as much as possible .

about VP9, You need this ：

ffmpeg -i <input> -c:v libvpx-vp9 -b:v 1M -maxrate 1M -minrate 1M <output>

fit ： Maintain a constant bit rate （duh）; Video streaming （ for example Twitch）
Not suitable for ： The archive ; Efficient use of bandwidth

2-Pass Average bit rate （2-Pass ABR）

Allow the encoder to perform twice （ Or more times ） By enabling it to estimate the content ahead in time . It can calculate the cost of encoding frames in the first pass , Then use the available bits more effectively in the second pass . This ensures that the output quality is the best under certain bit rate constraints . This also means that the bit rate will change over time , This is why this model It can also be called variable bit rate (VBR), Although this is a little vague .

by x264 Enable it ：

ffmpeg -i <input> -c:v libx264 -b:v 1M -pass 1 -f null /dev/null
ffmpeg -i <input> -c:v libx264 -b:v 1M -pass 2 <output>.mp4

about x265, Replace libx264 by libx265 And set... In the private option field pass Options ：

ffmpeg -i <input> -c:v libx265 -b:v 1M -x265-params pass=1 -f null /dev/null
ffmpeg -i <input> -c:v libx265 -b:v 1M -x265-params pass=2 <output>.mp4

about VP9, It's like x264：

ffmpeg -i <input> -c:v libvpx-vp9 -b:v 1M -pass 1 -f null /dev/null
ffmpeg -i <input> -c:v libvpx-vp9 -b:v 1M -pass 2 <output>.webm

This is the simplest way to stream a file . There are two warnings ： You don't know what the final quality is , So you have to do some tests to ensure that your bit rate is actually high enough for some complex content . Another disadvantage of this mode is that the bit rate may have local peaks , This means that you may send more data than your client receives .

Apply to ： Achieve a certain target bit rate ; Equipment code
Not suitable for ： If you need fast coding （ for example , live broadcast ）

Constant mass Constant Quality (CQ) / Constant rate factor Constant Rate Factor (CRF)

I discussed in more detail in another article Constant rate factor . It can basically provide you with constant quality throughout the coding process . This is a “ Once and for all ” Things about —— Just specify CRF, Then let the encoder finish the rest of the work .

ffmpeg -i <input> -c:v libx264 -crf 23 <output>
ffmpeg -i <input> -c:v libx265 -crf 28 <output>
ffmpeg -i <input> -c:v libvpx-vp9 -crf 30 -b:v 0 <output>

stay H.264 and H.265 in ,CRF The range is from 0 To 51（ and QP similar ）.23 yes x264 A good default for , and 28 yes x265 The default value of .18（ or x265 by 24） It should be visually transparent ; Any lower value may only waste file size .±6 The value of will result in about half or twice the original bit rate . about VP9,CRF It can be from 0 To 63. The recommended value is from 15 To 35.

The only disadvantage of this mode is that you don't know the fluctuation of the generated file size or bit rate .

Please note that , With the same result bit rate 2-pass and CRF The coding should be the same in quality . The main difference is that , Use 2-pass, You can control the file size （ if necessary ）, While using CRF, You just need to specify the required quality .

fit ： archives ; Achieve the best quality
It's not good for ： Streaming media ; Obtain a certain bit rate / file size

Constraint coding Constrained Encoding (VBV)

Video Buffering Verifier A method is provided to ensure that the bit rate is limited to a maximum . This is useful for streaming , Because you can now be sure that you will not send more bits than you promised in a specific time frame .VBV Both can be with 2-pass VBR Use it together （ Use it twice ）, with CRF Code together —— It can “ add to ” To the rate control mode that has been presented . The latter mode is also called “capped CRF”.

Use -maxrate and -bufsize Options on VBV To set the maximum bit rate and the expected client buffer size ：

ffmpeg -i <input> -c:v libx264 -crf 23 -maxrate 1M -bufsize 2M <output>
ffmpeg -i <input> -c:v libx265 -crf 28 -x265-params vbv-maxrate=1000:vbv-bufsize=2000 <output>

VP9 There are similar patterns , Don't cry VBV, But the thinking behind it is the same ：

ffmpeg -i <input> -c:v libvpx-vp9 -crf 30 -b:v 2M <output>

Be careful ： If you do this for a real-time streaming application and want to speed up the coding process , about x264 and x265 You can add -tune zerolatency and -preset ultrafast Options . They reduce the quality you get at a specific bit rate （ Compression efficiency ）, But it will significantly speed up the processing . about libvpx-vp9, You want to set -quality realtime and -speed 5. For more information , see also H.264 and VP9 guide .

To compare this method with the constrained ABR-VBV Code together ：

ffmpeg -i <input> -c:v libx264 -b:v 1M -maxrate 1M -bufsize 2M -pass 1 -f null /dev/null
ffmpeg -i <input> -c:v libx264 -b:v 1M -maxrate 1M -bufsize 2M -pass 2 <output>

about x265：

ffmpeg -i <input> -c:v libx265 -b:v 1M -x265-params pass=1:vbv-maxrate=1000:vbv-bufsize=2000 -f null /dev/null
ffmpeg -i <input> -c:v libx265 -b:v 1M -x265-params pass=2:vbv-maxrate=1000:vbv-bufsize=2000 <output>

about VP9：

ffmpeg -i <input> -c:v libvpx-vp9 -b:v 1M -maxrate 1M -bufsize 2M -pass 1 -f null /dev/null
ffmpeg -i <input> -c:v libvpx-vp9 -b:v 1M -maxrate 1M -bufsize 2M -pass 2 <output>

Be careful ： It can also be used here one-pass Method , according to x264 What developers say , this Usually with twice pass As good as , But it doesn't compress clips effectively .

How should you set bufsize？ It depends on how much variability you want the bit rate to have . A good default is that the buffer size is twice the maximum rate , However, the recommendations may vary depending on the flow settings . If your client buffer is small （ About a few seconds ）, Your bufsize Should be the same maxrate Same size . If you want to limit the bit rate of the stream , Please try to bufsize Set to half the maximum rate or less .

When you will VBV be applied to CRF When coding , The trick is to find one CRF value , On average, this value will produce the maximum bit rate you want , But not more . If your code is always “ Maximize ” Your maximum bit rate , Yours CRF It may be set too low . under these circumstances , The encoder will try to use bits that it does not have . On the other hand , If your CRF Higher , So that the bit rate does not always reach the maximum , You can still reduce it to get some quality . for example , Are you VBV In the case of CRF 18 Encoding . Your clip ended up with 3.0 Mbit/s The average bit rate of ends . But you want your VBV Set to limit clips to 1.5 Mbit/s, So you need to CRF Increase to about 24 To get only half the bit rate .

fit ： Streaming under bandwidth constraints ; live broadcast （ Use CRF,1-pass）;VoD Streaming media （ Target bit rate ,2-pass）
Not suitable for ： People who want to play ; archives

A comparative example

This is a quick comparison between different rate control algorithms . I shot it for free Big Buck Bunny and Tears of Steel Video sequence , And three different parts are selected （ The length is 30 second ）. Please note that , These sequences are uncompressed raw material . And then use libx264 And its default settings to encode video . The only difference is the different rate control mode . I set different target bit rates （750、1500、3000、7500 kbit/s） And maximum speed （ about VBV） and QP/CRF value （17、23、29、35）. You can go to GitHub Found on the Scripts for comparison .

Please note that , This comparison is by no means exhaustive or completely representative . Usually , You want to try many different sequences of various genres and different encoders . I plan to update these tests in the near future . For all that , These tests should let you understand the role of different patterns .

Let's start with different bit rate control modes . The left column is 3000 kbit/s, The right column is 7500 kbit/s. I exclude other target bit rates , Because they don't show much difference in the chart , Because the bit rate is already at such a low point , So that the encoder has not much choice in how to allocate it . These lines are from Big Buck Bunny (BBB) and Tears of Steel (ToS) Different clips of . This line represents LOESS smooth - It indicates how the bit rate changes with the duration of the clip .

You can see —— Especially the first four contents ——ABR（ Turquoise line ） and ABR+VBV（ violet ） Incorrectly estimated the complexity of the clip . in fact ,Big Buck Bunny Sequence to fade 、 Smooth gradients and low motion start , This means that it can be compressed with good quality without too many bits .2-pass The method correctly starts with a lower bit rate and saves bandwidth . The last third of the video clip contains a lot of spatial details , This makes 2-pass The pattern uses more bits it saves at the beginning .

For the second Big Buck Bunny clip , Different coding patterns are actually better aligned than expected , Although it can be seen again 2-pass How to cause the change of bit rate to be more extreme than other modes .

Of course , There are also some clips that are very easy to code （ Or the complexity changes little ）. ad locum , The rate control mode has little difference .

For quality based models （CQP and CRF）, I only show CRF/QP 17 and 23 Result , They are in the quality range “ good ” End （ It's like 3000 and 7500 kbit/s yes “ good ” Value , For full HD video ）. The order of curves is opposite —— Lower means better quality ：

ad locum , You can see that 2-pass The same trend ： Bit rate follows content complexity . however , Use CRF, It is more restricted , You can save bits where you don't need them . The most interesting example is the lower left corner ：CRF Than Constant QP Save bit rate , As it usually does , But it does so in a constant offset . I have to guess why —— Maybe this article will update some further analysis .

Generally speaking , We can see CRF Methods how to match content well , As long as we can know the average bit rate in advance …… This is it. CRF+VBV Where it works ：

For a given CRF Choose the right goal / The maximum bit rate is usually a guess , And it all depends on the source video . however , If done correctly , You will not limit the quality too much , Push it to the limit （ Such as 3000 kbit/s and CRF 17 The situation of ）. You don't want the bit rate to change too much .CRF 23 yes libx264 Default Settings , You can see , Given the appropriate target bit rate setting （ Such as 7500 kbit/s） Under the circumstances , This coding mode will change the bit rate enough to solve the difference of content complexity , While still keeping right VBV Compliance model .

summary

It is not easy to understand different rate control modes . Unfortunately , The simplest solution （ Specify only the bit rate ） Not recommended at all , but Web You've been using this method to propagate code samples .

To make a long story short , According to your use case , You should do the following ：

• archives ——CRF, Can provide you with the required quality .
• Streaming - Having a containing VBV Bit rate two-pass CRF perhaps ABR.
• Real time streaming —— Having a containing VBV Bit rate one-pass CRF perhaps ABR. Or if you can waste bits , Use CBR.
• Equipment code —— Usually two-pass ABR.

More reading materials ：

Handbrake Wiki： Constant quality and average bit rate
H.264 Excellence Video coding settings
x264 Qualitative overview of rate control methods
Google ： Detailed introduction VP9 Bit rate mode