The framing efficiency of setpreviewcallbackwithbuffer will become lower

The following code implements a video capture without preview , And every time 10 Second count the frame rate ：

class MainActivity : AppCompatActivity() {
    

    private lateinit var camera: Camera
    private val surfaceTexture = SurfaceTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES)
    private val width = 1920
    private val height = 1080
    private var fps = 0
    private var start = 0L

    override fun onCreate(savedInstanceState: Bundle?) {
    
        super.onCreate(savedInstanceState)
        println("width = $width height = $height")
        camera = Camera.open()
        camera.parameters = camera.parameters.apply {
    
            previewFormat = ImageFormat.NV21
            setPreviewSize(width, height)
        }

        camera.setPreviewTexture(surfaceTexture)
        camera.setPreviewCallback {
     data, camera ->
            fps++
            val current = System.currentTimeMillis()
            if (start == 0L) {
    
                start = current
            }
            if (current - start >= 10000) {
    
                println(" Frame rate ：${
      fps} frame /10 second ")
                fps = 0
                start = current
            }
        }
        camera.startPreview()
    }

    override fun onDestroy() {
    
        super.onDestroy()
        camera.setPreviewCallback(null)
        camera.stopPreview()
        camera.release()
    }

}

The millet 11 pro The operation effect is as follows ：

width = 1920 height = 1080
 Frame rate ：301 frame /10 second 
 Frame rate ：300 frame /10 second 
 Frame rate ：300 frame /10 second 
 Frame rate ：300 frame /10 second 
 Frame rate ：301 frame /10 second 
 Frame rate ：300 frame /10 second 
 Frame rate ：300 frame /10 second 

The average frame rate is 30 frame / second . I heard that using buffers can improve efficiency , So we changed the way of video capture to using buffer , as follows ：

class MainActivity : AppCompatActivity() {
    

    private lateinit var camera: Camera
    private val surfaceTexture = SurfaceTexture(GLES11Ext.GL_TEXTURE_EXTERNAL_OES)
    private val width = 1920
    private val height = 1080
    private val callbackBuffer = ByteArray((width * height * 3) shr 1)
    private var fps = 0
    private var start = 0L

    override fun onCreate(savedInstanceState: Bundle?) {
    
        super.onCreate(savedInstanceState)
        println("width = $width height = $height")
        camera = Camera.open()
        camera.parameters = camera.parameters.apply {
    
            previewFormat = ImageFormat.NV21
            setPreviewSize(width, height)
        }

        camera.setPreviewTexture(surfaceTexture)
        camera.addCallbackBuffer(callbackBuffer)
        camera.setPreviewCallbackWithBuffer {
     data, camera ->
            fps++
            val current = System.currentTimeMillis()
            if (start == 0L) {
    
                start = current
            }
            if (current - start >= 10000) {
    
                println(" Frame rate ：${
      fps} frame /10 second ")
                fps = 0
                start = current
            }
            camera.addCallbackBuffer(callbackBuffer)
        }
        camera.startPreview()
    }

    override fun onDestroy() {
    
        super.onDestroy()
        camera.setPreviewCallback(null)
        camera.stopPreview()
        camera.release()
    }

}

The millet 11 pro The operation results are as follows ：

width = 1920 height = 1080
 Frame rate ：276 frame /10 second 
 Frame rate ：293 frame /10 second 
 Frame rate ：298 frame /10 second 
 Frame rate ：299 frame /10 second 
 Frame rate ：295 frame /10 second 
 Frame rate ：297 frame /10 second 

You can see , After using the buffer , Frame rate decreased , Efficiency has not improved , It can only be said that it saves more memory . The difference in test results is more obvious on the company's mobile phone , Such as the test without buffer ：

width = 1920 height = 1080
 Frame rate ：189 frame /10 second 
 Frame rate ：249 frame /10 second 
 Frame rate ：247 frame /10 second 
 Frame rate ：250 frame /10 second 
 Frame rate ：173 frame /10 second 
 Frame rate ：162 frame /10 second 
 Frame rate ：147 frame /10 second 
 Frame rate ：173 frame /10 second 

Test condition after using buffer ：

width = 1920 height = 1080
 Frame rate ：133 frame /10 second 
 Frame rate ：181 frame /10 second 
 Frame rate ：133 frame /10 second 
 Frame rate ：129 frame /10 second 
 Frame rate ：141 frame /10 second 
 Frame rate ：139 frame /10 second 
 Frame rate ：133 frame /10 second 
 Frame rate ：135 frame /10 second 

You can see , After using buffer, the frame rate decreases significantly . If you use YV12 Format ,

Based on this test result , I don't use buffers in our company's project development , Because the project we developed is running on a special Android On the device , This Android As soon as the device starts, only one of our applications runs , There are no other applications , So it's OK to consume more memory , Anyway, memory is enough .

There are also particular about the choice of format , For example, on the company's mobile phone , Use NV21 Format , wide 640, high 480, Use buffer , The test results are as follows ：

width = 640 height = 480
 Frame rate ：192 frame /10 second 
 Frame rate ：240 frame /10 second 
 Frame rate ：236 frame /10 second 
 Frame rate ：239 frame /10 second 
 Frame rate ：238 frame /10 second 
 Frame rate ：238 frame /10 second 

Frame rate is OK , Then we only change one format , hold NV21 Change to YV12, The test results are as follows ：

width = 640 height = 480
 Frame rate ：181 frame /10 second 
 Frame rate ：182 frame /10 second 
 Frame rate ：179 frame /10 second 
 Frame rate ：177 frame /10 second 

You can see , Just change a format , The frame rate drops significantly , Can you think that the camera acquisition of this device is used NV21 Collected , When we use YV12 When the format , The framework needs to be NV21 To YV12 Format conversion , Therefore, the frame rate decreases ？ When I use Xiaomi 11 pro There is no such problem during the mobile phone test , Is it because the mobile phone is fast , Fast format conversion , So the frame rate is not delayed ？