Loop filtering based on Unet

This article is from Dahua in JVET Proposals of the working group JVET-Y0086《A Unet-Based Deep In-Loop Filter》

brief introduction

---

because DNN The effect is remarkable in image denoising and detail restoration , If the loop filter used in video coding can better reconstruct the image and improve the coding efficiency . The proposal is based on HDRUnet Loop filtering technology , among HDRUnet It's based on Unet Network of .

Network structure

chart 1 Network structure

chart 1 It's a network structure , For the brightness component, the network input size is 160x160, stay CTU Size 128x128 The foundation is filled outward in each direction 16 Pixel , At the same time, the corresponding chrominance block is sampled to 160x160 As input .

When dealing with chromaticity , The network input size is 80x80, In chromaticity CTU Size 64x64 The foundation is filled outward in each direction 8 Pixel , At the same time, the corresponding brightness block is sampled to 80x80 As input .

Brightness and chroma train different models respectively , The two chrominance components share a set of model parameters .

The model is applied to loop filtering SAO after ,ALF Before . And in SPS Control the opening of the tool through the flag bit , At the same time, the frame level has flag bits to indicate whether the luminance and chrominance components are enabled .

Training and deriving information

---

For different QP={22,27,32,37,42} Training models separately , So brightness and chroma have their own 5 A model . Model training uses PyTorch frame , And use Libtorch Integrated into the VTM11.0_NNVC.

Model training information is shown in table 1,

Model derivation information is shown in table 2,

experimental result

stay VTM11.0_NNVC On AI Under configuration , The method Y、U、V Of BD-Rate Respectively -6.08%、 -20.34%、 -21.13%

Interested parties, please pay attention to WeChat official account Video Coding