Construction of Hisilicon 3559 universal platform: draw a frame on the captured YUV image

Preface

   The second step of the long march ,YUV The recognition and coding of are still in progress , I'm familiar YUV The principle and storage method of format , We can combine the first step from vpss Channel intercepted YUV Try to modify the image , Superimpose your own algorithm , First, make simple and rough changes , Subsequent replacement is not , After all, we are “ universal ” platform , Hey

Get ready

   Generally speaking , The image storage mode of Hisilicon is determined by the following parameters （ You can refer to VIDEO_FRAME_S）

HI_U32 u32Width;
HI_U32 u32Height;
VIDEO_FIELD_E enField;
PIXEL_FORMAT_E enPixelFormat;
VIDEO_FORMAT_E enVideoFormat;
COMPRESS_MODE_E enCompressMode;

   among enPixelFormat Determines that a pixel is YUV still RGB, yes planar still packet, That is, our last article focused on YUV The arrangement and combination of various sampling and storage of pictures
   enVideoFormat Determines how the pixels of a graph are placed , such as linear, tile wait
   enCompressMode Define the structure of video compressed data format , Determines whether our image is compressed , Theoretically 100 Pixel information is not the first 100 individual （ It must be smaller after compression ）
   If you want to understand the image data in memory , You must follow the corresponding format
   Generally speaking use
  enVideoFormat = VIDEO_FORMAT_LINEAR
  enCompressMode = COMPRESS_MODE_NONE
   Then choose one more enPixelFormat, It is the image storage format used in general software
   Image storage format refers to the placement of pixels , It has nothing to do with the storage we mentioned in the previous article , It is a storage method that does not involve a pixel, such as packet/planar, for instance linear, A row of pixels in memory , It is the pixel representing a line of image , Regardless of whether a pixel is actually placed together in memory (packet), Or two separate parts （semi-planar）, perhaps 3 Parts of （planar）
and tile It is for more efficient placement of hardware processing , It is usually a piece in the actual image , For example 16x16 Or bigger （ Is usually 16x16 Multiple ） Put the pixels in a row , If the linear The way to see it is not the original image

HI_MPI_SYS_Mmap

   Hess' pit is still strange , We were in the first step HI_MPI_VPSS_GetChnFrame All the image information has been saved to the structure VIDEO_FRAME_INFO_S in , According to common sense, image information should be stored in the virtual address of image data , But unfortunately , Other members in the structure have not found exceptions yet , This rather important virtual address has caused a single moth , This virtual address is actually unavailable ！ The precautions in the manual are also not mentioned （ I haven't found it yet ）！ It requires us to reset according to the physical address in the structure HI_MPI_SYS_Mmap Of ！ In order to avoid the pit falling in the first step , Corresponding HI_MPI_SYS_Munmap Nature is also essential （ Some releases in Hisilicon are not mandatory , Back osd Part will mention ）

typedef struct hiVIDEO_FRAME_S {
    
    HI_U32              u32Width;           // The width of the image .
    HI_U32              u32Height;          // Height of the image .
    VIDEO_FIELD_E       enField;            // Frame field mode .
    PIXEL_FORMAT_E      enPixelFormat;      // Video image pixel format 
    VIDEO_FORMAT_E      enVideoFormat;      // Video image format .
    COMPRESS_MODE_E     enCompressMode;     // Video compression mode .
    DYNAMIC_RANGE_E     enDynamicRange;     // Dynamic range .
    COLOR_GAMUT_E       enColorGamut;       // Gamut range 
    HI_U32              u32HeaderStride[3]; // Image compression head span .
    HI_U32              u32Stride[3];       // Image data span .
    HI_U32              u32ExtStride[3];    //10bit  Image with data bit width , Some data formats are stored in the former 8bit  And after  2bit  Separate storage , This refers to the back  2bit  Data span .

    HI_U64              u64HeaderPhyAddr[3];// Compression header physical address 
    HI_U64              u64HeaderVirAddr[3];// Compressed header virtual address . Kernel state virtual address 
    HI_U64              u64PhyAddr[3];      // Physical address of image data .
    HI_U64              u64VirAddr[3];      // Virtual address of image data . Kernel state virtual address .
    HI_U64              u64ExtPhyAddr[3];   //10bit  Image with data bit width , Some data formats are stored in the former 8bit  And after  2bit  Separate storage , This refers to the back  2bit  The physical address of the data .
    HI_U64              u64ExtVirAddr[3];   //10bit  Image with data bit width , Some data formats are stored in the former 8bit  And after  2bit  Separate storage , This refers to the back  2bit  Virtual address of data . Kernel state virtual address .

    HI_S16              s16OffsetTop;       /*  Image top clipping width top offset of show area */
    HI_S16              s16OffsetBottom;    /*  Image bottom clipping width .bottom offset of show area */
    HI_S16              s16OffsetLeft;      /*  Image bottom clipping width .left offset of show area */
    HI_S16              s16OffsetRight;     /*  Clipping width on the right side of the image .right offset of show area */

    HI_U32              u32MaxLuminance;    // The maximum brightness of the display image .
    HI_U32              u32MinLuminance;    // The minimum brightness of the display image .

    HI_U32              u32TimeRef;         // Image frame serial number .
    HI_U64              u64PTS;             // Image timestamp 

    HI_U64              u64PrivateData;     // Private data .
    HI_U32              u32FrameFlag;       /*  The mark of the current frame , Use  FRAME_FLAG_E  The value tag inside , It can be operated by bit or .FRAME_FLAG_E, can be OR operation. */
    VIDEO_SUPPLEMENT_S  stSupplement;       // Supplementary information of the image .
} VIDEO_FRAME_S;
/*  Define the frame information structure of video image  */
typedef struct hiVIDEO_FRAME_INFO_S {
    
    VIDEO_FRAME_S stVFrame; // Video image frame .
    HI_U32        u32PoolId;// Video buffer pool  ID.
    MOD_ID_E      enModId;// Which hardware logic module writes the current frame data 
} VIDEO_FRAME_INFO_S;

y,v,u Offset calculation of

   With the correct code available base addr, A combination of YVU420SP Storage method , It can be modified violently ！

   Anyway, we know YUV Value range of , Assign a value casually ,uv The quantity of each is one quarter y The number of , And alternate , that v High sum of y Compared with half , generous and easygoing y comparison , If the corresponding y The width of w Words ,v The width of is naturally w-w%2,u yes v Offset by one

			unsigned char* yuvData = pVirAddr;
            offset = u32Stride*u32Height;
            /*YUV420SPtest ok */
            for (h = 300; h < (u32Height-500); h++) 
            {
    
                for (w = 300; w < (u32Stride-500); w++,yIndex++) 
                {
       
                    uvIndex = (h / 2) * u32Stride + w - w % 2;
                    yIndex = h*u32Stride+w;
                    yuvData[yIndex] =210;
                    // y = yuvData[yIndex] & 0xff ;
                    // printf("y is %d \r\n",y);
                    yuvData[offset + uvIndex] = 200;
                    // u = yuvData[offset + uvIndex] & 0xff;
                    // printf("u is %d \r\n",u);
                    yuvData[offset + uvIndex + 1] =200;
                    // v = yuvData[offset + uvIndex + 1] & 0xff;
                    // printf("v is %d \r\n",v);
                }
            }

The final code

   Write our final picture frame “ Algorithm ” Thread

/* * describe  ： Thread is used for processing demo And save the image information  * Parameters  ：arg  For custom structure video_process_s,VPSS_GRP and VPSS_CHN Used to pass parameters to HI_MPI_VPSS_GetChnFrame * Return value ： nothing  * Be careful  ：HI_MPI_VPSS_GetChnFrame You must release , Otherwise get again VB Will report a mistake  VIDEO_FRAME_INFO_S The virtual address in cannot be used directly , Must be via physical address HI_MPI_SYS_Mmap After mapping, it can be used as the data storage address   It also needs to be used up HI_MPI_SYS_Munmap demapping  */
HI_VOID deal_myself_yuvdump(HI_VOID *arg)
{
    
    // HI_S32 y,u,v;
    HI_CHAR szYuvName[128];
    HI_S32 s32Ret;
    HI_S32 h,w,offset,yIndex,uvIndex;
    HI_VOID *pVirAddr;
    HI_U32 u32Height, u32Stride;
    VIDEO_FRAME_INFO_S stVideoFrame;
    VIDEO_FRAME_INFO_S* pstVideoFrame = &stVideoFrame;
    video_process_s* pstPara;
    pstPara = (video_process_s*)arg;
    while(1)
        {
    
        s32Ret = HI_MPI_VPSS_GetChnFrame(pstPara->VpssGrp, pstPara->VpssChn, &stVideoFrame,1000);
        if(s32Ret != HI_SUCCESS)
        {
    
            // SAMPLE_PRT("VPSS_GetChnFrame err for %#x!\n",s32Ret);//while1 Too many prints 
        }
        // printf("u32TimeRef is %d\r\n",pstVideoFrame->stVFrame.u32TimeRef);
        else
        {
    
            // printf("enPixelFormat is %d\r\n",pstVideoFrame->stVFrame.enPixelFormat);
            // SAMPLE_PRT("VPSS_GetChnFrame success for %#x!\n",s32Ret);
            u32Height = pstVideoFrame->stVFrame.u32Height;
            u32Stride = pstVideoFrame->stVFrame.u32Stride[0];
            pVirAddr = NULL;
            pVirAddr = (unsigned char *)HI_MPI_SYS_Mmap((unsigned int)pstVideoFrame->stVFrame.u64PhyAddr[0], u32Stride*u32Height*3/2);
            snprintf(szYuvName, 128, "./YUV/YUV_chn%d_%dx%d.yuv",
                    pstPara->VpssChn,stVideoFrame.stVFrame.u32Width, stVideoFrame.stVFrame.u32Height);
            // printf("Dump YUV frame of AVS chn to file: \"%s\"\n", szYuvName);
            fflush(stdout);
            pfd = fopen(szYuvName, "wb");
            unsigned char* yuvData = pVirAddr;
            offset = u32Stride*u32Height;
            /*YUV420SPtest ok */
            for (h = 300; h < (u32Height-500); h++) 
            {
    
                for (w = 300; w < (u32Stride-500); w++,yIndex++) 
                {
       
                    uvIndex = (h / 2) * u32Stride + w - w % 2;
                    yIndex = h*u32Stride+w;
                    yuvData[yIndex] =210;
                    // y = yuvData[yIndex] & 0xff ;
                    // printf("y is %d \r\n",y);
                    yuvData[offset + uvIndex] = 200;
                    // u = yuvData[offset + uvIndex] & 0xff;
                    // printf("u is %d \r\n",u);
                    yuvData[offset + uvIndex + 1] =200;
                    // v = yuvData[offset + uvIndex + 1] & 0xff;
                    // printf("v is %d \r\n",v);
                }
            }
            /* Y test ok*/
            // for(h = 300;h < (u32Height-1000);h++)
            // {
    
            // for(w =300; w < (u32Stride-1000); w++)
            // {
    
            // Xil_Out8(pVirAddr+(h*u32Stride+w),200);
            // // Xil_Out8(pVirAddr+27649+((h*u32Stride+w)/2),200);
            // // value = Xil_In8(pVirAddr+u32offsetv+(h*u32Stride+w)); 
            // // printf("%d.%dpVirAddr number is 0x%x\r\n",h,w,value);
            // }
            // }

           
            sample_yuv_8bit_dump(&stVideoFrame.stVFrame, pfd);// By standard YUV Save format as file 
            HI_MPI_SYS_Munmap((void*)pVirAddr, u32Stride*u32Height*3/2); 
            // printf("video process success\r\n");
            s32Ret = HI_MPI_VENC_SendFrame(pstPara->VpssChn, &stVideoFrame,1000);
            // printf("after send enPixelFormat is %d\r\n",pstVideoFrame->stVFrame.enPixelFormat);
            // if(s32Ret != HI_SUCCESS)
            // {
    
            // SAMPLE_PRT("HI_MPI_VENC_SendFrame err for %#x!\n",s32Ret);//while1 Too many prints 
            // // goto EXIT_VENC_H264_STOP;
            // }
            // else
            // {
    
            // cnt++;
            // // SAMPLE_PRT("HI_MPI_VENC_SendFrame success %d!\n",cnt); 
            // }
            
            // printf("u32TimeRef is %d\r\n",pstVideoFrame->stVFrame.u32TimeRef);
            HI_MPI_VPSS_ReleaseChnFrame(pstPara->VpssGrp, pstPara->VpssChn, &stVideoFrame);
        }
    } 
    // EXIT_VENC_H264_STOP:
    // SAMPLE_COMM_VENC_Stop(pstPara->VpssChn);
    // SAMPLE_COMM_SYS_Exit();
}

Reference resources sample Built-in tools , transplant yuv Image saving function

HI_U32 u32Size = 0;
FILE* pfd = HI_NULL;
unsigned char* TmpBuff = HI_NULL;
HI_CHAR* pUserPageAddr[2] = {
    HI_NULL, HI_NULL};

/* * describe  ： Used to hold 8bit yuv Image information  * Parameters  ：pVBuf  The structure of image information  * pfd  Pointer to save the file  * Return value ： nothing  * Be careful  ： The saved format is standard YUV420 Format  */
HI_VOID sample_yuv_8bit_dump(VIDEO_FRAME_S* pVBuf, FILE* pfd)
{
    
    unsigned int w, h;
    char* pVBufVirt_Y;
    char* pVBufVirt_C;
    char* pMemContent;
    HI_U64 phy_addr;
    PIXEL_FORMAT_E  enPixelFormat = pVBuf->enPixelFormat;
    HI_U32 u32UvHeight;

    TmpBuff = (unsigned char*)malloc(MAX_FRM_WIDTH);

    if (NULL == TmpBuff)
    {
    
        printf("malloc fail !\n");
        return;
    }

    if (PIXEL_FORMAT_YVU_SEMIPLANAR_420 == enPixelFormat)
    {
    
        u32Size = (pVBuf->u32Stride[0]) * (pVBuf->u32Height) * 3 / 2;
        u32UvHeight = pVBuf->u32Height / 2;
    }
    else if (PIXEL_FORMAT_YVU_SEMIPLANAR_422 == enPixelFormat)
    {
    
        u32Size = (pVBuf->u32Stride[0]) * (pVBuf->u32Height) * 2;
        u32UvHeight = pVBuf->u32Height;
    }
    else
    {
    
        u32Size = (pVBuf->u32Stride[0]) * (pVBuf->u32Height);
        u32UvHeight = pVBuf->u32Height;
    }

    phy_addr = pVBuf->u64PhyAddr[0];

    pUserPageAddr[0] = (HI_CHAR*) HI_MPI_SYS_Mmap(phy_addr, u32Size);

    if (HI_NULL == pUserPageAddr[0])
    {
    
        free(TmpBuff);
        TmpBuff = HI_NULL;
        return;
    }

    pVBufVirt_Y = pUserPageAddr[0];
    pVBufVirt_C = pVBufVirt_Y + (pVBuf->u32Stride[0]) * (pVBuf->u32Height);

    /* save Y ----------------------------------------------------------------*/
    // fprintf(stderr, "saving......Y......");
    fflush(stderr);

    for (h = 0; h < pVBuf->u32Height; h++)
    {
    
        pMemContent = pVBufVirt_Y + h * pVBuf->u32Stride[0];
        fwrite(pMemContent, pVBuf->u32Width, 1, pfd);
    }

    if (PIXEL_FORMAT_YUV_400 != enPixelFormat)
    {
    
        fflush(pfd);
        /* save U ----------------------------------------------------------------*/
        // fprintf(stderr, "U......");
        fflush(stderr);

        for (h = 0; h < u32UvHeight; h++)
        {
    
            pMemContent = pVBufVirt_C + h * pVBuf->u32Stride[1];

            pMemContent += 1;

            for (w = 0; w < pVBuf->u32Width / 2; w++)
            {
    
                TmpBuff[w] = *pMemContent;
                pMemContent += 2;
            }

            fwrite(TmpBuff, pVBuf->u32Width / 2, 1, pfd);
        }

        fflush(pfd);

        /* save V ----------------------------------------------------------------*/
        // fprintf(stderr, "V......");
        fflush(stderr);

        for (h = 0; h < u32UvHeight; h++)
        {
    
            pMemContent = pVBufVirt_C + h * pVBuf->u32Stride[1];

            for (w = 0; w < pVBuf->u32Width / 2; w++)
            {
    
                TmpBuff[w] = *pMemContent;
                pMemContent += 2;
            }

            fwrite(TmpBuff, pVBuf->u32Width / 2, 1, pfd);
        }
    }

    fflush(pfd);

    // fprintf(stderr, "done %d!\n", pVBuf->u32TimeRef);
    fflush(stderr);

    HI_MPI_SYS_Munmap(pUserPageAddr[0], u32Size);
    pUserPageAddr[0] = HI_NULL;

    free(TmpBuff);
    TmpBuff = HI_NULL;

}

Without considering alignment , Naturally, it's done ！, It also further verifies what we learned in the last article yuv Format storage