OpenGL es shared context for multi-threaded rendering

OpenGL ES When sharing context , What resources can be shared ？

OpenGL ES Shared context for multithreaded rendering

EGL Review the concept

EGL yes OpenGL ES And the local window system （Native Window System） Communication interface between , Its main function ：

• Communicate with the native window system of the device ;
• Query the available types and configurations of the drawing surface ;
• Create drawing surface ;
• stay OpenGL ES And other graphics rendering API Synchronous rendering between ;
• Manage rendering resources such as texture maps .

OpenGL ES It's with the help of EGL Realized ,EGL Shielding the differences between different platforms （Apple Provide your own EGL API Of iOS Realization , Claim to be EAGL）.

Local window related API Provides access to the local window system interface , and EGL You can create a rendered surface EGLSurface , At the same time, it provides a graphics rendering context EGLContext, For state management , Next OpenGL ES You can draw on this rendered surface .

egl、opengles And equipment

In the picture ：

• Display(EGLDisplay) It's an abstraction of the actual display device ;
• Surface（EGLSurface） The memory area used to store images FrameBuffer The abstraction of , Include Color Buffer（ Color buffer ）, Stencil Buffer（ Template buffer ） ,Depth Buffer（ Deep buffer ）;
• Context (EGLContext) Storage OpenGL ES Some status information of the drawing ;

stay Android Development on the platform OpenGL ES When applied , class GLSurfaceView Has provided us with the right Display , Surface , Context Management of , namely GLSurfaceView Internal implementation of EGL Encapsulation , It's easy to use interfaces GLSurfaceView.Renderer The implementation of the , Use OpenGL ES API Render and draw , To a large extent, it has improved OpenGLES The convenience of development .

Of course, we can do it ourselves EGL Encapsulation , This article is about Native Layer pair EGL encapsulate , Without the help of GLSurfaceView , Realize background rendering of pictures , utilize GPU Complete efficient image processing .

About EGL More detailed use ends , You can refer to OpenGL ES 3.0 Development （ 6、 ... and ）：EGL

What resources can be shared when context is shared

When sharing context , What resources can be shared across threads ？ This is the focus of this article .

To take care of the patience of some readers , Here is the conclusion .

Resources that can be shared ：

• texture ;
• shader;
• program Shader program ;
• buffer Class object , Such as VBO、 EBO、 RBO etc. .

Resources that cannot be shared ：

• FBO Framebuffer object （ Do not belong to buffer class ）;
• VAO Vertex array object （ Do not belong to buffer class ）.

Here's the explanation , In resources that cannot be shared ,FBO and VAO It's a resource management object ,FBO Responsible for managing several buffers , It doesn't take up resources ,VAO Responsible for managing the VBO or EBO , It doesn't take up resources .

That's the conclusion , The conclusion verification will be carried out in the next section , We're going to open up a new rendering thread in addition to the main rendering thread , Then the main rendering thread generates the texture 、 program And other resources are shared with new rendering threads .

Shared context multithreaded rendering

Shared context multithreaded rendering

This section Will be shared outside the main rendering thread EGLContext The way to open up a new off screen rendering thread , After that, the texture generated by the main rendering thread 、 program 、VBO Resources are shared with new rendering threads , Finally, it will be saved （ New rendering thread ） The texture of the rendered result is returned to the main thread for screen rendering .

Shared context

stay EGL_VERSION_1_4 （Android 5.0） edition , Called directly in the current rendering thread eglGetCurrentContext You can get the context object directly EGLContext .

C++ ,Java Each layer has its own context object API Realization ：

//Java
EGL14.eglGetCurrentContext();

//C++
#include "egl.h"
eglGetCurrentContext();

We use in New Threads EGL When creating a rendering environment , From the main rendering thread sharedContext To create a context object for the new thread .

EGLContext context = eglCreateContext(mEGLDisplay, config,
                                              sharedContext, attrib2_list);

Because we are in the new thread to render to the area outside the screen , Need to create PbufferSurface .

EGLSurface eglSurface = eglCreatePbufferSurface(mEGLDisplay, mEGLConfig, surfaceAttribs);

International practice , We will EGL All operations are encapsulated in a class EglCore It's easy to use , Specific code can refer to the project at the end of the article .

Multithreaded rendering

analogy Android Java Layer of Looper class , We are C++ Realization Looper Used to create new threads and manage messages in threads .

class Looper {

public:
    Looper();
    Looper&operator=(const Looper& ) = delete;
    Looper(Looper&) = delete;
    virtual ~Looper();

    void postMessage(int what, bool flush = false);
    void postMessage(int what, void *obj, bool flush = false);
    void postMessage(int what, int arg1, int arg2, bool flush = false);
    void postMessage(int what, int arg1, int arg2, void *obj, bool flush = false);

    void quit();

    virtual void handleMessage(LooperMessage *msg);

private:
    void addMessage(LooperMessage *msg, bool flush);

    static void *trampoline(void *p);

    void loop(void);

    LooperMessage *head;
    pthread_t worker;
    sem_t headWriteProtect;
    sem_t headDataAvailable;
    bool running;

};

stay GLRenderLooper Class OnSurfaceCreated、 OnSurfaceChanged、 OnDrawFrame Used to handle corresponding events .

enum {
    MSG_SurfaceCreated,
    MSG_SurfaceChanged,
    MSG_DrawFrame,
    MSG_SurfaceDestroyed,
};

class GLRenderLooper : public Looper {
public:
    GLRenderLooper();
    virtual ~GLRenderLooper();

    static GLRenderLooper* GetInstance();
    static void ReleaseInstance();

private:
    virtual void handleMessage(LooperMessage *msg);

    void OnSurfaceCreated();
    void OnSurfaceChanged(int w, int h);
    void OnDrawFrame();
    void OnSurfaceDestroyed();

    bool CreateFrameBufferObj();

private:
    static mutex m_Mutex;
    static GLRenderLooper* m_Instance;

    GLEnv *m_GLEnv;
    EglCore *m_EglCore = nullptr;
    OffscreenSurface *m_OffscreenSurface = nullptr;
    GLuint m_VaoId;
    GLuint m_FboTextureId;
    GLuint m_FboId;
};

In function GLRenderLooper::OnSurfaceCreated in , utilize sharedContext establish OpenGL Rendering environment .

void GLRenderLooper::OnSurfaceCreated() {

	// utilize  sharedContext  establish  OpenGL  Off screen rendering environment 
    m_EglCore = new EglCore(m_GLEnv->sharedCtx, FLAG_RECORDABLE);
    SizeF imgSizeF = m_GLEnv->imgSize;
    m_OffscreenSurface = new OffscreenSurface(m_EglCore, imgSizeF.width, imgSizeF.height);
    m_OffscreenSurface->makeCurrent();

    glGenVertexArrays(1, &m_VaoId);
    glBindVertexArray(m_VaoId);

    glBindBuffer(GL_ARRAY_BUFFER, m_GLEnv->vboIds[0]);
    glEnableVertexAttribArray(0);
    glVertexAttribPointer(0, 3, GL_FLOAT, GL_FALSE, 3 * sizeof(GLfloat), (const void *)0);
    glBindBuffer(GL_ARRAY_BUFFER, GL_NONE);

    glBindBuffer(GL_ARRAY_BUFFER, m_GLEnv->vboIds[1]);
    glEnableVertexAttribArray(1);
    glVertexAttribPointer(1, 2, GL_FLOAT, GL_FALSE, 2 * sizeof(GLfloat), (const void *)0);
    glBindBuffer(GL_ARRAY_BUFFER, GL_NONE);

    glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, m_GLEnv->vboIds[2]);
    GO_CHECK_GL_ERROR();
    glBindVertexArray(GL_NONE);

    if (!CreateFrameBufferObj())
    {
        LOGCATE("GLRenderLooper::OnSurfaceCreated CreateFrameBufferObj fail");
    }
}

GLRenderLooper::OnDrawFrame Function , When the drawing is finished, pay attention to the swap buffer , Then the texture of the drawn result will be saved , Pass the callback function to the main thread for screen rendering .

void GLRenderLooper::OnDrawFrame() {
    LOGCATE("GLRenderLooper::OnDrawFrame");
    SizeF imgSizeF = m_GLEnv->imgSize;

    glBindFramebuffer(GL_FRAMEBUFFER, m_FboId);
    glViewport(0, 0, imgSizeF.width, imgSizeF.height);
    glUseProgram(m_GLEnv->program);
    glBindVertexArray(m_VaoId);
    glActiveTexture(GL_TEXTURE0);
    glBindTexture(GL_TEXTURE_2D, m_GLEnv->inputTexId);
    GLUtils::setInt(m_GLEnv->program, "s_TextureMap", 0);
    float offset = (sin(m_FrameIndex * MATH_PI / 80) + 1.0f) / 2.0f;
    GLUtils::setFloat(m_GLEnv->program, "u_Offset", offset);
    glDrawElements(GL_TRIANGLES, 6, GL_UNSIGNED_SHORT, (const void *)0);
    glBindVertexArray(0);
    glBindTexture(GL_TEXTURE_2D, 0);
	
	// Note the swap buffer 
    m_OffscreenSurface->swapBuffers();
    glBindFramebuffer(GL_FRAMEBUFFER, 0);

	// The texture of the drawn result will be saved  m_FboTextureId  Pass it to the main thread for screen rendering 	
    m_GLEnv->renderDone(m_GLEnv->callbackCtx, m_FboTextureId);
    m_FrameIndex++;
}

Back to the main rendering thread ,Init The resulting texture will be rendered in the main when 、 program 、VBO Resources and EGLContext To the new thread .

m_GLEnv.sharedCtx     = eglGetCurrentContext();
m_GLEnv.program       = m_FboProgramObj;
m_GLEnv.inputTexId    = m_ImageTextureId;
m_GLEnv.vboIds[0]     = m_VboIds[0];
m_GLEnv.vboIds[1]     = m_VboIds[2];
m_GLEnv.vboIds[2]     = m_VboIds[3];
m_GLEnv.imgSize       = imgSize;
m_GLEnv.renderDone    = OnAsyncRenderDone;// Main thread callback function 
m_GLEnv.callbackCtx   = this;

// Send shared resources to New Threads 
GLRenderLooper::GetInstance()->postMessage(MSG_SurfaceCreated, &m_GLEnv);

GLRenderLooper::GetInstance()->postMessage(MSG_SurfaceChanged, m_RenderImage.width, m_RenderImage.height);

When the main thread is rendering , First send rendering instructions to the new thread , Then wait for its rendering to finish , When the new thread finishes rendering, it calls OnAsyncRenderDone Function to inform the main thread to render on screen .

void SharedEGLContextSample::Draw(int screenW, int screenH)
{
	{
		// Send render instructions to the new thread , Then wait for its rendering to finish 
		unique_lock<mutex> lock(m_Mutex);
		GLRenderLooper::GetInstance()->postMessage(MSG_DrawFrame);
		m_Cond.wait(lock);
	}

	// The main thread is rendering on the screen 
	glViewport(0, 0, screenW, screenH);
	glUseProgram(m_ProgramObj);
	GO_CHECK_GL_ERROR();
	glBindVertexArray(m_VaoId);
	glActiveTexture(GL_TEXTURE0);
	glBindTexture(GL_TEXTURE_2D, m_FboTextureId);
	GLUtils::setInt(m_ProgramObj, "s_TextureMap", 0);
	GO_CHECK_GL_ERROR();
	glDrawElements(GL_TRIANGLES, 6, GL_UNSIGNED_SHORT, (const void *)0);
	GO_CHECK_GL_ERROR();
	glBindTexture(GL_TEXTURE_2D, GL_NONE);
	glBindVertexArray(GL_NONE);

}

void SharedEGLContextSample::OnAsyncRenderDone(void *callback, int fboTexId) {
	// When the new thread finishes rendering, it calls  OnAsyncRenderDone  Function to inform the main thread to render on screen 
	SharedEGLContextSample *ctx = static_cast<SharedEGLContextSample *>(callback);
	unique_lock<mutex> lock(ctx->m_Mutex);
	ctx->m_FboTextureId = fboTexId;
	ctx->m_Cond.notify_all();
}

The last thing to note is this ： Multithreaded rendering should ensure that shared resources such as textures will not be accessed at the same time , Otherwise, rendering errors will occur .

Refer to the following project for the complete code , choice Multi-Thread Render：

https://github.com/githubhaohao/NDK_OpenGLES_3_0