Detailed analysis of OpenGL es framework (8) -- OpenGL es Design Guide

Want to make APP Same thing as WeChat , Can run small programs smoothly ？ | Experience will send you to Xinjiang 、 Huawei 、 Cherry keyboard ！>>>

Version record

Version number	Time
V1.0	2017.10.01

Preface

OpenGL ES It's a powerful graphics library , It's cross platform graphics API, Belong to OpenGL A simplified version of .iOS The system can take advantage of OpenGL ES Send image data directly to GPU Rendering , This avoids from CPU The high performance consumption of computing and then sending it to the graphics card for rendering , Can bring better video effect and user experience . The next few articles will introduce iOS Systematic OpenGL ES frame . If you are interested, please read the above articles .
1. OpenGL ES Detailed analysis of the framework （ One ） —— Basic overview
2. OpenGL ES Detailed analysis of the framework （ Two ） —— About OpenGL ES
3. OpenGL ES Detailed analysis of the framework （ 3、 ... and ） —— Build for iOS Of OpenGL ES List of applications
4. OpenGL ES Detailed analysis of the framework （ Four ） —— To configure OpenGL ES The context of
5. OpenGL ES Detailed analysis of the framework （ 5、 ... and ） —— Use OpenGL ES and GLKit Drawing
6. OpenGL ES Detailed analysis of the framework （ 6、 ... and ） —— Draw to other rendering destinations
7. OpenGL ES Detailed analysis of the framework （ 7、 ... and ） —— multitasking , High resolution and other iOS function

OpenGL ES Design Guidelines - OpenGL ES Design Guide

Now you've got it in iOS Used in applications OpenGL ES Basic knowledge of , Use the information in this chapter to help you design your application's rendering engine for better performance . This chapter introduces the key concepts of the renderer design ; Subsequent chapters will expand this information based on specific best practices and performance techniques .

---

How to Visualize OpenGL ES - How to visualize OpenGL ES

This section introduces Visualization OpenGL ES Two perspectives of design ： As a client - Server architecture and pipeline . Both perspectives are useful in planning and evaluating the architecture of an application .

1. OpenGL ES as a Client-Server Architecture - OpenGL ES As a client - Server architecture

The figure below is visual OpenGL ES As a client - Server architecture . Your application will change state , Texture and vertex data and rendering commands are communicated to OpenGL ES client . The client converts the data into a format that the graphics hardware understands , And forward it to GPU. These processes increase the overhead of application graphics performance .

 

OpenGL ES client-server architecture

Achieving excellent performance requires careful management of this overhead . Well designed applications can reduce OpenGL ES Frequency of calls , Use data formats suitable for hardware to minimize translation costs , And carefully manage itself and OpenGL ES Data flow between .

2. OpenGL ES as a Graphics Pipeline - OpenGL ES As a graphics pipeline

The figure below is visual OpenGL ES As a graphics pipeline . Your application configures the graphics pipeline , Then execute the drawing command to send the vertex data to the pipeline . The successive stages of the pipeline run a vertex shader to process the vertex data , Combine vertices into primitives , Grid the original elements into fragments , Run the fragment shader to calculate the color and depth values for each clip , And the fragments are mixed into a frame buffer for display .

 

OpenGL ES graphics pipeline

Use pipes as Metal Model to determine what your application does to generate new frames . Your renderer design includes writing shader programs to handle the vertex and fragment phases of the pipeline , Organize vertex and texture data that you feed into these programs , And configuration drive pipeline fixed function stage OpenGL ES State machine .

Each stage in the graphics pipeline can calculate its results at the same time , for example , Your application may prepare new elements , A separate part of the graphics hardware performs vertex and fragment calculations on previously submitted geometry . However , The latter stage depends on the output of the earlier stage . If any pipeline stage performs too much work or too slowly , Other pipeline stages will be idle , Until the slowest stage is done . A well-designed application balances the work performed at each pipeline stage according to the graphics hardware function .

Important note ： When you tune the performance of your application , The first step is usually to determine which stage is the bottleneck , And why .

---

OpenGL ES Versions and Renderer Architecture - OpenGL ES Version and render Architecture

iOS Three versions of OpenGL ES. The new version offers more flexibility , Allows you to implement rendering algorithms that contain high-quality visual effects without affecting performance .

1. OpenGL ES 3.0

OpenGL ES 3.0 yes iOS 7 New features in . Your application can use OpenGL ES 3.0 In order to achieve advanced graphics programming technology , It used to be used only on desktop hardware and game consoles , To improve graphics performance and realistic visual effects .

OpenGL ES 3.0 Some of the main functions of are as follows . For a complete overview , see also OpenGL ES API Registry Medium OpenGL ES 3.0 standard .

OpenGL ES Shading Language Version 3.0 - OpenGL ES Color language version 3.0

GLSL ES 3.0 Added new features , Like a unified block ,32 Bit integers and other integer operations , Used to perform more general-purpose computing tasks in vertex and fragment shader programs . To use the new language in the shader program , Your shader source code must use #version 330 es The beginning of the order . OpenGL ES 3.0 Context and for OpenGL ES 2.0 The shaders written are compatible .

For more details , see also Adopting OpenGL ES Shading Language version 3.0 and OpenGL ES API Registry Medium OpenGL ES Coloring language 3.0 standard .

Multiple Render Targets - Multiple rendering targets

By enabling multiple render targets , You can create a fragment shader that writes multiple framebuffer attachments at the same time .

This feature can use advanced rendering algorithms , For example, delayed coloring , Your application first renders a set of textures to store geometric data , One or more shadow traverses read from these textures are then performed , And perform lighting calculations to output the final picture . Because this method pre computes the input to the lighting calculation , So the incremental performance cost of adding more lights to the scene is much less . The delay shading algorithm requires multiple render targets to support , As shown in the figure below , To achieve reasonable performance . otherwise , Rendering to multiple textures requires a separate drawing channel for each texture .

 

Example of fragment shader output to multiple render targets

You can use “ Create a framebuffer object ” Add multiple render targets in the procedure described in . You don't have to create a single color attachment for the framebuffer , It's about creating a few . then , call glDrawBuffers Function to specify which framebuffer The attachment , As shown in the following code .

// Setting up multiple render targets

// Attach (previously created) textures to the framebuffer.
glFramebufferTexture2D(GL_DRAW_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, _colorTexture, 0);
glFramebufferTexture2D(GL_DRAW_FRAMEBUFFER, GL_COLOR_ATTACHMENT1, GL_TEXTURE_2D, _positionTexture, 0);
glFramebufferTexture2D(GL_DRAW_FRAMEBUFFER, GL_COLOR_ATTACHMENT2, GL_TEXTURE_2D, _normalTexture, 0);
glFramebufferTexture2D(GL_DRAW_FRAMEBUFFER, GL_DEPTH_STENCIL_ATTACHMENT, GL_TEXTURE_2D, _depthTexture, 0);
 
// Specify the framebuffer attachments for rendering.
GLenum targets[] = {GL_COLOR_ATTACHMENT0, GL_COLOR_ATTACHMENT1, GL_COLOR_ATTACHMENT2};
glDrawBuffers(3, targets);

When your application issues a drawing command , The clip shader determines the color output for each pixel in each render target （ Or achromatic data ）. The following code shows a basic fragment shader , By assigning to and above Setting up multiple render targets The position matching fragment output variable set in the code , Present to multiple targets .

// Fragment shader with output to multiple render targets

#version 300 es
 
uniform lowp sampler2D myTexture;
in mediump vec2 texCoord;
in mediump vec4 position;
in mediump vec3 normal;
 
layout(location = 0) out lowp vec4 colorData;
layout(location = 1) out mediump vec4 positionData;
layout(location = 2) out mediump vec4 normalData;
 
void main()
{
    colorData = texture(myTexture, texCoord);
    positionData = position;
    normalData = vec4(normalize(normal), 1.0);
}

Multiple render targets can also be used for other advanced graphics techniques , For example, real-time reflection , Screen space ambient occlusion and volume lighting .

Transform Feedback - Switching feedback

The graphics hardware uses a highly parallel architecture optimized for vector processing . You can use the new transform feedback feature to make better use of this hardware , This allows you to capture the output of the vertex shader to GPU Buffer objects in memory . You can capture data from a rendering process , To use in another rendering process , Or disable parts of the graphics pipeline , And the transformation feedback is used in general calculation .

One technique that benefits from transformation feedback is the animated particle effect . The general architecture of rendering particle systems is shown in the following figure . First , The application sets the initial state of the particle simulation . then , For each frame rendered , The application runs its simulation steps , Update the position of each simulated particle , Direction and speed , Then draw the visible assets that represent the current state of the particles .

 

Overview of a particle system animation

Traditionally , Applications that implement particle systems are in CPU Run the simulation on , Store the simulation results in the vertex buffer , For rendering particle art . However , Transfer the contents of the vertex buffer to GPU Memory is time-consuming . Transforming feedback by optimizing modern GPU The function of parallel architecture in hardware , It solves this problem more effectively .

By changing feedback , You can design rendering engines to solve this problem more effectively . The following figure shows how the application is configured OpenGL ES Graphical piping for particle system animation Overview . because OpenGL ES Represent each particle and its state as a vertex ,GPU The vertex shader phase can run several particle simulations at a time . Because the vertex buffer containing the particle state data is reused between frames , So the data is transferred to GPU The expensive process of memory only happens once .

 

 

• At initialization , Create a vertex buffer , And fill in the data containing the initial state of all particles in the simulation .

• stay GLSL Vertex shader program to achieve particle simulation , And run it by drawing the contents of the vertex buffer containing the particle position data .

  • To enable transform feedback for rendering , Please call glBeginTransformFeedback function . （ Before resuming normal drawing, call glEndTransformFeedback()）
  • Use glTransformFeedbackVaryings Function to specify which shader output the transform feedback should capture , And use glBindBufferBase or glBindBufferRange Functions and GL_TRANSFORM_FEEDBACK_BUFFER Buffer type to specify the buffer to be captured .
  • By calling glEnable（GL_RASTERIZER_DISCARD） To disable rasterization （ And the rest of the pipeline ）.

• To render the displayed simulation results , Please use the vertex buffer containing the particle position as the input for the second painting pass , Turn on rasterization again （ And the rest of the pipes ）, And use vertex and fragment shaders suitable for rendering the visual content of the application .

• In the next frame , Use the vertex buffer output of the last simulation step as input to the next simulation step .

Other graphic programming techniques that can benefit from transformation feedback include skeletal animation （ It's also called peeling ） And ray travel .

2. OpenGL ES 2.0

OpenGL ES 2.0 Provides a flexible graphics pipeline with programmable shaders , And can be used for all current iOS equipment . stay OpenGL ES 3.0 Many of the features formally introduced in the specification can be passed through OpenGL ES 2.0 Extended to iOS equipment , So you can implement many advanced graphics programming techniques , Also compatible with most devices

3. OpenGL ES 1.1

OpenGL ES 1.1 Only basic fixed function graphics pipeline is provided . iOS Support OpenGL ES 1.1 Mainly for backward compatibility . If you are maintaining OpenGL ES 1.1 Applications , Please consider updating OpenGL ES Version code .

GLKit Frameworks can help you from OpenGL ES 1.1 Fixed function pipeline conversion to higher version . For more information , see also Using GLKit to Develop Your Renderer.

---

Designing a High-Performance OpenGL ES App - Design high performance OpenGL ES Applications

To make a long story short , Well designed OpenGL ES Application needs ：

• stay OpenGL ES Using parallelism in pipelines .
• Manage data flow between application and graphics hardware .

The figure below shows the use of OpenGL ES The process of animating the display application .

 

App model for managing resources

When app starts , The first thing to do is initialize the resources that will not change during the life cycle of the application . Ideally , The application encapsulates these resources into OpenGL ES In the object . The goal is to create any object that remains constant to the runtime of the application （ Even part of the application lifecycle , For example, the level duration in the game ）, Transactions increase initialization time for better rendering performance . Complex commands or state changes should be replaced with... That can be used for a single function call OpenGL ES object . for example , Dozens of function calls may be required to configure the fixed function pipeline . contrary , Compile graphics shaders at initialization time , And switch to the graphics shader through a single function call at run time . Create or modify expensive OpenGL ES Objects are almost always created as static objects .

The rendering loop will handle what you intend to present to OpenGL ES All items of context , Then the results are presented to the display . In an animated scene , Update some data per frame . In the internal rendering loop shown above , The application is updating rendering resources （ Create or modify in the process OpenGL ES object ） And submit the drawing commands that use these resources to alternate . The goal of this internal loop is to balance the workload , send CPU and GPU Parallel work , Prevent applications and OpenGL ES Access the same resources at the same time . stay iOS On , If the modification is not performed at the beginning or end of the frame , So modify OpenGL ES Objects can be expensive .

An important goal of this internal loop is to avoid transferring data from OpenGL ES Copy to application . Take the result from GPU Copied to the CPU It could be very slow . If the copied data is also later used as part of the process of rendering the current frame , As shown in the intermediate rendering loop , Your application will be blocked until all previously submitted drawing commands are completed .

After the application submits all the drawing commands required in the framework , Present the results to the screen . Non interactive applications copy the final image to the application memory for further processing .

Last , When your application is ready to exit or complete a major task , It can release OpenGL ES object , Provide additional resources for yourself or other applications .

Summarize the important features of this design ：

• Create static resources .
• The internal rendering loop alternates between modifying dynamic resources and submitting rendering commands . Try to avoid modifying dynamic resources , Except for the beginning or end of the frame .
• Avoid reading intermediate render results back to your application .

The rest of this chapter provides useful OpenGL ES Programming technology to achieve the function of this rendering cycle . Later sections will demonstrate how to apply these general techniques to OpenGL ES Specific areas of programming .

• Avoid Synchronizing and Flushing Operations
• Avoid Querying OpenGL ES State
• Use OpenGL ES to Manage Your Resources
• Use Double Buffering to Avoid Resource Conflicts
• Be Mindful of OpenGL ES State
• Encapsulate State with OpenGL ES Objects

---

Avoid Synchronizing and Flushing Operations - Avoid synchronization and refresh operations

OpenGL ES The specification does not require immediate command execution . Usually , The command will be queued to the command buffer , And then it's done by hardware . Usually ,OpenGL ES Waiting for the application to queue many commands , It is usually more efficient to send the command to the hardware batch . however , some OpenGL ES Function must immediately flush the command buffer . Other functions not only refresh the command buffer , It can also block , Until the previously submitted order is completed , Then you can control the application . The refresh and sync commands can only be used when this behavior is needed . Overusing refresh or sync commands can cause your application to stop waiting for hardware to finish rendering .

These situations require OpenGL ES Submit the command buffer to the hardware for execution .

• function glFlush Send command buffer to graphics hardware . Until it blocks the hardware command , But don't wait for the command to finish executing .
• function glFinish Refresh command buffer , Then wait for all previously submitted commands to complete execution on the graphics hardware .
• Function to retrieve the contents of the frame buffer （ Such as glReadPixels） Also waiting for the submitted command to complete .
• Command buffer full .

1. Using glFlush Effectively - To use effectively glFlush

On some desktops OpenGL In the implementation , Periodically call glFlush Functions can effectively balance CPU and GPU The job of , But in iOS Not so in China . from iOS The delay rendering algorithm based on tiling implemented by graphics hardware depends on buffering all vertex data in the scene immediately , Therefore, the hidden surface can be removed best . Usually ,OpenGL ES Applications can only call glFlush or glFinish function .

• When your application moves to the background , You should flush the command buffer , Because it is executed when your application is in the background OpenGL ES command , Lead to iOS Terminate your application . （ see also Implementing a Multitasking-Aware OpenGL ES App）
• If your application is shared across multiple contexts OpenGL ES object （ Such as vertex buffers or textures ）, Should be called glFlush Function to synchronize access to these resources . for example , After loading vertex data in a context , You should call glFlush function , To ensure that its content is ready to be retrieved by other contexts . When with others iOS API（ Such as Core Image） share OpenGL ES Object time , This suggestion also applies to .

2. Avoid Querying OpenGL ES State - Avoid querying OpenGL ES state

call glGet *（）, Include glGetError（）, You may need to OpenGL ES Execute the previous command before retrieving any state variables . This synchronization forces graphics hardware with CPU Run lock together , This reduces the chance of parallelization . To avoid that , Please maintain a copy of any state you need to query , And visit it directly , Instead of calling OpenGL ES.

When an error occurs ,OpenGL ES Set error flag . These and other errors appear in Xcode Medium OpenGL ES Frame Debugger Or in the instrument OpenGL ES In the analyzer . You should use these tools instead of glGetError function , If you call , Will reduce performance . Other inquiries , Such as glCheckFramebufferStatus（）,glGetProgramInfoLog（） and glValidateProgram（） And it's usually only useful for development and debugging . You should omit calls to these functions in the release version of your application .

---

Use OpenGL ES to Manage Your Resources - Use OpenGL ES Manage your resources

many OpenGL Data can be stored directly in OpenGL ES Render context and its associated shared group objects . OpenGL ES Implementation can convert data into the best format of graphics hardware . This can significantly improve performance , Especially for data that changes infrequently . Your application can also be directed to OpenGL ES Provides tips on how to use data . OpenGL ES Implementation can use these tips to process data more efficiently . for example , Static data may be placed in memory , Graphics processors can easily access , Or even into dedicated graphics memory .

---

Use Double Buffering to Avoid Resource Conflicts - Use double buffering to avoid resource conflicts

When your application and OpenGL ES Simultaneous access OpenGL ES Object time , There will be resource conflicts . When one participant tries to modify the... Used by another OpenGL ES Object time , They can block , Until the object is no longer in use . Once they start modifying objects , Other participants may not access the object , Until the modification is complete . perhaps ,OpenGL ES Objects may be implicitly copied , So that both participants can continue to execute the command . Both options are safe , But each option can become a bottleneck in your application . The figure below shows the problem . In this case , A single object has a texture ,OpenGL ES And your app to use . When an application tries to change the texture , You have to wait until the previously submitted drawing command is completed - CPU Talent and GPU Sync .

 

 

To solve this problem , Your application can do other work between changing objects and drawing . however , If your application doesn't have anything else to do , It should explicitly create two objects of the same size ; And a participant reads an object , Another participant modifies another object . The figure below shows the double buffering mode . When GPU When running on a texture ,CPU Will modify another texture . After initial start-up ,CPU or GPU I'm not free . Although for texture display , This solution is suitable for almost any type of OpenGL ES object .

 

Double-buffered texture data

Double buffering is enough for most applications , But two participants are required to complete the processing command in roughly the same time . To avoid blocking , You can add more buffers ; This enables traditional producers - Consumer model . If the producer completes before the consumer completes the processing order , It needs to be free buffer and continue to process commands . under these circumstances , Producers are idle only when consumers are lagging behind .

Double and triple buffers eliminate extra memory , To prevent the pipeline from stagnating . The extra use of memory can put pressure on other parts of the application . stay iOS On the device , There may be very little memory ; Your design may need to balance using more memory with other application optimizations .

---

Be Mindful of OpenGL ES State - Be careful OpenGL ES state

OpenGL ES Implementation to maintain a complex set of state data , Including the use of glEnable or glDisable Function setting switch , The current shader program and its uniform variable , The currently bound texture unit, the currently bound vertex buffer and its enabled vertex properties . The hardware has a current state , It's not compiled and cached in time . The switch state is expensive , So it's best to design your application to minimize state switches .

Do not set the state that has been set . After the function is enabled , It doesn't need to be enabled again . for example , If you call... With the same parameters more than once glUniform function ,OpenGL ES It may not be possible to check if the same unified state has been set . It just updates the status value , Even if the value is the same as the current value .

Avoid using special settings or closing routines to set states beyond what is necessary , Instead of putting such calls in the drawing loop . The set and close routines can also be used to turn on and off functions that achieve specific visual effects - for example , When drawing a wireframe outline around a textured polygon .

1. Encapsulate State with OpenGL ES Objects - Use OpenGL ES Object encapsulation state

In order to reduce state changes , Create multiple OpenGL ES State changes are collected from objects that can be bound through a single function call . for example , The vertex array object stores the configuration of multiple vertex attributes in a single object . see also Consolidate Vertex Array State Changes Using Vertex Array Objects.

2. Organize Draw Calls to Minimize State Changes - Organize drawing calls to minimize state changes

change OpenGL ES Status doesn't take effect immediately . contrary , When you issue a drawing command ,OpenGL ES Perform the necessary work , To draw a set of state values . You can reduce the need to reconfigure the graphics pipeline by minimizing state changes CPU Time . for example , Keep the state vector in your application , And only if your state changes between drawing calls, set the corresponding OpenGL ES state . Another useful algorithm is state sorting - Track the drawing operations you need to perform and the amount of state changes you need to perform , Then sort them , To execute the same state continuously .

OpenGL ES Of iOS The implementation can cache some configuration data needed for effective switching between states , But the initial configuration of each unique state set takes longer . To maintain consistent performance , You can use the “ Preheat ” Each state set you plan to use ：

• Enable the state configuration or shader you plan to use .
• Use this state configuration to draw a certain number of vertices .
  Refresh OpenGL ES Context , So that the drawing does not display in this pre heating stage .

Postscript

author ： Legend of swordsman
link ：https://www.jianshu.com/p/456c961164d9
source ： Simple books
The copyright belongs to the author . Commercial reprint please contact the author for authorization , Non-commercial reprint please indicate the source .