Explore cache configuration of Android gradle plug-in

What is configuration caching ？

Configuration caching is an improvement IDE And command line building speed . This is a Gradle 6.6 Version provides a highly experimental feature , It allows the build system to record a task's mapping information , And reuse it in the next build , To avoid configuring the whole project again . This function is also a continuation of the configuration phase improvements , These improvements introduce Lazy configuration (lazy configuration), To avoid unnecessary work during the configuration phase of the build . The importance of these improvements for rapid iterative development is self-evident , The latter is also Android Studio A use case that the team continues to focus on .

Performance improvements

The main goal of this feature is to speed up the build . stay Android edition  Santa Tracker  In engineering benchmarking analysis , For build processes with configuration caching enabled , We measured it in Android Studio The total build time in has been reduced 35% ( from 688ms To 443ms, The test platform is Linux, Use Intel Xeon Gold 6154 CPU @ 3.00GHz ). The following figure shows how to do this with and without configuration cache 100 The average total build time per build ( In Milliseconds ):

For some projects , The configuration phase may consume 10 Seconds or more , The effect of saving time is therefore more significant . Whether you're running a new build 、 Incremental build or update build , The cost of the configuration phase is the same . To measure the time spent in the configuration phase of your build , You can run in empty mode (dry run mode) Run the task , for example : ./gradlew :app:assembleDebug --dry-run.

To further avoid running the configuration process repeatedly , The configuration cache also allows tasks from the same project to run in parallel . before , Only by using  Worker API  Can run simultaneously , But because the configuration cache can ensure that the task is independent and cannot access the global shared state ( for example  Project  example ), So this behavior can be enabled by default . and , Dependency resolution results can be cached between runs , This helps to optimize the overall build time .

How to try ？

The configuration cache function is still in the experimental stage , We hope you can try it out and give us feedback . To use it in your build , It is necessary to ensure that all plug-ins used in all projects are compatible , It's to be safe ( back ) Serialize task graph . You may need to update some Gradle plug-in unit . You can use this  issue  To get a complete list of supported plug-ins , If the plug-in you are using is not in it , Please submit questions in their issue tracker , And from Gradle Link to the question  issue. 

the latest version Android Gradle The plug-in version is 4.1 ( At present for 4.1.0-rc03), But if you want to get all the bug fixes , Please try the latest 4.2 edition ( At present for 4.2.0-alpha13).Gradle The version of should be 6.6, And if you're using Kotlin, Please put Kotlin Gradle The plug-in is updated with the latest 1.4 edition ( relevant  Kotlin issue). Finally use the following code to update gradle.properties:

org.gradle.unsafe.configuration-cache=true
#  Use this mark with care , Because some plug-ins are not fully compatible 
org.gradle.unsafe.configuration-cache-problems=warn

View all Android Gradle The plug-in version , Please refer to the following page :

https://maven.google.com/web/index.html#com.android.tools.build:gradle

If configuration caching is enabled , You should be able to run the first time through Android Studio Of Build Output window or command line to see  "Calculating task graph as no configuration cache is available for tasks…" ( There is no configuration available for the current task , Generating mission map ...) word ; The configuration cache will be reused in the second run , So the output will contain "Reusing configuration cache. ( Reuse configuration cache )".

Whatever problems you encounter , Can be in  Android Studio issue track or  Gradle issue track We've been fed back .

How it works ？

Want to learn more about configuration caching , We start by understanding the configuration phase of the build . Even if you turn on the configuration cache , The first build will still go through this process . In the configuration phase , All the projects involved ( In evaluation  settings.gradle  When you get ) Will be configured based on the evaluation results of their build files . Usually all plug-ins are applied first , meanwhile DSL The object is instantiated ; Next, we will continue to evaluate the build file , and DSL Object will be assigned the value you specified . When the build file evaluation is complete , Would call Android Gradle plug-in unit ( And many other plug-ins that follow the same pattern ) Of  Project.afterEvaluate  Callback . During the call of this callback ,Android Gradle The plug-in does most of its work , This includes creating variations and registering tasks .

In evaluation DSL And after registering the task , The next phase builds a task map . The tasks you ask to perform and the tasks they depend on are fully configured . This process will continue until you reach the leaf task that you don't depend on . This phase of configuration will output a task diagram ,Gradle The scheduling mechanism in will use this task graph to run build operations . When the task map is completed , The configuration cache stores it on disk ( stay Gradle 6.6 In root engineering  .gradle/configuration-cache directory  Under the table of contents ) . It can serialize all the  Gradle-managed type ( Such as  FileCollection、Property、Provider) And all user-defined serializable types . At the end of this phase , The status of each task will be fully recorded and preserved .

At the second build , hypothesis Gradle Cache that can reuse records , The task graph of the requested task will be loaded 、 skip DSL assessment , Task configuration, etc . This means that all tasks will be instantiated , And all of their properties will be loaded from the cache . From this moment on , The build process is basically the same as a cache free build , The difference is that tasks can be run in parallel by default and the advantages of reusing dependency resolution results in the cache .

In order to guarantee the correctness ,Gradle Will continuously track all inputs that affect the cached task graph , Including build files 、 The task to be performed and the configuration process for Gradle And access to system properties . A request to run a different set of tasks results in a different task graph , So you need to create a new cache record . An example of a state that needs to be disabled is :  You have modified build File or buildSrc, And pass a different value to the environment variable or system property . To detect such changes , The build system will create a cache task diagram used by build A snapshot of the file ; Besides , It also detects buildSrc Are there any tasks that have not been updated in . Last , Any value that affects the configuration phase should be packaged as Gradle-managed type , This helps build the system to keep track of the variables used in the configuration phase .

Use compatible Gradle API

All of the applications in the build Gradle Plug ins must be compatible with the configuration cache ,Gradle And so a new set of API. Here's how we configure caching and new API The constraints brought about by the investigation :

Use... In tasks Project example

Gradle The most common compatibility problem in plug-ins comes from the use of  Task.getProject(). When using the configuration cache , To keep each task completely independent , The task will not be able to access this shared state . because  Project  Instances can access  TaskContainer、ConfigurationContainer  And other objects that will not be populated during cache enabled runtime , This leads to an invalid state , So it's necessary to disable it . Introduced a lot of alternative API, For example, it is used to delay the creation of objects  ObjectFactory, There are also interfaces that can be used to get the distribution of project file systems , such as  ProjectLayout, If you need to start a process in the build , have access to  ExecOperations. You can refer to complete API list To do the migration work .

visit Gradle/ System Attributes and environment variables

If you use system properties 、Gradle attribute 、 Environment variables or extra files to specify the logical input for the build , What will be the result ？ The build system is already tracking build File modification , But any extra value that affects the task graph should use  ProviderFactory API To get . The following example shows how to get the  enableTask  System property value , And how to get system properties that are only input for tasks anotherFlag. If the value of the former changes , Then the cache fails ; And if the value of the latter changes , The cache will be reused , And the task is not up to date :

val systemProperty = project.providers.systemProperty("enableTask").forUseAtConfigurationTime()
if (systemProperty.orNull == "enabled") {
    project.tasks.register("myTask", …) {
        it.anotherFlag.set(project.providers.systemProperty("anotherFlag"))
    }
}

In the internal ,Gradle The value provider that resolves in the configuration phase (value provider) Keep track of , Each value provider is treated as a build logic input . in addition , Unless calls  Provider.forUseAtConfigurationTime(), Otherwise, the provider cannot be resolved , This makes it very difficult to introduce configuration phase input unexpectedly . As mentioned earlier , whatever Gradle Will be in build Disable configuration cache when file changes , This characteristic is related to  ProviderFactory API Together we make sure that Gradle You can capture everything that affects the task diagram .

Share work between tasks

If you want to be able to share some work between tasks , for example :  Avoid multiple connections to network servers or multiple resolution of certain information , Then you can use compatible configuration cache Shared build services To implement . It's like a mission , Build services can contain input information , And the content will be serialized after the first run . The run of the cache will simply deserialize the parameters and instantiate the build services required by the task . The added benefit of building a service is that it fits well into the build lifecycle , If you want to release some resources after the build is complete , Then use... In your build service  AutoCloseable  This function can be realized . Because it cannot be safely serialized to disk , Adding build monitor is not compatible with configuration cache .

From migration Android Gradle Lessons learned from plug-ins

Trying to make Android Gradle Plug in compatible configuration cache process , We learned something that might be useful to plug-ins and script writers .

First , After enabling the configuration cache , If you see something like this in the build output , Don't be discouraged. , Because many questions are repetitive , It can be solved easily :

428 problems were found reusing the configuration cache, 4 of which seem unique.

（ After reusing the configuration cache , Found out  428  Dealing with problems , among  4  It looks special ）

By migrating to the new API, We can solve many problems easily . for example :

Old code

abstract class MyTask: DefaultTask() {
    @TaskAction
    fun process() {
        project.exec(…)
        project.logger().log(…)
    }
}

Migrated code

abstract class MyTask: DefaultTask() {
   
   @get:Inject
   abstract val execOperations: ExecOperations
   
   @TaskAction
   fun process() {
       execOperations.exec(…)
       this.logger.log(…)
   }
}

If you still use Project example , Then you need to find an alternative API. For the most part , There will be a compatible API, You just need to migrate directly .

It is not convenient to create another object when serializing it , As a substitute , They will be created when needed in our task operations . for example , In the following example , We don't have to force Handler Types can be serialized , Because we only create it when we need it :

Old code

abstract class Mytask: DefaultTask() {
    private val handler: Handler by lazy { createHandler(someInput) }
    
    @TaskAction
    fun process() {
        handler.doSomething(…)
    }
}

Migrated code

abstract class Mytask: DefaultTask() {
    
    @TaskAction
    fun process() {
        val handler = createHandler(someInput)
    }
}

When creating a task , Please make sure that the task input correctly reflects everything the task needs during its execution . Avoid accessing environment objects or anything that can be accessed from  Project  Other objects accessed by instance . for example : If your plug-in creates a configuration , Please use it as  FileCollection  Pass it on to the task . If you need to build a directory location , Please record it in task Properties of :

Old code

abstract class MyTask: DefaulTask() {
    private val userConfiguration: MyDslObjects
    
    @InputFiles
    fun getClasses(): FileCollection {
        return project.configurations.getByName(userConfiguration.name)
    }
  
    @Internal
    fun getBuildDir(): File {
        return project.buildDir
    }
  
    @TaskAction
    fun process() { … }
}

Migrated code

abstract class MyTask: DefaulTask() {
    @get:InputFiles
    abstract val classes: ConfigurableFileCollection
   
    @get:Internal
    abstract val buildDir: DirectoryProperty
   
    @TaskAction
    fun process() { … }
}

project.tasks.register("myTask", MyTask::class.java) {
    it.classes.from(project.configurations.getByName(userConfiguration.name))
    it.buildDir.set(project.layout.buildDirectory)
}

Android Gradle A common pattern that plug-ins once relied on , It's initializing some objects on first use , Store it in a static field , And use the build listener to clear these states when the build is complete . As mentioned above , For this use case, you should use Shared build services . See the example below to see how to use it :

abstract MyBuildService: BuildService<BuildServiceParameters.None>, AutoCloseable {
    
    fun doAndCacheSomeComplexWork() { ... }
 
    override fun close() {
        //  Clear all States , Free memory 
    }
}

abstract class MyTask: DefaultTask() {
    @get:Internal
    abstract val myService: Property<MyBuildService>
}

The last piece of advice is , When you implement a custom serializable type , Pay attention to the serialized content . Make sure you don't serialize derived properties , And make these properties temporary or use functions instead . for instance , At cache runtime , You will be responsible for  allLines  Property gets an old value , So this operation is necessary .

Old code

class StringsFromFiles(private val inputs: FileCollection) {
    val allLines = inputFiles.files.flatMap { it.readLines() }
}

Migrated code

class StringsFromFiles(private val inputs: FileCollection):  Serializable {
    
    fun getAllLines() {
        return inputFiles.files.flatMap { it.readLines() }
    }
}

The configuration cache is still in the experimental stage , We hope you can try and give us feedback . You can  Android Studio issue track or  Gradle Of issue track Report any problems you have .

Happy coding ！