KBEA-00130 - Best Practices for Android Builds

This KB article applies to ElectricAccelerator 9.0 and newer versions.

Overview

This article describes the best practices for running Android builds. It includes descriptions of the ElectricAccelerator template files that you customize to optimize your builds.

Recommended Agent Deployment and Sizing

For the fastest Android builds, you should have at least 48 ElectricAccelerator agents, unless you use JobCache to replay a cached result. In internal tests, Electric Cloud has found that using local cores and agents using ElectricAccelerator Developer Edition combined with remote agents from a back-end cluster is a cost-efficient way to use numerous cores while still making the infrastructure available to a large developer community.

RAM Requirements for Agent Hosts

The recommended total amount of RAM for an agent host is at least 2 GB per ElectricAccelerator agent plus the amount of RAM normally needed by your build. For example, if you run four agents and your build normally needs 16 GB, you need ((2 * 4) + 16) = 24 GB.

The 16 GB that is normally needed for Android builds is quoted from the following sources:

https://source.android.com/source/building.html

https://groups.google.com/forum/?hl=en#!topic/android-Zbuilding/N1exifTpPfA|https://groups.google.com/forum/?hl=en#!topic/android-building/N1exifTpPfA

Template Files for Build Optimization

ElectricAccelerator provides a set of files that work together to provide optimal settings for Android 6.x (Android M) and Android 7.x (Android N). You customize these files for your builds.

For ElectricAccelerator 9.0, these files are in the attached unsupported_conf.tgz file. For ElectricAccelerator 9.0.1, these files are also available in the /opt/ecloud/i686_Linux/unsupported/conf directory. 

eadroid.sh

eadroid.sh is a bash script that runs your build. Two versions are provided: One for Android M and another for Android N.

The top few lines of this file specify details such as the locations of your Android source code and your history file. Following are the top few lines in the template file for Android N:

   ...

# Edit these:
BUILD_KEY=androidn_1
ANDROID_SOURCE="/c/src/android/AndroidN"
HISTORY="$ANDROID_SOURCE/emake.data"
JAVA_DIR="/c/src/android/jdk1.8.0_72" # Android N
LOGS_DIR="$ANDROID_SOURCE/logs" # must exist already

...
  • BUILD_KEY distinguishes this build from other builds so that log files and annotation files have build-specific names. You must set it to a different value with each build. For example, you can set it to the output from the date command:
       BUILD_KEY ="androidn_$(date +%Y%m%d-%H%M%S)"
  • ANDROID_SOURCE points to the top-level directory of your Android source tree. This directory contains the makefile that starts the build.
  • HISTORY points to your history file. Change this setting only if your build stores history files elsewhere.
  • JAVA_DIR points to your JDK. For Android N, you must use JDK1.8.0_72.
  • LOGS_DIR points to your log files and annotation files. This directory must already exist.

The script also includes settings for addendum makefiles and the setting for a file introduced in ElectricAccelerator 9.0 that contains the pragmas for Ninja targets.

cachejavadoc.mk

cachejavadoc.mk contains pragma jobcache javadoc pragmas, which enable caching for rules that run Javadoc. The Javadoc caching feature can improve build performance by caching and reusing files that Javadoc generates.   

Following is the cachejavadoc.mk template file:

   #pragma jobcache javadoc
   out/target/common/docs/doc-comment-check-timestamp :

#pragma jobcache javadoc
out/target/common/docs/api-stubs-timestamp :

#pragma jobcache javadoc
out/target/common/docs/system-api-stubs-timestamp :

#pragma jobcache javadoc
out/target/common/docs/apache-http-stubs-gen-timestamp :

By default, the cache becomes obsolete when *.java pragma files are added to (or removed from) directories that are subject to wildcard matching by commands in the rule body. You can make the cache sensitive to additional wildcard patterns by using one or more -readdir options:

   #pragma jobcache javadoc -readdir pattern

where pattern is the pattern that you want to ignore. Javadoc caching supports multiple wildcard (glob) patterns.

noautodep.mk

noautodep.mk contains “hints” that the ElectricAccelerator eDepend feature uses to tailor dependency management for better Android build performance. Following is the noautodep.mk template file:

   #pragma noautodep */.git/*
   $(local-intermediates-dir)/libbcc-stamp.c :

   #pragma noautodep */out/target/product/generic/system/bin/cat
   $(linked_module) :

   #pragma nolookup noproguard.classes-with-local.dex
   #pragma nolookup noproguard.classes.dex
   #pragma nolookup javalib.jar
   #pragma nolookup classes-full-debug.jar
   out/target/common/docs/doc-comment-check-timestamp :

sensitivity.mk

sensitivity.mk contains a pragma to instruct parse avoidance to detect the dependence of a parse result upon path wildcard match results. This pragma makes the parse cache sensitive to the existence (or absence) of specific files in directories that it reads as well as in new subdirectories.

Following is the sensitivity.mk template file:

#pragma jobcache parse \

   -readdir Android.mk \
-readdir *.java \
-readdir *.c \
-readdir I*.aidl \
-readdir *.logtags \
-readdir *.proto \
-readdir *.rs \
-readdir *.fs \
-readdir *.html

cachekati.mk

Because output target names from Android N's kati tool do not use a well-known pattern, you cannot enable JobCache for kati via the eMake --emake-jobcache command-line option. Instead, you must apply the #pragma jobcache kati pragma inside a makefile such as the cachekati.mk template file.

cachekati.mk contains settings for caching kati target files (.ninja files if using the Ninja build system). Following is the cachekati.mk template file:

   #pragma jobcache kati 
$(KATI_BUILD_NINJA) :

emake.pragmas

emake.pragmas is a pragma addendum file. Using pragmas for builds with eMake Ninja emulation enabled (--emake-emulation-ninja) requires an addendum file that contains the pragmas for Ninja targets. (Pragma addendum files are optional with any other emulation mode.)

This file contains #pragma settings similar to those in other addendum makefiles, but it works for Ninja emulation, which has no addendum mechanism. It marks certain targets as “runlocal” for performance. It also disables certain auto-dependencies to match Android's natural behavior so that a build number change does not automatically cause a rebuild of large parts of the system.

Following is the emake.pragmas template file for Android M:

   #pragma runlocal
out/target/product/generic/obj/PACKAGING/systemimage_intermediates/system.img:
#pragma runlocal
out/target/product/generic/system.img:

Following is an excerpt from the emake.pragmas template file for Android N:

   #pragma runlocal
out/target/product/generic/obj/PACKAGING/systemimage_intermediates/system.img:
#pragma runlocal
out/target/product/generic/system.img:
#pragma jobcache kati
out/build-aosp_arm.ninja:
#pragma noautodep ./out/build_number.txt
out/host/common/obj/JAVA_LIBRARIES/dexdeps_intermediates/classes-full-debug.jar:
#pragma noautodep ./out/build_number.txt
out/host/linux-x86/obj32/STATIC_LIBRARIES/libaapt_intermediates/Resource.o:
    ...
   #pragma noautodep ./out/build_number.txt
out/host/linux-x86/obj/STATIC_LIBRARIES/libaapt_intermediates/AaptUtil.o:
#pragma noautodep ./out/build_number.txt
out/target/product/generic/obj/PACKAGING/systemimage_intermediates/system.img:
#pragma noautodep ./out/build_number.txt
out/host/linux-x86/obj32/STATIC_LIBRARIES/libaapt_intermediates/AaptConfig.o:
#pragma noautodep ./out/build_number.txt
out/host/linux-x86/obj/STATIC_LIBRARIES/libaapt_intermediates/Resource.o:
#pragma noautodep ./out/build_number.txt
out/host/linux-x86/obj32/STATIC_LIBRARIES/libaapt_intermediates/AaptAssets.o:

Optimizing Your Build

To optimize an Android build, complete the following steps.

1. Upgrade ElectricAccelerator to at least version 9.0. For instructions, see the ElectricAccelerator Installation and Configuration Guide at http://docs.electric-cloud.com/accelerator_doc/AcceleratorIndex.html.
 
2. If you use ElectricAccelerator 9.0, extract the attached unsupported_conf.tgz tar file to the <install_dir>/i686_Linux/unsupported directory. For example, extract it to /opt/ecloud/i686_Linux/unsupported.
 
3. Clean your build environment. To do so, remove the “out” directory and any residual eMake history files (the default history file name is emake.data).

4. Set up your environment as normal. For example:
   source build/envsetup.sh
   lunch aosp_arm-eng  
5. Run a “learning” build to generate a history file to record the build dependencies. A learning build also populates the cache.
 
6. (Optional) Copy the eadroid.sh build script to your desired location:
  • For Android M, copy the script from
   /opt/ecloud/i686_Linux/unsupported/conf/android/6.0.0/eadroid.sh
  • For Android N, copy the script from
 /opt/ecloud/i686_Linux/unsupported/conf/android/7.0.0/eadroid.sh 
7. Customize the
  • eadroid.sh build script to indicate the location of your Android source code, history file, and so on
  • cachejavadoc.mk file as needed to cache and reuse files that Javadoc generates
  • noautodep.mk file as needed to tailor dependency management for better performance
  • sensitivity.mk file as needed to instruct parse avoidance to detect the dependence of a parse result upon path wildcard match results
  • (kati on Android N or newer versions) cachekati.mk file as needed to cache kati target files such as Ninja files
  • (Required for Ninja targets) emake.pragmas file as needed to add the pragmas

8. Run a make “clean” and then rerun the build with the script from step 5. ElectricAccelerator uses the learning build from that step to optimize performance.

Note: If you want to determine why a cached result was not used, use the Job Cache Misses report in ElectricInsight version 5.0 and newer versions. For details, see the “Reports” chapter in the ElectricInsight User Guide at http://docs.electric-cloud.com/accelerator_doc/AcceleratorIndex.html.

Overview of File Patching for Android

Some aspects of the android build process conflict with accelerator's ability to virtualize builds and one of these is a tool called jack which compiles java source into dex bytecode. Jack is a java program and the android project has made it run as a server during the build so that its startup time is a penalty that only has to be paid once.

When a jack server is started by a job on an agent it sees the filesystem according to the needs of the current job that's running on that agent.  If a job on agent X requests actions on a server run by agent Y then there could be a problem as the jack server's view of the client filesystem may not match that of the job.

One way to get around this is to patch the scripts that start and run jack so that there is in effect one running under each agent.  In this scenario all the jobs that want to use jack can use the one that's local - that's running on the same agent. The change is actually quite simple - we tell jack to create its settings file in (and read it from) the location pointed to by $TMPDIR rather than in $HOME because TMPDIR is set uniquely for each agent.  It then chooses its own unique port and writes it to the settings file.

Electric Accelerator supports a feature known as file patching to do this automatically.  When a file is requested by an agent, the emake client checks its name, size and md5sum and if they match it supplies a "patch" rather than the original content. These patches are stored in a special format in /opt/ecloud/i686_Linux/patches. 

Electric Accelerator is supplied with patches for the jack script and one other supporting script (jack-admin) which are suitable for AOSP, the open source version of Android, at 6.0, 7.0 and 7.1.  Unfortunately some customers have made minor changes to these scripts which means that emake cannot recognise the file to patch with 100% certaintly and it so does nothing. 

How to Create Patches

Firstly one needs have the customers original copy of the jack and jack startup scripts. These can be found in:

  prebuilts/sdk/tools/jack
  prebuilts/sdk/tools/jack-admin

One should then add these files to Electric Accelerator's patch database with the addpatch.sh script in the unsupported/addpatch directory of the Electric Accelerator installation.

This script displays a path to the database entry e.g.

  /opt/ecloud/i686_Linux/patches/jack/4860/17f21ce31a981ae587e9e189da2edc9a/patch

This file, the "Patchfile", should be modified. The aim is to ensure that there is one jack server per agent and the following diffs are provided as examples for AOSP:

  unsupported/addpatch/android/android-6.0.1_r1/jack.diff
  unsupported/addpatch/android/android-6.0.1_r1/jack-admin.diff
  unsupported/addpatch/android/android-7.0.0_r1/jack.diff
  unsupported/addpatch/android/android-7.0.0_r1/jack-admin.diff

These diff files can be applied with the 'patch' tool from linux in this manner:

   cd /opt/ecloud/i686_Linux && 
   patch patches/jack/4860/17f21ce31a981ae587e9e189da2edc9a/patch < \
   unsupported/addpatch/android/android-6.0.1_r1/jack.diff

NOTE: the first parameter to patch is the output from the 'addpatch' script.

If the customer's versions of jack and jack-admin differ too markedly from the AOSP versions then these patches might not succeed and the effort will have to be made to apply them manually.

How to Check if the Patch Works?

Without doing a full android build it is possible to check that the patches are working by writing a little makefile similar to the following and running it from the top-level Android directory:

all:
		cat  prebuilts/sdk/tools/jack
		cat  prebuilts/sdk/tools/jack-admin

You can then visually check that the output shows your modified script rather than the original.

Before Building with a Patch

The Android build system makes copies of the jack and jack-admin scripts under the "out" directory. File patching doesn't affect uptodateness checks so these copies won't be updated just because a patch has been created for their "source". Thus it is best to do a "from-clean" build after creating patches.

See Also

Applies to

  • ElectricAccelerator 9.0 and newer versions

Attachments

Have more questions? Submit a request

Comments

Powered by Zendesk