Whenever relevant, enable incremental builds and add short-circuit checks to improve performance.
Incremental
Many toolchains default to enabling incremental builds and simply require the prior build output and source files. Others may require environment variables or flags to enable an incremental build. Check the relevant toolchain documentation.
Only files within ${CI_PROJECT_DIR}
, clone root, are cached so ensure that relevant files end there.
Once any incremental settings are handled, ensure that the Intelligent Cache incremental
setting is also enabled. The incremental setting defaults to enabled
for jobs containing a cache
stanza and may be verified using the
log line.
job:
variables:
CEDARCI_INCREMENTAL: "true"
Short-circuit
For tools or tasks without incremental support a short-circuit can be added to avoid the task when the result is already available.
Consider the following example in which atool
is downloaded and executed. If atool
was already downloaded during the
previous instance, of an incremental job, there is no value in downloading it again.
#!/bin/sh
CHECKSUM="579e3b739ede5d133cc2fbf89dcca013"
if [ ! -f "manifest.${CHECKSUM}" ] ; then
# If the manifest is not already present then download and extract
# the tar containing the files: manifest.${CHECKSUM}, and atool.
wget "https://example.com/artifact.${CHECKSUM}.tar.xz"
file="artifact.${CHECKSUM}.tar.xz"
tar -xf "${file}"
rm "${file}"
fi
# Always execute on source files since they may have changed.
./atool
This technique can be adapted to accommodate a wide variety of tasks. Arbitrary files may be written to signify
the completion of tasks to be skipped in future runs. Be sure to use appropriate techniques to avoid skipping to often
(ex. updating the atool
version).
Such checks are also generally backwards compatible since the checked files will not exist outside of Cedar CI runners
and thus the task will always be performed. Tasks that cannot be modified while maintaining backwards compatibility can
be wrapped with conditions. CEDARCI=true
is present on enhanced runners to assist with this.