Third Party Dependencies
All third party dependencies are built independent of the main build at the moment. This is done to ensure:
- we are using the versions of software we intend to use with Ark, irrespective of distro
- we build everything statically (where possible)
- the third party packages are built as hermetically as possible
- we don’t waste time rebuilding third party packages on user’s machines
Some of this could be alleviated by moving to something like Bazel, and rewriting the third party build systems to work within Bazel. Right now, we use cmake, and didn’t have the resources to rewrite all of our third party dependencies.
In general, care was taken to ensure that you could use distro-related packages in place of Ark packages, but it’s not something regularly tested.
Rebuilding Third Party Dependencies
To build the dependencies, you need docker installed on your machine. Run:
cd ~/ark/packages ./build-hermetic.sh ~/ark/third_party --package
Note if you have additional
third_party directories you should reference
those as well. For example, if you have project specific dependencies, you
may wish to provide:
cd ~/ark/packages ./build-hermetic.sh ~/ark/third_party /path/to/my/project/third_party--package
Using Custom-Built Third Party Dependencies
You can make use of these dependencies by setting an environment variable
make.sh to point at the built dependencies.
Commonly, this looks like:
You do not have to make clean or anything like that; everything that needs to rebuild will rebuild automatically.
Fully Replacing the Ark Toolchains
The Ark build system determines which compiler toolchain, sysroot, and third
party bundles to pull down based on the contents of the
.packages.conf file that
is located in the
ARK_ROOT (root of the Ark repository) directory. It can be
overridden by placing a
.packages.conf file next to your
env.sh symlink in your
This file contains a set of URLs that are used to determine where to pull packages from, along with a set of GUIDs that determine the packages to download.
Ark itself is only tested against the packages that are in the
that we provide. See the comments in the
.packages.conf file for details on
subdirectory layout if you wish to host your own packages.
The overall architecture of this build is similar to cmake’s
Each dependency is represented by a YAML file and several patch files. If the any of this content changes, the entire dependency is cleaned up and rebuilt from scratch.
Everything is built within a docker container with minimal dependencies installed. Additionally, every package that is built is passed in appropriate CFLAGS (or equivalent) that contain references to the compiler, linker, sysroot, and paths to the third party packages that you declared a dependency on.
The YAML syntax used for defining a build is not yet documented, but loosely
follows the cmake
Packages are built into
Each package is given a
hash that is computed from looking at its YAML file and various
patches. This is the next level of the tree, looking like this:
Finally, each package has three levels of directories underneath it:
source- The unpacked source of the third party dependency, with patches applied
build- The directory for the out-of-source build
install- Where the contents of the dependency is installed to
Within the packages directory for your architecture (
packages/x86_64 for example), there
exist three files.
The first is
env, which contains an environment that you can source into your bash
shell. This sets up various paths and variables such that you can build dependencies
outside of the build system.
For example, you can build gtest by running:
cd ~/ark . env.sh cd ~/ark/packages/.toolchain/packages/x86_64 . env cd gtest/bd5cf648d8bf19c230973c9dda2f69c41e96108bc69f7c260d686d28e39e6e61/build ninja install
Feel free to make local changes in the
source directory of gtest and rebuild
with the above command.
WarningAll of these changes will be lost if someone else builds the packages, or if you change anything in your YAML file. This is just for rapid testing or debugging.
The easiest way we’ve found to make patches is to check out the package that you wish to patch, and create a git repository within it:
git init git add * .git* git commit -a -m Initial
You can then use the diff command to generate patches:
git diff > ~/ark/third_party/gazebo/build-fixes.patch
You may need to apply all of the existing patches to ensure your patch applies cleanly, depending on where you are making changes.