Release Packages

Overview

A release package is an archive (typically .tar.zst) that contains everything necessary to run a system. For example, in the case of a pipeline, it would include the core pipeline along with any necessary configuration.

These are typically defined in a YAML configuration file. You can then use the ark-deployment-packager tool to package the content up into an archive, and optionally upload to the Catalog.

Releases are stored on the Catalog, and the Supervisor system will take care of downloading and unpacking the releases.

Package Configuration

You configure a package through a “deployable” YAML file. As an example:

---
config:
  default:
    launch_parameters:
      - name: "Nano Data Collection Pipeline"
        entrypoint: "data_collection_nano"
        startup_policy: OnStartup
        restart_policy: Always
    config_package_paths:
      - "ark/data_collection/data_collection_nano.yml"

This provides the name of the pipeline, an entrypoint (the binary that is executed), and then a pointer to the config package path. The config package path is processed and all necessary dependencies are brought in.

startup_policy allows you to configure if the process will start automatically on startup or not. Set to ‘OnStartup’ to start the process when the supervisor starts, setting it to ‘OnDemand’ means that you have to manually start the process.

restart_policy allows you to configure if the process will restart. Set to ‘Always’ to automatically restart it if it ever exits. Set it to ‘OnFailure’ to only restart it automatically if it exits due to a crash. Setting this to ‘Never’ will cause it to never restart.

You can also specify additional command line parameters through the command_line declaration in the launch_parameters block.

Deployment Packager Tool

This tool consumes a deployable YAML file and produces a ZIP file package. It also allows you to include additional files into the ZIP file.

For example:

./build/ark-deployment-packager \
  --deployable ./ark/data_collection/deployable.yml \
  --add-file ./path/to/extra/file.txt \
  --add-file ./another/extra/file.txt \
  --working-directory /home/ark \

You can also use the --catalog-url argument to upload and register the release with the Catalog.

The --supervisor-url will actually upload the generated ZIP file to a Supervisor, instantly deploying it.

When adding files, the file will be copied into the root of the release ZIP file. You can copy it into an alternate location by specifying --add-file <SRC>:<DEST> (in other words, colon-separate out the source and destination location). Make sure to keep a trailing slash on directory destinations if you are copying an entire directory.

Notice

See the section below on using cmake to define a release package, that is often the better way to use this tool for releases.

Timeouts

You can set the ARK_SUPERVISOR_TIMEOUT environment variable to adjust the timeout of uploading a binary to either Catalog or the vehicle. This allows you to override the default timeout when building a release through cmake.

cmake Configuration

You can define a cmake target for a release, to make generation and pushing of releases a bit easier:

add_release(
    data_collection_nano_release
    DEPENDS
      data_collection_nano
      ark-platform-supervisor
      ark-fallback-utility
    RELEASE_NAME
      "My Name"
    KEYS
      ReleaseKey
    DEFINES
      Key=$<... cmake generator expression...>
    PACKAGE_LOCATION
      ${CMAKE_BINARY_DIR}/something.tar.gz
    CONFIGS
      deployable.yml)

In this example, we generate a new target known as data_collection_nano_release. This pulls in the data_collection_nano, ark-platform-supervisor, and ark-fallback-utility targets (and depends on them). It builds against the deployable.yml file located in the current source directory, populating any keys in it labeled Key with the given generator expression.

You can optionally specify a PACKAGE_LOCATION, otherwise, the default is used (the target name plus “.zip”).

You can optionally specify a CATEGORY variable, which provides a release category to the packager (typically done with the --category flag).

You can optionally specify a CONFIGS parameter to copy over the config package (and all of its dependent files) into the ZIP file.

You can also specify a VENVS parameter to copy the virtualenv (by Ark build system name) into the ZIP file. Note that there is some coupling between virtualenvs and the build operating system.

The RELEASE_NAME variable is optional but allows you to customize the ’name’ of the software release that you are producing.

Finally, it tries to sign the package with the ReleaseKey. You can do this if you have access to the S3 bucket where keys are stored. This will sign your package and make sure that is uploaded to the Catalog. If you provided an ORGANIZATION GUID, the package will be assigned to that organization automatically.