Containers

The catalog can manage the container repositories used for your various ingest and batch jobs for you.

Any user can create a container repository in their organization. They can then get login credentials for the repository, and from there, upload docker images using standard tooling.

Container information is not visible in the catalog frontend; you must use the ark-catalog-container-tool to work with container repositories.

Listing Repositories

You can list all of the repositories that you have access to by using the ark-catalog-container-tool and the ’list repositories’ operation:

~/ark$ ./build/ark-catalog-container-tool --list-repositories
Organization                         URL
2ea25035-6aa4-42fb-9955-92288ea1b972 095412845506.dkr.ecr.us-east-1.amazonaws.com/prod/2ea25035-6aa4-42fb-9955-92288ea1b972/catalog/extract-video
2ea25035-6aa4-42fb-9955-92288ea1b972 095412845506.dkr.ecr.us-east-1.amazonaws.com/prod/2ea25035-6aa4-42fb-9955-92288ea1b972/catalog/extract-gps
2ea25035-6aa4-42fb-9955-92288ea1b972 095412845506.dkr.ecr.us-east-1.amazonaws.com/prod/2ea25035-6aa4-42fb-9955-92288ea1b972/catalog/extract-annotations

From here you can see that three repositories are available to you, and it includes their full URL (the URL you would tag and push to).

Creating Repositories

Any user can create a repository in their organization(s). Any user in that organization can see that repository (or push to it).

To create a repository, use the container tool:

~/ark$ ./build/ark-catalog-container-tool --create-repository test/repo --organization <GUID>

This will create a repository with the name ’test/repo’ in the given organization. The full URL will be returned. Note that the URL will have ’test/repo’ in it, but will also include other paths, including your organization.

Logging in

You can log into the remote ECR by using the login operation:

./build/ark-catalog-container-tool --login catalog/extract-video --organization 2ea25035-6aa4-42fb-9955-92288ea1b972

This will log into the catalog/extract-video under the standard TBD organization. A base64-encoded string will be returned that can be used for your password. You can pipe it into docker as such:

./build/ark-catalog-container-tool ... | docker login --username AWS --password stdin

Note that the cmake docker rules take care of this step for you.

Uploading Images

The easiest way to do this is to use the built-in cmake docker rules. See the docker page for more details on the command.

Once you have your docker target defined, you can typically execute something like this:

./make.sh ark-extract-catalog-video-docker

This will handle automatically logging in to the appropriate registry, building the dependencies and container, and then pushing it up to the catalog.