During the course of a simulation run or batch job, it’s not unusual to produce artifacts. These can include things such as metrics, images, videos, or any other file content you think is related to this run. For example, it is very common to associate a video with a simulation run.
When running pipelines, the
ark::catalog::UploadArtifactStage is added automatically to your pipeline
and will take care of uploading those artifacts for you. Simply write them out to a location on disk,
and then send off a
ark::catalog::UploadArtifactRequest message. This should consist of:
name- The name of the artifact that will appear in the catalog
local_location- The path on disk of the artifact to upload
remote_key_prefix- The category of data (such as
remote_key_name- A name for this object on storage (such as
The artifacts will be uploaded when possible, and they will be associated with your simulation job (or artifact that is being ingested). Users can access these through the ’external attachments' display in the catalog.