ark::catalog::UploadFileStage

Defined in header “ark/catalog/stages/upload_file_stage.hh”.

A stage that can work in conjunction with the telemetry stage or upload artifact stage to automatically upload file / metrics bundles to the Catalog. Useful for pushing things like parquet-files to Catalog.

Methods

  • UploadFileStage(std::string name)
    Constructor.

  • ~UploadFileStage()
    Destructor.

  • void initialize(pipeline::StageInterface & interface)
    Initializes the stage, preparing subscribers/publishers.

  • void start()
    Starts the stage, creating the background thread that uploads to the Catlaog.

  • void shutdown()
    Shuts down the stage, stopping all threads.

  • void finalize()
    Shuts down the stage, stopping all threads and completes any remaining uploads.