ark::catalog::MetricQueue

Defined in header “ark/catalog/metric_queue.hh”.

---

This class will store timestamped rbuf messages in a series of buckets, and can be used to periodically produce parquet files that can be sent to the catalog or written to disk.

Use this for accumulating metrics, onboard or offboard. The intent of this is to allow deterministic creation of parquet files.

Methods

• MetricQueue()
  Constructor. Creates an empty queue.

• MetricQueue(std::chrono::nanoseconds bucket_interval)
  Constructor. Creates a queue with the given bucketing interval.

• MetricQueue(std::chrono::nanoseconds bucket_interval, std::chrono::nanoseconds linger_duration)
  Constructor. Creates a queue with the given bucketing interval and linger duration. The linger duration defines how long we will keep around clean buckets before they are pruned. Longer numbers reduce the risk of clobbering existing files, which shorter numbers use less memory.

• MetricQueue(MetricQueueConfig config)
  Constructor. Initializes a queue with the given configuration.

• void push(const std::chrono::system_clock::time_point & row_time, const RbufType & row)
  Records the given item. Computes a bucket timestamp based on the bucketing interval provided in configuration.

  Note that metrics are not persisted when pushed; you must manually call persist().

• void persist()
  Records any existing metrics/buckets out to storage.

• size_t bucket_count()
  Returns the count of buckets stored in this queue.

• size_t row_count()
  Returns the count of rows stored over all buckets in this queue.

• std::vector< UploadFileRequest > flush(const std::filesystem::path & file_output_path, const std::string & bucket_name, const std::string & table_name, const std::string & robot_serial)
  Writes all of the parquet rows. This will record parquet rows to the given file path (if provided), and return a list of upload file requests for the Catalog (if a bucket is provided).

  Data will be marked as ‘clean’, so it will not be pushed a second time, unless more data is written out to the bucket.

• void prune(const std::chrono::system_clock::time_point & now)
  Prunes any buckets that are older than the threshold for keeping around lingering data, based on the timestamp given above.

  Metrics will be automatically persisted once pruning is complete.

• void clear()
  Clears all existing buckets. Any persisted metrics will also be cleared.