ark::serialization::ParquetWriter

Defined in header “ark/serialization/parquet/parquet_writer.hh”.

This class can consume flat rbuf messages and convert them into rows within a parquet file. It’s typically used for emitting metrics. It uses reflection to retrieve values from the rbuf messages, and so performance is not as fast as writing directly from the structures.

Methods

  • ParquetWriter(const std::filesystem::path & output_path, const std::string & message_schema, const std::string & message_type, compression::CompressionType compression)
    Constructor. The given path is used as the path to write the Parquet file out to. The message schema/type come from rbuf.

  • ParquetWriter(core::ByteBuffer & output_buffer, const std::string & message_schema, const std::string & message_type, compression::CompressionType compression)
    Constructor. Wraps the given byte buffer, emitting parquet data to it. You maintain ownership of the buffer. The message schema/type come from rbuf.

  • ParquetWriter(ParquetWriter && other)
    Move constructor.

  • ~ParquetWriter()
    Destructor.

  • void close()
    Closes the underlying file. No further writes can take place.

  • void write(const RbufType & input)
    Writes the given message out as a row to the parquet file.

  • void write(const core::ByteBuffer & buffer)
    Writes the given message out as a row to the parquet file.