ark::serialization

Enums

  • JsonReadValueType (int32_t)
    Potential types that a JSON value can be.

  • RbufConstantType (int32_t)
    The type of constant value held.

  • FieldType (int32_t)
    Indicates the type of the field this indicates what primitive ’type’ this field is (for example, if its a string or integer).

  • FieldAttribute (uint32_t)
    This is a bitmask containing various attributes that could be set for fields.

  • SchemaAttribute (uint32_t)
    This is a bitmask containing variou sattributes that could be set for schemas.

  • EnumType (int8_t)
    Indicates an enumeration type (bitmask or plain values).

Typedefs

Defined in “ark/serialization/field_value.hh”:

  • using ObjectPtr = std::shared_ptr< Object >
    Shared-pointer wrapper for Object.

Defined in “ark/serialization/schema.hh”:

  • using FieldDescriptorPtr = std::shared_ptr< FieldDescriptor >
    A pointer for fields.

Variables

Defined in “ark/serialization/json_reader.hh”:

  • concept GettableJsonType= std::is_same_v<ObjectType, std::string> ||
    std::is_same_v<ObjectType, bool> ||
    std::is_same_v<ObjectType, uint8_t> ||
    std::is_same_v<ObjectType, uint16_t> ||
    std::is_same_v<ObjectType, uint32_t> ||
    std::is_same_v<ObjectType, uint64_t> ||
    std::is_same_v<ObjectType, int8_t> ||
    std::is_same_v<ObjectType, int16_t> ||
    std::is_same_v<ObjectType, int32_t> ||
    std::is_same_v<ObjectType, int64_t> ||
    std::is_same_v<ObjectType, float> ||
    std::is_same_v<ObjectType, double> ||
    std::is_same_v<ObjectType, core::Guid> ||
    std::is_same_v<ObjectType, core::ByteBuffer> ||
    std::is_same_v<ObjectType, std::chrono::steady_clock::time_point> || std::is_same_v<ObjectType, std::chrono::system_clock::time_point> || std::is_same_v<ObjectType, std::chrono::nanoseconds>

    This concept can be used to ensure a template parameter is a supported type that we can get to.

  • concept GettableEnumType= std::is_enum_v
    This concept can be used to ensure a template parameter is an enumerated type.

  • concept ValueBasedEnumType= ark::serialization::is_value_based_enum::value
    This concept can be used to ensure a template parameter is a value-based enumerated type.

  • concept DeserializableObject= ark::serialization::is_serializable::value && !std::is_same_v<ObjectType, ark::core::Guid> && !std::is_same_v<ObjectType, ark::core::ByteBuffer>
    This concept can be used to ensure a template parameter is an serializable type.

Defined in “ark/serialization/json_writer.hh”:

  • concept Enumeration= std::is_enum_v
    This concept can be used to ensure a template parameter is an enumerated type.

  • concept SerializableObject= ark::serialization::is_serializable::value && !std::is_same_v<ObjectType, ark::core::Guid> && !std::is_same_v<ObjectType, ark::core::ByteBuffer>
    This concept can be used to ensure a template parameter is an serializable type.

Defined in “ark/serialization/magic.hh”:

  • constexpr uint8_t BITSTREAM_HEADER_MAGIC= 0xd0
    Magic bits that are at the top of every bitstream header.

  • constexpr uint8_t BITSTREAM_GROUP_MAGIC= 0xe0
    Magic bits that start every group.

  • constexpr uint8_t BITSTREAM_HAS_FOLLOWING_SECTION= 0x04
    Indicates (in the magic block) if there is a following block.

Classes

Functions

Declared in “ark/serialization/byte_buffer.hh”:

  • void serialize< core::ByteBuffer >(OutputStream & stream, const core::ByteBuffer & source)
    Supports serializations serializes the given byte buffer into the specified output stream.

  • void deserialize< core::ByteBuffer >(InputStream & stream, core::ByteBuffer & destination)
    Supports deserializations deserializes the given byte buffer into the specified output stream.

  • void hash< core::ByteBuffer >(Hasher & hasher, const core::ByteBuffer & source)
    Supports hashing. Hashes the given GUID into the ongoing hash.

Declared in “ark/serialization/cast.hh”:

  • OutputType cast_field_value(const FieldValue & value)
    Casts the given field value to the specified templated output type, based on the type that is contained within the field value itself.

    Throws if the value inside the field value is non-castable.

Declared in “ark/serialization/file.hh”:

  • std::shared_ptr< serialization::Object > read_generic_object_from_serialized_file(const std::filesystem::path & path)
    Reads an unknown object from the given file, returning it as a generic object. Uses schema information to deserialize it.

  • void write_serializable_type_to_file(const std::filesystem::path & path, const Type & type, core::WriteOptions options)
    Writes the given serializable type out to the specified file. This is just a helper routine wrapping serialize and writing to files.

    You can pass in additional write options if you wish to ensure data is ‘power safe’ (such as by passing in the WriteOptions::Synchronize flag).

  • core::ByteBuffer write_serializable_type_to_buffer(const Type & type)
    Writes the given serializable type out a buffer, returning the results as a byte buffer. This can then be written to disk and read in via routines in this file.

    This may make copies of the data during serialization, so is not as efficient as writing directly to a file.

  • Type read_serializable_type_from_file(const std::filesystem::path & path)
    Reads the given serializable type from the specified file. The path is loaded in from memory, and then deserialize is invoked on the resulting bytes.

Declared in “ark/serialization/generator_helpers.hh”:

  • std::string enum_field_to_base_type(const SchemaRegistry & registry, const FieldDescriptor & field)
    Resolves the given enumeration field in the registry and emits its base type as a string.

  • std::string enum_class_name(const SchemaRegistry & registry, const FieldDescriptor & field)
    Fetches the name of the enum, throws if the field isn’t an enum type.

  • std::string object_to_base_type(const SchemaRegistry & registry, const std::string & object_type)
    Resolves the given object field in the registry and emits its base type as a string.

  • std::string object_field_to_base_type(const SchemaRegistry & registry, const FieldDescriptor & field)
    Resolves the given object field in the registry and emits its base type as a string.

  • std::string array_field_to_base_type(const SchemaRegistry & registry, const FieldDescriptor & field)
    Resolves the given array field container type in the registry and emits its base type as a string.

  • bool has_field_type(const SchemaRegistry & registry, FieldType type)
    Returns true if we have any fields of the given type throughout the given registry.

  • bool has_field_attribute(const SchemaRegistry & registry, FieldAttribute attribute)
    Returns true if we have any fields with the given attribute throughout the given registry.

Declared in “ark/serialization/guid.hh”:

  • void serialize< core::Guid >(OutputStream & stream, const core::Guid & source)
    Supports serializations serializes the given GUID into the specified output stream.

  • void deserialize< core::Guid >(InputStream & stream, core::Guid & destination)
    Supports deserializations deserializes the given GUID into the specified output stream.

  • void hash< core::Guid >(Hasher & hasher, const core::Guid & source)
    Supports hashing. Hashes the given GUID into the ongoing hash.

Declared in “ark/serialization/helpers.hh”:

  • core::ByteBuffer serialize(const Type & type)
    This is a helper routine that allows you to serialize an object and get back a byte buffer, as a one-liner.

  • Type deserialize(const core::ByteBuffer & buffer)
    This is a helper routine that allows you to deserialize an object from a byte buffer, as a one liner.

  • Type deserialize(const std::string_view & buffer)
    This is a helper routine that allows you to deserialize an object from a string, as a one liner.

  • Type from_json(const std::string_view & buffer)
    This is a helper routine that can translate a JSON string into an object, with a one-liner.

  • Type from_json(const serialization::JsonBlob & blob)
    This is a helper routine that can translate a JSON blob into an object, with a one-liner.

Declared in “ark/serialization/json_patch.hh”:

  • Type apply_json_merge_patch(const Type & original, const std::string_view & patch)
    This allows you to take an existing serializable object and apply a ‘merge patch’ to it (for example, for changing a subset of the values in it).

  • Type apply_json_patch(const Type & original, const std::string_view & patch)
    This allows you to take an existing serializable object and apply a ‘JSON patch’ to it (for example, for changing a subset of the values in it).

Declared in “ark/serialization/json_reader.hh”:

  • const char * to_string(JsonReadValueType type)
    Converts the read value type into a string.

  • void get_to(const JsonReadOnlyValue & value, std::vector< bool > & destination)
    Retrieves the current value into the given vector.

  • void from_json(JsonObjectIterator iterator, Type & destination)
    A templated type that allows you to decode JSON into the given object, from an object iterator.

  • void get_to(const JsonReadOnlyValue & value, ObjectType & destination)
    Retrieves the current value into the given object, throwing if there is a type mismatch that cannot be converted automatically.

  • void get_to(const JsonReadOnlyValue & value, EnumType & destination)
    Retrieves the current value into the given object, throwing if there is a type mismatch that cannot be converted automatically.

  • void get_to(const JsonReadOnlyValue & value, std::optional< Type > & destination)
    Retrieves the current value into the given object, throwing if there is a type mismatch that cannot be converted automatically.

  • void get_to(const JsonReadOnlyValue & value, std::vector< Type > & destination)
    Retrieves the current value into the given vector.

  • void get_to(const JsonReadOnlyValue & value, std::array< Type, Size > & destination)
    Retrieves the current value into the given array.

  • void get_to(const JsonReadOnlyValue & value, std::map< std::string, Value > & destination)
    Retrieves the current value into the given dictionary.

  • void get_to(const JsonReadOnlyValue & value, std::map< core::Guid, Value > & destination)
    Retrieves the current value into the given dictionary.

  • void get_to(const JsonReadOnlyValue & value, std::map< Key, Value > & destination)
    Retrieves the current value into the given dictionary, converting the string to an enumerated value.

    Retrieves the current value into the given dictionary.

  • void get_to(const JsonReadOnlyValue & value, IndexedVariant< VariantSubtypes… > & destination)
    Retrieves the current value into the given variant.

  • void get_to(const JsonReadOnlyValue & value, std::variant< VariantSubtypes… > & destination, const std::array< uint8_t, sizeof…(VariantSubtypes)> & index)
    Retrieves the current value into the given variant.

Declared in “ark/serialization/json_writer.hh”:

  • void to_json(JsonDocumentWriter & writer, JsonObjectWriter root_object, const Type & source)
    Writes a serializable object into the given root object (overwriting fields, existing fields are not re-initialized).

Declared in “ark/serialization/object.hh”:

  • FieldValue string_to_field_value(const std::string & value, const SchemaRegistry & registry, const Schema & schema, const FieldDescriptor & field)
    Converts a string to a field value, based on the type. This will automatically convert the string into the appropriate type (uint32_t, std::string, etc), based on the field type specified.

  • std::chrono::nanoseconds duration_from_string(const std::string & value)
    Converts a string into a duration, applying rbuf-style formatting rules. This means that the string can end with a suffix to indicate units. In other words, if the string has no suffix, it will be interpreted as nanoseconds. ‘_s’ will be interpreted as seconds, ‘_ms’ will be interpreted as ‘milliseconds’, ‘_m’ will be interpreted as minutes, and ‘_h’ will be interpreted as hours.

  • std::string field_value_to_string(const FieldValue & value, const SchemaRegistry & registry, const Schema & schema, const FieldDescriptor & field)
    Converts the given field value to a string, based on the field type and schema.

Declared in “ark/serialization/rbuf_depfiles.hh”:

  • std::string emit_depfiles(const std::filesystem::path & rbuf_path, const std::filesystem::path & primary_output_path, const serialization::RbufParseResult & parse_result)
    Emit a depfile (as content that could be written to a file), containing all the dependencies of the primary output file, in the GCC depfile format suitable for being read from Ninja.

Declared in “ark/serialization/rbuf_grammar.hh”:

  • const char * get_rbuf_grammar()
    Returns the PEG grammar used by the rbuf parser. Kept in a separate file to make rebuilds on grammar changes more managable.

Declared in “ark/serialization/rbuf_parser.hh”:

  • RbufParseResult parse_rbuf(const std::string & input, RbufOptions options)
    Takes in data in the ‘rbuf’ format, and parses it into an appropriate schema structure. From there, you can either deserialize data encoded with that schema, or generate code that can parse data more efficiently.

Declared in “ark/serialization/schema.hh”:

  • const char * field_type_to_string(FieldType type)
    A free-function that converts a field type into a human-readable string.

  • FieldType field_type_from_string(const std::string & type)
    A free-function that converts a string into a field type.

  • const char * enum_type_to_string(EnumType type)
    A free-function that converts an enumeration type into a human-readable string.

  • EnumType enum_type_from_string(const std::string & type)
    A free-function that converts a string into an enumeration type.

  • bool is_field_type_fundamental(FieldType type)
    Returns true if the given field type is a fundamental, false otherwise. A fundamental is defined as something like a uint or double, rather than a string or list.

  • bool is_field_type_complex(FieldType type)
    Returns true if the given field type is ‘complex’, as in, is an object, list, container, or something that is nested. Primitives, strings, GUIDs, etc, are all considered “simple”.

  • void serialize< FieldDescriptor >(OutputStream & stream, const FieldDescriptor & source)
    Supports serialization of the schemas.

  • void deserialize< FieldDescriptor >(InputStream & stream, FieldDescriptor & destination)

  • void serialize< EnumDescriptor >(OutputStream & stream, const EnumDescriptor & source)

  • void deserialize< EnumDescriptor >(InputStream & stream, EnumDescriptor & destination)

  • void serialize< OptionalGroup >(OutputStream & stream, const OptionalGroup & source)

  • void deserialize< OptionalGroup >(InputStream & stream, OptionalGroup & destination)

  • void serialize< Schema >(OutputStream & stream, const Schema & source)

  • void deserialize< Schema >(InputStream & stream, Schema & destination)

  • void serialize< SourceLocation >(OutputStream & stream, const SourceLocation & source)

  • void deserialize< SourceLocation >(InputStream & stream, SourceLocation & destination)

  • JsonBlob to_json< FieldDescriptor >(const FieldDescriptor & source)
    Supports conversion to/from JSON.

  • void from_json< FieldDescriptor >(const JsonBlob & json, FieldDescriptor & destination)

  • JsonBlob to_json< EnumDescriptor >(const EnumDescriptor & source)

  • void from_json< EnumDescriptor >(const JsonBlob & json, EnumDescriptor & destination)

  • JsonBlob to_json< OptionalGroup >(const OptionalGroup & source)

  • void from_json< OptionalGroup >(const JsonBlob & json, OptionalGroup & destination)

  • JsonBlob to_json< Schema >(const Schema & source)

  • void from_json< Schema >(const JsonBlob & json, Schema & destination)

  • JsonBlob to_json< SourceLocation >(const SourceLocation & source)

  • void from_json< SourceLocation >(const JsonBlob & json, SourceLocation & destination)

  • constexpr FieldAttribute operator|(FieldAttribute lhs, FieldAttribute rhs)
    Or operator for our bitmask.

  • constexpr FieldAttribute operator&(FieldAttribute lhs, FieldAttribute rhs)
    And operator for our bitmask.

  • constexpr FieldAttribute operator~(FieldAttribute attribute)
    Inverse operator for our bitmask.

  • constexpr SchemaAttribute operator|(SchemaAttribute lhs, SchemaAttribute rhs)
    Or operator for our bitmask.

  • constexpr SchemaAttribute operator&(SchemaAttribute lhs, SchemaAttribute rhs)
    And operator for our bitmask.

Declared in “ark/serialization/schema_registry.hh”:

Declared in “ark/serialization/serializers.hh”:

  • void serialize(OutputStream & stream, const Type & source)
    A function that allows you to serialize a given C++ structure into the specified OutputStream. This will static assert unless the is_serializable trait is true.

  • void deserialize(InputStream & stream, Type & destination)
    A function that allows you to deserialize a given C++ structure from the specified InputStream and into the given structure.

    This will static assert unless the is_serializable trait is true.

  • JsonBlob to_json(const Type & source)
    A function that allows you to dump the given structure to JSON.

    This will static assert unless the is_serializable trait is true.

  • void from_json(const JsonBlob & json, Type & destination)
    A function that allows you read JSON into the given type.

    This will static assert unless the is_serializable trait is true.

  • std::string schema_protocol()
    A function that can return the schema type for a serialized type. This will be returned as an arbitrary string (e.g. rbuf, proto, rosmsg, etc).

    This will statically assert until the has_schema trait is true.

  • std::string schema()
    A function that can return the schema for a serialized type. This will be returned as an arbitrary string, suitable for passing into the Schema object.

    This will statically assert until the has_schema trait is true.

  • std::string type_name()
    A function that can return the type name for the schema of the serialized type. This is used as a mechanism to uniquely identify the type within schema registries.

    This will statically assert until the has_schema trait is true.

  • void hash(Hasher & hasher, const Type & source)
    This function will use the specified Hasher to generate a hash for the object, recursively.

    This will statically assert until the has_schema trait is true.

  • std::vector< ReturnType > values()
    This function returns the values of an enumeration. This will statically assert unless the has_schema trait is true.

  • size_t count()
    This function returns the a count of the values of an enumeration. This will statically assert unless the has_schema trait is true. Returns the number of potential values known to this system; enums coming in from other systems (ie, via forwards/backwards compat) may have differing numbers.

  • int64_t max()
    This function returns the maximum value of an enumeration. This will statically assert unless the has_schema trait is true. Returns the maximum value from the values known to this system; enums coming in from other systems (ie, via forwards/backwards compat) may have differing numbers.

Declared in “ark/serialization/timespec.hh”:

  • uint64_t to_packed_timespec(const typename Clock::time_point & timestamp)
    Converts the given timepoint into a uint64 value, which consists of the lower bits being set as seconds, and the lower bits being set as nanoseconds.

  • Clock::time_point from_packed_timespec(uint64_t timestamp)
    Converts the given uint64 value to a timepoint, assuming the lower bits are set as seconds, and lower bits as nanoseconds.