Web Log (awlog)
Ark now provides the
awlog format (for Ark Web Log). This is primarily intended to create
optimized single-file logs that are suitable for playback in our
webviz suite of tooling,
but they do contain much of the same core functionality as
ark::logging logs (the primary
features lost are zero-copy creation of snippets and any type of amendment support).
Presently, all data in an
awlog is compressed with
zstd, but that is expected to be made
configurable at some point.
There are two primary
awlog-related tools right now:
ark-awlog-tool- A tool to inspect
ark-awlog-convert- Converts an
ark::logginglog into an
Many of the command line options for
ark-awlog-tool are identical to
a quick example of inspecting a log:
./build/ark-awlog-tool ./391d3934-1d10-4000-a4d3-98c2e90d553e.awlog -l MIN MAX COUNT RATE SIZE(MB) TYPE NAME 236.747 238.542 10 5.6 0.00 ark::debuglog::DebugLogOutputGroup /debuglog 233.170 242.101 9 1.0 0.00 ark::logging::LogWriterStageStatistics /logger/statistics 233.588 242.989 142 15.1 6.27 ark::image::Image /multisense/forward/left/compressed_image 233.747 242.757 10 1.1 0.00 ark::time::GlobalTimeEstimate /time/global_time 233.007 242.997 1372 137.3 0.16 ark::PushTransformMessage3dStamped /transform_network/transform 233.453 238.474 2 0.4 0.00 ark::version::VersionInformation /version_information
You can also print information out:
/build/ark-awlog-tool ./391d3934-1d10-4000-a4d3-98c2e90d553e.awlog -i Log Id: 391d3934-1d10-4000-a4d3-98c2e90d553e Common Id: aee21401-b25f-4bbb-9d97-c51e8d5bf5f2 Name: TestPlanRetail Start: 0.000000 End: 242.996527 Duration: 242.996527
Any metadata that is associated with the log will also be printed in this case.
Finally, you can dump index entries out:
./build/ark-awlog-tool ./391d3934-1d10-4000-a4d3-98c2e90d553e.awlog -e [242.654744] /multisense/forward/left/compressed_image (46337 bytes) [242.696959] /transform_network/transform (106 bytes) [242.706583] /transform_network/transform (106 bytes) [242.714025] /transform_network/transform (150 bytes) [242.721551] /multisense/forward/left/compressed_image (46367 bytes) [242.746373] /transform_network/transform (106 bytes) [242.756619] /transform_network/transform (106 bytes) [242.756850] /time/global_time (21 bytes)
Converting Ark Logs
You can use the
ark-awlog-convert tool to convert an
ark::logging log into an
takes parameters for the input log and output log, and then you can further filter things down
by time range and allowed/blocked messages.
This allows you to losslessly translate logs, or to take snippets of logs (keeping only certain
channels or time ranges). Since
awlog files are typically used for the web, it can be a nice
way to reduce the size of your logs to make them download faster.
Logging from the Pipeline
You can also use the
ark::awlog::AwlogWriterStage to write out
awlog files directly from
your simulation. This can be useful for running batch simulations or other translation jobs, where
you can generate an
awlog from simulated data, upload it to AWS, and then let others view the
results (without having your exact build of software).
See the stage documentation for more details.