Part 9 (Logging)

In this section, we’ll go over adding a log writer to your pipeline, so that your output messages can be logged and replayed later.

Adding Log Writer

The first thing we’ll do is update the ${APP_ROOT}/tutorial/pipeline/tutorial_pipeline.cc file to add in a LogWriterStage. Add this include:

#include "ark/logging/stages/log_writer_stage.hh"

Then add this stage to your pipeline:

pipeline.add_stage<ark::logging::LogWriterStage>();

We’ll also need to update our CMakeLists.txt file, such that it looks like:

add_executable(tutorial_pipeline tutorial_pipeline.cc)

target_link_libraries(
    tutorial_pipeline
    PRIVATE first_stage 
            second_stage 
            ark::main_offboard 
            ark::http_server_stage 
            ark::debuglog 
            ark::log_writer_stage)

Try building and running now:

~/ark$ ./build/tutorial_pipeline
(2022-02-02 14:41:50.804) [error] (realtime_executor.cc:202) Exception during initialization: Exception while initializing LogWriterStage: Tried to request the config for 'LogWriterStage', but that config was not registered.
(2022-02-02 14:41:50.804) [error] (main_offboard.cc:340) Unhandled error during pipeline execution: Exception while initializing LogWriterStage: Tried to request the config for 'LogWriterStage', but that config was not registered.
(2022-02-02 14:41:50.804) [error] (tutorial_pipeline.cc:30) Exiting due to error: Exception while initializing LogWriterStage: Tried to request the config for 'LogWriterStage', but that config was not registered.

It looks like we need to create a configuration for the log writer, so that it knows what to log.

Configuring Log Writer

Let’s create the log writer configuration for this pipeline. Edit the ${APP_ROOT}/tutorial/pipeline/log_writer_config.yml file:

---
config:
  metadata:
    stage_config:
      comms_namespace: "/logger"

  default:
    destination_path: "/tmp/ark_logs"
    initial_log_name: "MyLog"

    columns:
      - column_name: "data"
        compression_type: "lz4"
        included_channels:
          - name: "/second01/stats"
          - name: "/second02/stats"
          - name: "/my_strings"

This configuration says to write out logs to /tmp/ark_logs, and to include all of the channels that you are publishing in your pipeline. These are included in a single column, because the data is fairly small.

Another thing to note is that we are using LZ4 compression. The logger supports logging with or without compression, and supports LZ4 and Zstd compression transparently.

Finally, let’s update our pipeline config. Add the following to the bottom:

    LogWriterStage:
      path: "tutorial/pipeline/log_writer_config.yml"

As you can see, we added a new LogWriterStage section, pointing to our config. If you run your pipeline now, you will see that it begins logging immediately.

~/ark$ ./build/tutorial_pipeline
(2022-02-02 14:47:49.549) [warn] (log_writer_stage.cc:1559) Pipeline configuration for OrganizationConfig is not found.
(2022-02-02 14:47:49.550) [info] (http_server_stage.cc:874) HTTP Server is listening on "0.0.0.0:8080".
(2022-02-02 14:47:49.567) [info] (log_writer_stage.cc:812) Writing log out to '5a8b9212-fb29-451d-a2f9-2c5a036570bf'.
(2022-02-02 14:47:50.067) [info] (second_stage.cc:39) SecondStage01 stage received message: 'At 0.518397979, my message is 'Configured Message!''
(2022-02-02 14:47:50.067) [info] (second_stage.cc:39) SecondStage02 stage received message: 'At 0.518397979, my message is 'Configured Message!''

Let it run for a few seconds. Take note of the GUID it is writing out to (5a8b9212-fb29-451d-a2f9-2c5a036570bf in this example, but it will change each time you run your pipeline). This is the GUID of the log that is being captured.

You can check out the log (at any time) by running:

~/ark$ ./build/ark-logtool /tmp/ark_logs/manifests/5a8b9212-fb29-451d-a2f9-2c5a036570bf -l
MIN            MAX            COUNT      RATE       SIZE(MB)   DISK(MB)   FLAG  TYPE                                          NAME
0.519          4.519          9          2.3        0.00       0.00             MyString                                      /my_strings
1.019          4.019          4          1.3        0.00       0.00             SecondStageStats                              /second01/stats
1.019          4.019          4          1.3        0.00       0.00             SecondStageStats                              /second02/stats

Great, we can see the three channels, each with their type name. The MIN/MAX here refers to the time range that these objects exist over.

You can dump one of the channels with:

~/ark$ ./build/ark-logtool /tmp/ark_logs/manifests/5a8b9212-fb29-451d-a2f9-2c5a036570bf -dc /second01/stats
[
{
  "messages_received": 1,
  "stage_name": "SecondStage01"
},
{
  "messages_received": 3,
  "stage_name": "SecondStage01"
},
{
  "messages_received": 5,
  "stage_name": "SecondStage01"
},
{
  "messages_received": 7,
  "stage_name": "SecondStage01"
}
]

Notice that the format is appropriate for using jq or similar tools for further processing. Use the --help option to see other options for looking into logs, including log metadata, indexes, etc.

In Part 10, we’ll control the log writer with ark-put.