GUI (C++) Overview

The C++ GUI toolkit is built around Qt 6.2, and is located in the ark/gui folder.

Example:

GUI

All of the plugins are organized into individual docked widgets, which allow you to control their layout. You can stack plugins ontop of each other to collapse them into tabs.

Core Concepts

All plugins inherit from the Plugin class. Just like a Stage, you create your interfaces in the initialize routine. Plugins also have namespaces.

One difference is that with plugins, all subscribers currently execute on the Qt event loop, allowing you to update Qt widgets right when subscribers are called.

Rendering

The plugin interface will provide a RendererInterface pointer to you. From there, you can publish RendererLayer objects, which contain primitives that you’d like to have rendered in the 3D panel.

You do not need to update these every frame; the geometry is saved and redrawn each time the renderer updates. Think of these as simple geometry buffers.

Stages can render things as well (in fact, this is more common). Subscribe to the channel ark::gui::RENDERER_LAYER_NAME (provided in the ark/gui/well_known_gui_channels.hh header), and emit RendererLayer objects on it. These objects will be rendered when received by the GUI.

Transform Network

The GUI provides basic transform network support. See Transform Network for more information.

The user can select the “rendering frame” that the 3D data is transformed to before rendering. This can be set in the GUI setting when creating the application or through the dropdown list during runtime.

If a rendering frame is selected but there is no transform available between the objects frame_id and the rendering frame than the object is not drawn.

If an object has no frame_id set (i.e. empty string) than it will be rendered relative to the current rendering frame origin (no matter which frame is selected).

If no rendering frame is selected (i.e. empty string) then the frame_id in the layer objects are ignored and everything is drawn relative to the origin.

Communications

You may add_publisher and add_subscriber in your plugin, just like you would in a Stage. These two functions are available in the PluginInterface, which is passed in to your plugin’s initialize routine.

There are a few caveats with these routines right now:

  • you cannot run ark-spy on publishers created in the GUI
  • you can only send serializable messages over GUI channels

Expect some of these restrictions to be lifted in the future.

Unlike stages, you can connect to publishers/subscribers at any time. Do so by taking PluginCommsManager from your PluginInteface via the comms_manager API. You can hold on to this comms manager, and then invoke the add_publisher or add_subscriber APIs at any time.

For example:

#include "ark/gui/plugin_comms_manager.hh"

void MyPlugin::initialize(const ark::gui::PluginInterface &interface)
{
    auto comms_manager = interface.comms_manager();

    comms_manager->add_publisher<MyMessage>("/channel/name");
}

GUI Channels

There are a few well-known channels in the GUI. Their names are provided in the ark/gui/well_known_gui_channels.hh header file.

  • /gui/renderer_layer - All primitives to be rendered go over this channel
  • /gui/plugins/channel_info - Information on available publishers in the connected pipeline
  • /gui/plugins/comms_info - Statistics on connection and websocket state

You can use the channel_info channel to dynamically discover channels that are available. With that information, and the PluginCommsManager, you can dynamically construct connections for different sources of data when they become available.

Settings

Two APIs are available to plugins that with to save/restore their state:

  • save_settings(QSettings &settings);
  • restore_settings(QSettings &settings);

The save_settings API is invoked periodically and right before shutdown. It gives you a chance to save your state to disk.

The restore_settings is invoked on startup, giving you a chance to read settings and restore your state. This can be helpful so that users of your tools don’t need to re-enter or reconfigure data each time they restart the application.

For example:

#include "ark/gui/plugin.hh"
#include <QSettings>

class MyPlugin
{
public:
    void save_settings(QSettings &settings) override
    {
        settings.setValue("my_saved_key_name", "A value to save");
    }

    void restore_settings(QSettings &settings) override
    {
        auto value = settings.value("my_saved_key_name");

        // "value" should now contain "A value to save" if settings had
        // previously been saved.
    }
};

Layout is also always saved and restored.

If you wish to totally start from scratch, you can use a command line option to load the GUI up and ignore all existing state:

./my_gui_app --reset-settings

Designer

You can construct your plugins with the Qt designer. To use the hermetic Qt designer (provided with our third party deps), type:

./scripts/designer

cmake

If you have a .ui file, you can link it into a plugin like so:

qt_wrap_ui(ui_source <my_ui_file.ui>)

Then just reference the ${ui_source} variable in your plugin library.

Some platforms do not have Qt support at the moment (namely, aarch64 and arm). To help allow the tree to build in either system, use this macro:

build_qt_gui(gui)

Use this instead of add_subdirectory. It will only build the GUI components if Qt is enabled on the platform you are building for.

Simple Plugin

A very simple plugin example looks like:

#include "ark/gui/plugin.hh"

class MyPlugin : public ark::gui::Plugin
{
public:
    MyPlugin() : ark::gui::Plugin("MyPlugin") {}

    void initialize(ark::gui::PluginInterface &interface) override
    {
        ark::pipeline::SubscriberConfiguration<Message> config;

        config.channel_name = "message";
        config.callback = [this](auto ...args) { this->update(args...); };

        interface.add_subscriber(config);
    }

    void update(const std::shared_ptr<const Message> &message)
    {
        my_widget->setText(message->text);
    }
};

This plugin will subscribe to the ‘message’ channel, and then set an imaginary my_widget’s text to be the text within that message periodically.

Creating a GUI Appliciation

Creating a GUI application is similar to creating a pipeline. You create a ark::gui::GuiApplication object, and then call register_plugin to register your various plugins.

A simple example:

#include "ark/gui/gui_application.hh"

int main (int argc, const char **argv)
{
    ark::gui::GuiApplication app("My App Name", argc, argv);

    application.register_plugin<MyPlugin>();
    application.register_central_plugin<AnotherCentralPlugin>();
    application.register_right_plugin<AnotherRightPlugin>();
    application.register_bottom_plugin<AnotherBottomPlugin>();

    return app.execute();
}

Note that there are four methods for adding plugins, corresponding to locations:

  • register_plugin - Registers a plugin with no widgets (an invisible plugin)
  • register_central_plugin - Plugin appears in the central tabbed pane
  • register_right_plugin - Plugin appears in the right tabbed pane
  • register_bottom_plugin - Plugin appears in the bottom tabbed pane

Note that plugins can be moved into any Qt docking spot, or between docking spots, once the use has control. These registrations refer to the default locations only.

Linking

You can link against ark::gui if you wish to acutally use Qt to display windows. If you simply wish to publish renderer layers or plugin data, you only need to link against ark::renderer.

The ark::gui target will bring in all Qt (and X11/GL) dependencies. When deploying to a robot, you typically do not want to have these dependencies. For that reason, your stages and pipelines should almost always prefer to link to ark::renderer.

Local Debug Rendering

Often it can be useful to render directly from any function for debugging purposes without setting up all the infrastructure required to connect to the external GUI (e.g. in unit tests, command line tools, etc).

There are a number of free functions provided to support this in ark/gui/local_debug.hh.

All the show functions open a separate window (based on the provided name). The functions are thread safe and non-blocking (you can use the wait_key function to achieve blocking if desired).

NOTE: This is intended only for debugging purposes and should not be used in any production applications.

Display images:

auto im = ark::image::jpeg_decompress("ark/gui/gui/images/TBD-ARK_primary.jpg");
ark::gui::show(std::move(im), "Debug Image");

Debug Image

Display layers:

auto builder = ark::gui::LayerBuilder("/layer");
builder.text2()
    .text("Hello World")
    .font_size(20)
    .alignment(ark::gui::TextAlignment::LineLeft)
    .anchor(ark::gui::HudAnchor::BottomLeft);
builder.axis2();
builder.cube()
    .fill_color(ark::gui::to_rgba(ark::gui::ColorName::maroon))
    .edge_color(ark::gui::to_rgba(ark::gui::ColorName::blue))
    .transform(ark::Transform3d::trans(3, 2, 0));
builder.cube()
    .fill_color(ark::gui::to_rgba(ark::gui::ColorName::blue))
    .edge_color(ark::gui::to_rgba(ark::gui::ColorName::maroon))
    .transform(ark::Transform3d::trans(-3, 2, 0));
ark::gui::show(builder.build(), "Debug Layer");

Debug Layer

Wait for key press:

// Return after any key stroke or 10s elapses.
ark::gui::wait_key(std::chrono::seconds(10));

There are two libraries you can link against to make use of this functionality. At a minimum, you should link against ark::gui_local_debug. This allows your code to compile, but showing windows will be non-functional. Link against the full ark::gui to gain the ability to show these popup windows.

These are split between libraries so that you can make use of local debug functionality in your stages/algorithms, but you only need to link against Gl, Qt, or X11 for your debug/offboard processes, not the onboard or production side.