GUI (Web) Overview

A skeleton for a web-based GUI framework is provided in the ark/gui/web directory. This framework is built with the following technology:

  • esbuild for packaging
  • Vue 3.0 for components/plugins
  • ThreeJS for visualization
  • Golden Layout for layout

There are a few useful libraries as well:

  • ArkPipelineSocket - a wrapper for receiving messages from a pipeline
  • CommunicationsManager - a singleton for communicating with a pipeline
  • serialize/deserialize - helpers to serialize/deserialize rbuf objects

All of the above generally assumes that messages are flowing from the pipeline over a websocket and encoded with rbuf.

Starting the Sample

To start the sample application, run these commands:

cd ~/ark
. env.sh
cd ark/gui/web
yarn watch

This will run the webserver in ‘watch’ mode. It will then be listening on http://localhost:1234.

This will automatically try to connect to any pipeline running on “localhost:8080”. If you wish to connect to an alternate pipeline, specify a host query parameter, such as:

http://localhost:1234/?host=192.168.1.1:8080

Connection and retries will happen automatically. Any transforms or “renderer layers” published from your pipeline will be automatically rendered, just as in the C++ GUI. Further, camera/video streams (if encoded in JPEG or H264) will be automatically rendered in the camera image plugin.

At the end, you should have something that looks like this:

GUI

Extending

We export a handful of packages that can be used within any web GUI. These can be used as NPM modules (preferably), or copied into your repository.

These are stored in ark/gui/web/packages. They consist of:

  • ark-web-comms - A comms wrapper that can talk to pipelines
  • ark-web-serialization - Helper APIs for deserializing Javascript rbuf code
  • ark-web-renderer - A three.js based rendering component that consumes ark::gui::RendererLayer objects and renders them
  • ark-web-awlog - A component for reading awlog files.
  • parcel-transformer-rbuf - Allows using rbufs directly in Parcel by compiling them transparently during build
  • esbuild-plugin-rbuf - Allows using rbufs directly in esbuild by compiling them transparently during build

If you wish to use the entire GUI framework, it is most likely easiest to copy the Vue system that is located in ark/gui/web/ui and extend it locally. There are not provisions for extending it via external repositories at this time.

For creating new web applications, if you wish to use our esbuild framework, you can symlink the ark/gui/web/esbuild-tools path into your web directory. This provides helper APIs for building, watching, and serving generic web applications.