Priorities in Stages

There are a few methods for controlling priorities in stages. They all rely on the underlying host operating system to respect priorities. This all applies to the real time executor only.

Further, you can specify processor affinities for thread pools, allowing you to pin different pools to subsets of cores on the machine, giving you fine grained control on what stages run on what cores.

Overall Pipeline Priority

You can configure the “primary thread pool” of the real time executor to run at a different priority level (or contain different numbers of threads).

This is done through the config package, and pipeline metadata. For example, in your pipeline config package:

package:
  configs:
    Pipeline:
      metadata:
        realtime_config:
          thread_pool_priority: 3
          thread_pool_thread_count: 6

If you leave thread_pool_thread_count at zero, it will automatically create an appropriate number of threads for your machine (typically the same as the number of physical cores). Otherwise, it creates as many threads as you ask for.

Setting the thread_pool_priority will cause the overall pool scheduling class to be adjusting to round robin, with that priority as the chosen priority.

All stages run in this pool by default.

Secondary Thread Pools

You can request that a stage runs in a different thread pool – this means that all of its callbacks and timers will be invoked in a different pool, which could have different priorities.

Be wary of priority inversion when doing this. All the normal callback rules apply, and there is nothing special you need to do aside from placing your stage in this pool.

This can be useful if your callbacks are doing disk I/O, or you simply want to run them higher/lower priority then the primary pool.

package:
  configs:
    Pipeline:
      metadata:
        realtime_config:
          thread_pools:
            - name: "Secondary"
              thread_count: 2
              thread_pool_priority: 1
              scheduler: Other
              stages:
                - "FirstStageName"
                - "SecondStageName"

You can define as many thread pools as you like. Stages must exist in the pipeline to be part of this pool. If you have many stages, and just one thread, they will all block on each other.

Processor Affinity

You can configure the processor affinity of both the primary thread pool and any secondary thread pools independently. This allows you to control which processors a thread pool will run on (so, for example, you can configure your pipeline to only run on some of the available cores, or split up thread pools to run on different cores to reduce the chance of conflicting).

Do so by setting the thread_pool_processor_affinity string. This is set as a range of cores, starting from zero (matching logical processor zero). For example, setting the value to “0” will pin the thread pool to only run on core 0. You can also use values such as “0-3,8-11” to say “run on cores 0,1,2,3,8,9,10,11”.

As an example:

package:
  configs:
    Pipeline:
      metadata:
        realtime_config:
          thread_pool_thread_count: 2
          thread_pool_processor_affinity: "0-1"

          thread_pools:
            - name: "Secondary"
              thread_count: 4
              thread_pool_processor_affinity: "3-6"
              stages:
                - "FirstStageName"
                - "SecondStageName"

            - name: "Tertiary"
              thread_count: 1
              thread_pool_processor_affinity: "8"
              stages:
                - "ThirdStageName"

The ranges are inclusive, so this schedules the primary pool on cores 0 and 1, the secondary pool on cores 3, 4, 5, and 6, and the tertiary pool on core 8.

Stages with Threads

Some stages (such as communications interfaces, or other stages that are inherently non-deterministic), start threads up.

In this case, just use the core function ark::core::set_this_thread_priority to safely set your thread’s priority. If this fails, an exception will be thrown.