API Overview

All of the catalog APIs are accessible through REST. You can POST JSON messages to the various URLs, and you will get JSON responses back.

API Metrics

Allows you to request metrics about the API gateway:

URL: /api/metrics

Input:

None, just use a standard GET on this URL.

Output:

{
  "endpoint_request_count": {
    "/api/metrics": 1
  },
  "instance_identifier": "9deb6425-73fd-417c-a199-054e210daeb9",
  "internal_server_error_count": 0,
  "peak_connection_count": 1,
  "resident_memory_usage_mb": 2.015625,
  "total_request_count": 1,
  "uptime_s": 4.9202638
}

Querying Logs

Allows you to query the catalog for logs that match criteria.

URL: /api/logs/query

Input:

None, just use a standard GET on this URL. Note that this will change in the near future, when you can actually query logs…

Output:

{
  "logs": [
    {
      "common_identifier": "716ec956-a307-4971-9e37-2988c2df62e4",
      "ingest_time_utc": 1608343535,
      "log_duration": 189000000000,
      "log_identifier": "2fa2d58b-589e-4e01-bef8-0883f4e16f86",
      "name": "run_down_street",
      "size_mb": 2231
    },
    ...
  ]
}

Describing a Log

This allows you to get additional details about a particular log, including its job ingest status. It is typically used if you want to lookup a particular log, rather than doing a broader query.

URL: /api/logs/describe

Input:

{
  "log_identifier": "2fa2d58b-589e-4e01-bef8-0883f4e16f86"
}

Output:

{
  "job_states": [
    {
      "complete": true,
      "job_definition_identifier": 0,
      "job_name": "Video Extraction",
      "successful": true
    }
  ],
  "metadata": {
    "common_identifier": "716ec956-a307-4971-9e37-2988c2df62e4",
    "log_duration": 189000000000,
    "log_identifier": "2fa2d58b-589e-4e01-bef8-0883f4e16f86",
    "name": "run_down_street",
    "size_mb": 2231
  }
}

Retrieving Job Output

Retrieve a particular job’s output (text output).

URL: /api/jobs/output

Input:

{
  "log_identifier": "2fa2d58b-589e-4e01-bef8-0883f4e16f86", 
  "job_identifier": 0
}

Output:

[2020-12-03 13:47:12 -0500] [libvpx @ 0x38f4240] v1.9.0
[2020-12-03 13:47:22 -0500] Encoding libvpx/webm, 12.69% through the log.
[2020-12-03 13:47:32 -0500] Encoding libvpx/webm, 29.30% through the log.
[2020-12-03 13:47:42 -0500] Encoding libvpx/webm, 45.42% through the log.
...

Refreshing the Database Schema

Allows you to update the database schema to the latest version (non-destructive).

URL: /api/database/refresh

Input:

None, just use a standard GET on this URL.

Output:

None, use the HTTP response type to understand if this was successful or not.

Listing Available Ingest Jobs

Allows you to retrieve a list of all available ingest jobs.

URL: /api/jobs/list

Input:

None, just use a standard GET on this URL.

Output:

{
  "jobs": [
    {
      "container_image": "<Docker Container Image>:latest",
      "cpu_count": 1,
      "job_identifier": 1,
      "memory_size_mb": 1024,
      "name": "My Custom Job"
    },
    ...
  ]
}

Registering Ingest Job

Registers a new ingest job with the system. All existing logs will immediately be backfilled.

URL: /api/jobs/add

Input:

{
  "name": "My Custom Job",
  "container_image": "<Docker Container Image>:latest",
  "cpu_count": 2,
  "memory_size_mb": 2048
}

Output:

{
  "success": true,
  "message": "Added successfully."
}