Configuring Pass Detector

This microservice does not require GPU resources. Pass Detector processes records from a source topic and writes the result to its target topic.

Pass Detector can run in batch mode. When running in batch mode, the microservice stops after processing the last record. Otherwise it keeps running and waits for more records.

Environment Variables

  • KAFKA_PASSDET_MS_PROPERTY_FILE_PATHS: property file list

Properties

The values of properties with the .file extension are the paths of the .json files, while the values of properties with the .data extension are the .json configurations themselves.

Note Setting .data and .file properties are exclusive to each other, setting both of them results in an error.

For an example of Pass Detector properties, see Pass Detector Template Properties.

ultinous.service.kafka.passdet.ms.config.data

Propertyultinous.service.kafka.passdet.ms.config.data
DescriptionPass Detector microservice configuration, and optionally authentication and functional configuration. Content must be in one-liner JSON format described in Superconfig Topic Definitions.
RequiredRequired if ultinous.service.kafka.passdet.ms.config.file not given.

ultinous.service.kafka.passdet.ms.config.file

Propertyultinous.service.kafka.passdet.ms.config.file
DescriptionPath to a microservice configuration file.
The file contents must be in JSON format, according to ultinous.service.kafka.passdet.ms.config.data above.
RequiredRequired if ultinous.service.kafka.passdet.ms.config.data not given.

Microservice Configuration Example:

{
  "source_options": {
    "start":"START_DATETIME",
    "start_date_time": "2019-05-31T10:30:00.000 +02:00",
    "end":"END_NEVER"
  },
  "source": {
    "broker_list": "127.0.0.1:6499",
    "name": "some_kafka_topic.TrackChangeRecord.json"
  },
  "target_options": {
    "handling": "REPLACE"
  },
  "target": {
    "broker_list": "127.0.0.1:6499",
    "name": "some_other_kafka_topic.PassDetectionRecord.json"
  },
  "config_data": { "...": "..." }
}

In this example, the sources are set to be read from the given date and never stop reading them. One track change source is defined with broker list and name. Existing messages in the target stream are replaced.

For an example of the properties of the config_data field, see the Example of a PassDetConfigRecord.

ultinous.service.kafka.passdet.config.data

Propertyultinous.service.kafka.passdet.config.data
DescriptionPass Detector functional configuration.
The PassDetConfigRecord definition should be in JSON format as defined in the Kafka configuration proto, see the example below.
RequiredOptional
Value Typestring

ultinous.service.kafka.passdet.config.file

Propertyultinous.service.kafka.passdet.config.file
DescriptionPath to a functional configuration file.
The file contents must be in JSON format, according to ultinous.service.kafka.passdet.config.data above.
RequiredOptional
Value Typestring

Example of a PassDetConfigRecord

 {
   "passLines": [
     {
       "id": "pass_1",
       "poly": [
         {"x": 100, "y": 500},
         {"x": 1870, "y": 500}
       ]
     },
     {
       "id": "pass_2",
       "poly": [
         {"x": 960, "y": 980},
         {"x": 1000, "y": 560},
         {"x": 960, "y": 100}
       ]
     }
   ]
 }

Note UVAP provides an easy-to-use tool for pass line configuration called Stream Configurator. For further information, see Stream Configuration.

ultinous.service.kafka.passdet.auth.defs.data

Propertyultinous.service.kafka.passdet.auth.defs.data
DescriptionAuthentication definition data in one-liner JSON format corresponding to the definitions described in Superconfig Authentication Definitions.
RequiredOptional
NoteRequired if ID based authentication is used in ultinous.service.kafka.passdet.ms.config.*

ultinous.service.kafka.passdet.auth.defs.file

Propertyultinous.service.kafka.passdet.auth.defs.file
DescriptionAuthentication definition file path. Content of the file should be in JSON format described described in Superconfig Authentication Definitions.
RequiredOptional
NoteRequired if ID based authentication is used in ultinous.service.kafka.passdet.ms.config.*

ultinous.service.kafka.passdet.monitoring.port

Propertyultinous.service.kafka.passdet.monitoring.port
DescriptionMonitoring server port.
RequiredRequired
Value Typeuint16

ultinous.service.kafka.passdet.monitoring.threads

Propertyultinous.service.kafka.passdet.monitoring.threads
DescriptionMonitoring server thread pool size.
RequiredN/A
Value Typeuint16
Default Value1

Changing the Configuration

It is possible to modify the configuration while Pass Detector is running. When you modify the configuration files on disk, Pass Detector will not automatically reload them. Instead, this can be achieved by posting a HTTP reload request through the monitoring port of the microservice. (The monitoring port is given in the Pass Detector property file, e.g. the Pass Detector Template Properties.)

For example:

$ curl -X POST http://localhost:6495/reload

will reload the configuration files.

Note The values of the ultinous.service.kafka.passdet.monitoring.port and ultinous.service.kafka.passdet.monitoring.threads properties cannot be changed.

The result of the reload operation is the same as if the service was stopped and restarted with the new configuration. However, it can be done using the microservice API, without maintenance privileges.

Empty Configuration

It is also possible to start Pass Detector with an "empty" configuration and switch to a valid configuration later. In this case, the service will be running and healthy but it will not process any inputs until an acceptable configuration is reloaded.

Attention In order for the service to run, even the "empty" configuration must contain a valid value for ultinous.service.kafka.passdet.monitoring.port.

Invalid Configuration

Invalid configuration is treated the same way as empty configuration, i.e. Pass Detector will run without processing any inputs. Note, however, that at least ultinous.service.kafka.passdet.monitoring.port needs to be provided for the service to run.

If changing the configuration results in any error, e.g. because of invalid configuration, the reload request returns the error code 500. In this case, Pass Detector will also continue running without processing any inputs.

Output Schemas

The schema definition of TrackChangeRecord and PassDetectionRecord can be found in the Kafka data proto.

Example of a PassDetectionRecord from Kafka:

PASS_CANDIDATE:
{
  "timestamp": 1564493689341,
  "key": "pass_01",
  "value": {
    "type": "PASS_CANDIDATE",
    "pass_candidate": {
      "pass": {
        "id": {
          "track_key": "1564493689041_50187",
          "serial": 0
        },
        "pass_line_id": "pass_01",
        "cross_dir": "RL",
        "section_idx": 0,
        "cross_point": {
          "x": 511,
          "y": 500
        }
      },
      "is_extrapolated": false
    }
  },
  "headers": "[('format', b'json'), ('type', b'PassDetectionRecord')]"
}
PASS_REALIZED:
{
  "timestamp": 1564493689341,
  "key": null,
  "value": {
    "type": "PASS_REALIZED",
    "pass_realized": {
      "pass_event_ref": {
        "track_key": "1564493689041_50187",
        "serial": 0
      }
    }
  },
  "headers": "[('format', b'json'), ('type', b'PassDetectionRecord')]"
},
HEARTBEAT:
{
  "timestamp": 1564493689341,
  "key": null,
  "value": {
    "type": "HEARTBEAT"
  },
  "headers": "[('format', b'json'), ('type', b'PassDetectionRecord')]"
}
END_OF_TRACK:
{
  "timestamp": 1564493689341,
  "key": null,
  "value": {
    "type": "END_OF_TRACK",
    "end_of_track": {
      "track_key": "1564493689041_50187"
    }
  },
  "headers": "[('format', b'json'), ('type', b'PassDetectionRecord')]"
}

Note: The key is empty in the following cases:

  • The record is only a HEARTBEAT message.
  • The record is an end-of-track signal.
  • The record is a Pass Realization (see below).

Dataflow Architecture of UVAP Pass Detection Example

Pass Realization A TrackChangeRecord triggering a Pass Detection is not always derived from a "real" detection. The track change can also come from a prediction of the next track position. In this case, the is_extrapolated field of the PassDetectionRecord is true. After the predicted detection the track usually continues with a "real" detection.

In this case a new PassDetectionRecord is inserted into the target topic with an empty key but with the appropriate non-empty track_key.

See the Kafka data proto.

Feature Demo

For the pass detection feature demo, see Pass Detection Demo.

← Configuring TrackerConfiguring Reidentifier →