Configuring Detection Filter

This microservice does not require GPU resources.

The Detection Filter microservice processes the record from a source topic and writes the result to its target topic.

Detection Filter can run in batch mode. When running in batch mode, the microservice stops after processing the last record. Otherwise it keeps running and waits for more records.

Environment Variables

  • KAFKA_DETECTION_FILTER_MS_PROPERTY_FILE_PATHS: property files path list.

Detection Filter reads its configuration from the property files in the above list during startup. See the list of properties below.

Properties

The values of properties with the .file extension are the paths of the .json files, while the values of properties with the .data extension are the .json configurations themselves.

Attention! Setting .data and .file properties are exclusive to each other, meaning setting both ultinous.service.kafka.detection.filter.ms.config.data and ultinous.service.kafka.detection.filter.ms.config.file results in an error.

ultinous.service.kafka.detection.filter.ms.config.data

Propertyultinous.service.kafka.detection.filter.ms.config.data
DescriptionDetection Filter microservice configuration, and optionally authentication and detection filter configuration. Content must be in one-liner JSON format described in Superconfig Topic Definitions.
RequiredRequired

ultinous.service.kafka.detection.filter.ms.config.file

Propertyultinous.service.kafka.detection.filter.ms.config.file
DescriptionDetection Filter microservice configuration, and optionally authentication and detection filter configuration. Content must be the path to the JSON format configuration file.
RequiredRequired

Example:

{
  "source_options": {
    "start":"START_DATETIME",
    "start_date_time": "2019-05-31T10:30:00.000 +02:00",
    "end":"END_NEVER"
  },
  "source": {
    "broker_list": "127.0.0.1:6499",
    "name": "some_kafka_topic.ObjectDetectionRecord.json"
  },
  "target_options": {
    "handling": "REPLACE"
  },
  "target": {
    "broker_list": "127.0.0.1:6499",
    "name": "some_other_kafka_topic.ObjectDetectionRecord.json"
  },
  "config_data": { "...": "..." }
}

In this example, the sources are set to be read from the given date and never stop reading them. One feature vector source is defined with broker list and name. The target stream is being replaced.

For an example of the properties of the config_data field, see the Example of a DetectionFilterConfigRecord from Kafka.

ultinous.service.kafka.detection.filter.auth.defs.data

Propertyultinous.service.kafka.detection.filter.auth.defs.data
DescriptionAuthentication definition data in one-liner JSON format corresponding to the definitions described in Superconfig Authentication Definitions.
RequiredOptional
NoteRequired if ID based authentication is used in ultinous.service.kafka.detection.filter.ms.config.*

ultinous.service.kafka.detection.filter.auth.defs.file

Propertyultinous.service.kafka.detection.filter.auth.defs.file
DescriptionAuthentication definition file path. Content of the file should be in JSON format described described in Superconfig Authentication Definitions.
RequiredOptional
NoteRequired if ID based authentication is used in ultinous.service.kafka.detection.filter.ms.config.*

ultinous.service.kafka.detection.filter.config.data

Propertyultinous.service.kafka.detection.filter.config.data
DescriptionDetection filter configuration.
RequiredOptional
NoteRequired if config_data is not defined in ultinous.service.kafka.detection.filter.ms.config.*

ultinous.service.kafka.detection.filter.config.file

Propertyultinous.service.kafka.detection.filter.config.file
DescriptionDetection filter configuration.
RequiredOptional
NoteRequired if config_data is not defined in ultinous.service.kafka.detection.filter.ms.config.*

ultinous.service.kafka.detection.filter.monitoring.port

Propertyultinous.service.kafka.detection.filter.monitoring.port
DescriptionMonitoring server port.
RequiredRequired

ultinous.service.kafka.detection.filter.monitoring.threads

Propertyultinous.service.kafka.detection.filter.monitoring.threads
DescriptionMonitoring server thread pool size.
RequiredOptional

Microservice Configuration

Configuration records are separated into the following three values:

Example of a DetectionFilterConfigRecord from Kafka

{
  "min_confidence": 0.699999988079071,
  "detection_types": [
    "PERSON_HEAD"
  ],
  "negative_areas": [
    {
      "vertices": [
        {
          "x": 800,
          "y": 600
        },
        {
          "x": 1060,
          "y": 600
        },
        {
          "x": 1100,
          "y": 620
        },
        {
          "x": 900,
          "y": 620
        },
        {
          "x": 970,
          "y": 610
        }
      ]
    }
  ],
  "positive_areas": [
    {
      "vertices": [
        {
          "x": 1185,
          "y": 593
        },
        {
          "x": 1185,
          "y": 625
        },
        {
          "x": 1000,
          "y": 625
        },
        {
          "x": 1000,
          "y": 593
        }
      ]
    }
  ]
}

In this example the min_confidence field contains information on the confidence value. The detection_types field specifies that only head detections are valid. The negative_areas and positive_areas fields define the filtered areas.

Example Configuration

{
  "source_options": {},
  "source": {
    "broker_list": "127.0.0.1:6499",
    "name": "OutputTopicsForTest.OUT1.ObjectDetectionRecord.json"
  },
  "target_options": {
    "handling": "REPLACE"
  },
  "target": {
    "broker_list": "127.0.0.1:6499",
    "name": "OutputTopicsForTest.OUT2.ObjectDetectionRecord.json"
  },
  "config_data": {
    "min_confidence": 0.699999988079071,
    "positive_areas": [
      {
        "vertices": [
          {
            "x": 1185,
            "y": 593
          },
          {
            "x": 1185,
            "y": 625
          },
          {
            "x": 1000,
            "y": 625
          },
          {
            "x": 1000,
            "y": 593
          }
        ]
      }
    ]
  }
}

Changing the Configuration

It is possible to modify the configuration while Detection Filter is running. When you modify the configuration files on disk, Detection Filter will not automatically reload them. Instead, this can be achieved by posting a HTTP reload request through the monitoring port of the microservice. (The monitoring port is given in the Detection Filter property file, e.g. the Detection Filter base template property file.)

For example:

$ curl -X POST http://localhost:6498/reload

will reload the configuration files.

Note The values of the ultinous.service.kafka.detection.filter.monitoring.port and ultinous.service.kafka.detection.filter.monitoring.threads properties cannot be changed.

The result of the reload operation is the same as if the service was stopped and restarted with the new configuration. However, it can be done using the microservice API, without maintenance privileges.

Empty Configuration

It is also possible to start Detection Filter with an "empty" configuration and switch to a valid configuration later. In this case, the service will be running and healthy but it will not process any inputs until an acceptable configuration is reloaded.

Attention In order for the service to run, even the "empty" configuration must contain a valid value for ultinous.service.kafka.detection.filter.monitoring.port.

Invalid Configuration

Invalid configuration is treated the same way as empty configuration, i.e. Detection Filter will run without processing any inputs. Note, however, that at least ultinous.service.kafka.detection.filter.monitoring.port needs to be provided for the service to run.

If changing the configuration results in any error, e.g. because of invalid configuration, the reload request returns the error code 500. In this case, DetectionFilter will also continue running without processing any inputs.

← Configuring Feature Vector ClusteringConfiguring Video Capture →