Setting the Retention Period

Kafka has a default retention period set to 168 hours. The *.Image.jpg topics require a large amount of storage space because they contain all frames in JPEG image format. To change the retention period of JPEG topics run the set_retention.sh script.

Attention! By default, the retention period is 0 which means nothing is deleted. Without these settings, the *.Image.jpg topics use a lot of storage.

To set retention period:

$ "${UVAP_HOME}"/scripts/set_retention.sh --retention-unit [UNIT] \
   --retention-number [NUMBER]

Where:

  • [UNIT]:

    A unit of time. Replace [UNIT] (including brackets) with one of the following time units:

    • ms for milliseconds
    • second for seconds
    • minute for minutes
    • hour for hours
    • day for days

  • [NUMBER]:

    A parameter defining duration. Replace [NUMBER] (including brackets) with a number, that (together with the retention unit) defines the retention time to set.

For example, to set the retention period to 15 minutes:

$ "${UVAP_HOME}"/scripts/set_retention.sh --retention-unit minute \
  --retention-number 15

Expected output:

INFO: These topics will change:
base.cam.0.anonymized_original.Image.jpg
base.cam.0.original.Image.jpg
Completed Updating config for entity: topic 'base.cam.0.anonymized_original.Image.jpg'.
Topic:base.cam.0.anonymized_original.Image.jpg  PartitionCount:1    ReplicationFactor:1 Configs:retention.ms=900000
    Topic: base.cam.0.anonymized_original.Image.jpg Partition: 0    Leader: 1001    Replicas: 1001  Isr: 1001
Completed Updating config for entity: topic 'base.cam.0.original.Image.jpg'.
Topic:base.cam.0.original.Image.jpg PartitionCount:1    ReplicationFactor:1 Configs:retention.ms=900000
    Topic: base.cam.0.original.Image.jpg    Partition: 0    Leader: 1001    Replicas: 1001  Isr: 1001