Hi Team,
When configuring historical monitoring for memsql using KAFKA and memsql pipe lines, is there any way we can set the purging policy for these either in kafka or memsql pipelines or in memsql database to keep the retention for already processed messages and also for the upcoming new messages so that we need to view or validate the data only from particular point of time
Need the information on how to set the retention policy both in kafka and also for tables related in monitoring database
Hello,
Our existing monitoring solution does not have a built-in data retention policy. You must configure this on your own (e.g., via a cron job) on your monitoring database.
That being said, we are about to release a new version of our monitoring solution (within the next month) that no longer requires Kafka, and also allows you to set a retention period for your monitoring data. This solution uses the same dashboards and MemSQL exporter. It simply removes the pusher & Kafka and replaces it with a MemSQL pipeline instead (as well as introducing a retention-period variable in the configuration) Please stay tuned for the release of this solution.
Best,
Roxanna