When are files in 'blobs' directory that are no longer valid cleaned up?

1

Hi everyone,

When are files in ‘blobs’ directory that are no longer valid cleaned up?

I confirmed that it was cleaned up when I ran snapshot.
Besides this, are there any other events?

In other words, I wonder when the GC is triggered.

Best regards,
KH.

2

Which directory specifically? There are several directories that contain blobs.

You mentioned snapshots, so maybe that’s what you’re talking about. SingleStore automatically cleans up old snapshots as part of its normal operation.

The engine variable snapshot_trigger_size sets a threshold for the transaction log’s size (in bytes) that must be reached before the next snapshot is taken (SNAPSHOT DATABASE · SingleStore Documentation). Old snapshots are cleaned up automatically over time as new snapshots are taken.

Keep in mind that if your snapshot directory is growing without control, you may need to increase your disk space or consider adjusting the snapshots-to-keep configuration to limit the number of snapshots retained(The snapshot directory has a lot of capacity).

3

Hi,

These are segment files in the “<node_base_directory>/data/blobs/” directory.
Even if a table is truncated or dropped, the table’s segment files here still exist.

I would like to know when it will be removed.
I have “snapshot database …;” I confirmed that it was removed by running the command.
Are there other events that trigger file remove?

Best regards,
KH.

4

Oh, okay. Columnstore data blobs in the “<node_base_directory>/data/blobs/” directory will be deleted asynchronously after snapshots_to_keep snapshots have been taken on the database since the data in the data in the blob was deleted.

If you run additional snapshots, to pass the snapshots_to_keep threshold, then the cleanup of blogs eligible to be deleted will start within a few seconds. This is not necessary – you should just wait for snapshots to happen naturally.