Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .typos.toml
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ AKS = "AKS"
# tech terms
excluder = "excluder"
flate = "flate"
consts = "consts"

[files]
extend-exclude = [
Expand Down
83 changes: 83 additions & 0 deletions docs/user-guide/deployments-administration/maintenance/gc.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
---
keywords: [GC, Garbage Collection, Mito, Repartition, GreptimeDB]
description: GC keeps SST/index files until all references are released, protecting long queries and repartition workflows.
---
# Configuration

GreptimeDB GC delays physical deletion of SST/index files until all references (running queries, [repartition](../manage-data/table-sharding.md#Repartition) cross-region file refs) are released. The configuration contains two parts:

- Metasrv Configuration
- Datanode Configuration

## How it works

- **Roles**: Meta decides when/where to clean; datanodes perform the actual delete while keeping in-use files safe.
- **Safety windows**: `lingering_time` holds known-removed files a bit longer; `unknown_file_lingering_time` is a rare-case guard.
- **Listing modes**: Fast mode removes files the system already marked; full listing walks storage to catch stragglers/orphans.

```mermaid
flowchart LR
A[Meta schedules GC] --> B[Pick regions]
B --> C[Send GC task]
C --> D[Datanode cleans files]
D --> E{Fast or Full}
E --> F[Fast: remove marked files]
E --> G[Full: walk storage for orphans]
F --> H[Cleanup recorded]
G --> H
```

## Metasrv Configuration

On the Metasrv side, GC schedules cleanup tasks for regions and coordinates when to run GC.

```toml
[gc]
enable = true # Enable meta GC scheduler. default to be false; must match datanode.
gc_cooldown_period = "5m" # Minimum gap before the same region is GCed again.
```

### Options

| Configuration Option | Description |
| --- | --- |
| `enable` | Enable the meta GC scheduler. Must match datanode GC enablement. |
| `gc_cooldown_period` | Minimum interval before the same region is scheduled for GC again; keep datanode `lingering_time` longer than this. |

## Datanode Configuration

The Datanode side performs the actual deletion while protecting files still in use.

```toml
[[region_engine]]
[region_engine.mito]
[region_engine.mito.gc]
enable = true # Turn on datanode GC worker; must match meta.
lingering_time = "10m" # Keep known-removed files this long for active queries.
unknown_file_lingering_time = "1h" # Keep files without expel time; rare safeguard.
```

### Options

| Configuration Option | Description |
| --- | --- |
| `enable` | Enable the datanode GC worker. Must match meta GC `enable`. |
| `lingering_time` | How long to keep manifest-removed files before deletion to protect long follower-region queries/cross-region references; set longer than `gc_cooldown_period`. Use `"None"` to delete immediately. |
| `unknown_file_lingering_time` | Safety hold for files without expel time (not tracked in manifest). Should be generous; these cases are rare. |

:::warning
`gc.enable` must be set consistently on metasrv and all datanodes. Mismatched flags cause GC to be skipped or stuck.
:::

## When to enable

- GC only applies when tables use object storage; tables on local filesystems ignore GC settings.
- Turn on GC if need to repartition so cross-region references can drain safely before deletion.
- For clusters with long-running follower-region queries, turn on GC and set `lingering_time` longer than `gc_cooldown_period` so files created or referenced during a GC cycle stay alive (in-use or lingering) until at least the next cycle.
- Leave GC off if you are not repartitioning and do not need delayed deletion.

## Operational notes

- GC is designed for object storage backends (with list/delete support); ensure your store credentials and permissions allow listing and deletion.
- Deleted files live in object storage until GC removes them; ensure storage listing/deletion permissions are in place.
- After enabling, restart metasrv and datanodes to apply config changes.
Original file line number Diff line number Diff line change
Expand Up @@ -128,7 +128,7 @@ ALTER TABLE sensor_readings SPLIT PARTITION (

:::caution Note
Repartitioning is only supported in distributed clusters.
You must enable shared object storage and GC, and ensure all datanodes can access the same object store before running repartitioning operations.
You must enable shared object storage and [GC](docs/user-guide/deployments-administration/maintenance/gc.md), and ensure all datanodes can access the same object store before running repartitioning operations.
:::

## Insert data into the table
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
---
keywords: [GC, 垃圾回收, Mito, Repartition, GreptimeDB]
description: GC 会在所有引用释放后才删除 SST/索引文件,以保护长查询和重分区流程。
---
# 配置

GreptimeDB GC 会延迟删除 SST/索引文件,直到所有引用(运行中的查询、[repartition](../manage-data/table-sharding.md#Repartition) 的跨 region 文件引用)释放。配置包含两部分:

- Metasrv 配置
- Datanode 配置

## 工作原理

- **角色**:Meta 决定何时/何处清理;Datanode 负责实际删除,同时保护正在使用的文件。
- **安全窗口**:`lingering_time` 会额外保留已移除文件;`unknown_file_lingering_time` 用于极少见的保护场景。
- **列举模式**:快速模式删除系统已标记的文件;全量列举遍历对象存储以发现滞留/孤儿文件。

```mermaid
flowchart LR
A[Meta schedules GC] --> B[Pick regions]
B --> C[Send GC task]
C --> D[Datanode cleans files]
D --> E{Fast or Full}
E --> F[Fast: remove marked files]
E --> G[Full: walk storage for orphans]
F --> H[Cleanup recorded]
G --> H
```

## Metasrv 配置

在 Metasrv 侧,GC 负责为各个 region 安排清理任务,并协调 GC 的运行时机。

```toml
[gc]
enable = true # 开启 meta GC 调度器;必须与 datanode 一致。
gc_cooldown_period = "5m" # 同一 region 再次 GC 的最小间隔。
```

### 配置

| 配置项 | 说明 |
| --- | --- |
| `enable` | 启用 meta GC 调度器,必须与 datanode 的 GC 开关一致。 |
| `gc_cooldown_period` | 同一 region 再次被调度 GC 的最小间隔;请保证 datanode 的 `lingering_time` 大于该值。 |

## Datanode 配置

Datanode 负责实际删除,同时保护仍在使用中的文件。

```toml
[[region_engine]]
[region_engine.mito]
[region_engine.mito.gc]
enable = true # 开启 datanode GC worker;必须与 meta 一致。
lingering_time = "10m" # 已移除文件在活跃查询期间保留时长。
unknown_file_lingering_time = "1h" # 未记录 expel time 的文件保留时长;罕见保护。
```

### 配置

| 配置项 | 说明 |
| --- | --- |
| `enable` | 启用 datanode GC worker,必须与 meta GC 的 `enable` 一致。 |
| `lingering_time` | manifest 中已移除文件在删除前的保留时长,用于保护长时间 follower-region 查询/跨 region 引用;请设置为大于 `gc_cooldown_period`。设为 `"None"` 表示立即删除。 |
| `unknown_file_lingering_time` | 对缺少 expel time 的文件的安全保留时间(未在 manifest 中追踪)。建议设置为较长值;此类情况较少。 |

:::warning
`gc.enable` 必须在 metasrv 与所有 datanode 上保持一致。开关不一致会导致 GC 被跳过或卡住。
:::

## 何时启用

- GC 仅在表使用对象存储时生效;本地文件系统上的表会忽略 GC 设置。
- 如果需要重分区,请开启 GC,以便跨 region 引用在删除前安全释放。
- 对于有长时间 follower-region 查询的集群,开启 GC 并将 `lingering_time` 设为大于 `gc_cooldown_period`,确保 GC 周期内创建或引用的文件保持存活(在用或 lingering)直到至少下个周期。
- 如果不进行重分区且不需要延迟删除,可保持 GC 关闭。

## 运维注意事项

- GC 面向对象存储后端(需支持 list/delete);确保存储凭据与权限允许列举和删除。
- 删除的文件会在对象存储中保留直到 GC 清理;确保具备列举/删除权限。
- 启用后重启 Metasrv 与 Datanode 以使配置生效。
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
---
keywords: [GC, 垃圾回收, Mito, Repartition, GreptimeDB]
description: GC 会在所有引用释放后才删除 SST/索引文件,以保护长查询和重分区流程。
---
# 配置

GreptimeDB GC 会延迟删除 SST/索引文件,直到所有引用(运行中的查询、[repartition](../manage-data/table-sharding.md#Repartition) 的跨 region 文件引用)释放。配置包含两部分:

- Metasrv 配置
- Datanode 配置

## 工作原理

- **角色**:Meta 决定何时/何处清理;Datanode 负责实际删除,同时保护正在使用的文件。
- **安全窗口**:`lingering_time` 会额外保留已移除文件;`unknown_file_lingering_time` 用于极少见的保护场景。
- **列举模式**:快速模式删除系统已标记的文件;全量列举遍历对象存储以发现滞留/孤儿文件。

```mermaid
flowchart LR
A[Meta schedules GC] --> B[Pick regions]
B --> C[Send GC task]
C --> D[Datanode cleans files]
D --> E{Fast or Full}
E --> F[Fast: remove marked files]
E --> G[Full: walk storage for orphans]
F --> H[Cleanup recorded]
G --> H
```

## Metasrv 配置

在 Metasrv 侧,GC 负责为各个 region 安排清理任务,并协调 GC 的运行时机。

```toml
[gc]
enable = true # 开启 meta GC 调度器;必须与 datanode 一致。
gc_cooldown_period = "5m" # 同一 region 再次 GC 的最小间隔。
```

### 配置

| 配置项 | 说明 |
| --- | --- |
| `enable` | 启用 meta GC 调度器,必须与 datanode 的 GC 开关一致。 |
| `gc_cooldown_period` | 同一 region 再次被调度 GC 的最小间隔;请保证 datanode 的 `lingering_time` 大于该值。 |

## Datanode 配置

Datanode 负责实际删除,同时保护仍在使用中的文件。

```toml
[[region_engine]]
[region_engine.mito]
[region_engine.mito.gc]
enable = true # 开启 datanode GC worker;必须与 meta 一致。
lingering_time = "10m" # 已移除文件在活跃查询期间保留时长。
unknown_file_lingering_time = "1h" # 未记录 expel time 的文件保留时长;罕见保护。
```

### 配置

| 配置项 | 说明 |
| --- | --- |
| `enable` | 启用 datanode GC worker,必须与 meta GC 的 `enable` 一致。 |
| `lingering_time` | manifest 中已移除文件在删除前的保留时长,用于保护长时间 follower-region 查询/跨 region 引用;请设置为大于 `gc_cooldown_period`。设为 `"None"` 表示立即删除。 |
| `unknown_file_lingering_time` | 对缺少 expel time 的文件的安全保留时间(未在 manifest 中追踪)。建议设置为较长值;此类情况较少。 |

:::warning
`gc.enable` 必须在 metasrv 与所有 datanode 上保持一致。开关不一致会导致 GC 被跳过或卡住。
:::

## 何时启用

- GC 仅在表使用对象存储时生效;本地文件系统上的表会忽略 GC 设置。
- 如果需要重分区,请开启 GC,以便跨 region 引用在删除前安全释放。
- 对于有长时间 follower-region 查询的集群,开启 GC 并将 `lingering_time` 设为大于 `gc_cooldown_period`,确保 GC 周期内创建或引用的文件保持存活(在用或 lingering)直到至少下个周期。
- 如果不进行重分区且不需要延迟删除,可保持 GC 关闭。

## 运维注意事项

- GC 面向对象存储后端(需支持 list/delete);确保存储凭据与权限允许列举和删除。
- 删除的文件会在对象存储中保留直到 GC 清理;确保具备列举/删除权限。
- 启用后重启 Metasrv 与 Datanode 以使配置生效。
1 change: 1 addition & 0 deletions sidebars.ts
Original file line number Diff line number Diff line change
Expand Up @@ -394,6 +394,7 @@ const sidebars: SidebarsConfig = {
'user-guide/deployments-administration/maintenance/prevent-metadata-changes',
'user-guide/deployments-administration/maintenance/sequence-management',
'user-guide/deployments-administration/maintenance/table-reconciliation',
'user-guide/deployments-administration/maintenance/gc',
]
},
'user-guide/deployments-administration/troubleshooting',
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,83 @@
---
keywords: [GC, Garbage Collection, Mito, Repartition, GreptimeDB]
description: GC keeps SST/index files until all references are released, protecting long queries and repartition workflows.
---
# Configuration

GreptimeDB GC delays physical deletion of SST/index files until all references (running queries, [repartition](../manage-data/table-sharding.md#Repartition) cross-region file refs) are released. The configuration contains two parts:

- Metasrv Configuration
- Datanode Configuration

## How it works

- **Roles**: Meta decides when/where to clean; datanodes perform the actual delete while keeping in-use files safe.
- **Safety windows**: `lingering_time` holds known-removed files a bit longer; `unknown_file_lingering_time` is a rare-case guard.
- **Listing modes**: Fast mode removes files the system already marked; full listing walks storage to catch stragglers/orphans.

```mermaid
flowchart LR
A[Meta schedules GC] --> B[Pick regions]
B --> C[Send GC task]
C --> D[Datanode cleans files]
D --> E{Fast or Full}
E --> F[Fast: remove marked files]
E --> G[Full: walk storage for orphans]
F --> H[Cleanup recorded]
G --> H
```

## Metasrv Configuration

On the Metasrv side, GC schedules cleanup tasks for regions and coordinates when to run GC.

```toml
[gc]
enable = true # Turn on meta GC scheduler; must match datanode.
gc_cooldown_period = "5m" # Minimum gap before the same region is GCed again.
```

### Options

| Configuration Option | Description |
| --- | --- |
| `enable` | Enable the meta GC scheduler. Must match datanode GC enablement. |
| `gc_cooldown_period` | Minimum interval before the same region is scheduled for GC again; keep datanode `lingering_time` longer than this. |

## Datanode Configuration

The Datanode side performs the actual deletion while protecting files still in use.

```toml
[[region_engine]]
[region_engine.mito]
[region_engine.mito.gc]
enable = true # Turn on datanode GC worker; must match meta.
lingering_time = "10m" # Keep known-removed files this long for active queries.
unknown_file_lingering_time = "1h" # Keep files without expel time; rare safeguard.
```

### Options

| Configuration Option | Description |
| --- | --- |
| `enable` | Enable the datanode GC worker. Must match meta GC `enable`. |
| `lingering_time` | How long to keep manifest-removed files before deletion to protect long follower-region queries/cross-region references; set longer than `gc_cooldown_period`. Use `"None"` to delete immediately. |
| `unknown_file_lingering_time` | Safety hold for files without expel time (not tracked in manifest). Should be generous; these cases are rare. |

:::warning
`gc.enable` must be set consistently on metasrv and all datanodes. Mismatched flags cause GC to be skipped or stuck.
:::

## When to enable

- GC only applies when tables use object storage; tables on local filesystems ignore GC settings.
- Turn on GC if need to repartition so cross-region references can drain safely before deletion.
- For clusters with long-running follower-region queries, turn on GC and set `lingering_time` longer than `gc_cooldown_period` so files created or referenced during a GC cycle stay alive (in-use or lingering) until at least the next cycle.
- Leave GC off if you are not repartitioning and do not need delayed deletion.

## Operational notes

- GC is designed for object storage backends (with list/delete support); ensure your store credentials and permissions allow listing and deletion.
- Deleted files live in object storage until GC removes them; ensure storage listing/deletion permissions are in place.
- After enabling, restart metasrv and datanodes to apply config changes.
3 changes: 2 additions & 1 deletion versioned_sidebars/version-1.0-sidebars.json
Original file line number Diff line number Diff line change
Expand Up @@ -393,7 +393,8 @@
"user-guide/deployments-administration/maintenance/recovery-mode",
"user-guide/deployments-administration/maintenance/prevent-metadata-changes",
"user-guide/deployments-administration/maintenance/sequence-management",
"user-guide/deployments-administration/maintenance/table-reconciliation"
"user-guide/deployments-administration/maintenance/table-reconciliation",
"user-guide/deployments-administration/maintenance/gc"
]
},
"user-guide/deployments-administration/troubleshooting",
Expand Down