harbor 如何做到物理删除镜像 harbor镜像清理脚本

一、背景

相比于nexus，harbor的一大优点是方便及时清理无用的docker镜像。本文就harbor怎么设置清理，梳理一下具体的操作办法。

harbor 版本是 v2.9.0

二、目标

随着我们推送至仓库的镜像越来越多，带来的一个最大运维问题就是存储空间的浪费，需要我们能够把旧的镜像清理出去。这就要求我们按推送的时间倒序排列，保留最新推送的多少个，从而删除较早推送的记录。

三、设置

已使用存储空间为2.54GB
library/vue-cli镜像已推送的总数为4个
library/node镜像已推送的总数为3个

所以，我们测试规则中的保留数不宜过多，暂定为3个。
预期效果是：library/vue-cli的最早的一个镜像将被清理掉，而 library/node 因为没有超过阈值，所以不会变化。

1、策略

策略支持多个规则，不过我们不需要那么多。我这里选择的是第一个默认规则，保留最近推送的3个。

然后执行的频率是每周，这个频率看每个人的使用场景。

2、模拟运行

查看清理的明细日志：

2023-10-10T01:02:19Z [INFO] [/pkg/retention/job.go:86]: Run retention process.
 Repository: library/vue-cli
 Rule Algorithm: or
 Dry Run: true
 2023-10-10T01:02:21Z [INFO] [/pkg/retention/job.go:101]: Load 4 candidates from repository library/vue-cli
 2023-10-10T01:02:21Z [INFO] [/pkg/retention/job.go:212]:
 | Digest | Tag | Kind | Labels | PushedTime | PulledTime | CreatedTime | Retention |
 |-------------------------------------------------------------------------|-------------------------|-------|--------|---------------------|---------------------|---------------------|-----------|
 | sha256:9ebe43b637dbda81e0513ef21b950a4fcf3319482fe6f25e16653b41406370ef | 3.3.0_build202309150905 | image | | 2023/10/09 01:21:56 | 2023/10/09 09:40:53 | 2023/10/09 01:21:56 | RETAIN |
 | sha256:dce192888562d266e6959776f1fd772e89f1f654044fbd445e4475d5ba4d7f7e | 3.3.0_build202303140655 | image | | 2023/10/09 01:20:50 | | 2023/10/09 01:20:49 | RETAIN |
 | sha256:165463708f354f8d96ba083b7fbbb1f25b9c8533cbc01117e7648c195733cc8d | 3.3.0_build202109241209 | image | | 2023/10/09 01:20:11 | | 2023/10/09 01:20:11 | RETAIN |
 | sha256:5a9e9334f5db76e592ca22c6b31755ce666742ff0a7dc7834b53c5bc7eaae5e4 | 3.3.0_build202108200549 | image | | 2023/10/09 01:20:01 | | 2023/10/09 01:20:01 | DEL |

日志是按创建时间排序排列，可以看出，标签为3.3.0_build202108200549的镜像将被移除，它的推送时间是2023/10/09 01:20:01，为最早推送的一个，因此符合我们对清理的预期。

作为对比，另外一个镜像的清理日志就不会出现DEL的操作。（因为library/node总共只有3个，而清理的规则也恰好是3个，意味着没有超过清理的阈值）

2023-10-10T00:59:41Z [INFO] [/pkg/retention/job.go:86]: Run retention process.
 Repository: library/node
 Rule Algorithm: or
 Dry Run: true
 2023-10-10T00:59:43Z [INFO] [/pkg/retention/job.go:101]: Load 3 candidates from repository library/node
 2023-10-10T00:59:43Z [INFO] [/pkg/retention/job.go:212]:
 | Digest | Tag | Kind | Labels | PushedTime | PulledTime | CreatedTime | Retention |
 |-------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------|-------|--------|---------------------|---------------------|---------------------|-----------|
 | sha256:1b7e5ae382ed76a8e39a4afaa968f4b68f00501db7e8b91e5870117b881207c9 | 3.3.0_build202310090525 | image | | 2023/10/09 05:30:15 | 2023/10/10 00:41:02 | 2023/10/09 05:30:14 | RETAIN |
 | sha256:9ebe43b637dbda81e0513ef21b950a4fcf3319482fe6f25e16653b41406370ef | 3.3.0_build202310090127,3.3.0_build202309210924,3.3.0_build202309150958,3.3.0_build202309150956 | image | | 2023/10/09 01:27:10 | 2023/10/09 01:30:42 | 2023/10/09 01:25:00 | RETAIN |
 | sha256:59167f718a1f6daecb900ffbb4f5b09f0dca709c5d7e180f33bbaeac076cdb7b | 16.20.2-bullseye | image | | 2023/10/09 01:24:56 | | 2023/10/09 01:24:56 | RETAIN |

3、立即执行

在你使用模拟运行确认无误后，一定会要点击“立即运行”。

但是运行完成后，持续时间是4秒，可harbor的存储空间并无变化。（仍旧是2.54GB）

四、清理服务

点击“立即清理垃圾”，看到下面的任务记录。

2023-10-10T01:42:45Z [INFO] [/jobservice/job/impl/gc/garbage_collection.go:599]: ID-3 MediaType-application/vnd.docker.container.image.v1+json ManifestMediaType-application/vnd.docker.distribution.manifest.v2+json RepositoryName-library/vue-cli Digest-sha256:5a9e9334f5db76e592ca22c6b31755ce666742ff0a7dc7834b53c5bc7eaae5e4 CreationTime-2023-10-10 01:38:52
 2023-10-10T01:42:45Z [INFO] [/jobservice/job/impl/gc/garbage_collection.go:341]: [05491b08-f601-4ee7-a5bf-c32a76cd96ca][3/11] delete the manifest with registry v2 API: library/vue-cli, application/vnd.docker.distribution.manifest.v2+json, sha256:5a9e9334f5db76e592ca22c6b31755ce666742ff0a7dc7834b53c5bc7eaae5e4
 2023-10-10T01:42:45Z [INFO] [/jobservice/job/impl/gc/garbage_collection.go:370]: [05491b08-f601-4ee7-a5bf-c32a76cd96ca][3/11] delete manifest from storage: sha256:5a9e9334f5db76e592ca22c6b31755ce666742ff0a7dc7834b53c5bc7eaae5e4
 2023-10-10T01:42:45Z [INFO] [/jobservice/job/impl/gc/garbage_collection.go:398]: [05491b08-f601-4ee7-a5bf-c32a76cd96ca][3/11] delete artifact blob record from database: 3, library/vue-cli, sha256:5a9e9334f5db76e592ca22c6b31755ce666742ff0a7dc7834b53c5bc7eaae5e4
 2023-10-10T01:42:45Z [INFO] [/jobservice/job/impl/gc/garbage_collection.go:406]: [05491b08-f601-4ee7-a5bf-c32a76cd96ca][3/11] delete artifact trash record from database: 3, library/vue-cli, sha256:5a9e9334f5db76e592ca22c6b31755ce666742ff0a7dc7834b53c5bc7eaae5e4

五、查看清理后

观察镜像library/vue-cli 的数量，可以看出，最早的标签已被清理。

六、保留最近多少天被拉取过的artifact

由于xxx/api-platform镜像只推送过一次镜像，为了演示“立即运行”的清理，我使用另外一个规则，“保留最近多少天被拉取过的artifact”。（因为xxx/api-platform镜像好久没被拉取过，所以预期将被清理）

从上图也可以看出，它将保留0个，也即全部清理。

2023-10-10T01:57:03Z [INFO] [/pkg/retention/job.go:86]: Run retention process.
 Repository: xxx/api-platform
 Rule Algorithm: or
 Dry Run: false
 2023-10-10T01:57:05Z [INFO] [/pkg/retention/job.go:101]: Load 1 candidates from repository xxx/api-platform
 2023-10-10T01:57:05Z [INFO] [/pkg/retention/job.go:212]:
 | Digest | Tag | Kind | Labels | PushedTime | PulledTime | CreatedTime | Retention |
 |-------------------------------------------------------------------------|-------|-------|--------|---------------------|------------|---------------------|-----------|
 | sha256:f7f0721d6ce20278942c97b59c361a0256fc8330a1c21fe0044bc85e6ce91d58 | 1.0.2 | image | | 2023/10/09 01:26:25 | | 2023/10/09 01:26:25 | DEL |

七、总结

这里通过清理的对比实验，可以看出“策略”–》“立即运行”，也可以清理掉无用的构建物；那么外面单独的菜单–“清理服务”是起何作用？

清理服务，从日志内容可以看到，它清理的是垃圾：“xx个 blob(s) 和 xx个 manifest(s) 将被删除”，另外它还可以清理harbor本身的日志记录。

2023-10-10T02:13:03Z [INFO] [/jobservice/job/impl/gc/garbage_collection.go:162]: Garbage Collection parameters: [delete_untagged: false, dry_run: true, time_window: 2, workers: 1]
2023-10-10T02:13:03Z [INFO] [/jobservice/job/impl/gc/garbage_collection.go:172]: start to run gc in job.
2023-10-10T02:13:03Z [WARNING] [/jobservice/job/impl/gc/garbage_collection.go:213]: no removed artifacts.
2023-10-10T02:13:04Z [INFO] [/jobservice/job/impl/gc/garbage_collection.go:261]: blob eligible for deletion: sha256:d5db42f314af162c5dc93a6c89a5bfeda6307f9acf3c2290d41c32668855016d
2023-10-10T02:13:04Z [INFO] [/jobservice/job/impl/gc/garbage_collection.go:261]: blob eligible for deletion: sha256:076545cf68e620c922789f9321e2de7f8eb88b850906d53194eeb5c77f1ff23d
2023-10-10T02:13:04Z [INFO] [/jobservice/job/impl/gc/garbage_collection.go:261]: blob eligible for deletion: sha256:330d755f124c8b8e56a11ad2b053fb4eee18a0399b6ec8acc777bb712ba12b30
2023-10-10T02:13:04Z [INFO] [/jobservice/job/impl/gc/garbage_collection.go:261]: blob eligible for deletion: sha256:eac1b95df832dc9f172fd1f07e7cb50c1929b118a4249ddd02c6318a677b506a
2023-10-10T02:13:04Z [INFO] [/jobservice/job/impl/gc/garbage_collection.go:261]: blob eligible for deletion: sha256:47aa3ed2034c4f27622b989b26c06087de17067268a19a1b3642a7e2686cd1a3
2023-10-10T02:13:04Z [INFO] [/jobservice/job/impl/gc/garbage_collection.go:261]: blob eligible for deletion: sha256:0a34e63c37606f57dbba92a722b19bfb743d619a48ea93de80548ca5198ad22d
2023-10-10T02:13:04Z [INFO] [/jobservice/job/impl/gc/garbage_collection.go:261]: blob eligible for deletion: sha256:e29a0832e742d9ab0040e9bab8f502b7890d9d17bc6a140432deee5659710e27
2023-10-10T02:13:04Z [INFO] [/jobservice/job/impl/gc/garbage_collection.go:261]: blob eligible for deletion: sha256:f7f0721d6ce20278942c97b59c361a0256fc8330a1c21fe0044bc85e6ce91d58
2023-10-10T02:13:04Z [INFO] [/jobservice/job/impl/gc/garbage_collection.go:261]: blob eligible for deletion: sha256:5bd4f271ae1c8e7db16cdd6948b5c2736852247ff16414102dd568e5f11f94b5
2023-10-10T02:13:04Z [INFO] [/jobservice/job/impl/gc/garbage_collection.go:261]: blob eligible for deletion: sha256:17497ec24c6e61e3f16afadd5e376e9bf737edb2175e62864d26806bf328a496
2023-10-10T02:13:04Z [INFO] [/jobservice/job/impl/gc/garbage_collection.go:273]: 9 blobs and 1 manifests eligible for deletion
2023-10-10T02:13:04Z [INFO] [/jobservice/job/impl/gc/garbage_collection.go:274]: The GC could free up 373 MB space, the size is a rough estimation.
2023-10-10T02:13:04Z [INFO] [/jobservice/job/impl/gc/garbage_collection.go:200]: success to run gc in job.