删除磁盘（删）
删除未绑定磁盘
[SOLVED] – Removing disk image | Proxmox Support Forum
我们从虚拟机配置中删除了该磁盘，但没有删除实际的磁盘。
运行 qm rescan 命令，磁盘将再次关联到虚拟机，然后我们再执行手动删除；
修改磁盘（改）
磁盘扩容
qm resize 100 virtio0 +5G # 将磁盘增加 5G qm resize 100 virtio0 5G # 将磁盘设为 5G
参考 Resize disks – Proxmox VE 文档，以获取后续的 磁盘扩容、文件系统扩容 相关内容。
调整带宽
Disk Throttling Settings and Default Values | Proxmox Support Forum
qm set 168010105 --scsi0 file=local-lvm:vm-168010105-disk-0,iops_rd=180,iops_wr=180,mbps_rd=40,mbps_wr=40
通过 Web 界面，配置磁盘，选择 Bandwidth 选项卡，修改其中的 Read limit、Write limit 字段；
I’m experimenting with the VM Disk Throttle settings. So far here is my understanding:
1）Read limit (MB/s): total read bandwidth per second allowed (I’m using 115 to simulate a HDD)
2）Write limit (MB/s): total write bandwidth per second allowed (I’m using 115 to simulate a HDD)
3）Read limit (ops/s): number of read io operations per second (iops) (I’m using 300 to simulate a HDD doing sequential reads)
4）Read limit (ops/s): number of write io operations per second (iops) (I’m using 100 to simulate a HDD doing sequential writes)
In terms of the Read max burst (MB) (similar theory for Write max burst (MB), Read max burst (ops), Write max burst (ops)). This value refers to a leaky bucket algorithm which I’m assuming works like so:
1）You set it for something equal to or larger than the “Read limit (MB/s)” say 500.
2）This will allow you to burst up to 500MB at the max speed or the storage system.
3）Once you hit that limit you have to wait for the bucket to refill from the value you set for “Read Limit (MB/s)”. So for a value of 115 this would take 4.34s. This is assuming you have no IO happening after the burst otherwise that will delay the refill.
NFS
unable to activate storage ‘iso-image’ – directory ‘xxx’ does not exist or is unreachable (500)
[SOLVED] – Storage status unknown | Proxmox Support Forum
[SOLVED] – Storage status unknown | Proxmox Support Forum
问题描述：突然发现 NFS 无法正常访问，提示如上错误；
原因分析：我们曾修改 PVE 管理接口的地址，由于 NFS 限制，导致我们无法正常访问；
解决方案：调整 NFS 访问控制，允许新地址进行访问；
SSD Trim Support
[SOLVED] – Trim/discard With CEPH Rbd | Proxmox Support Forum
TRIM Support on KVM Virtual Machines – ZenCoffee Blog – random notes, guides, and thoughts…
超融合（Ceph）
Deploy Hyper-Converged Ceph Cluster – Proxmox VE
通过命令行安装 Ceph 存储：
# pveceph install
创建 OSD 实例
需要选择节点，然后再添加磁盘；
删簇 OSD 实例
Proxmox Ceph remove OSD – How to do it via Proxmox VE GUI and CLI?
通过图形化界面删除：
1）Firstly, we select the Proxmox VE node in the tree.
2）Next, we go to Ceph >> OSD panel. Then we select the OSD to remove. And click the OUT button.
3）When the status is OUT, we click the STOP button. This changes the status from up to down.
4）Finally, we select the More drop-down and click Destroy.
通过命令行删除：
1）ceph osd out {osd-num}
2）查看 OSD 数据迁移状态：ceph -w
3）systemctl stop ceph-osd@{osd-num}
4）ceph osd crush remove {name}
5）ceph auth del osd.{osd-num}
6）ceph osd rm {osd-num}
清理为删除的 Monitor 实例
[SOLVED] – CEPH how to delete dead monitor? | Proxmox Support Forum
如果为清理 Ceph 而直接删除节点，则会导致 Monitor 状态异常。
删除未使用的 Monitor 实例：
# ceph -s ... # ceph mon remove xxxxxxx
问题：虽然 Monitor 已删，但是仍旧在 GUI 中显示；
[SOLVED] – Ghost monitor in CEPH cluster | Page 2 | Proxmox Support Forum
linux – How to remove systemd services – Super User
Re-adding Ceph Node | Proxmox Support Forum
[SOLVED] – CEPH how to delete dead monitor? | Proxmox Support Forum
Reinstall/remove dead monitor | Proxmox Support Forum
systemctl disable ceph-mon@<name-of-mon>.service rm -rf /etc/systemd/system/ceph-mon.target.wants/ceph-mon@<name-of-mon>.service rm -rf /var/lib/ceph/mon/<name of monitor> vim /etc/pve/ceph.conf ...（1）删除对应的 Monitor 条目； ...（2）删除 [global].mon_host 中对应地址；
常见问题处理
所有 LV 进入 INACTIVE 状态
local-LVM not available after Kernel update on PVE 7 | Page 2 | Proxmox Support Forum
2023213 – Autoactivation of thin pool during boot fails
问题描述：
当物理节点重启后，该节点虚拟机无法正常启动，虚拟机实例的磁盘进入 INACTIVE 状态；
原因分析：
thin_check 执行时间过长，导致 pvscan 被结束。
解决方案：
# 问题修复 lvchange -an pve/data_tmeta lvchange -an pve/data_tdata vgchange -ay pve # Workaround vim /etc/lvm/lvm.conf ... thin_check_options = [ "-q", "--skip-mappings" ] update-initramfs -u
创建 OSD 失败：… unable to find a keyring … entity osd.0 exists but key does not match …
[SOLVED] – PVE7 unable to create OSD | Proxmox Support Forum
entity osd.5 exists but key does not match | Proxmox Support Forum
在添加 OSD 设备时，提示如下错误：
create OSD on /dev/sdi (bluestore) wiping block device /dev/sdi 200+0 records in 200+0 records out 209715200 bytes (210 MB, 200 MiB) copied, 0.464955 s, 451 MB/s Running command: /bin/ceph-authtool --gen-print-key Running command: /bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring -i - osd new 435258e3-31c8-4fc7-958e-3d37f881e388 stderr: 2022-10-13T01:37:26.795+0800 7f9276af5700 -1 auth: unable to find a keyring on /etc/pve/priv/ceph.client.bootstrap-osd.keyring: (2) No such file or directory stderr: 2022-10-13T01:37:26.795+0800 7f9276af5700 -1 AuthRegistry(0x7f927005b868) no keyring found at /etc/pve/priv/ceph.client.bootstrap-osd.keyring, disabling cephx stderr: Error EEXIST: entity osd.0 exists but key does not match --> RuntimeError: Unable to create a new OSD id TASK ERROR: command 'ceph-volume lvm create --cluster-fsid fc27b60a-41ba-4540-987e-0cd0f7d1b1f4 --data /dev/sdi' failed: exit code 1
按照 [SOLVED] – PVE7 unable to create OSD | Proxmox Support Forum 提示，进行修复：
cp /etc/pve/priv/ceph.client.bootstrap-osd.keyring /etc/pve/priv/ceph.client.bootstrap-osd.keyring.backup # 该文件应该是不存在的 ceph auth get client.bootstrap-osd > /etc/pve/priv/ceph.client.bootstrap-osd.keyring
但是，再次添加 OSD 设备，提示如下错误：
create OSD on /dev/sdi (bluestore) wiping block device /dev/sdi 200+0 records in 200+0 records out 209715200 bytes (210 MB, 200 MiB) copied, 0.47554 s, 441 MB/s Running command: /bin/ceph-authtool --gen-print-key Running command: /bin/ceph --cluster ceph --name client.bootstrap-osd --keyring /var/lib/ceph/bootstrap-osd/ceph.keyring -i - osd new a2f8be83-dfc5-4843-a984-a511f815f96b stderr: Error EEXIST: entity osd.0 exists but key does not match --> RuntimeError: Unable to create a new OSD id TASK ERROR: command 'ceph-volume lvm create --cluster-fsid fc27b60a-41ba-4540-987e-0cd0f7d1b1f4 --data /dev/sdi' failed: exit code 1
按照 entity osd.5 exists but key does not match | Proxmox Support Forum 提示，我们发现是操作不规范：旧的 OSD 没有正确删除；
解决方案：ceph auth del osd.0