欢迎您关注我的公众号【尚雷的驿站】
****************************************************************************
公众号：尚雷的驿站
CSDN ：https://blog.csdn.net/shlei5580
墨天轮：https://www.modb.pro/u/2436
PGFans：https://www.pgfans.cn/user/home?userId=4159
****************************************************************************
一、前言
最近遇到一个需求，有套NetAPP存储已过保一段时间，目前作为测试环境使用，但最近这套存储出现故障，一条链路状态异常。导致其上配置的多路径有两条链路非Active状态，系统组担心会影响到数据库的使用。最近刚好有套过保的3par存储，在配置和性能方面都要优于这套过保的NetAPP存储，希望我配合进行存储的替换。
目前这套NetAPP存储，目前在其上部署的有4套测试环境，其中有三套都是单机在使用，可以方便替换，其中有套Oracle 19C RAC测试库，虽然是测试库，但当前生产对应的很多测试都是在其上进行验证测试，非常重要，需要提前和项目组沟通，并要避免数据的丢失和损坏。
之前我曾验证过EMC存储和3Par存储的替换，因为EMC日常使用的是EMC自带的存储，3Par存储通常使用操作系统自带multipah多路径软件，在同一台服务器上安装时会有不兼容的情况。
但目前这套NetAPP使用的multipah多路径软件,和3par还是可以兼容的。
为防止在替换过程中会出现一些意外，我这边还是提前做了一些相关备份操作，将多路径软及OCR信息进行了手工备份，并将数据库数据进行了备份。
并通知了项目组，计划在下班后进行操作。
二、环境信息
这套19C RAC库操作系统为Centos 7.9，当前存储使用3个多T。
[root@local-test-xxxx~]# cat /etc/redhat-release
CentOS Linux release 7.9.2009 (Core)
[root@local-test-mlandb1 ~]# su - grid
Last login: Mon Aug 28 13:29:37 CST 2023
[grid@local-test-xxxx~]$ ./asm/asm
CURDATE
-------------------
2023-08-28 13:44:07
Disk Group Sector Block Allocation
Name Size Size Unit Size State Type Total Size (MB) Used Size (MB) FREE_MB Pct. Used
---------------- ------- ------- ------------ ----------- ------ --------------- -------------- ---------- ---------
CRS 512 4,096 4,194,304 MOUNTED NORMAL 30,672 1,132 29540 3.69
DG 512 4,096 4,194,304 MOUNTED EXTERN 3,583,968 3,188,400 395568 88.96
--------------- --------------
Grand Total: 3,614,640 3,189,532
ASM使用的是Oracle ASMLIB包安装的ASM磁盘。
[root@local-test-xxxx~]# oracleasm -V
oracleasm version 2.1.11
[root@local-test-xxxx~]# oracleasm listdisks
VOL01
VOL02
VOL03
VOL04
VOL05
VOL06
VOL07
OCR信息如下：
[grid@local-test-xxxx ~]$ ocrcheck
Status of Oracle Cluster Registry is as follows :
Version : 4
Total space (kbytes) : 491684
Used space (kbytes) : 84332
Available space (kbytes) : 407352
ID : 231456364
Device/File Name : +CRS
Device/File integrity check succeeded
Device/File not configured
Device/File not configured
Device/File not configured
Device/File not configured
Cluster registry integrity check succeeded
Logical corruption check bypassed due to non-privileged user
SQL> select GROUP_NUMBER,NAME,TYPE,STATE,total_mb/1024 total_gb,free_mb/1024 free_gb,round((free_mb/total_mb)*100,2)||'%' pct_free from v$asm_diskgroup;
1 CRS NORMAL MOUNTED 23.9882813 22.9648438 95.73%
2 DG EXTERN MOUNTED 7000.41016 3983.82813 56.91%
ASM磁盘信息如下
SQL> select GROUP_NUMBER,free_mb,total_mb,FAILGROUP,disk_number,MOUNT_STATUS,mode_status,STATE,HEADER_STATUS,name,PATH from v$asm_disk order by 4,5;
1 6844 10224 CRS_0001 3 CACHED ONLINE NORMAL MEMBER CRS_0001 /dev/oracleasm/disks/VOL01
1 6848 10224 CRS_0002 4 CACHED ONLINE NORMAL MEMBER CRS_0002 /dev/oracleasm/disks/VOL02
1 6848 10224 CRS_0003 5 CACHED ONLINE NORMAL MEMBER CRS_0003 /dev/oracleasm/disks/VOL03
2 291368 512076 DG_00004 0 CACHED ONLINE NORMAL MEMBER DG_0000 /dev/oracleasm/disks/VOL04
2 291368 512076 DG_00005 1 CACHED ONLINE NORMAL MEMBER DG_0001 /dev/oracleasm/disks/VOL05
2 874144 1536192 DG_00006 3 CACHED ONLINE NORMAL MEMBER DG_0003 /dev/oracleasm/disks/VOL06
2 582744 1024108 DG_00007 2 CACHED ONLINE NORMAL MEMBER DG_00007 /dev/oracleasm/disks/VOL07
7 rows selected.
**多路径信息备份： **
[root@local-test-xxxx~]# multipath -ll > /root/multipath_xxx.txt
数据库备份,使用expdp对数据库进行导出备份
OCR备份：
-- root用户
[root@local-test-mlandbxxx ~]# /u01/app/19.3.0/grid/bin/ocrconfig -export /tmp/ocr_20230808.dmp
然后通知系统管理员为这套RAC 分配3Par存储。为何原存储进行区分，新分配的存储没块大小都和之前不同，便于区分，存储分配后，先逐台重启了两台数据库服务器识别到新加的存储。
然后使用dd if=/dev/zero of=/dev/mapper/xxx bs=1024k count=10000 对新加的存储进行格式化，然后进行fdisk分区。
之后通过如下oracleasm命令创建新的ASM磁盘，新创建的ASM磁盘名称和原磁盘名称区分开来。
[root@local-test-xxxx~]# oracleasm createdisk VOL1 /dev/mapper/mpathi1
[root@local-test-xxxx~]# oracleasm createdisk VOL2 /dev/mapper/mpathj1
[root@local-test-xxxx~]# oracleasm createdisk VOL3 /dev/mapper/mpathk1
[root@local-test-xxxx~]# oracleasm createdisk VOL4 /dev/mapper/mpathl1
[root@local-test-xxxx~]# oracleasm createdisk VOL5 /dev/mapper/mpathm1
---然后再二个节点扫盘识别到新加的ASM存储盘。
[root@local-test-xxxx2 ~]# oracleasm scandisks
三、磁盘替换
对于数据盘替换，这块之前已经做了好多次，心里有把握，现将新的ASM磁盘通过如下方式添佳到数据磁盘组：
su - grid
sqlplus /as sysasm
SQL> alter diskgroup DG add disk '/dev/oracleasm/disks/VOL4' rebalance power 11;
SQL> alter diskgroup DG add disk '/dev/oracleasm/disks/VOL5' rebalance power 11;
SQL> SQL> select * from v$asm_operation;
2 REBAL COMPACT WAIT 11 11 0 0 0 0 0
2 REBAL REBALANCE RUN 11 11 5962 71713 3900 16 0
2 REBAL REBUILD DONE 11 11 0 0 0 0
替换OCR盘，之前替换10G RAC OCR盘的时候比较麻烦，需要修改的配置较多，但替换19C OCR盘还是第一次，心里也是没有把握，于是查询了网上一些资料，看到JikeXu曾写过 11g https://cloud.tencent.com/developer/article/1696479 OCR替换，不知道19C是不是也可以，并向其进行了请教。
SQL> alter diskgroup CRS add disk '/dev/oracleasm/disks/VOL1';
alter diskgroup CRS add disk '/dev/oracleasm/disks/VOL1'
*
ERROR at line 1:
ORA-15032: not all alterations performed
ORA-15410: Disks in disk group CRS do not have equal size.
因为新的OCR盘和原盘大小不同，无法直接添加，恩墨彗星性老师给我一个他曾写的文章：https://www.modb.pro/db/543611 对于ORA-15410 Disks in disk group OCR_VOT do not have equal size 有相应的解释和处理办法。我本次没有去尝试。
于是决定采用JiekeXU提到的方法将全部OCR添加进去并替换掉原有的OCR盘。
su - grid
sqlplus / as sysasm
SQL> alter diskgroup CRS add disk '/dev/oracleasm/disks/VOL1','/dev/oracleasm/disks/VOL2','/dev/oracleasm/disks/VOL3' drop disk 'CRS_0000','CRS_0001','CRS_0002';
Diskgroup altered.
SQL> select GROUP_NUMBER,free_mb,total_mb,FAILGROUP,disk_number,MOUNT_STATUS,mode_status,STATE,HEADER_STATUS,name,PATH from v$asm_disk order by 4,5;
1 9844 10224 CRS_0003 3 CACHED ONLINE NORMAL MEMBER CRS_0003 /dev/oracleasm/disks/VOL1
1 9848 10224 CRS_0004 4 CACHED ONLINE NORMAL MEMBER CRS_0004 /dev/oracleasm/disks/VOL2
1 9848 10224 CRS_0005 5 CACHED ONLINE NORMAL MEMBER CRS_0005 /dev/oracleasm/disks/VOL3
2 291368 512076 DG_0000 0 CACHED ONLINE NORMAL MEMBER DG_0000 /dev/oracleasm/disks/VOL04
2 291368 512076 DG_0001 1 CACHED ONLINE NORMAL MEMBER DG_0001 /dev/oracleasm/disks/VOL05
2 874144 1536192 DG_0003 3 CACHED ONLINE NORMAL MEMBER DG_0003 /dev/oracleasm/disks/VOL07
2 1107116 1945584 DG_0004 4 CACHED ONLINE NORMAL MEMBER DG_0004 /dev/oracleasm/disks/VOL4
2 932304 1638384 DG_0005 5 CACHED ONLINE NORMAL MEMBER DG_0005 /dev/oracleasm/disks/VOL5
2 582744 1024108 VOL06 2 CACHED ONLINE NORMAL MEMBER VOL06 /dev/oracleasm/disks/VOL06
0 0 0 0 CLOSED ONLINE NORMAL FORMER /dev/oracleasm/disks/VOL02
0 0 0 1 CLOSED ONLINE NORMAL FORMER /dev/oracleasm/disks/VOL01
0 0 0 2 CLOSED ONLINE NORMAL FORMER /dev/oracleasm/disks/VOL03
12 rows selected.
原来三块OCR盘确实都被替换了。
四、附录
4.1 替换OCR盘日志信息
以下是OCR盘替换时查看到的ASM日志信息
替换OCR盘的日志信息如下：
2023-08-08T16:00:10.337765+08:00
SQL> alter diskgroup CRS add disk '/dev/oracleasm/disks/VOL1','/dev/oracleasm/disks/VOL2','/dev/oracleasm/disks/VOL3' drop disk 'CRS_0000','CRS_0001','CRS_0002'
2023-08-08T16:00:10.339009+08:00
NOTE: GroupBlock outside rolling migration privileged region
NOTE: Assigning number (1,3) to disk (/dev/oracleasm/disks/VOL1)
NOTE: Assigning number (1,4) to disk (/dev/oracleasm/disks/VOL2)
NOTE: Assigning number (1,5) to disk (/dev/oracleasm/disks/VOL3)
NOTE: requesting all-instance membership refresh for group=1
2023-08-08T16:00:11.457532+08:00
NOTE: Attempting voting file relocation on diskgroup CRS
WARNING: Read Failed. group:1 disk:3 AU:0 offset:0 size:4096
path:Unknown disk
incarnation:0xf0f0ed7e asynchronous result:'I/O error' ioreason:2818 why:11
subsys:Unknown library krq:0x7f5e6fbe5938 bufp:0x7f5e75183000 osderr1:0x434c5344 osderr2:0x0
IO elapsed time: 0 usec Time waited on I/O: 0 usec
WARNING: Read Failed. group:1 disk:4 AU:0 offset:0 size:4096
path:Unknown disk
incarnation:0xf0f0ed7f asynchronous result:'I/O error' ioreason:2818 why:11
subsys:Unknown library krq:0x7f5e6fbe5490 bufp:0x7f5e75181000 osderr1:0x434c5344 osderr2:0x0
IO elapsed time: 0 usec Time waited on I/O: 0 usec
WARNING: Read Failed. group:1 disk:5 AU:0 offset:0 size:4096
path:Unknown disk
incarnation:0xf0f0ed80 asynchronous result:'I/O error' ioreason:2818 why:11
subsys:Unknown library krq:0x7f5e751809e0 bufp:0x7f5e6fc13000 osderr1:0x434c5344 osderr2:0x0
IO elapsed time: 0 usec Time waited on I/O: 0 usec
NOTE: Successful voting file relocation on diskgroup CRS
2023-08-08T16:00:12.534839+08:00
NOTE: Disk 3 in group 1 is assigned fgnum=4
NOTE: Disk 4 in group 1 is assigned fgnum=5
NOTE: Disk 5 in group 1 is assigned fgnum=6
NOTE: discarding redo for group 1 disk 3
NOTE: discarding redo for group 1 disk 4
NOTE: discarding redo for group 1 disk 5
NOTE: initializing header (replicated) on grp 1 disk CRS_0003
NOTE: initializing header (replicated) on grp 1 disk CRS_0004
NOTE: initializing header (replicated) on grp 1 disk CRS_0005
NOTE: initializing header on grp 1 disk CRS_0003
NOTE: initializing header on grp 1 disk CRS_0004
NOTE: initializing header on grp 1 disk CRS_0005
NOTE: requesting all-instance disk validation for group=1
2023-08-08T16:00:12.605238+08:00
NOTE: skipping rediscovery for group 1/0xd4a01dae (CRS) on local instance.
2023-08-08T16:00:12.727464+08:00
NOTE: running client discovery for group 1 (reqid:15321278565476053588)
NOTE: requesting all-instance disk validation for group=1
2023-08-08T16:00:12.731123+08:00
NOTE: skipping rediscovery for group 1/0xd4a01dae (CRS) on local instance.
2023-08-08T16:00:12.829109+08:00
NOTE: running client discovery for group 1 (reqid:15321278565476065544)
NOTE: Allocated 1 DD reserve extents for group 1 (CRS)
NOTE: Allocated 6 VAT reserve extents for group 1 (CRS).
NOTE: Adding disk 3 (CRS_0003) to grp 1 (CRS) (2556 AUs)
2023-08-08T16:00:13.804894+08:00
NOTE: Adding disk 4 (CRS_0004) to grp 1 (CRS) (2556 AUs)
2023-08-08T16:00:14.387378+08:00
NOTE: Adding disk 5 (CRS_0005) to grp 1 (CRS) (2556 AUs)
2023-08-08T16:00:14.992385+08:00
GMON updating for reconfiguration, group 1 at 21 for pid 34, osid 30275
2023-08-08T16:00:15.003705+08:00
NOTE: group 1 PST updated.
2023-08-08T16:00:15.068630+08:00
NOTE: membership refresh pending for group 1/0xd4a01dae (CRS)
NOTE: Attempting voting file refresh on diskgroup CRS
NOTE: Refresh completed on diskgroup CRS. Found 3 voting file(s).
NOTE: Voting file relocation is required in diskgroup CRS
2023-08-08T16:00:15.129060+08:00
GMON querying group 1 at 22 for pid 27, osid 6491
NOTE: cache opening disk 3 of grp 1: CRS_0003 path:/dev/oracleasm/disks/VOL1
NOTE: cache opening disk 4 of grp 1: CRS_0004 path:/dev/oracleasm/disks/VOL2
NOTE: cache opening disk 5 of grp 1: CRS_0005 path:/dev/oracleasm/disks/VOL3
2023-08-08T16:00:15.552478+08:00
NOTE: Attempting voting file refresh on diskgroup CRS
NOTE: Refresh completed on diskgroup CRS. Found 3 voting file(s).
NOTE: Voting file relocation is required in diskgroup CRS
NOTE: Attempting voting file relocation on diskgroup CRS
NOTE: Successful voting file relocation on diskgroup CRS
2023-08-08T16:00:15.616116+08:00
GMON querying group 1 at 23 for pid 27, osid 6491
2023-08-08T16:00:15.657735+08:00
SUCCESS: refreshed membership for 1/0xd4a01dae (CRS)
2023-08-08T16:00:15.658273+08:00
SUCCESS: alter diskgroup CRS add disk '/dev/oracleasm/disks/VOL1','/dev/oracleasm/disks/VOL2','/dev/oracleasm/disks/VOL3' drop disk 'CRS_0000','CRS_0001','CRS_0002'
2023-08-08T16:00:15.659229+08:00
NOTE: starting rebalance of group 1/0xd4a01dae (CRS) at power 1
NOTE: starting process ARBA
Starting background process ARBA
2023-08-08T16:00:15.706992+08:00
ARBA started with pid=46, OS id=48198
NOTE: starting process ARB0
Starting background process ARB0
2023-08-08T16:00:15.729429+08:00
ARB0 started with pid=47, OS id=48200
NOTE: assigning ARBA to group 1/0xd4a01dae (CRS) to compute estimates
NOTE: assigning ARB0 to group 1/0xd4a01dae (CRS) with 1 parallel I/O
2023-08-08T16:00:16.056816+08:00
NOTE: header on disk 0 advanced to format #2 using fcn 0.0
NOTE: F1X0 on disk 0 (fmt 2) relocated at fcn 0.149388: AU 10 -> AU 0
NOTE: header on disk 1 advanced to format #2 using fcn 0.0
NOTE: F1X0 on disk 1 (fmt 2) relocated at fcn 0.149388: AU 10 -> AU 0
NOTE: header on disk 2 advanced to format #2 using fcn 0.0
NOTE: F1X0 on disk 2 (fmt 2) relocated at fcn 0.149388: AU 10 -> AU 0
NOTE: header on disk 3 advanced to format #2 using fcn 0.0
NOTE: F1X0 on disk 3 (fmt 2) relocated at fcn 0.149388: AU 0 -> AU 10
NOTE: header on disk 4 advanced to format #2 using fcn 0.0
NOTE: F1X0 on disk 4 (fmt 2) relocated at fcn 0.149388: AU 0 -> AU 10
NOTE: header on disk 5 advanced to format #2 using fcn 0.0
NOTE: F1X0 on disk 5 (fmt 2) relocated at fcn 0.149388: AU 0 -> AU 10
NOTE: 08/08/23 16:00:15 CRS.F1X0 copy 1 relocating from 0:10 to 4:10 at FCN 0.149388
NOTE: 08/08/23 16:00:15 CRS.F1X0 copy 2 relocating from 1:10 to 3:10 at FCN 0.149388
NOTE: 08/08/23 16:00:15 CRS.F1X0 copy 3 relocating from 2:10 to 5:10 at FCN 0.149388
2023-08-08T16:00:18.760349+08:00
NOTE: Attempting voting file refresh on diskgroup CRS
NOTE: Refresh completed on diskgroup CRS. Found 3 voting file(s).
NOTE: Voting file relocation is required in diskgroup CRS
NOTE: Attempting voting file relocation on diskgroup CRS
NOTE: voting file allocation (replicated) on grp 1 disk CRS_0003
NOTE: voting file allocation on grp 1 disk CRS_0003
NOTE: voting file allocation (replicated) on grp 1 disk CRS_0004
NOTE: voting file allocation on grp 1 disk CRS_0004
NOTE: voting file allocation (replicated) on grp 1 disk CRS_0005
NOTE: voting file allocation on grp 1 disk CRS_0005
NOTE: voting file deletion (replicated) on grp 1 disk CRS_0000
NOTE: voting file deletion on grp 1 disk CRS_0000
NOTE: voting file deletion (replicated) on grp 1 disk CRS_0001
NOTE: voting file deletion on grp 1 disk CRS_0001
NOTE: voting file deletion (replicated) on grp 1 disk CRS_0002
NOTE: voting file deletion on grp 1 disk CRS_0002
NOTE: Successful voting file relocation on diskgroup CRS
2023-08-08T16:00:21.170262+08:00
NOTE: stopping process ARB0
NOTE: stopping process ARBA
NOTE: Starting expel slave for group 1/0xd4a01dae (CRS)
2023-08-08T16:00:21.201736+08:00
NOTE: GroupBlock outside rolling migration privileged region
NOTE: requesting all-instance membership refresh for group=1
2023-08-08T16:00:21.304682+08:00
GMON updating for reconfiguration, group 1 at 24 for pid 49, osid 48289
2023-08-08T16:00:21.308250+08:00
NOTE: group 1 PST updated.
SUCCESS: grp 1 disk CRS_0000 emptied
SUCCESS: grp 1 disk CRS_0001 emptied
SUCCESS: grp 1 disk CRS_0002 emptied
NOTE: process _x000_+asm1 (48289) initiating offline of disk 0.4042321254 (CRS_0000) with mask 0x7e in group 1 (CRS) without client assisting
NOTE: process _x000_+asm1 (48289) initiating offline of disk 1.4042321255 (CRS_0001) with mask 0x7e in group 1 (CRS) without client assisting
NOTE: process _x000_+asm1 (48289) initiating offline of disk 2.4042321253 (CRS_0002) with mask 0x7e in group 1 (CRS) without client assisting
NOTE: initiating PST update: grp 1 (CRS), dsk = 0/0xf0f0ed66, mask = 0x6a, op = clear mandatory
NOTE: initiating PST update: grp 1 (CRS), dsk = 1/0xf0f0ed67, mask = 0x6a, op = clear mandatory
NOTE: initiating PST update: grp 1 (CRS), dsk = 2/0xf0f0ed65, mask = 0x6a, op = clear mandatory
2023-08-08T16:00:21.335368+08:00
GMON updating disk modes for group 1 at 25 for pid 49, osid 48289
NOTE: group CRS: updated PST location: disks 0003 0004 0005
2023-08-08T16:00:21.342612+08:00
NOTE: PST update grp = 1 completed successfully
NOTE: initiating PST update: grp 1 (CRS), dsk = 0/0xf0f0ed66, mask = 0x7e, op = clear mandatory
NOTE: initiating PST update: grp 1 (CRS), dsk = 1/0xf0f0ed67, mask = 0x7e, op = clear mandatory
NOTE: initiating PST update: grp 1 (CRS), dsk = 2/0xf0f0ed65, mask = 0x7e, op = clear mandatory
2023-08-08T16:00:21.343143+08:00
GMON updating disk modes for group 1 at 26 for pid 49, osid 48289
2023-08-08T16:00:21.345868+08:00
NOTE: cache closing disk 0 of grp 1: CRS_0000
2023-08-08T16:00:21.349547+08:00
NOTE: cache closing disk 1 of grp 1: CRS_0001
2023-08-08T16:00:21.349885+08:00
NOTE: cache closing disk 2 of grp 1: CRS_0002
2023-08-08T16:00:21.351878+08:00
NOTE: PST update grp = 1 completed successfully
NOTE: Expelling disk 0 (CRS_0000) from grp 1 (CRS)
NOTE: Expelling disk 1 (CRS_0001) from grp 1 (CRS)
NOTE: Expelling disk 2 (CRS_0002) from grp 1 (CRS)
2023-08-08T16:00:21.354440+08:00
GMON updating for reconfiguration, group 1 at 27 for pid 49, osid 48289
2023-08-08T16:00:21.354951+08:00
NOTE: cache closing disk 0 of grp 1: (not open) CRS_0000
2023-08-08T16:00:21.355051+08:00
NOTE: cache closing disk 1 of grp 1: (not open) CRS_0001
2023-08-08T16:00:21.355145+08:00
NOTE: cache closing disk 2 of grp 1: (not open) CRS_0002
2023-08-08T16:00:21.357594+08:00
NOTE: group 1 PST updated.
NOTE: grp 1 disk 0 expelled from the PST.
NOTE: grp 1 disk 1 expelled from the PST.
NOTE: grp 1 disk 2 expelled from the PST.
NOTE: Erasing header on /dev/oracleasm/disks/VOL01
NOTE: Erasing header on /dev/oracleasm/disks/VOL02
NOTE: Erasing header on /dev/oracleasm/disks/VOL03
2023-08-08T16:00:21.625721+08:00
NOTE: membership refresh pending for group 1/0xd4a01dae (CRS)
NOTE: Attempting voting file refresh on diskgroup CRS
NOTE: Refresh completed on diskgroup CRS. Found 3 voting file(s).
NOTE: Voting file relocation is required in diskgroup CRS
2023-08-08T16:00:21.687195+08:00
GMON querying group 1 at 28 for pid 27, osid 6491
GMON querying group 1 at 29 for pid 27, osid 6491
2023-08-08T16:00:21.708706+08:00
NOTE: Disk CRS_0000 in mode 0x0 marked for de-assignment
NOTE: Disk CRS_0001 in mode 0x0 marked for de-assignment
NOTE: Disk CRS_0002 in mode 0x0 marked for de-assignment
SUCCESS: refreshed membership for 1/0xd4a01dae (CRS)
2023-08-08T16:00:22.580833+08:00
SUCCESS: rebalance completed for group 1/0xd4a01dae (CRS)
NOTE: Attempting voting file refresh on diskgroup CRS
NOTE: Refresh completed on diskgroup CRS. Found 3 voting file(s).
NOTE: Voting file relocation is required in diskgroup CRS
NOTE: Attempting voting file relocation on diskgroup CRS
NOTE: Successful voting file relocation on diskgroup CRS
4.2 剔除数据盘日志信息
2023-08-08T16:15:21.792257+08:00
NOTE: requesting all-instance membership refresh for group=2
2023-08-08T16:15:21.820517+08:00
NOTE: membership refresh pending for group 2/0xd4c01daf (DG)
2023-08-08T16:15:21.822852+08:00
GMON querying group 2 at 30 for pid 27, osid 6491
2023-08-08T16:15:21.844719+08:00
SUCCESS: refreshed membership for 2/0xd4c01daf (DG)
2023-08-08T16:15:21.845335+08:00
SUCCESS: alter diskgroup DG drop disk DG_0000 rebalance power 11 nowait
2023-08-08T16:15:23.305285+08:00
NOTE: Attempting voting file refresh on diskgroup DG
NOTE: Refresh completed on diskgroup DG. No voting file found.
2023-08-08T16:15:23.328193+08:00
NOTE: starting rebalance of group 2/0xd4c01daf (DG) at power 11
NOTE: starting process ARBA
Starting background process ARBA
2023-08-08T16:15:23.377695+08:00
ARBA started with pid=39, OS id=10549
NOTE: starting process ARB0
Starting background process ARB0
2023-08-08T16:15:23.403147+08:00
ARB0 started with pid=47, OS id=10551
NOTE: assigning ARBA to group 2/0xd4c01daf (DG) to compute estimates
NOTE: assigning ARB0 to group 2/0xd4c01daf (DG) with 11 parallel I/Os
2023-08-08T16:15:23.603553+08:00
NOTE: F1X0 on disk 0 (fmt 2) relocated at fcn 0.14556462: AU 10 -> AU 0
NOTE: header on disk 4 advanced to format #2 using fcn 0.0
NOTE: F1X0 on disk 4 (fmt 2) relocated at fcn 0.14556462: AU 0 -> AU 209615
NOTE: 08/08/23 16:15:23 DG.F1X0 copy 1 relocating from 0:10 to 4:209615 at FCN 0.14556462
