Category: ASM

How to Increase ASM Disk Size in VirtualBox

FRA diskgroup has two ASM disks, we just want to have one disk and also need increase the FRA diskgroup size, here are the steps.

  • Drop ASM disk
  • Clone the disk, because ASM disk file is “Format variant: fixed default”. After cloning, the format changes to “Format variant: dynamic default”.
  • Resize the disk
  • Clone back the disk
  • Partition the disk
  • Create ASM disk
  • Add back into Disk Group

1)Check ASM disks in FRA diskgroup.

SQL>  select GROUP_NUMBER, STATE, TOTAL_MB,FREE_MB,NAME,PATH  
       from  v$asm_disk 
       where GROUP_NUMBER in ( select GROUP_NUMBER 
                                from v$asm_diskgroup 
                               where name='FRA');

GR STATE    TOTAL_MB FREE_MB  NAME      PATH
-- -------- ----- -------- ---------- ------------------------------
2  NORMAL   5119      4954 FRA_0001  /dev/oracleasm/disks/ASM_DISK6
2  NORMAL   5119      4958 FRA_0000  /dev/oracleasm/disks/ASM_DISK5

2) Drop asm disk  FRA_0001.

SQL> ALTER DISKGROUP FRA drop disk FRA_0001;

Diskgroup altered.
SQL> select * from v$asm_operation ;

no rows selected
SQL> select STATE, TOTAL_MB,FREE_MB,NAME,PATH 
      from v$asm_disk where GROUP_NUMBER=2;

STATE  TOTAL_MB FREE_MB NAME     PATH
----- -------- ------- -------- -------------------------------------
NORMAL 5119   4795  FRA_0000 ASM_DISK5 /dev/oracleasm/disks/ASM_DISK5

3)Release asm6.vdi disk file from menu “Host Media Manager”. It detached this file from both racnode1 &2 VMs.

4)Resize asm6.vdi from 5G to 50G.

We can just create a new vdi file. But here for practice purpose, we just choose hard way to resize the fixed size vdi file.

c:\Program Files\Oracle\VirtualBox>VBoxManage showhdinfo D:\virtuallab\asm6.vdi
UUID: 8a52a736-6f03-4766-b656-4e6496cce4eb
Parent UUID: base
State: created
Type: normal (base)
Location: D:\virtuallab\asm6.vdi
Storage format: VDI
Format variant: fixed default
Capacity: 5120 MBytes
Size on disk: 5122 MBytes
Encryption: disabled
Property: AllocationBlockSize=1048576

We must clone it into new one file, dynamically allocated.

c:\Program Files\Oracle\VirtualBox>VBoxManage.exe clonehd 
                   d:\virtuallab\asm6.vdi d:\virtuallab\asm6_new.vdi
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
Clone medium created in format 'VDI'. UUID: e3f94421-f905-4dee-bc8e-91649d187d35
c:\Program Files\Oracle\VirtualBox>VBoxManage showhdinfo D:\virtuallab\asm6_new.vdi
UUID: e3f94421-f905-4dee-bc8e-91649d187d35
Parent UUID: base
State: created
Type: normal (base)
Location: D:\virtuallab\asm6_new.vdi
Storage format: VDI
Format variant: dynamic default
Capacity: 5120 MBytes
Size on disk: 168 MBytes
Encryption: disabled
Property: AllocationBlockSize=1048576

Resize asm6_new.vdi to 50GB, and change into fixed size for ASM disk purpose.

c:\Program Files\Oracle\VirtualBox>VBoxManage modifyhd 
                          d:\virtuallab\asm6_new.vdi --resize 50000
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%

Remove asm6.vdi from “Host Media Manager”, and clone back with fixed size now.

c:\Program Files\Oracle\VirtualBox>VBoxManage.exe clonehd 
  d:\virtuallab\asm6_new.vdi d:\virtuallab\asm6.vdi --variant Fixed
0%...10%...20%...30%...40%...50%...60%...70%...80%...90%...100%
Clone medium created in format 'VDI'. UUID: d0875c78-ac96-49cc-a4d0-2e55422b2560


c:\Program Files\Oracle\VirtualBox>VBoxManage showhdinfo D:\virtuallab\asm6.vdi
UUID: d0875c78-ac96-49cc-a4d0-2e55422b2560
Parent UUID: base
State: created
Type: normal (base)
Location: D:\virtuallab\asm6.vdi
Storage format: VDI
Format variant: fixed default
Capacity: 50000 MBytes
Size on disk: 50002 MBytes
Encryption: disabled
Property: AllocationBlockSize=1048576

Partition disk.

[root@racnode1 ~]# fdisk /dev/sde
Welcome to fdisk (util-linux 2.23.2).

Changes will remain in memory only, until you decide to write them.
Be careful before using the write command.


Command (m for help): p

Disk /dev/sde: 52.4 GB, 52428800000 bytes, 102400000 sectors
Units = sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disk label type: dos
Disk identifier: 0xe5f3fc7c

Device Boot Start End Blocks Id System
/dev/sde1 2048 10485759 5241856 83 Linux

Command (m for help): d
Selected partition 1
Partition 1 is deleted

Command (m for help): 1
1: unknown command
Command action
a toggle a bootable flag
b edit bsd disklabel
c toggle the dos compatibility flag
d delete a partition
g create a new empty GPT partition table
G create an IRIX (SGI) partition table
l list known partition types
m print this menu
n add a new partition
o create a new empty DOS partition table
p print the partition table
q quit without saving changes
s create a new empty Sun disklabel
t change a partition's system id
u change display/entry units
v verify the partition table
w write table to disk and exit
x extra functionality (experts only)

Command (m for help): n
Partition type:
p primary (0 primary, 0 extended, 4 free)
e extended
Select (default p): p
Partition number (1-4, default 1):
First sector (2048-102399999, default 2048):
Using default value 2048
Last sector, +sectors or +size{K,M,G} (2048-102399999, default 102399999):
Using default value 102399999
Partition 1 of type Linux and of size 48.8 GiB is set

Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.

WARNING: Re-reading the partition table failed with error 16: Device or resource busy.
The kernel still uses the old table. The new table will be used at
the next reboot or after you run partprobe(8) or kpartx(8)
Syncing disks.
[root@racnode1 ~]#
[root@racnode1 ~]# oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...
[root@racnode1 ~]# oracleasm listdisks
ASM_DISK1
ASM_DISK2
ASM_DISK3
ASM_DISK5
ASM_DISK6
ASM_OCR_VOTE1
[root@racnode1 ~]# oracleasm deletedisk ASM_DISK6;
Clearing disk header: done
Dropping disk: done

[root@racnode1 ~]# oracleasm createdisk  ASM_DISK6  /dev/sde1
Writing disk header: done
Instantiating disk: done
[root@racnode1 ~]# blkid
/dev/sda1: UUID="fc21c76e-90ae-4d7f-9b1a-52412af3cf78" TYPE="xfs"
/dev/sda2: UUID="ifffmk-GSXU-NcKC-vww1-HMF6-Nc36-A6sewK" TYPE="LVM2_member"
/dev/sda3: UUID="DG6gQe-VQki-1oT2-V1wk-1etX-6qkp-7QP3cX" TYPE="LVM2_member"
/dev/sdb1: LABEL="ASM_DISK1" TYPE="oracleasm"
/dev/sdc1: LABEL="ASM_DISK2" TYPE="oracleasm"
/dev/sdd1: LABEL="ASM_DISK3" TYPE="oracleasm"
/dev/sde1: LABEL="ASM_DISK6" TYPE="oracleasm"
/dev/sdf1: LABEL="ASM_DISK5" TYPE="oracleasm"
/dev/sdg1: LABEL="ASM_OCR_VOTE1" TYPE="oracleasm"
/dev/sr0: UUID="2019-12-10-15-09-23-93" LABEL="VBox_GAs_6.1.0" TYPE="iso9660"
/dev/mapper/ol-root: UUID="b4379983-435f-4012-b054-89e6834c283a" TYPE="xfs"
/dev/mapper/ol-swap: UUID="3bfcff72-20c3-4e1e-ad38-20de7cca2050" TYPE="swap"
[root@racnode1 ~]# lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sdf 8:80 0 5G 0 disk
└─sdf1 8:81 0 5G 0 part
sdd 8:48 0 5G 0 disk
└─sdd1 8:49 0 5G 0 part
sdb 8:16 0 5G 0 disk
└─sdb1 8:17 0 5G 0 part
sr0 11:0 1 56.8M 0 rom
sdg 8:96 0 50G 0 disk
└─sdg1 8:97 0 50G 0 part
sde 8:64 0 48.8G 0 disk
└─sde1 8:65 0 5G 0 part
sdc 8:32 0 5G 0 disk
└─sdc1 8:33 0 5G 0 part
sda 8:0 0 96G 0 disk
├─sda2 8:2 0 29.5G 0 part
│ ├─ol-swap 252:1 0 8G 0 lvm [SWAP]
│ └─ol-root 252:0 0 61G 0 lvm /
├─sda3 8:3 0 66G 0 part
│ ├─ol-swap 252:1 0 8G 0 lvm [SWAP]
│ └─ol-root 252:0 0 61G 0 lvm /
└─sda1 8:1 0 500M 0 part /boot

Add disk back into diskgroup FRA.

SQL> ALTER DISKGROUP FRA ADD DISK '/dev/oracleasm/disks/ASM_DISK6' 
                                               NAME FRA_ASM_DISK6;

Diskgroup altered.
SQL> select STATE, TOTAL_MB,FREE_MB,NAME,PATH
from v$asm_disk where GROUP_NUMBER=2; 

STATE  TOTAL_MB FREE_MB NAME          PATH
------ -------- ------- ------------- ------------------------------
NORMAL    5119  5086    FRA_0000      /dev/oracleasm/disks/ASM_DISK5
NORMAL   49999 49706    FRA_ASM_DISK6 /dev/oracleasm/disks/ASM_DISK6

Drop the small size asm disk ASM_DISK5.

SQL>  ALTER DISKGROUP FRA drop DISK FRA_0000;

Diskgroup altered.

[root@racnode1 ~]#  oracleasm deletedisk ASM_DISK5;
Disk "ASM_DISK5" defines an unmarked device
Dropping disk: done

[root@racnode1 ~]#  oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...

[root@racnode1 ~]#  oracleasm listdisks
ASM_DISK1
ASM_DISK2
ASM_DISK3
ASM_DISK6
ASM_OCR_VOTE1
[root@racnode1 ~]#
SQL>select STATE, TOTAL_MB,FREE_MB,NAME,PATH
    from v$asm_disk where GROUP_NUMBER=2;

STATE TOTAL_MB   FREE_MB NAME         PATH
------ -------- ------- ------------- ------------------------------
NORMAL 49999    49706   FRA_ASM_DISK6 /dev/oracleasm/disks/ASM_DISK6

ORA-15032 ORA-59303 ORA-15221: ASM operation requires compatible.asm of 11.1.0.0.0 or higher

For upgrading 12.1.0.2 GI to 12.2.0.1 GI, after running rootupgrade.sh, there are some disk groups are not mounted.

NAME                  STATE       COMPATIBILITY      V
-------------------- ----------- ------------------- -
GIMR                  DISMOUNTED  0.0.0.0.0          N
FRA                   DISMOUNTED  0.0.0.0.0          N
DATA                  MOUNTED     12.1.0.0.0         N
OCR_VOTE              MOUNTED     11.2.0.4.0         Y
SQL> alter diskgroup FRA mount;
alter diskgroup FRA mount
*
ERROR at line 1:
ORA-15032: not all alterations performed
ORA-59303: The attribute compatible.asm (10.1.0.0.0) of the 
           diskgroup being mounted should be 11.2.0.2.0 or higher.
ORA-15221: ASM operation requires compatible.asm of 11.1.0.0.0 
           or higher
SQL> alter diskgroup FRA mount restricted;

Diskgroup altered.

SQL> alter diskgroup FRA set attribute 'compatible.asm'='11.2.0.4';

Diskgroup altered.

SQL> alter diskgroup FRA dismount;

Diskgroup altered.

SQL> alter diskgroup GIMR mount restricted;

Diskgroup altered.

SQL> alter diskgroup GIMR set attribute 'compatible.asm'='11.2.0.4';

Diskgroup altered.

SQL> alter diskgroup GIMR dismount;

Diskgroup altered.

SQL> alter diskgroup FRA mount;

Diskgroup altered.

SQL> alter diskgroup GIMR mount;

Diskgroup altered.



SQL>   select NAME, STATE, COMPATIBILITY  from  v$asm_diskgroup;

NAME                           STATE       COMPATIBILITY
------------------------------ ----------- -------------------
DATA                           MOUNTED     12.1.0.0.0
FRA                            MOUNTED     11.2.0.4.0
OCR_VOTE                       MOUNTED     11.2.0.4.0
GIMR                           MOUNTED     11.2.0.4.0

ORA-00449: background process ‘RBAL’ unexpectedly terminated with error 448

When startup a RAC database or its instances, and get “ORA-00449: background process ‘RBAL’ unexpectedly terminated with error 448”.

Checked all cluster resources are ONLINE.

SQL> startup 
ORA-00449: background process 'RBAL' unexpectedly terminated with error 448
SQL>

Subscribe to get access

Read more of this content when you subscribe today.

Subscribe

Drop ASM Diskgroup

Here is an example of how to drop an ASM disk group.

On racnode1:

SQL> connect / as sysasm
Connected.
SQL> drop diskgroup ocr;
drop diskgroup ocr
*
ERROR at line 1:
ORA-15039: diskgroup not dropped
ORA-15073: diskgroup OCR is mounted by another ASM instance

On racnode2 ASM instance.

SQL> alter diskgroup ocr dismount;

alter diskgroup OCR dismount
*
ERROR at line 1:
ORA-15032: not all alterations performed
ORA-15260: permission denied on ASM disk group

$ sqlplus / as sysasm

SQL> alter diskgroup ocr dismount;

Diskgroup altered.

Back to racnode1 ASM instance, and drop the diskgroup ocr, which is not longer used and required. There are options:

INCLUDING CONTENTS:You must specify this clause if the disk group contains any files.
EXCLUDING CONTENTS: To ensure that Oracle ASM drops the disk group only when the disk group is empty. This is the default. If the disk group is not empty, then an error will be returned.
FORCE: Clears the headers on the disk belonging to a disk group that cannot be mounted by the Oracle ASM instance. The disk group cannot be mounted by any instance of the database.

SQL> drop diskgroup ocr;

drop diskgroup ocr
*
ERROR at line 1:
ORA-15039: diskgroup not dropped
ORA-15053: diskgroup "OCR" contains existing files
SQL>  drop diskgroup OCR INCLUDING CONTENTS;

Diskgroup dropped.

Find the ASM disk label:

[root@racnode2 ~]# /sbin/blkid
...
..
.
/dev/sdi1: LABEL="ASM_OCR_VOTE" TYPE="oracleasm"
...
..
..

Delete the asm disk “ASM_OCR_VOTE” on one node only:

[root@racnode1 ~]# oracleasm deletedisk ASM_OCR_VOTE
Clearing disk header: done
Dropping disk: done
[root@racnode1 ~]#

[root@racnode1 bin]# oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Cleaning disk "ASM_OCR_VOTE"
Scanning system for ASM disks...

Remove vdi file from VirtualBox.

  1. go to every VM –>settings –>storage, choose the right vdi file and removes seleted storage attachment.
  2. go to Virtual Media Manager –>hard disks Tab –> choose the right one, and  click ‘Remove’.

Reconfigure ASMLIB after Unbreakable Enterprise Kernel (UEK) Upgraded from UEKR3 to UEKR5 on Oracle Linux 7

After unbreakable enterprise kernel (UEK) is upgraded from release 3 to release 5 on Oracle Linux 7,  the oracle asm is not working well:

[root@racnode1 ~]# oracleasm listdisks
[root@racnode1 ~]# oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...
Instantiating disk "ASM_DISK1"
Unable to instantiate disk "ASM_DISK1"
Instantiating disk "ASM_DISK2"
Unable to instantiate disk "ASM_DISK2"
Instantiating disk "ASM_DISK3"
Unable to instantiate disk "ASM_DISK3"
Instantiating disk "ASM_DISK4"
Unable to instantiate disk "ASM_DISK4"
Instantiating disk "ASM_DISK5"
Unable to instantiate disk "ASM_DISK5"
Instantiating disk "ASM_DISK6"
Unable to instantiate disk "ASM_DISK6"
Instantiating disk "ASM_DISK7"
Unable to instantiate disk "ASM_DISK7"
Instantiating disk "ASM_OCR_VOTE"
Unable to instantiate disk "ASM_OCR_VOTE"
[root@racnode1 ~]# oracleasm listdisks

Check oracleasm packages , and found there are one or two are missing:

[root@racnode1 ~]# rpm -qa|grep -i oracleasm
oracleasm-support-2.1.11-2.el7.x86_64
[root@racnode1 ~]#

The Oracle ASMLib kernel driver is now included in the Unbreakable Enterprise Kernel. No driver package needs to be installed when using this kernel.

Check Oracle ASMLib kernel driver has been installed.

[root@racnode1 ~]# lsmod|grep asm
oracleasm              61440  1

[root@racnode1 ~]# modinfo oracleasm
filename:       /lib/modules/4.14.35-1902.8.4.el7uek.x86_64/kernel/drivers/block/oracleasm/oracleasm.ko.xz
description:    Kernel driver backing the Generic Linux ASM Library.
author:         Joel Becker, Martin K. Petersen <martin.petersen@oracle.com>
version:        2.0.8
license:        GPL
srcversion:     DF8809442FB655948FEC227
depends:
retpoline:      Y
intree:         Y
name:           oracleasm
vermagic:       4.14.35-1902.8.4.el7uek.x86_64 SMP mod_unload modversions
signat:         PKCS#7
signer:
sig_key:
sig_hashalgo:   md4
parm:           use_logical_block_size:Prefer logical block size over physical (Y=logical, N=physical [default]) (bool)
[root@racnode1 ~]#

For RHEL linux, install  kmod-oracleasm package:

[root@racnode1 ~]# yum install kmod-oracleasm

Download and install oracleasmlib-2.0.12-1.el7.x86_64.rpm package from below url, if it is unavailable.

https://www.oracle.com/linux/downloads/linux-asmlib-v7-downloads.html

Check the required packages are all available now:

[root@racnode1 ~]# rpm -qa|grep -i oracleasm
oracleasmlib-2.0.12-1.el7.x86_64
oracleasm-support-2.1.11-2.el7.x86_64
kmod-oracleasm-2.0.8-26.0.1.el7.x86_64 ( for RHEL )

Configure ASM

[root@racnode1 ~]# oracleasm configure -i
Configuring the Oracle ASM library driver.

This will configure the on-boot properties of the Oracle ASM library
driver. The following questions will determine whether the driver is
loaded on boot and what permissions it will have. The current values
will be shown in brackets ('[]'). Hitting <ENTER> without typing an
answer will keep that current value. Ctrl-C will abort.

Default user to own the driver interface [grid]:
Default group to own the driver interface [oinstall]:
Start Oracle ASM library driver on boot (y/n) [y]:
Scan for Oracle ASM disks on boot (y/n) [y]:
Writing Oracle ASM library driver configuration: done
[root@racnode1 ~]#

Re-initialize oracleasm:

[root@racnode1 sysconfig]# oracleasm init
Creating /dev/oracleasm mount point: /dev/oracleasm
Loading module "oracleasm": oracleasm
Configuring "oracleasm" to use device physical block size
Mounting ASMlib driver filesystem: /dev/oracleasm

Scan and list ASM disks:

[root@racnode1 sysconfig]# oracleasm status
Checking if ASM is loaded: yes
Checking if /dev/oracleasm is mounted: yes


[root@racnode1 sysconfig]# oracleasm scandisks
Reloading disk partitions: done
Cleaning any stale ASM disks...
Scanning system for ASM disks...
Instantiating disk "ASM_DISK1"
Instantiating disk "ASM_DISK2"
Instantiating disk "ASM_DISK3"
Instantiating disk "ASM_DISK4"
Instantiating disk "ASM_DISK5"
Instantiating disk "ASM_DISK6"
Instantiating disk "ASM_DISK7"
Instantiating disk "ASM_OCR_VOTE"


[root@racnode1 sysconfig]# oracleasm listdisks
ASM_DISK1
ASM_DISK2
ASM_DISK3
ASM_DISK4
ASM_DISK5
ASM_DISK6
ASM_DISK7
ASM_OCR_VOTE