that uses ASM as a Storage option on RedHat EL 4 / CentOS 4
By Bhavin Hingu
| |
Creating
and Configuring ASM instance and diskgroups AND creating Database that uses ASM as a Storage option on RedHat EL 4 / CentOS 4 By Bhavin Hingu |
NOTE: Creating ASM Instance on
RedHat AS 3 / AS 2.1/ CentOS 4 /CentOS 3 works same
way as that of on
Rehdat EL 4
Implementing
Automatic Storage Management involves allocating partitioned disks
for Oracle Database with
preferences for striping and mirroring.
Automatic Storage Management manages the disk space for you.
This
helps avoid the need for traditional disk management tools such as
Logical Volume Managers (LVM), file
systems, and the numerous
commands necessary to manage both. The synchronization between
Automatic
Storage Management and the database instance is handled by
Oracle Cluster Synchronization Services (CSS).
Tasks covered:
ASM
Creatioin/implementation Using UNIX IO:
ASM
Creation/Configuration Using Oracle's ASMLib IO:
Creating database
through dbca that uses ASM storage option.:
To
include devices in a diskgroup, you can specify either whole-drive
device names or partition device. Based on the
Redunduncy
Level, you need more devices (or partitions). I have two extra
disks attached to my machine one is internal
harddrive
(IDE) and one is external (SCSI) hd.
NOTE: Oracle recommends that you create a single whole-disk partition on each disk that you want to use.
The device name varies besed on the type of the disk.
|
Disk Type |
Device Name Format |
Description |
|
/dev/hdxn |
In this example, x is a letter that identifies the IDE disk and n is the partition number. For example, /dev/had is the first disk on the first IDE bus. |
|
|
/dev/sdxn |
In this example, x is a letter that identifies the SCSI disk and n is the partition number. For example, /dev/sda is the first disk on the first SCSI bus. |
|
|
/dev/rd/cxdypz /dev/ida/cxdypz |
Depending on the RAID controller, RAID devices can have different device names. In the examples shown, x is a number that identifies the controller, y is a number that identifies the disk, and z is a number that identifies the partition. For example, /dev/ida/c0d1 is the second logical drive on the first controller. |
I
have created 4 physical paritions on /dev/sda and 4 on /dev/hdb just
just so that It seems that I have more disk
available
for experiments. If you are going to create asm disks on production
server, then it is highly recommended that you
create
single partition on the whole device. One of the reason is you have
one controller per disk in this case so as faster IO.
[root@shree ~]# fdisk /dev/hdb
The number of cylinders for this disk is set to 14946.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)
Command (m for help): p
Disk
/dev/hdb: 122.9 GB, 122942324736 bytes
255
heads, 63 sectors/track, 14946 cylinders
Units
= cylinders of 16065 * 512 = 8225280 bytes
Device
Boot Start End Blocks Id System
Command
(m for help): n
Command
action
e extended
p primary partition (1-4)
p
Partition
number (1-4): 1
First
cylinder (1-14946, default 1):<RETURN>
Using
default value 1
Last
cylinder or +size or +sizeM or +sizeK (1-14946, default
14946): +10000M
Command
(m for help): n
Command
action
e extended
p primary partition (1-4)
p
Partition
number (1-4): 2
First
cylinder (1218-14946, default 1218):<RETURN>
Using
default value 1218
Last
cylinder or +size or +sizeM or +sizeK (1218-14946, default
14946): +40000M
Command
(m for help): n
Command
action
e extended
p primary partition (1-4)
p
Partition
number (1-4): 3
First
cylinder (6082-14946,
default 6082):<RETURN>
Using
default value 6082
Last
cylinder or +size or +sizeM or +sizeK (6082-14946, default
14946): +40000M
Command
(m for help): n
Command
action
e extended
p primary partition (1-4)
p
Partition
number (1-4): 4
First
cylinder (10946-14946,
default 10946):<RETURN>
Using
default value 10946
Last
cylinder or +size or +sizeM or +sizeK (10946-14946, default
14946): +40000M
Command (m for help): p
Command
(m for help): w
The
partition table has been altered!
Calling
ioctl() to re-read partition table.
Syncing
disks.
The
kernel still uses the old table.
The new table will be used at the next reboot.
NOTE:
You do not need to reboot the machine just to activate the created
partitions tables available to kernel.
You
can use the below command instead of reboorting the machine:
[root@shree ~]# partprobe
The same way, I partitioned the /dev/sda and the final partition table looks like below:
ASM feature support two different types of IO.
Standard UNIX IO.
ASMLib IO.
This document covers both the IO types.
ASM
Creation/Implementation Using UNIX IO:
[root@shree
~]# cat /etc/sysconfig/rawdevices
#
raw device bindings
#
format: <rawdev> <major> <minor>
# <rawdev> <blockdev>
#
example: /dev/raw/raw1 /dev/sda1
# /dev/raw/raw2 8 5
/dev/raw/raw1
/dev/sda1
/dev/raw/raw2
/dev/sda2
/dev/raw/raw3
/dev/sda3
/dev/raw/raw4
/dev/sda4
/dev/raw/raw5
/dev/hdb4
[root@shree
~]# service rawdevices restart
Also,
you need to change the ownership of these devices to oracle user. Raw
devices are refreshed with the default
permissions
and ownership every time you reboot your system. For this reason, I
add these lines to the /etc/rc.local so
that
every time machine reboots, this devices are assigned correct
ownership/permissions
[root@shree
~]# chown oracle.dba /dev/raw/raw1
[root@shree
~]# chown oracle.dba /dev/raw/raw2
[root@shree
~]# chown oracle.dba /dev/raw/raw3
[root@shree
~]# chown oracle.dba /dev/raw/raw4
[root@shree
~]# chown oracle.dba /dev/raw/raw5
[root@shree
~]# chmod 660 /dev/raw/raw1
[root@shree
~]# chmod 660 /dev/raw/raw2
[root@shree
~]# chmod 660 /dev/raw/raw3
[root@shree
~]# chmod 660 /dev/raw/raw4
[root@shree
~]# chmod 660 /dev/raw/raw5
Please add the below lines to the /etc/rc.local
for
i in `seq 1 5`
do
chown
oracle.dba /dev/raw/raw$i
chmod
660 /dev/raw/raw$i
done
Creating ASM Instance and diskgroups using dbca:
To
Create an ASM Instance using dbca, please connect as oracle user and
type dbca.
Follow
these steps to create an ASM
instance and diskgroups.
Creating
ASM Instance and diskgroups manually without dbca:
Using
vi editor or any other editor you like, create the init+ASM.ora file
under the $ORACLE_HOME/dbs
directory
and add the below lines into this file.
background_dump_dest='/u01/app/admin/+ASM/bdump'
core_dump_dest='/u01/app/admin/+ASM/cdump'
instance_type='asm'
large_pool_size=12M
remote_login_passwordfile='SHARED'
user_dump_dest='/u01/app/admin/+ASM/udump'
[oracle@shree
~]$ cat $ORACLE_HOME/dbs/init+ASM.ora
background_dump_dest='/u01/app/admin/+ASM/bdump'
core_dump_dest='/u01/app/admin/+ASM/cdump'
instance_type='asm'
large_pool_size=12M
remote_login_passwordfile='SHARED'
user_dump_dest='/u01/app/admin/+ASM/udump'
[oracle@shree
~]$
[oracle@shree
~]$ export ORACLE_SID=+ASM
[oracle@shree
~]$ sqlplus / as sysdba
SQL*Plus: Release 10.2.0.1.0 - Production on Thu Dec 1 14:06:35 2005
Copyright (c) 1982, 2005, Oracle. All rights reserved.
Connected to an idle instance.
SQL> create spfile from pfile;
File created.
SQL> startup mount
ASM
instance started
Total
System Global Area 83886080 bytes
Fixed
Size 1217836 bytes
Variable
Size 57502420 bytes
ASM
Cache 25165824 bytes
ORA-15110:
no diskgroups mounted
SQL> alter system set asm_diskstring = '/dev/raw/raw1', '/dev/raw/raw2', '/dev/raw/raw3', '/dev/raw/raw4', '/dev/raw/raw5';
System altered.
SQL> alter system set asm_diskgroups = 'DATA_GRP';
System
altered.
SQL>
create diskgroup data_grp
2 failgroup data_grp_f1
disk '/dev/raw/raw4'
3 failgroup data_grp_f2
disk '/dev/raw/raw5';
Diskgroup
created.
SQL>
set linesize 100
SQL>
col path format a15
SQL>
select name, path from v$asm_disk where name is not null;
NAME
PATH
---------------
---------------
DATA_GRP_0001 /dev/raw/raw5
DATA_GRP_0000 /dev/raw/raw4
SQL>
select name, type, total_mb, free_mb from v$asm_diskgroup;
NAME
TYPE TOTAL_MB FREE_MB
---------------
------ ---------- ----------
DATA_GRP
NORMAL 62776
62701
Use ASM storage option
to an Existing Database which is currently using Filesystem option:
SQL> set linesize 100
SQL> col path format a15
SQL> col name format a50
SQL> select name from v$datafile;
NAME
--------------------------------------------------
/u01/app/oradata/db102/system01.dbf
/u01/app/oradata/db102/undotbs01.dbf
/u01/app/oradata/db102/sysaux01.dbf
/u01/app/oradata/db102/users01.dbf
SQL> select name, path from v$asm_disk where name is not null;
no rows selected
SQL> create tablespace indx01 datafile '+DATA_GRP';
Tablespace created.
SQL> drop tablespace indx01;
Tablespace dropped.
SQL> create tablespace indx01 datafile '+DATA_GRP' SIZE 100m extent
management local uniform size 1m;
Tablespace created.
SQL> drop tablespace indx01;
Tablespace dropped.
SQL> create tablespace indx01
2 datafile '+DATA_GRP' SIZE 100m
3 extent management local
4 segment space management auto
5 uniform size 1m;
Tablespace created.
SQL> select name from v$datafile;
NAME
--------------------------------------------------
/u01/app/oradata/db102/system01.dbf
/u01/app/oradata/db102/undotbs01.dbf
/u01/app/oradata/db102/sysaux01.dbf
/u01/app/oradata/db102/users01.dbf
+DATA_GRP/db102/datafile/indx01.258.576105687
ASM
Creation/Implementation Using Oracle's ASMLib:
[root@shree
~]# uname -a
Linux shree 2.6.9-11.0.0.10.3.EL #1 Tue Jul 5 12:20:09 PDT 2005 i686
athlon i386 GNU/Linux
[root@shree
~]# uname -mi
i686
i386
You
must install the following Packages, where version is the version of the
ASM library driver, arch
is the
system architecture and kernel
is the version of
the kernel that you are using.
oracleasm-support-version.arch.rpm
oracleasm-kernel-verson.arch.rpm
orcleasmlib-version.arch.rpm
I
downloaded the below rpms and istalled them as root user ( su - root if
not logged in as root).
[root@shree
asmlib]# rpm -Uvh oracleasm-support-2.0.1-1.i386.rpm \
>
oracleasm-2.6.9-22.EL-2.0.0-1.i686.rpm \
>
oracleasmlib-2.0.1-1.i386.rpm
Preparing...
########################################### [100%]
1:oracleasm-support
########################################### [ 33%]
2:oracleasm-2.6.9-22.EL
########################################### [ 67%]
3:oracleasmlib
########################################### [100%]
[root@shree asmlib]#
[root@shree
rhel3]# rpm -e oracleasm-support-2.0.0-1
[root@shree rhel3]# rpm -Uvh oracleasm_support_2.0.0_1.i386.rpm \
> oracleasm-2.4.21-27.0.2.ELorafw1-1.0.4-1.i686.rpm \
> oracleasmlib_2.0.0_1.i386.rpm
Preparing...
########################################### [100%]
1:oracleasm-support
########################################### [ 33%]
2:oracleasm-2.4.21-27.0.2########################################### [
67%]
3:oracleasmlib
########################################### [100%]
[root@shree rhel3]#
Enter the following command to run oracleasm init script with configure
option.
[root@shree
rhel3]# /etc/init.d/oracleasm configure
Configuring the Oracle ASM library driver.
This will configure the on-boot properties of the Oracle ASM library
driver. The following questions will determine whether the driver
is
loaded on boot and what permissions it will have. The current
values
will be shown in brackets ('[]'). Hitting <ENTER> without
typing an
answer will keep that current value. Ctrl-C will abort.
Default user to own the driver interface []: oracle
Default group to own the driver interface []: dba
Start Oracle ASM library driver on boot (y/n) [n]: y
Fix permissions of Oracle ASM disks on boot (y/n) [y]:
Writing Oracle ASM library driver
configuration:
[ OK ]
Creating /dev/oracleasm mount
point:
[ OK ]
Loading module
"oracleasm":
[ OK ]
Mounting ASMlib driver
filesystem:
[ OK ]
Scanning system for ASM
disks:
[ OK ]
[root@shree rhel3]#
[root@shree
root]# /etc/init.d/oracleasm createdisk DSK1 /dev/sda1
Marking disk "/dev/sda1" as an ASM
disk:
[ OK ]
[root@shree root]# /etc/init.d/oracleasm createdisk DSK2 /dev/hdb4
Marking disk "/dev/hdb4" as an ASM
disk:
[ OK ]
[root@shree root]#
[root@shree root]# /etc/init.d/oracleasm listdisks
DSK1
DSK2
[root@shree root]#
[root@shree
~]# cat /etc/sysconfig/rawdevices
#
raw device bindings
#
format: <rawdev> <major> <minor>
# <rawdev> <blockdev>
#
example: /dev/raw/raw1 /dev/sda1
# /dev/raw/raw2 8 5
/dev/raw/raw1
/dev/sda1
/dev/raw/raw2
/dev/sda2
/dev/raw/raw3
/dev/sda3
/dev/raw/raw4
/dev/sda4
/dev/raw/raw5
/dev/hdb4
[root@shree
~]# service rawdevices restart
Also,
you need to change the ownership of these devices to oracle user.
[root@shree
~]# chown oracle.dba /dev/raw/raw1
[root@shree
~]# chown oracle.dba /dev/raw/raw2
[root@shree
~]# chown oracle.dba /dev/raw/raw3
[root@shree
~]# chown oracle.dba /dev/raw/raw4
[root@shree
~]# chown oracle.dba /dev/raw/raw5
[root@shree
~]# chmod 660 /dev/raw/raw1
[root@shree
~]# chmod 660 /dev/raw/raw2
[root@shree
~]# chmod 660 /dev/raw/raw3
[root@shree
~]# chmod 660 /dev/raw/raw4
[root@shree
~]# chmod 660 /dev/raw/raw5
Please
add the below lines to the /etc/rc.local so that these are set at every
boot.
for
i in `seq 1 5`
do
chown
oracle.dba /dev/raw/raw$i
chmod
660 /dev/raw/raw$i
done
Creating ASM Instance and Diskgroups using dbca:
To
Create an ASM Instance using dbca, please connect as oracle user and
type dbca.
Follow
these steps to create an ASM
instance and diskgroups.
Creating
ASM Instance and Diskgroup manually without dbca:
[root@shree
root]# /etc/init.d/oracleasm createdisk DSK1 /dev/sda1
Marking disk "/dev/sda1" as an ASM
disk:
[ OK ]
[root@shree root]# /etc/init.d/oracleasm createdisk DSK2 /dev/hdb4
Marking disk "/dev/hdb4" as an ASM
disk:
[ OK ]
[root@shree root]#
[root@shree root]# /etc/init.d/oracleasm listdisks
DSK1
DSK2
[root@shree root]#
[oracle@shree
~]$ mkdir -p $ORACLE_BASE/admin/+ASM
[oracle@shree
~]$ cd $ORACLE_BASE/admin/+ASM
[oracle@shree
+ASM]$ mkdir bdump
[oracle@shree
+ASM]$ mkdir udump
[oracle@shree
+ASM]$ mkdir cdump
[oracle@shree
+ASM]$ mkdir pfile
Using
vi editor or any other editor you like, create the init+ASM.ora file
under the $ORACLE_HOME/dbs
directory
and add the below lines into this file.
asm_diskgroups='PROD_DB_GRP'
asm_diskstring='ORCL:*'
background_dump_dest='/u01/app/admin/+ASM/bdump'
core_dump_dest='/u01/app/admin/+ASM/cdump'
instance_type='asm'
large_pool_size=12M
remote_login_passwordfile='SHARED'
user_dump_dest='/u01/app/admin/+ASM/udump'
[oracle@shree
~]$ cat $ORACLE_HOME/dbs/init+ASM.ora
asm_diskgroups='PROD_DB_GRP'
asm_diskstring='ORCL:*'
background_dump_dest='/u01/app/admin/+ASM/bdump'
core_dump_dest='/u01/app/admin/+ASM/cdump'
instance_type='asm'
large_pool_size=12M
remote_login_passwordfile='SHARED'
user_dump_dest='/u01/app/admin/+ASM/udump'
[oracle@shree
~]$
[oracle@shree
~]$ export ORACLE_SID=+ASM
[oracle@shree
~]$ sqlplus / as sysdba
SQL*Plus: Release 10.2.0.1.0 - Production on Sun Dec 4 21:17:35 2005
Copyright (c) 1982, 2005, Oracle. All rights reserved.
Connected to an idle instance.
SQL> create spfile from pfile;
File created.
SQL> startup mount
ASM
instance started
Total
System Global Area 83886080 bytes
Fixed
Size 1217836 bytes
Variable
Size 57502420 bytes
ASM
Cache 25165824 bytes
ORA-15032: not all alterations performed
ORA-15063: ASM discovered an insufficient number of disks for diskgroup
"PROD_DB_GRP"
SQL> show parameter disk
NAME
TYPE VALUE
------------------------------------ -----------
------------------------------
asm_diskgroups
string PROD_DB_GRP
asm_diskstring
string ORCL:*
disk_asynch_io
boolean TRUE
SQL>
create diskgroup data_grp
2 failgroup f1
disk 'ORCL:DSK1'
3 failgroup f2
disk 'ORCL:DSK2';
Diskgroup
created.
SQL>
set linesize 100
SQL>
col name format a15
SQL>
col path format a15
SQL>
select name, path from v$asm_disk where name is not null;
NAME
PATH
---------------
---------------
DSK1
ORCL:DSK1
DSK2
ORCL:DSK2
SQL>
select name, type, total_mb, free_mb from v$asm_diskgroup;
Creating
database through dbca that Uses ASM storage options:
Follow this steps to create a database
that uses the ASM as a storage Option.