~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
For Understanding Grid and ASM Concepts:
Grid:
https://docs.oracle.com/cd/B16240_01/welcome.html
ASM:
http://docs.oracle.com/cd/E11882_01/server.112/e18951/asmcon.htm#OSTMG03601
For Implementations:
1> To migrate from filesystem to ASM check doc id: Steps To Migrate/Move a Database From Non-ASM to ASM And Vice-Versa (Doc ID 252219.1)
2> For Grid Installations:
a) Grid Basic Installation:
https://docs.oracle.com/cd/E11857_01/install.111/e15838.pdf
b) Grid Advanced Installation:
https://docs.oracle.com/cd/E11857_01/install.111/e16847.pdf
ASM 11.2 Configuration KIT (ASM 11gR2 Installation & Configuration, Deinstallation, Upgrade, ASM Job Role Separation (Standalone Only). (Doc ID 1092213.1)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ASM is now part of Grid Infrastructure, which includes, Clusterware, ASM and ACFS. So now you’ll Install Grid Infrastructure to use ASM.
-In 11gR2 there are two options for install –
GI for Standalone Server (aka Oracle Restart) and GI for Clusterware
Q. How does the database instance "connects" to the asm instance and put files within ?
A. The database communicates with ASM instance using the ASMB (umblicusprocess) process. Once the database obtains the necessary extents from extent map, all database IO going forward is processed through by the database processes, bypassing ASM.
Detailed RDBMS and ASM Instance Interaction:
(1) Database opens file
(1A) OPEN: Database issues open of a database file.
(1B) Extent Map : ASM sends the extent map for the file to database instance. Starting with 11g, the RDBMS only receives first 60 extents the remaining extents in the extent map are paged in on demand, providing a faster open.
(2) Database reads the file
(2A) Read : Database now reads directly from disk.
(2B) I/O Completes : Database now reads directly from disk.
(3) Database Creates file
(3A) Create : RDBMS foreground initiates a create tablespace for example.
(3B) Allocates file : ASM does the allocation for its essentially reserving the allocation units for the file creation.
(3C) Extent Map : Once allocation phase is done, the extent map is sent to the RDBMS.
(3D) Commit : The RDBMS initialization phase kicks in. In this phase the initializes all the reserved AUs.
(3E) If file creation is successful, then the RDBMS commits the file creation. Going forward all I/Os are done by the RDBMS.
RDBMS does I/O directly to the raw disk devices. ASM is not in the I/O path so ASM does not impede the database file access. Since the RDBMS instance is performing raw I/O, the I/O is as fast as possible.
Q. Is it so straight forward that anyone can put data into ASM without any authentication or connection details ?
A. Operating System Authentication happens for Oracle ASM
Membership in the operating system group designated as the OSASM group provides operating system authentication for the SYSASM system privilege. OSASM is provided exclusively for Oracle ASM. Initially, only the user that installs ASM is a member of the OSASM group, if you use a separate operating system group for that privilege. However, you can add other users. Members of the OSASM group are authorized to connect using the SYSASM privilege and have full access to Oracle ASM, including administrative access to all disk groups that are managed by that Oracle ASM instance.
On Linux and UNIX systems, asmadmin, asmoper and asmdba is the default operating system group designated as OSASM, OSOPER, and OSDBA respectively for Oracle ASM.
On Windows systems, ORA_ASMADMIN, ORA_ASMDBA, and ORA_ASMOPER are the operating system groups designated for OSASM, OSDBA and OSOPER respectively for Oracle ASM.
For more details please refer 5.1.4.4 Oracle Automatic Storage Management Groups for Job Role Separation : https://docs.oracle.com/database/121/LADBI/usr_grps.htm#LADBI2820.
Q. Is there any permission, configuration , or OS groups permission that need to be done ? Do i need to allow oracle into the asmadmin group ?
A. Yes, You will need to add only asmdba group to oracle if using different user for asm instance; Else add asmadmin and asmdba group as if using same oracle user for both instances.
Operating system authentication using membership in the group or groups designated as OSDBA, OSOPER, and OSASM is valid on all Oracle platforms.
A typical deployment could be as follows:
ASM administrator:
User : grid
Group: oinstall,asmdba(OSDBA),asmadmin(OSASM)
Database administrator:
User : oracle
Group: oinstall,asmdba(OSDBA of ASM),dba(OSDBA)
ASM disk ownership : grid:oinstall
Remember that Database instance connects to ASM instance as sysdba. The user id the database instance runs as needs to be the OSDBA group of the ASM instance.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
q1) given your explanation on the OS authentication, if asm instance is running as grid user and oracle instance is running as oracle. what group should be added to oracle in order for oracle to access the disks (for storing and retrieving of data). is it osdba or osasm ? (i must emphasize the operation required is to store and retrieve data into/outof asm instance)
A. You will need to add asmdba group to Oracle in above case.
q2) can i say that without the appropriate privilege specified above, the database instance will not be able to access asm instance ?
A. Without the appropriate privilege specified, the database instance will not be able to access asm instance. As an ASM instance does not have a data dictionary, so access to the instance is restricted to users who can authenticate with the operating system.
Summary of the groups and privileges:
asmadmin : Using this group and the SYSASM system privileges enables the separation of SYSDBA database administration privileges from Oracle ASM storage administration privileges. Members of the OSASM group are authorized to connect using the SYSASM privilege and have full access to Oracle ASM, including administrative access to all disk groups that the Oracle ASM instance manages.
asmdba : This group grants access for the database to connect to Oracle ASM. During installation, the Oracle Installation Users are configured as members of this group. After you create an Oracle Database, this groups contains the contains Oracle Home Users of those database homes.
asmoper (optional) : Members of this group are granted SYSOPER system privileges on the Oracle ASM instance, which permits a user to perform operations such as startup, shutdown, mount, dismount, and check disk group. This group has a subset of the privileges of the OSASM group. This group does not have any members after installation, but you can manually add users to this group after the installation completes.
During the installation of Oracle Database, all groups mentioned in the table are populated for proper operation of Oracle products. You must not remove any group member populated by Oracle.
q3) how does the oracle instance in fact able to detect the existence of the "asm" instance running in the background, since both their oracle_home and oracle_sid are different.
A. RDBMS and ASM Interactions:
When an ASM instance is started it registers with the Cluster Synchronisation Services (CSS) daemon running as occsd.bin. This is part of Oracle Clusterware, but the occsd.bin runs in a single instance configuration as well.
Whenever an RDBMS instance needs for the first time to access files that reside on ASM storage and the RDBMS has not needed prior access to the ASM instance, the RDBMS needs to find the connection details for the ASM instance.
These connection details are NOT stored anywhere and you do not need to configure them. The RDBMS instance contacts CSS and gets passed the connect string for the ASM instance. The RDBMS then connects as sysdba to the ASM instance using OS authentication.
It’s actually the RDBMS’s ASMB background process that connects to the ASM instance and remains connected while there are ASM files open. Termination of this ASMB background process is fatal for the instance.
Summarizing, RDBMS finds CSS which passes connect string for the ASM Instance. On successfull connection, RDBMS connects as sysdba to ASM using OS authentication.
q4) is it via the listener ?
A. No
Role of listeners in StandAlone Grid Environment Only-
DB Listener : Listener under DB home : For Database Connections.
ASM Listener : Listener under Grid home : To listen for ASM Connections.. for example if you want to manage ASM from OEM.
Best practices in RAC and Restart environments is it to only have one listener (the one of GI Home). There is no need to have an additional listener defined in the database home. While you can separate them, there is really no additional value in doing so.
q5) do i have set any parameter to specify where the asm instance is in the database ?
A. No. ASM Instance and RDBMS interactions are facilitated by css daemon process and ASMB Background process ASMB.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
q1) I must emphasize that i have my oracle database installed before i install grid. Will the database instance still know how to access the asm via css daemon ? Do i have to register my database instance with oracle restart to do so ?
A. Yes, You need to check CSS before Installing ASM Instance. Please find the steps:
1. Check CSS must be running before any ASM Instance
Check CSS Running or Not?
cd $ORACLE_HOME/bin
crsctl check css
2. If its not running, you should configure CSS process by running $ORACLE_HOME/bin/localconfig script:
login as a root
$ORACLE_HOME/bin/localconfig add
Sample Output:
/etc/oracle does not exist. Creating it now.
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
Configuration for local CSS has been initialized
Adding to inittab
Startup will be queued to init within 90 seconds.
Checking the status of new Oracle init process...
Expecting the CRS daemons to be up within 600 seconds.
CSS is active on these nodes.
RAC1
CSS is active on all nodes.
Oracle CSS service is installed and running under init(1M)
q2) how does the database instance know that it will need to access the ASM ? is it when the storage destination is specify with a "+xxx" e.g. +DATA ?
A. Yes, css daemon process registers ASM Instance and ASMB Background process makes the connections after successful authorization . Obviously, after the filesystem is migrated to ASM Storage (+xxx) Database and ASM Instance interacts.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
q1) With the steps you have provided, is it just for creation/starting of the css daemon ?
Will the oracle instance register with the css daemon as well or will the oracle instance detects the existence of a css daemon ?
A. For migration from Non-ASM to ASM, we would need css daemon to be up and running. css is a daemon process of Oracle.
q2) in the documentation for oracle grid for cluster, for job role separation, i do not see that grid user is assigned with the dba priviledge but in oracle restart for grid single installation it seems like dba is granted to grid user as well.
Can you confirm the permission for grid and oracle in oracle cluster rac environment and oracle grid single installation ?
Grid cluster
oracle - dba, oper, asmdba, asmadmin, asmoper, oinstall
grid - asmdba, asmadmin,asmoper oinstall -- need dba ?
Grid Single installation
oracle - dba, oper, asmdba, asmadmin, asmoper, oinstall
grid - asmdba, asmadmin,asmoper oinstall -- need dba ?
A. For both the above cases dba is granted to grid user.
As mentioned earlier, it is not mandatory; for example: user grid with only group ASMDBA is not able to restart oracle instance but just asm instance. Group dba can be granted to grid user if you want to use Oracle Restart feature.
As in RAC environment, Clusterware takes care of Oracle restart; but you can still assign dba privilege to be on safer side.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Scenario 1) install oracle database instance 1st, then install oracle grid for single instance database
1) grid user = dba, oinstall, asmadmin, asmdba
A -> ASM instance should be created and up and running as a pre-requisite before start of the migration of Non-ASM to ASM Database.
2) oracle user = dba, oinstall, asmdba --->
q1) does oracle need asmadmin for the oracle instance to access the asm instance ?
A-> asmadmin is not required if using different user for Grid (grid) and RDBMS Instances (oracle).
3) in order for oracle instance to access asm instance ->
q2) do we need to register the database instance with oracle restart or anything ? or just make sure that the CSS is running ?
A-> Yes, You just need to ensure CSS is configured and running.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ q4. ) will there be any issue for the db instance to connect to the asm instance if there are not sharing the same listener ?
A. No, as stated earlier:
Role of listeners in StandAlone Grid Environment Only-
DB Listener : Listener under DB home : For Database Connections.
ASM Listener : Listener under Grid home : To listen for ASM Connections.. for example if you want to manage ASM from OEM.
Best practices in RAC and Restart environments is it to only have one listener (the one of GI Home). There is no need to have an additional listener defined in the database home. While you can separate them, there is really no additional value in doing so.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Configuring the listener using netca or manually from new cluster ORACLE_HOME or Grid Infrastructure home in case of 11gr2.
Stop the single instance listeners when they are running on one of the clustered nodes in case they are using the same listener ports.
If there is multiple database is running and you don’t want to stop all database, Create new LISTENER with different port.
Ideally, use the ‘LISTENER’ as name for that listener or you can change.
Setting up the listener Steps:
a. Create a new listener (ASM Instance)
b. Set local_listener parameter in asm and database instance (as listener isn’t on default port)
sql> alter system set LOCAL_LISTENER=LISTENER scope=both sid=’*';
System altered.
eg:
On ASM instance issue the following:
SQL> ALTER SYSTEM SET local_listener='(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=server1.domain.com)(PORT=1531))))' scope=both;
SYSTEM altered
On your database instance you would issue similarly:
SQL> ALTER SYSTEM SET local_listener='(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=server1.domain.com)(PORT=1541))))' scope=both;
SYSTEM altered.
c. Resolve this local_listener in the tnsnames.ora
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
q5) how can we allow 1 listener to listen to both service ?
By default, the Oracle Database Configuration Assistant (DBCA) does not configure the TNS listener to accept client requests to an ASM instance.
Connecting to ASM through the existing TNS Listener.
1> Modify the listener.ora for the ASM Instances
LISTENER_LINUX1 =
(DESCRIPTION_LIST =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = linux1-vip)(PORT = 1521)(IP = FIRST))
(ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.9.100)(PORT = 1521)(IP = FIRST))
)
)
SID_LIST_LISTENER_LINUX1 =
(SID_LIST =
(SID_DESC =
(SID_NAME = PLSExtProc)
(ORACLE_HOME = /u01/app/oracle/product/11.2.0/db_1)
(PROGRAM = extproc)
)
(SID_DESC =
(GLOBAL_DBNAME = +ASM)
(SID_NAME = +ASM1)
(ORACLE_HOME = /u02/app/oracle/product/11.2.0/db_1)
)
)
2> Also, Add ASM Entries to the tnsnames.ora File on the Client Machine
3> Bounce the TNS Listener and tnspings the services
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
q1) for disks to be used by ASM, do we still need to fdisk it ?
or we can assign the disk device as a whole (using udev with proper rights, user, group etc). Meaning do i assign e.g /dev/sda or fdisk and assign /dev/sda1 ?
A) Yes, To use a disk for ASM, you must create partition and stamp the disks as ASM disks. There are ASM libraries to stamp disks as ASM. These are distributed on your Linux kernel version.
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel. Building a new DOS disklabel. Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won’t be recoverable.
In order to use a disk (eg SAN) in ASM, the disk must have a partition table. Oracle recommends creating exactly one partition for each disk containing the entire disk.
You can use any physical disk for ASM, as long as it is partitioned. Use IDE, SCSI, or RAID devices.
Oracle recommends that you create a single whole-disk partition on each disk that you want to use.
Use either fdisk or parted to create a single whole-disk partition on the disk devices that you want to use.
q2) if i am using external redundancy, in the event that 1 of the asm disk has failed -> does that mean that data cannot be retrieved any more ?
or only affected tables with extents in the disk will fail ?
A) No data cannot be retreived in case asm disk corruption with no RAID configuration. When an ASM disk becomes unavailable, an external redundancy disk group would be simply dismounted.
Hence, OS Data will not be available; you will need to add another ASM disk and recover the data.
q3) When ASM do stripping, what does it actually strip ? If i create a datafile of 5mb, is it the extents that will be stripped across the disks ?
A) It is important for you to understand, ASM stripes at the database segment/extent level, for example SAN would stripe at the filesystem level.
q4) With the above scenario (external redundancy with 1 disk failed) but I have external RAID on and assuming that the lost data is rebuilt on the additional RAID disk
Will ASM be affected or it will be transparent to ASM ?
I am using udev (via UUID) to present disk devices to ASM. So i am not sure if I have external RAID on (e.g. RAID 1 or 5) and the lost data is rebuilt or on new disk
(do i need to present the new disk to ASM as well ?)
A) Raid is not redundancy for data. Yes, the terminology is a little different, ASM stripes at the database segment/extent level, RAID would stripe at the filesystem level, but the concept is the same.
ASM is done at the extent level of database segments instead of some OS stripe size.
If initially you have external redundancy, in case of failure of disk; it gets dismounted forcefully; So yes you need to present the new disk to ASM as well and recover the data.
q5) when creating db using dbca and specifying ASM as the storage, it prompted me to enter the password for ASMSNMP, why does dbca or the database need access to ASMSNMP ?
A) ASMSNMP is a less privileged user that is primarily used by Oracle Enterprise Manager to monitor Oracle ASM instances.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Q1) Are you able to elaborate further on why the need for partitioning/fdisk for the disk ?
I have tried using a raw disk (without partitioning) on ASM and it works
A) Yes, you can use raw disk. It is just the best practice as recommended by Oracle as there are Devices that does not contains disklabel.
So In order to use such disk in ASM, the disk must have a partition table as Building a new DOS disklabel.
You can create an ASM diskgroup using one of the following storage resources:
1) Raw disk partition—A raw partition can be the entire disk drive or a section of a disk drive. However, the ASM disk cannot be in a partition that includes the partition table because the partition table can be overwritten.
2) Logical unit numbers (LUNs)—Using hardware RAID functionality to create LUNs is a recommended approach. Storage hardware RAID 0+1 or RAID5, and other RAID configurations, can be provided to ASM as ASM disks.
3) Raw logical volumes (LVM)—LVMs are supported in less complicated configurations where an LVM is mapped to a LUN, or an LVM uses disks or raw partitions. LVM configurations are not recommended by Oracle because they create a duplication of functionality. Oracle also does not recommended using LVMs for mirroring because ASM already provides mirroring.
4) NFS NAS files—If you have a certified NAS device, then you can create zero-padded files in an NFS mounted directory and use those files as disk devices in an Oracle Automatic Storage Management (Oracle ASM) diskgroup.
Q2) Assuming if i have 3 disks in a diskgroup (with external redundancy and no raid configured);
if 1 of the disks failed, does that mean my whole diskgroup is unusable and no data can be retrieved at all ?
A) With no RAID configured, when 1 disk fails, the diskgroup will get dismounted forcefully. And, data (as OS filesystem files) will not be available which will require recovery.
Q3) You mentioned that ASM stripped at Segment/Extent level.
Multiple continuous Oracle blocks = 1 extent.
Multiple extents = 1 Segment
If I have 3 disk, do you mean that for 1 segment/table with 3 extents, extent#1 will be in disk1, extent#2 on disk2 and extent#3 on disk3 ?
How does Oracle assign the extent for usage ? (round robin) ?
A) To stripe data, Oracle ASM separates files into stripes and spreads data evenly across all of the disks in a disk group. It has two methods of striping, Fine & Coarse.
Fine Striping:- Fine striping writes 128 KB data to each ASM Disk in the diskgroup in a round robin fashion, 128 KB goes to the first disk, then the next 128 KB, goes to the next disk, etc. According to manual, The fine-grained stripe size always equals 128 KB in any configuration; this provides lower I/O latency for small I/O operations.” Small I/O operations sure sounds like a good candidate for redo logs, control files etc.
Coarse Striping:-With coarse grained striping ASM writes data to each disk in the same round robin fashion, but writes chunks in the size of the ASM instance’s allocation unit (AU) size, default is 1MB.
Further, starting from 11gR2, Online redo log files are no more in Fine striping contradictory to 10g, as Oracle silently changed the template to Coarse striping.
A note has been written on the same:- Redo Log Striping In 11.2 ASM, is Coarse Or Fine? [ID 1269158.1]
Q3a) In terms of Oracle blocks in ASM setup, is it still mapped to OS size block or directly to harddisk sectors ?
e.g. 1 oracle block = x hdd sectors ?
or 1 oracle block = x OS blocks
A) In simple expainations mapping happens at block level: When setting up ASM, Allocation Unit (AU) is allocated (takes the default value unless specified) And, During RDBMS setup, Oracle Block is setup.
Oracle Data block is the way through which Oracle maintains your datafiles. The ASM AU is the way through which the blocks are allocated from the shared pool when the file modifications request come from the db.
Please understand, ASM is Oracle's recommended storage management solution (Volume Manager) that provides an alternative to conventional volume managers, file systems, and raw devices.
ASM uses disk groups to store datafiles; an ASM disk group is a collection of disks that ASM manages as a unit. Within a disk group, ASM exposes a file system interface for Oracle database files. The content of files that are stored in a disk group are evenly distributed, or striped, to eliminate hot spots and to provide uniform performance across the disks. The performance is comparable to the performance of raw devices.
Every Oracle ASM disk is divided into allocation units (AU). An allocation unit is the fundamental unit of allocation within a disk group. A file extent consists of one or more allocation units. An Oracle ASM file consists of one or more file extents.
About Allocation Unit, AU:
An allocation unit is the minimum segment of disk that can be assigned to a file. Each extent is an integral number of AU. Each file contain integral number of extents
The default AU size is set to 1MB for ASM . There are ways to set the AU using FINE grained striping(128K) and COARSE grained stripe is equal to AU size selected
Q4) How is raid not redundancy for data. Can you elaborate further
if i am present a LUN/device (which is make up underlying disks with RAID5 configured) to ASM and adds it to diskgroup +DATA.
The diskgroup has only this LUN/device. if an underlying disk in the RAID failed , the LUN is still represented as usual. Will ASM be affected at all ?
A) It applies to all RAID disks. You can use any RAID disks - create an ASM diskgroup for those disks and tell ASM not to mirror it, as that diskgroup resides on externally redundant disks.
ASM does not know what RAID levels a disk is. It only knows that it is a disk that it can use. The HOW TO USE, you need to explicitly tell ASM.
You need to tell ASM whether you want to use that disk as part of externally redundant diskgroup as below:
To create an Oracle ASM disk group where redundancy is provided by the storage array, first create the RAID-protected logical unit numbers (LUNs) in the storage array, and then create the Oracle ASM disk group using the EXTERNAL REDUNDANCY clause:
CREATE DISKGROUP DATA EXTERNAL REDUNDANCY DISK
'/devices/lun1','/devices/lun2','/devices/lun3','/devices/lun4';
In the event of a disk failure the RAID controller switches to one of the mirrors to prevent system failure, while in ASM alert log you will see notifications.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
For Understanding Grid and ASM Concepts:
Grid:
https://docs.oracle.com/cd/B16240_01/welcome.html
ASM:
http://docs.oracle.com/cd/E11882_01/server.112/e18951/asmcon.htm#OSTMG03601
For Implementations:
1> To migrate from filesystem to ASM check doc id: Steps To Migrate/Move a Database From Non-ASM to ASM And Vice-Versa (Doc ID 252219.1)
2> For Grid Installations:
a) Grid Basic Installation:
https://docs.oracle.com/cd/E11857_01/install.111/e15838.pdf
b) Grid Advanced Installation:
https://docs.oracle.com/cd/E11857_01/install.111/e16847.pdf
ASM 11.2 Configuration KIT (ASM 11gR2 Installation & Configuration, Deinstallation, Upgrade, ASM Job Role Separation (Standalone Only). (Doc ID 1092213.1)
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
ASM is now part of Grid Infrastructure, which includes, Clusterware, ASM and ACFS. So now you’ll Install Grid Infrastructure to use ASM.
-In 11gR2 there are two options for install –
GI for Standalone Server (aka Oracle Restart) and GI for Clusterware
Q. How does the database instance "connects" to the asm instance and put files within ?
A. The database communicates with ASM instance using the ASMB (umblicusprocess) process. Once the database obtains the necessary extents from extent map, all database IO going forward is processed through by the database processes, bypassing ASM.
Detailed RDBMS and ASM Instance Interaction:
(1) Database opens file
(1A) OPEN: Database issues open of a database file.
(1B) Extent Map : ASM sends the extent map for the file to database instance. Starting with 11g, the RDBMS only receives first 60 extents the remaining extents in the extent map are paged in on demand, providing a faster open.
(2) Database reads the file
(2A) Read : Database now reads directly from disk.
(2B) I/O Completes : Database now reads directly from disk.
(3) Database Creates file
(3A) Create : RDBMS foreground initiates a create tablespace for example.
(3B) Allocates file : ASM does the allocation for its essentially reserving the allocation units for the file creation.
(3C) Extent Map : Once allocation phase is done, the extent map is sent to the RDBMS.
(3D) Commit : The RDBMS initialization phase kicks in. In this phase the initializes all the reserved AUs.
(3E) If file creation is successful, then the RDBMS commits the file creation. Going forward all I/Os are done by the RDBMS.
RDBMS does I/O directly to the raw disk devices. ASM is not in the I/O path so ASM does not impede the database file access. Since the RDBMS instance is performing raw I/O, the I/O is as fast as possible.
Q. Is it so straight forward that anyone can put data into ASM without any authentication or connection details ?
A. Operating System Authentication happens for Oracle ASM
Membership in the operating system group designated as the OSASM group provides operating system authentication for the SYSASM system privilege. OSASM is provided exclusively for Oracle ASM. Initially, only the user that installs ASM is a member of the OSASM group, if you use a separate operating system group for that privilege. However, you can add other users. Members of the OSASM group are authorized to connect using the SYSASM privilege and have full access to Oracle ASM, including administrative access to all disk groups that are managed by that Oracle ASM instance.
On Linux and UNIX systems, asmadmin, asmoper and asmdba is the default operating system group designated as OSASM, OSOPER, and OSDBA respectively for Oracle ASM.
On Windows systems, ORA_ASMADMIN, ORA_ASMDBA, and ORA_ASMOPER are the operating system groups designated for OSASM, OSDBA and OSOPER respectively for Oracle ASM.
For more details please refer 5.1.4.4 Oracle Automatic Storage Management Groups for Job Role Separation : https://docs.oracle.com/database/121/LADBI/usr_grps.htm#LADBI2820.
Q. Is there any permission, configuration , or OS groups permission that need to be done ? Do i need to allow oracle into the asmadmin group ?
A. Yes, You will need to add only asmdba group to oracle if using different user for asm instance; Else add asmadmin and asmdba group as if using same oracle user for both instances.
Operating system authentication using membership in the group or groups designated as OSDBA, OSOPER, and OSASM is valid on all Oracle platforms.
A typical deployment could be as follows:
ASM administrator:
User : grid
Group: oinstall,asmdba(OSDBA),asmadmin(OSASM)
Database administrator:
User : oracle
Group: oinstall,asmdba(OSDBA of ASM),dba(OSDBA)
ASM disk ownership : grid:oinstall
Remember that Database instance connects to ASM instance as sysdba. The user id the database instance runs as needs to be the OSDBA group of the ASM instance.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
q1) given your explanation on the OS authentication, if asm instance is running as grid user and oracle instance is running as oracle. what group should be added to oracle in order for oracle to access the disks (for storing and retrieving of data). is it osdba or osasm ? (i must emphasize the operation required is to store and retrieve data into/outof asm instance)
A. You will need to add asmdba group to Oracle in above case.
q2) can i say that without the appropriate privilege specified above, the database instance will not be able to access asm instance ?
A. Without the appropriate privilege specified, the database instance will not be able to access asm instance. As an ASM instance does not have a data dictionary, so access to the instance is restricted to users who can authenticate with the operating system.
Summary of the groups and privileges:
asmadmin : Using this group and the SYSASM system privileges enables the separation of SYSDBA database administration privileges from Oracle ASM storage administration privileges. Members of the OSASM group are authorized to connect using the SYSASM privilege and have full access to Oracle ASM, including administrative access to all disk groups that the Oracle ASM instance manages.
asmdba : This group grants access for the database to connect to Oracle ASM. During installation, the Oracle Installation Users are configured as members of this group. After you create an Oracle Database, this groups contains the contains Oracle Home Users of those database homes.
asmoper (optional) : Members of this group are granted SYSOPER system privileges on the Oracle ASM instance, which permits a user to perform operations such as startup, shutdown, mount, dismount, and check disk group. This group has a subset of the privileges of the OSASM group. This group does not have any members after installation, but you can manually add users to this group after the installation completes.
During the installation of Oracle Database, all groups mentioned in the table are populated for proper operation of Oracle products. You must not remove any group member populated by Oracle.
q3) how does the oracle instance in fact able to detect the existence of the "asm" instance running in the background, since both their oracle_home and oracle_sid are different.
A. RDBMS and ASM Interactions:
When an ASM instance is started it registers with the Cluster Synchronisation Services (CSS) daemon running as occsd.bin. This is part of Oracle Clusterware, but the occsd.bin runs in a single instance configuration as well.
Whenever an RDBMS instance needs for the first time to access files that reside on ASM storage and the RDBMS has not needed prior access to the ASM instance, the RDBMS needs to find the connection details for the ASM instance.
These connection details are NOT stored anywhere and you do not need to configure them. The RDBMS instance contacts CSS and gets passed the connect string for the ASM instance. The RDBMS then connects as sysdba to the ASM instance using OS authentication.
It’s actually the RDBMS’s ASMB background process that connects to the ASM instance and remains connected while there are ASM files open. Termination of this ASMB background process is fatal for the instance.
Summarizing, RDBMS finds CSS which passes connect string for the ASM Instance. On successfull connection, RDBMS connects as sysdba to ASM using OS authentication.
q4) is it via the listener ?
A. No
Role of listeners in StandAlone Grid Environment Only-
DB Listener : Listener under DB home : For Database Connections.
ASM Listener : Listener under Grid home : To listen for ASM Connections.. for example if you want to manage ASM from OEM.
Best practices in RAC and Restart environments is it to only have one listener (the one of GI Home). There is no need to have an additional listener defined in the database home. While you can separate them, there is really no additional value in doing so.
q5) do i have set any parameter to specify where the asm instance is in the database ?
A. No. ASM Instance and RDBMS interactions are facilitated by css daemon process and ASMB Background process ASMB.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
q1) I must emphasize that i have my oracle database installed before i install grid. Will the database instance still know how to access the asm via css daemon ? Do i have to register my database instance with oracle restart to do so ?
A. Yes, You need to check CSS before Installing ASM Instance. Please find the steps:
1. Check CSS must be running before any ASM Instance
Check CSS Running or Not?
cd $ORACLE_HOME/bin
crsctl check css
2. If its not running, you should configure CSS process by running $ORACLE_HOME/bin/localconfig script:
login as a root
$ORACLE_HOME/bin/localconfig add
Sample Output:
/etc/oracle does not exist. Creating it now.
Successfully accumulated necessary OCR keys.
Creating OCR keys for user 'root', privgrp 'root'..
Operation successful.
Configuration for local CSS has been initialized
Adding to inittab
Startup will be queued to init within 90 seconds.
Checking the status of new Oracle init process...
Expecting the CRS daemons to be up within 600 seconds.
CSS is active on these nodes.
RAC1
CSS is active on all nodes.
Oracle CSS service is installed and running under init(1M)
q2) how does the database instance know that it will need to access the ASM ? is it when the storage destination is specify with a "+xxx" e.g. +DATA ?
A. Yes, css daemon process registers ASM Instance and ASMB Background process makes the connections after successful authorization . Obviously, after the filesystem is migrated to ASM Storage (+xxx) Database and ASM Instance interacts.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
q1) With the steps you have provided, is it just for creation/starting of the css daemon ?
Will the oracle instance register with the css daemon as well or will the oracle instance detects the existence of a css daemon ?
A. For migration from Non-ASM to ASM, we would need css daemon to be up and running. css is a daemon process of Oracle.
q2) in the documentation for oracle grid for cluster, for job role separation, i do not see that grid user is assigned with the dba priviledge but in oracle restart for grid single installation it seems like dba is granted to grid user as well.
Can you confirm the permission for grid and oracle in oracle cluster rac environment and oracle grid single installation ?
Grid cluster
oracle - dba, oper, asmdba, asmadmin, asmoper, oinstall
grid - asmdba, asmadmin,asmoper oinstall -- need dba ?
Grid Single installation
oracle - dba, oper, asmdba, asmadmin, asmoper, oinstall
grid - asmdba, asmadmin,asmoper oinstall -- need dba ?
A. For both the above cases dba is granted to grid user.
As mentioned earlier, it is not mandatory; for example: user grid with only group ASMDBA is not able to restart oracle instance but just asm instance. Group dba can be granted to grid user if you want to use Oracle Restart feature.
As in RAC environment, Clusterware takes care of Oracle restart; but you can still assign dba privilege to be on safer side.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Scenario 1) install oracle database instance 1st, then install oracle grid for single instance database
1) grid user = dba, oinstall, asmadmin, asmdba
A -> ASM instance should be created and up and running as a pre-requisite before start of the migration of Non-ASM to ASM Database.
2) oracle user = dba, oinstall, asmdba --->
q1) does oracle need asmadmin for the oracle instance to access the asm instance ?
A-> asmadmin is not required if using different user for Grid (grid) and RDBMS Instances (oracle).
3) in order for oracle instance to access asm instance ->
q2) do we need to register the database instance with oracle restart or anything ? or just make sure that the CSS is running ?
A-> Yes, You just need to ensure CSS is configured and running.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ q4. ) will there be any issue for the db instance to connect to the asm instance if there are not sharing the same listener ?
A. No, as stated earlier:
Role of listeners in StandAlone Grid Environment Only-
DB Listener : Listener under DB home : For Database Connections.
ASM Listener : Listener under Grid home : To listen for ASM Connections.. for example if you want to manage ASM from OEM.
Best practices in RAC and Restart environments is it to only have one listener (the one of GI Home). There is no need to have an additional listener defined in the database home. While you can separate them, there is really no additional value in doing so.
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Configuring the listener using netca or manually from new cluster ORACLE_HOME or Grid Infrastructure home in case of 11gr2.
Stop the single instance listeners when they are running on one of the clustered nodes in case they are using the same listener ports.
If there is multiple database is running and you don’t want to stop all database, Create new LISTENER with different port.
Ideally, use the ‘LISTENER’ as name for that listener or you can change.
Setting up the listener Steps:
a. Create a new listener (ASM Instance)
b. Set local_listener parameter in asm and database instance (as listener isn’t on default port)
sql> alter system set LOCAL_LISTENER=LISTENER scope=both sid=’*';
System altered.
eg:
On ASM instance issue the following:
SQL> ALTER SYSTEM SET local_listener='(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=server1.domain.com)(PORT=1531))))' scope=both;
SYSTEM altered
On your database instance you would issue similarly:
SQL> ALTER SYSTEM SET local_listener='(DESCRIPTION=(ADDRESS_LIST=(ADDRESS=(PROTOCOL=TCP)(HOST=server1.domain.com)(PORT=1541))))' scope=both;
SYSTEM altered.
c. Resolve this local_listener in the tnsnames.ora
+++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
q5) how can we allow 1 listener to listen to both service ?
By default, the Oracle Database Configuration Assistant (DBCA) does not configure the TNS listener to accept client requests to an ASM instance.
Connecting to ASM through the existing TNS Listener.
1> Modify the listener.ora for the ASM Instances
LISTENER_LINUX1 =
(DESCRIPTION_LIST =
(DESCRIPTION =
(ADDRESS = (PROTOCOL = TCP)(HOST = linux1-vip)(PORT = 1521)(IP = FIRST))
(ADDRESS = (PROTOCOL = TCP)(HOST = 192.168.9.100)(PORT = 1521)(IP = FIRST))
)
)
SID_LIST_LISTENER_LINUX1 =
(SID_LIST =
(SID_DESC =
(SID_NAME = PLSExtProc)
(ORACLE_HOME = /u01/app/oracle/product/11.2.0/db_1)
(PROGRAM = extproc)
)
(SID_DESC =
(GLOBAL_DBNAME = +ASM)
(SID_NAME = +ASM1)
(ORACLE_HOME = /u02/app/oracle/product/11.2.0/db_1)
)
)
2> Also, Add ASM Entries to the tnsnames.ora File on the Client Machine
3> Bounce the TNS Listener and tnspings the services
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
q1) for disks to be used by ASM, do we still need to fdisk it ?
or we can assign the disk device as a whole (using udev with proper rights, user, group etc). Meaning do i assign e.g /dev/sda or fdisk and assign /dev/sda1 ?
A) Yes, To use a disk for ASM, you must create partition and stamp the disks as ASM disks. There are ASM libraries to stamp disks as ASM. These are distributed on your Linux kernel version.
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel. Building a new DOS disklabel. Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won’t be recoverable.
In order to use a disk (eg SAN) in ASM, the disk must have a partition table. Oracle recommends creating exactly one partition for each disk containing the entire disk.
You can use any physical disk for ASM, as long as it is partitioned. Use IDE, SCSI, or RAID devices.
Oracle recommends that you create a single whole-disk partition on each disk that you want to use.
Use either fdisk or parted to create a single whole-disk partition on the disk devices that you want to use.
q2) if i am using external redundancy, in the event that 1 of the asm disk has failed -> does that mean that data cannot be retrieved any more ?
or only affected tables with extents in the disk will fail ?
A) No data cannot be retreived in case asm disk corruption with no RAID configuration. When an ASM disk becomes unavailable, an external redundancy disk group would be simply dismounted.
Hence, OS Data will not be available; you will need to add another ASM disk and recover the data.
q3) When ASM do stripping, what does it actually strip ? If i create a datafile of 5mb, is it the extents that will be stripped across the disks ?
A) It is important for you to understand, ASM stripes at the database segment/extent level, for example SAN would stripe at the filesystem level.
q4) With the above scenario (external redundancy with 1 disk failed) but I have external RAID on and assuming that the lost data is rebuilt on the additional RAID disk
Will ASM be affected or it will be transparent to ASM ?
I am using udev (via UUID) to present disk devices to ASM. So i am not sure if I have external RAID on (e.g. RAID 1 or 5) and the lost data is rebuilt or on new disk
(do i need to present the new disk to ASM as well ?)
A) Raid is not redundancy for data. Yes, the terminology is a little different, ASM stripes at the database segment/extent level, RAID would stripe at the filesystem level, but the concept is the same.
ASM is done at the extent level of database segments instead of some OS stripe size.
If initially you have external redundancy, in case of failure of disk; it gets dismounted forcefully; So yes you need to present the new disk to ASM as well and recover the data.
q5) when creating db using dbca and specifying ASM as the storage, it prompted me to enter the password for ASMSNMP, why does dbca or the database need access to ASMSNMP ?
A) ASMSNMP is a less privileged user that is primarily used by Oracle Enterprise Manager to monitor Oracle ASM instances.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Q1) Are you able to elaborate further on why the need for partitioning/fdisk for the disk ?
I have tried using a raw disk (without partitioning) on ASM and it works
A) Yes, you can use raw disk. It is just the best practice as recommended by Oracle as there are Devices that does not contains disklabel.
So In order to use such disk in ASM, the disk must have a partition table as Building a new DOS disklabel.
You can create an ASM diskgroup using one of the following storage resources:
1) Raw disk partition—A raw partition can be the entire disk drive or a section of a disk drive. However, the ASM disk cannot be in a partition that includes the partition table because the partition table can be overwritten.
2) Logical unit numbers (LUNs)—Using hardware RAID functionality to create LUNs is a recommended approach. Storage hardware RAID 0+1 or RAID5, and other RAID configurations, can be provided to ASM as ASM disks.
3) Raw logical volumes (LVM)—LVMs are supported in less complicated configurations where an LVM is mapped to a LUN, or an LVM uses disks or raw partitions. LVM configurations are not recommended by Oracle because they create a duplication of functionality. Oracle also does not recommended using LVMs for mirroring because ASM already provides mirroring.
4) NFS NAS files—If you have a certified NAS device, then you can create zero-padded files in an NFS mounted directory and use those files as disk devices in an Oracle Automatic Storage Management (Oracle ASM) diskgroup.
Q2) Assuming if i have 3 disks in a diskgroup (with external redundancy and no raid configured);
if 1 of the disks failed, does that mean my whole diskgroup is unusable and no data can be retrieved at all ?
A) With no RAID configured, when 1 disk fails, the diskgroup will get dismounted forcefully. And, data (as OS filesystem files) will not be available which will require recovery.
Q3) You mentioned that ASM stripped at Segment/Extent level.
Multiple continuous Oracle blocks = 1 extent.
Multiple extents = 1 Segment
If I have 3 disk, do you mean that for 1 segment/table with 3 extents, extent#1 will be in disk1, extent#2 on disk2 and extent#3 on disk3 ?
How does Oracle assign the extent for usage ? (round robin) ?
A) To stripe data, Oracle ASM separates files into stripes and spreads data evenly across all of the disks in a disk group. It has two methods of striping, Fine & Coarse.
Fine Striping:- Fine striping writes 128 KB data to each ASM Disk in the diskgroup in a round robin fashion, 128 KB goes to the first disk, then the next 128 KB, goes to the next disk, etc. According to manual, The fine-grained stripe size always equals 128 KB in any configuration; this provides lower I/O latency for small I/O operations.” Small I/O operations sure sounds like a good candidate for redo logs, control files etc.
Coarse Striping:-With coarse grained striping ASM writes data to each disk in the same round robin fashion, but writes chunks in the size of the ASM instance’s allocation unit (AU) size, default is 1MB.
Further, starting from 11gR2, Online redo log files are no more in Fine striping contradictory to 10g, as Oracle silently changed the template to Coarse striping.
A note has been written on the same:- Redo Log Striping In 11.2 ASM, is Coarse Or Fine? [ID 1269158.1]
Q3a) In terms of Oracle blocks in ASM setup, is it still mapped to OS size block or directly to harddisk sectors ?
e.g. 1 oracle block = x hdd sectors ?
or 1 oracle block = x OS blocks
A) In simple expainations mapping happens at block level: When setting up ASM, Allocation Unit (AU) is allocated (takes the default value unless specified) And, During RDBMS setup, Oracle Block is setup.
Oracle Data block is the way through which Oracle maintains your datafiles. The ASM AU is the way through which the blocks are allocated from the shared pool when the file modifications request come from the db.
Please understand, ASM is Oracle's recommended storage management solution (Volume Manager) that provides an alternative to conventional volume managers, file systems, and raw devices.
ASM uses disk groups to store datafiles; an ASM disk group is a collection of disks that ASM manages as a unit. Within a disk group, ASM exposes a file system interface for Oracle database files. The content of files that are stored in a disk group are evenly distributed, or striped, to eliminate hot spots and to provide uniform performance across the disks. The performance is comparable to the performance of raw devices.
Every Oracle ASM disk is divided into allocation units (AU). An allocation unit is the fundamental unit of allocation within a disk group. A file extent consists of one or more allocation units. An Oracle ASM file consists of one or more file extents.
About Allocation Unit, AU:
An allocation unit is the minimum segment of disk that can be assigned to a file. Each extent is an integral number of AU. Each file contain integral number of extents
The default AU size is set to 1MB for ASM . There are ways to set the AU using FINE grained striping(128K) and COARSE grained stripe is equal to AU size selected
Q4) How is raid not redundancy for data. Can you elaborate further
if i am present a LUN/device (which is make up underlying disks with RAID5 configured) to ASM and adds it to diskgroup +DATA.
The diskgroup has only this LUN/device. if an underlying disk in the RAID failed , the LUN is still represented as usual. Will ASM be affected at all ?
A) It applies to all RAID disks. You can use any RAID disks - create an ASM diskgroup for those disks and tell ASM not to mirror it, as that diskgroup resides on externally redundant disks.
ASM does not know what RAID levels a disk is. It only knows that it is a disk that it can use. The HOW TO USE, you need to explicitly tell ASM.
You need to tell ASM whether you want to use that disk as part of externally redundant diskgroup as below:
To create an Oracle ASM disk group where redundancy is provided by the storage array, first create the RAID-protected logical unit numbers (LUNs) in the storage array, and then create the Oracle ASM disk group using the EXTERNAL REDUNDANCY clause:
CREATE DISKGROUP DATA EXTERNAL REDUNDANCY DISK
'/devices/lun1','/devices/lun2','/devices/lun3','/devices/lun4';
In the event of a disk failure the RAID controller switches to one of the mirrors to prevent system failure, while in ASM alert log you will see notifications.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
No comments:
Post a Comment