Gmail - raw mapping, ‘https://mail.google.com/mail/u/0/?ui=2&ik-9230c51212&jsver“CS8IN. M1 Gmail raw mapping 1 message ankit modi Sat, Jun 4, 2011 at 4:04 PM. To: ankitmodimea , Ankit Modi Configuring raw devices (multipath) for Oracle Clusterware 10g Release 2 (10.2.0) on RHELS/OELS [ID 564580.1] Modified 24-JAN-2010 Type HOWTO — Status PUBLISHED In this Document Goal Solution Deprecation of Support for Raw Devices A Bit About Udev and Device Name Persistency Muttipath, Raw and Udev Configuring raw devices (multipath) for Oracle Clusterware 10g Release 2 (10.2.0) on RHELS/OELS: ‘Assumptions 1. Configure SCSI_ID to Return Unique Device Identifiers 1a, Whitelist SCSI devices 1b. List all SCSI (Clusterware) devices 1c. Obtain Clusterware device unique SCSI identifiers 2. Configure Multipath for Persistent Naming of Clusterware Devices 2a. Configure Mutipathing 2b. Verify Multipath Devices 3. Create Raw Devices 4. Test Raw Device Accessibilty 5. Script Creation of Raw Bindings and Permissions 6. Test the Raw Device Script 7. Install Oracle 10gR2 Clusterware References. Applies to: Linux OS - Version: 5.0 to 5.0 Linux x86 Linux x86-64 Linux Itanium, Linux Kernel - Version: 5.0 to 5.0 Goal This article is intended for Oracle on Linux Database and System Administrators, particularly those intending to install (or migrate to) Oracle Real Application Clusters 10g Release 2 (10.2.0) on Red Hat/Oracle Enterprise Linux 5 (ELS). The article is intended to focus (on the configuration of raw devices against multipathed devices on ELS in preparation for RAC Clusterware usage, rather than on multipathing or installation of the Clusterware, Examples were taken from a working system of the following configuration: * Enterprise Linux 5 (GA) - 2.6.18-8.e15 * Oracle Clusterware 10g Release 2 (10.2.0) * Shared storage for Clusterware files served via iSCSI 1 of 1S 11/24/2017 6:09 PMGmail - raw mapping, dof 1S Note: this document differs to Note.465001.1 that describes configuration of raw devices against single pathed devices. This Note describes configuration of raw devices against muttipathed devices. Solution Deprecation of Support for Raw Devices In versions prior to ELS, applications such as Oracle, could access unstructured data on block devices by binding to them via character raw devices, such as /deviraw/raw1 using the raw(8) command. Persistent device assignments could be configured using the Jetc/sysconfigitawdevices file in conjunction with the rawdevicos service. ‘Support for raw devices was initially deprecated in the Linux 2.6 kernel (ELS < UA) in favour of directio (O_DIRECT) access, however was later undeprecated from ELS Us (initscripts-8.45.30-2). For details of the deprecation and undeprecation of support for rawio, refer to Linux kernel/version documentation including * jusr/sharefdoc/kernel-doc-2,6. 18/Documentation/feature-removal-schedule. bt * Red Hat Enterprise Linux 4/5 Release notes Both the /etc/sysconfigirawdevices file (EL4) and Jetc/udevirules.d/60-raw.rules file (ELS) similarly discuss deprecation of raw. ‘OCFS2, Oracle's Cluster Filesystem version 2 (htto://oss.oracle.com/projects/ocfs2), is an extent based, POSIX-compliant file system that provides for shared, O_DIRECT file access. For certified ports and distributions, Oracle extends free support of OCFS2 users with an Oracle database license for use in storing Oracle datafiles, redologs, archivelogs, control files, voting disk (CRS), cluster registry (OCR), etc. along with shared Oracle home. Bit About Udev and Device Name Persistency Unlike devlabel in the 2.4 kernel, udev (the 2.6 kernel device file naming scheme) dynamically creates device file names at boot time. This can, however, give rise to the possibilty that device file names may change - a device that may once have been named /devisdd say, may be named /devisdf, say, after reboot. Without specific configuration, if udev is left to dynamically name devices, the potential exists for devices referred to, or inadvertently accessed by, their arbitrary kernel-assigned name (e.g. Oracle Clusterware files; Cluster Registry, Voting disks, etc.) to become corrupt. Multipath, Raw and Udev ‘The necessity for high availabilty access to storage is well understood. For singlepath environments, raw devices can easily be configured via udev rules as described in Note.465001.1. For multipath environments, however, configuration of raw devices against multipathed devices via udev is more complex. In fact, significant modification of defauit udev rules can introduce issues with supportabilty. Therefore, other means are recommended to achieve configuration of raw devices against muttipathed devices with multipath device naming persistency. Configuring raw devices (mutipath) for Oracle Clusterware 10g Release ‘https://mail.google.com/mail/u/0/?ui=2&ik-9230c51212&jsver“CS8IN. 11/24/2017 6:09 PMGmail - raw mapping, 3 of 15 2 (10.2.0) on RHELS/OELS ‘The following procedure outlines the steps necessary to configure persistent multipath device naming and creation of raw devices (including permissions) in preparation for Oracle 10gR2 (10.2.0) Clusterware devices. From Oracie11g Release 1 (11.1.0), Clusterware files may be placed on either block or raw devices located on shared disk partitions, therefore the following procedure only strictly applies when using Oracle 10gR2 (10.2.0) and mutipatting, ‘Therefore, take this opportunity to consider whether you wish to proceed using 10gR2 or 11gR1 Clusterware to manage your 10gR2 databases - multipath device configuration for Oracle 11g Clusterware is described in Note 605828.1. The following procedure may also be used as a basis for configuring raw devices on EL4 (Update 2 or higher). Unless otherwise stated, all steps should be performed on ‘each cluster node and as a privileged user. ‘Assumptions. ‘The following procedure assumes the following to have occured: * Clusterware devices have been created on supported shared storage * Clusterware devices have been appropriately sized according to Oracle10g Release 2 (10.2.0) RAC documentation * Clusterware devices have been partitioned * All cluster nodes have multipath access to shared devices * Cluster nodes are configured to satisfy Oracle Universal Installer (OUI) requirements 1. Configure SCSI_ID to Return Unique Device Identifiers 1a, Whitelist SCSI devices Before being able to configure udev to explicitly name devices, ‘scsi_id(8) should first be configured to return SCSI device identifiers. Modify the /etciscsi_id.config file - add or replace the b parameterivalue pair (if exists) with option=-g, for 1b. List all SCSI (Clusterware) devices Clusterware devices must be visible and accessible to all cluster nodes. Typically, cluster node operating systems need to be updated in order to see newly provisioned (or modified) devices on shared storage i.e, use '/sbin/partprobe ' or ‘Isbin/sfdisk -r ’, etc., or simply reboot. Resolve any issues preventing cluster nodes from correctly seeing or accessing Clusterware devices before proceeding. Run the fdisk(8) and/or ‘cat /procipartitions' commands to ensure Clusterware devices are visible, for example: # cat /procipartitions major minor #blocks name 8 0 6291456 sda 8 1 5735173 sdat 8 2 664242 sda2 ‘https://mail.google.com/mail/u/0/?ui=2&ik-9230c51212&jsver“CS8IN. 11/24/2017 6:09 PMGmail - raw mapping, 4 of 15 16 987966 sdb 17 987681 sdb1 32 987966 sdc 33. 987681 sdct 48 987966 sid 49 987681 sdd1 64 987966 sde 65 987681 sde1 80 987966 sdf 81 987681 sdit 96 987966 sig 97 987681 sdg1 112 1004031 sdh 113 1003873 sdht 128 1004031 sdi 129 1003873 sdit 144 1004031 sdj 145 1003873 sdjt 160 1004031 sdk. 161 1003873 sdk1 176 1004031 sdi 177 1003873 sdit 192 1004031 sdm 193 1003873 sdmt ‘Above, though perhaps not entirely evident, the kernel has assigned two device files per multipathed device i.e. devices /devisdb and Idevisde both refer to the same device/LUN on shared storage, as do Idevisdd and /devisde and so on. Note, at this point, each cluster node may refer to the would-be Clusterware devices by different device file names - this is expected ‘1c, Obtain Clusterware device unique SCSI identifiers Run the scsi_id(8) command against Clusterware devices from one Cluster node to obtain their unique device identifiers. When running the scsi_id(8) command with the -s argument, the device path and name passed should be that relative to sysfs directory /sys! ie. Iblock/ when referring to /sys/biock/. Record the unique SCSI identifiers of Clusterware devices - these are required later (Step 2a.), for example: # fori in “cat /proc/partitions | awk {‘print $4'} |grep sd’; do echo “HHH Si: “sosi_id -9 -u-s /block/$i"; done ‘iit sdb: 1494564000000000000000000010000005¢3900000d000000 tit sdb iit sde: 1494564000000000000000000010000005¢3900000d000000 tit sd ### edd: 149455400000000000000000001000000843900000d000000 HH odd ### sde: 149455400000000000000000001000000843900000d000000 HHH sdet ### sdf: 149455400000000000000000001000000ae3900000d000000 HH sd HH sdg: 149455400000000000000000001000000ae3900000d000000 HH sda} ### sdh; 149455400000000000000000001000000403900000d000000 HH sdht ‘https://mail.google.com/mail/u/0/?ui=2&ik-9230c51212&jsver“CS8IN. 11/24/2017 6:09 PM