Ask the Expert - Mr. Database
by Eric Gross, DBA
This Month's Question:

Dear Mr. Database:

I'm having difficulty getting the best storage options for RAC - any suggestions?

- Jonathan Palmer,
Salt Lake City, UT

Of course there’s difficulty with this - configuring the storage requirements for RAC databases properly is an intricate task because there are so many moving parts. Once you step outside of the local disks on servers, there are two major choices for RAC, both of which add significant complexity to the environment as a whole - Storage Area Networks (SAN) and Network Attached Storage (NAS).

A SAN is an architecture where hosts and storage communicate freely between the various devices over a shared network architecture, which can be Fibre Channel or Ethernet. The SAN storage device is “carved” up into discrete disks called LUNs (for Logical Unit Number), and each LUN is made available to one or more hosts. While a SAN can be configured to allow multiple hosts to access the same LUNs simultaneously, traditional filesystems (like UFS, ext3, etc.), do not support shared access to the same data. This makes it totally unsuitable for RAC, and the other option for RAC - raw devices - is complex enough to be deprecated by Oracle in favor of one of the options described below.

Clustered Filesystems (CFS) work like traditional filesystems, except that they allow multiple nodes to access the same filesystem concurrently; they allow each LUN to be used in a distributed fashion where many nodes can use the same data store while maintaining data integrity. Oracle’s preferred CFS is the Oracle Cluster File System Version 2 (OCFS2) which is a general-purpose file system – that is, it provides all of the functionality of a traditional filesystem, unlike some clustered filesystems where they are highly specialized, or will not work with all applications. There have been significant improvements over the original OCFS, so if you haven’t evaluated this product recently, it might be worth another check. You can find more information about it at Oracle’s open source website at http://oss.oracle.com.

Automatic Storage Management (ASM) is another storage option used to improve manageability over raw devices. ASM is similar to a volume manager, like Veritas, except that it is volume management at a database level, not on a filesystem layer. With ASM, you create a diskgroup on top of a number of disks, and it presents itself as a filesystem to the database only. From the operating system perspective, the OS just sees a number of disks, but the database sees a filesystem. Since ASM is at the database level, it has certain limitations versus OS-level filesystems. For starters, ASM can only be used to store certain types of files, which means it will never exist as the only shared storage for a cluster. In order to handle all the directories and files that need to be shared between all nodes of a RAC cluster, ASM can be combined with other storage technologies.

If ASM cannot handle all the requirements for RAC, then why use it you ask? Because using ASM offers some useful features for your databases such as automatic balancing of data between various LUNs. With ASM, diskgroups are logical constructs created by grouping together one or more LUNs. Each ASM diskgroup can be used by an unlimited number of databases within a single cluster – a single diskgroup cannot be mounted by more then one cluster at the same time. A downside of ASM is the opaque nature of the contents – since the ASM diskgroup just looks like raw disk, there’s no visible filesystem. Storage personnel are familiar with the standard tools available to determine the usage of the space, but when ASM is in use, all that can be determined without logging into Oracle is that the space is allocated.

NFS is quite a different beast at every level. While specialized hardware is required to connect to a SAN, NAS connectivity is enabled via the standard network that is already in place on each host. The beauty of NAS is that there’s no need for clustered filesystems, ASM, raw devices, or anything else – the storage array handles all of those concerns, leaving the host free to be a database server. Administration is also almost unrelated to management of SAN storage - hosts “mount” file systems on the storage device. The only sticky bit about NAS storage is that there’s a number of vendors, but only certain vendors are supported and certified by Oracle. The most common NAS (and most widely supported vendor) is NetApp, which is supported by Oracle on all platforms they support. At its simplest, a single mount, or filesystem, can be created on the storage device to store all binaries and data for a cluster.

Determining which of the above options should be used, and how to mix-and-match to meet all requirements is our job as database administrators. Beyond that, the implementation of the storage devices and requisite networking is the role of the storage professional. Extensive communication between these two groups is essential to pulling off a smooth implementation.

But there’s a great “out-of-the-box” solution: GridApp’s D2500 Database Appliance. The D2500 limits the storage management that must be performed to an absolute minimum – the configuration of the storage platform itself. The appliance comes preconfigured for all storage options and takes care of all the requirements inside and outside the database - there’s no manual intervention or expertise necessary. All the storage professional needs to do is properly expose the storage that is to be used in the cluster and let you, the database administrator, know the required details. When provisioning using the appliance, inputs exist for the storage properties to be defined. And that’s all there is to it!

GridApp's monthly newsletters are Op/Eds about our data-centric world. We welcome your questions, insights and opinions, so please email to mrdatabase@gridapp.com.

<< back