DOC HOME SITE MAP MAN PAGES GNU INFO SEARCH PRINT BOOK
 
[Next] [Previous] [Top] [Contents] [Index]

VxVM System Administrator's Guide

Volume Administration

Chapter 4


Introduction

This chapter provides information on how to maintain a system configuration under VxVM control. It includes information about how to create, remove, and maintain VxVM objects (volumes, plexes, and subdisks). Information is presented in the form of both overviews of relevant utilities and more detailed descriptions on how to perform particular operations using the utilities. Online backup information is also included.


Note: Most Volume Manager commands require appropriate privileges.


The following topics are covered in this chapter:

VxVM Operations

Once the software has been installed and properly initialized, VxVM can be used to perform system and configuration management operations on its objects: disks, disk groups, subdisks, plexes, and volumes. Volumes are composed of two types of objects -- plexes and subdisks. A plex is composed of a series of subdisks linked together in an address space. A subdisk is a portion of a physical disk and is defined by disk media, offset, and length.

Disks and disk groups must be initialized and defined to the Volume Manager before volumes can be created. Once disks and disk groups are defined, volumes can be created in either of the following ways:

This chapter contains detailed descriptions of Volume Manager operations, along with examples that illustrate the use of VxVM utilities. Many of the concepts and procedures in this chapter are normally "hidden" from the casual user or are performed automatically through use of the higher level commands such as vxassist. Some of the commands and procedures documented in this chapter do not normally need to be performed by system administrators.

VxVM Utility Descriptions

The following sections describe VxVM utilities that are most commonly used to perform system administration and maintenance functions. Detailed information for each of these utilities can be found in their respective manual pages. Utility-specific examples are included later in the chapter.

Using vxassist

The vxassist command provides a convenient approach to volume management. vxassist acts as an automated one-step interface to Volume Manager operations. Unlike most other VxVM commands, vxassist does not require a thorough understanding of Volume Manager concepts. vxassist is capable of accomplishing alone many tasks that would otherwise require the use of a sequence of several other VxVM utilities. vxassist automatically takes care of all underlying and related operations that would otherwise need to be performed manually by the user (in the form of other commands).

vxassist does not conflict with the existing VxVM utilities or preclude their use. Objects created by vxassist are compatible and inter-operable with objects created manually using the other VxVM utilities and interfaces, and vice versa.

vxassist does the following:

For detailed information about how to use vxassist, refer to the vxassist(1M) manual page.

vxassist Command Features

The vxassist command provides a convenient, one-step interface to the Volume Manager and is especially useful for basic and commonly used administrative operations.

In general, it is more convenient to use vxassist than a series of other VxVM commands. Some of the advantages of using vxassist include:

How vxassist Works

The vxassist utility allows users to create and modify simple volumes. The user specifies the basic volume creation or modification requirements and vxassist proceeds to perform all of the necessary underlying tasks.

The amount of information that vxassist requires from the user is minimal because vxassist obtains most of the information it needs from other sources. vxassist obtains information about the existing objects and their layouts from the objects themselves. For operations requiring new disk space, vxassist seeks out available disk space and tries to allocate it in the configuration that conforms to the layout specifications and offers the best use of free space.

The vxassist command typically takes the following form:

vxassist keyword volume_name [attributes...]
where keyword selects the specific operation to perform. The first argument after any vxassist keyword is always a volume name. The volume name is followed by a set of attributes. For details on available vxassist keywords and attributes, refer to the vxassist(1M) manual page.

vxassist creates and manipulates volumes based on a set of established defaults, but also allows the user to supply preferences for each operation.

vxassist Defaults

The vxassist invocation is designed to be as simple as possible, while allowing its behavior to be tailored when necessary. vxassist uses a set of tunable parameters, which can be specified in defaults files or at the command line. The tunable parameters default to reasonable values if they are not mentioned anywhere. Any tunables listed on the command line override those specified elsewhere. The tunable parameters are specified as follows:

Defaults File

The default behavior of vxassist is controlled by the tunables specified in the /etc/default/vxassist file. The format of the defaults file is a list of attribute=value pairs separated by new lines. These attribute=value pairs are the same as those specified as options on the vxassist command line (refer to the vxassist(1M) manual page for details).

The following is a sample vxassist defaults file:

# by default:
#  create unmirrored, unstriped volumes
#  allow allocations to span drives
#  with RAID-5 create a log, with mirroring don't create a log
#  align allocations on cylinder boundaries
	layout=nomirror,nostripe,span,nocontig,raid5log,noregionlog,
	diskalign

#  use the fsgen usage type, except when creating RAID-5 volumes
	usetype=fsgen

#  allow only root access to a volume
	mode=u=rw,g=,o=
	user=root
	group=root

#  when mirroring, create two mirrors
	nmirror=2

#  for regular striping, by default create between 2 and 8 stripe
#  columns
	max_nstripe=8
	min_nstripe=2

#  for RAID-5, by default create between 3 and 8 stripe columns
	max_nraid5stripe=8
	min_nraid5stripe=3

#  create 1 log copy for both mirroring and RAID-5 volumes, by
	default
	nregionlog=1
	nraid5log=1

#  by default, limit mirroring log lengths to 32Kbytes
	max_regionloglen=32k

#  use 64K as the default stripe unit size for regular volumes
	stripe_stwid=64k

#  use 16K as the default stripe unit size for RAID-5 volumes
	raid5_stwid=16k

Manipulating the Volume Configuration Daemon With vxdctl

The volume configuration daemon (vxconfigd) is the interface between the VxVM utilities and the kernel vxconfig device driver. The vxconfig device is a special device file created by the Volume Manager that interacts with vxdctl to make system configuration changes.

Some vxdctl operations involve modifications to the volboot file, which indicates the locations of some root configuration copies.

The vxdctl utility is the interface to vxconfigd and is used for:

For detailed information about how to use vxdctl, refer to the vxdctl(1M) manual page.

Removing and Modifying Entities With vxedit

The vxedit utility has two functions:

In general, VxVM objects that are associated with other objects are not removable by vxedit. This means that vxedit cannot remove:

For detailed information about how to use vxedit, refer to the vxedit(1M) manual page.

Creating VxVM Objects With vxmake

The vxmake utility is used to add a new volume, plex, or subdisk to the set of objects managed by VxVM. vxmake adds a new record for that object to the VxVM database. Records can be created entirely from parameters specified on the command line, or they can be created using a description file.

If operands are specified on the command line, then the first operand is a keyword that determines the kind of object to be created, the second operand is the name given to that object, and additional operands specify attributes for the object. If no operands are specified on the command line, then a description file is used to specify what records to create.

A description file is a file that contains plain text which describes the objects to be created with vxmake. A description file can contain several commands, and can be edited to perform a list of operations. The description file is read from standard input, unless the -d description_file option specifies a filename. The following is a sample description file:

#rectyp #name       #options
sd      disk3-01    disk=disk3 offset=0 len=10000
sd      disk3-02    disk=disk3 offset=25000 len=10480
sd      disk4-01    disk=disk4 offset=0 len=8000
sd      disk4-02    disk=disk4 offset=15000 len=8000
sd      disk4-03    disk=disk4 offset=30000 len=4480
plex    db-01       layout=STRIPE ncolumn=2 stwidth=16k
                    sd=disk3-01:0/0,disk3-02:0/10000,disk4-01:1/0,\ 
							disk4-02:1/8000, disk4-03:1/16000
sd      ramd1-01    disk=ramd1 len=640
                    comment="Hot spot for dbvol
plex    db-02       sd=ramd1-01:40320
vol     db          usetype=gen plex=db-01,db-02
                    readpol=prefer prefname=db-02
                    comment="Uses mem1 for hot spot in last 5m 

This description file specifies a volume with two plexes. The first plex has five subdisks on physical disks. The second plex is preferred and has one subdisk on a volatile memory disk.

For detailed information about how to use vxmake, refer to the vxmake(1M) manual page.

Correcting Plex Problems With vxmend

The vxmend utility performs miscellaneous VxVM usage-type-specific operations on volumes, plexes, and subdisks. These operations are used to fix simple problems in configuration records (such as clearing utility fields, changing volume or plex states, and offlining or onlining volumes or plexes).

vxmend is used primarily to escape from a state that was accidentally reached. The offline and online functions can be performed with disk-related commands.

For detailed information about how to use vxmend, refer to the vxmend(1M) manual page.

Performing Plex Operations With vxplex

The vxplex utility performs VxVM operations on plex or on volume-and-plex combinations. The first operand is a keyword that determines the specific operation to perform. The remaining operands specify the configuration objects to which the operation is to be applied.

The vxplex utility can be used to do the following:

For detailed information about how to use vxplex, refer to the vxplex(1M) manual page.

Printing Configuration Information With vxprint

The vxprint utility provides a flexible method of displaying information from records in a VxVM configuration database. This command can be used to display partial or complete information about any or all objects. The format can be hierarchical to clarify relationships between VxVM objects. vxprint can tailor its output for use by UNIX system utilities such as awk, sed, or grep.

For detailed information about how to use vxprint, refer to the vxprint(1M) manual page.

Performing Subdisk Operations With vxsd

The vxsd utility is used to maintain subdisk-mirror associations. vxsd can associate a subdisk with a mirror or dissociate a subdisk from its associated mirror, to move the contents of a subdisk to another subdisk, to split one subdisk into two subdisks that occupy the same space as the original, or to join two contiguous subdisks into one.


Note: Users should be aware that some vxsd operations can take a considerable amount of time to complete.


For detailed information about how to use vxsd, refer to the vxsd(1M) manual page.

Printing Volume Statistics With vxstat

The vxstat utility prints statistics information about VxVM objects and block devices under VxVM control. vxstat reads the summary statistics for Volume Manager objects and formats them to the standard output. These statistics represent VxVM activity from the time the system was initially booted or from the last time statistics were cleared. If no VxVM object name is specified, then statistics from all volumes in the configuration database are reported.

For detailed information about how to use vxstat, refer to the vxstat(1M) manual page and Chapter 2, "VxVM Performance Monitoring."

Tracing Volume Operations With vxtrace

The vxtrace utility prints kernel I/O error or I/O error trace event records on the standard output or writes them to a file in binary format. Binary trace records written to a file can be read back and formatted by vxtrace as well.

If no operands are given, then either all error trace data or all I/O trace data on all virtual disk devices are reported. With error trace data, it is possible to select all accumulated error trace data, to wait for new error trace data, or both (the default). Selection can be limited to a specific disk group, to specific VxVM kernel I/O object types, or to particular named objects or devices.

For detailed information about how to use vxtrace, refer to the vxtrace(1M) manual page and Chapter 2, "VxVM Performance Monitoring."

Performing Volume Operations With vxvol

The volume operations that the vxvol utility performs are:

Starting a volume changes its kernel state from DISABLED or DETACHED to ENABLED.

Stopping a volume changes its state from ENABLED or DETACHED to DISABLED (however, it is seldom useful to stop a volume).

One of these read policies can be selected:

For detailed information about how to use vxvol, refer to the vxvol(1M) manual page.

Subdisk Operations

Subdisks are low-level building blocks of a VxVM configuration. The following sections describe each of the operations that can be performed in relation to subdisks. These subdisk operations are:

Creating Subdisks

The command to create VxVM objects is vxmake. The steps to create a subdisk include specifying the:

To create a subdisk, use the following command:

vxmake sd name disk,offset,len 
For example, the command to create a subdisk labeled disk02-01 that starts at the beginning of disk disk02 and has a length of 8000 sectors looks like this:

vxmake sd disk02-01 disk02,0,8000 
Commands take sizes in sectors. Adding a suffix (such as k, m, or g) changes the unit of measure.


Note: To preserve (encapsulate) data that exists on the disk, a plex and volume must be created to cover that data.


Removing Subdisks

Subdisks are generally removed when making changes to the system configuration. To remove a subdisk, use the following command:

vxedit rm subdisk_name 
For example, the command line to remove a subdisk labeled disk02-01 looks like this:

vxedit rm disk02-01

Displaying Subdisk Information

The vxprint utility displays information about VxVM objects. To display general information for all subdisks, use the following command:

vxprint -st
The -s option specifies information about subdisks. The -t option prints a single-line output record that depends on the type of object being listed.

To display complete information about a particular subdisk, use the following command:

vxprint -l subdisk_name
For example, the command to obtain all information on a subdisk labeled disk02-01 looks like this:

vxprint -l disk02-01
This command provides the following information:

Disk group: rootdg

Subdisk:  disk02-01
info:     disk=disk02 offset=0 len=205632
assoc:    vol=mvol plex=mvol-02 (offset=0)
flags:    enabled
device:   device=c2b0t0d1s7 path=/dev/dsk/c2b0t0d1s4 \
  				diskdev=32/68

Associating Subdisks

Associating a subdisk with a plex places the amount of disk space defined by the subdisk at a specific offset within the plex. In all cases, the entire area that the subdisk fills must not be occupied by any portion of another subdisk. There are several different ways that subdisks can be associated with plexes, depending on the overall state of the configuration.

If the system administrator has already created all the subdisks needed for a particular plex, subdisks are associated at plex creation by using the command:

vxmake plex plex_name sd=subdisk_name,... 
For example, the following command creates the plex home-1 and associates subdisks disk02-01, disk02-00, and disk02-02 with the plex home-1 during the plex creation process:

vxmake plex home-1 sd=disk02-01,disk02-00,disk02-02 
Subdisks are associated in order starting at offset 0. Using a command like this one eliminates the need to specify the multiple commands necessary to create the plex and then associate each of the subdisks with that plex. In this example, the subdisks are associated to the plex in the order they are listed (after the sd=); the disk space defined as disk02-01 will be first, the disk space of disk02-00 is second, and disk02-02 is third.

This method of associating subdisks is convenient during initial configuration. Subdisks can also be associated with a plex that already exists. One or more subdisks can be associated with an existing plex as follows:

vxsd assoc plex_name sd_name [sd_name2 sd_name3 ...]
For example, the command line to associate subdisks labeled disk02-01, disk02-00, and disk02-02 with a plex labeled home-1 looks like this:

vxsd assoc home-1 disk02-01 disk02-00 disk02-01 
If the plex is not empty, the new subdisks are added after any subdisks that are already associated with the plex, unless the -l option is specified with the command. The -l option provides a way to associate subdisks at a specific offset within the plex.

The -l option is needed in a case where a system administrator has created a sparse plex for a particular volume, and wishes to make this plex complete. To make the plex complete, it is necessary to create a subdisk of exactly the size needed to fill the hole in the sparse plex, and then associate the subdisk with the plex by specifying the offset of the beginning of the hole in the plex. Use the following command to accomplish this task:

vxsd -l offset assoc sparse_plex_name exact_size_subdisk 

Note: The subdisk must be exactly the right size because VxVM does not allow for the space defined by two subdisks to overlap within a single plex.


For striped subdisks, a column number and column offset for the subdisk can be specified:

vxsd -l column_#/offset assoc plex_name sd_name ...
If only one number is specified with the -l option for striped plexes, the number is interpreted as a column number and the subdisk is associated at the end of the column.

Associating Log Subdisks

Log subdisks are subdisks that are defined for and added to a plex that is to become part of a volume using Dirty Region Logging. Dirty Region Logging is enabled for a volume when the volume is mirrored and has at least one log subdisk.

For a description of Dirty Region Logging, refer to "Dirty Region Logging" in "Chapter 1" and "Dirty Region Logging Guidelines" in Chapter 2. Log subdisks are ignored as far as the usual plex policies are concerned, and are only used to hold the dirty region log.


Note: Only one log subdisk can be associated with a plex. Because this log subdisk is frequently written, care should be taken to position it on a disk that is not heavily used. Placing a log subdisk on a heavily-used disk can result in degradation of system performance.


To add a log subdisk to an existing plex, use the following command:

vxsd aslog plex  subdisk
where subdisk is the name of the subdisk to be used as a log subdisk. The plex must be associated with a mirrored volume before DRL will take effect.

For example, the command line to associate a subdisk labeled disk02-01 with a plex labeled vol01-02 (which is already associated with volume vol01) looks like this:

vxsd aslog vol01-02 disk02-01

Note: You can also use the following command to add a log subdisk to an existing volume:
vxassist addlog volume_name disk
This command automatically creates a log subdisk within a log plex for the specified volume.


Dissociating Subdisks

To break an established relationship between a subdisk and the plex to which it belongs, the subdisk is dissociated from the plex. A subdisk is dissociated when the subdisk is to be removed or used in another plex. To dissociate a subdisk, use the following command:

vxsd dis subdisk_name 
To dissociate a subdisk labeled disk02-01 from the plex with which it is currently associated, use the following command:

vxsd dis disk02-01

Note: Subdisks can also be removed with the command:
vxsd -orm dis subdisk_name


Changing Subdisk Information

The vxedit utility changes information related to subdisks. To change information relating to a subdisk, use the following command:

vxedit set field=value ... subdisk_name   
For example, the command line to change the comment field of a subdisk labeled disk02-01 looks like this:

vxedit set comment= "new comment" disk02-01
The subdisk fields that can be changed using vxedit are:

Moving Subdisks

Moving a subdisk copies the disk space contents of a subdisk onto another subdisk. If the subdisk being moved is associated with a plex, then the data stored on the original subdisk is copied to the new subdisk, the old subdisk is dissociated from the plex, and the new subdisk is associated with the plex, at the same offset within the plex as the source subdisk. To move a subdisk, use the following command:

vxsd mv old_subdisk_name new_subdisk_name 
For the subdisk move operation to perform correctly, the following conditions must be met:

Splitting Subdisks

Splitting a subdisk divides an existing subdisk into two subdisks. To split a subdisk, use the command:

vxsd -s size split subdisk_name newsd1 newsd2
where subdisk_name is the name of the original subdisk, newsd1 is the name of the first of the two subdisks that will be created, and newsd2 is the name of the second subdisk to be created. The -s option is required to specify the size of the first of the two subdisks that will be created. The second subdisk will take up the remaining space used by the original subdisk.

If the original subdisk is associated with a plex before the operation, upon completion of the split, both of the resulting subdisks will be associated with the same plex.

To split the original subdisk into more than two subdisks, repeat the previous command as many times as necessary on the resulting subdisks.

Joining Subdisks

Joining a subdisk combines two or more existing subdisks into one subdisk. To join subdisks, the subdisks must be contiguous on the same disk; if the selected subdisks are associated, they must be associated with the same plex, and be contiguous in that plex. Join a subdisk with the command:

vxsd join subdisk1 subdisk2 new_subdisk 

Plex Operations

Plexes are logical groupings of subdisks that create an area of disk space independent of any physical disk size. Replication (mirroring) of disk data can be accomplished by creating multiple plexes for a single volume, where each plex contains an identical copy of the volume's data. Since each plex must reside on different disks, the replication provided by mirroring prevents data loss in the event of a single-point disk-subsystem failure. Multiple plexes also provide increased data integrity and reliability.

Plex operations include:

Creating Plexes

Plexes are created by identifying subdisks and associating them to the plex that you want to create. The vxmake command creates VxVM objects.

To create a plex from existing subdisks, use the following command:

vxmake plex plex_name sd=subdisk_name,...
For example, the command to create a concatenated plex labeled vol01-02 using two existing subdisks labeled disk02-01 and disk02-02 is:

vxmake plex vol01-02 sd=disk02-01,disk02-02
To create a striped plex, additional attributes need to be specified. For example, the command to create a striped plex named pl-01 with a stripe width of 32 sectors and 2 columns is:

vxmake plex pl-01 layout=STRIPE stwidth=32 ncolumn=2\ sd=disk01-01,disk02-01

Backup Using Mirroring

If a volume is mirrored, backup can be performed on that volume by taking one of the volume's mirrors offline for a period of time. This eliminates the need for extra disk space for the purpose of backup only. However, it also eliminates redundancy of the volume for the duration of the time needed for the backup.


Note: Note that the information in this section does not apply to RAID-5.


Perform a backup of a mirrored volume on an active system as follows:

1. Optionally ask users to stop activity for a short time, in order to improve the consistency of the backup.

2. Dissociate one of the volume's mirrors (vol01-01, for this example):

	vxplex dis vol01-01 
3. Create a new, temporary volume using the dissociated plex:

	vxmake -U gen voltemp plex=vol01-01 
4. Start the temporary volume:

	vxvol start tempvol
5. Perform appropriate backup procedures, using the temporary volume.

6. Stop the temporary volume:

	vxvol stop tempvol
7. Dissociate the backup plex from its temporary volume:

	vxplex dis vol01-01 
8. Reassociate the backup plex with its original volume in order to regain redundancy of the volume:

	vxplex att vol01 vol01-01
9. Remove the temporary volume:

	vxedit rm voltemp
For information on an alternative online backup method using the vxassist command, refer to "Performing Online Backup."

Associating Plexes

A plex becomes a participating mirror for a volume by associating the plex with the volume. To associate a plex with an existing volume, use the following command:

vxplex att volume_name plex_name
For example, the command to associate a plex labeled vol01-02 with a volume labeled vol01 looks like this:

vxplex att vol01 vol01-02
Alternately, if the volume has not been created, a plex (or multiple plexes) can be associated with the volume to be created as part of the volume create command:

vxmake -U usetype vol volume_name plex=plex_name1, plex_name2...
For example, the command line to create a mirrored, fsgen-type volume labeled home and associate two existing plexes labeled home-1 and home-2 looks like this:

vxmake -Ufsgen vol home plex=home-1,home-2 

Note: You can also use the following command on an existing volume to add and associate a plex:
vxassist mirror volume_name


Dissociating and Removing Plexes

When a plex is no longer needed, it can be removed. Examples of operations that require a plex to be removed are:

A plex can be dissociated from the associated volume and removed with the following command:

vxplex -o rm dis plex_name  
To dissociate and remove a plex labeled vol01-02, use the following command:

vxplex -o rm dis vol01-02
This removes the plex vol01-02 and all associated subdisks.


Note: You can first dissociate the plex and subdisks, then remove them with the commands:

vxplex dis plex_name
vxedit
-r rm plex_name

Together, these commands accomplish the same as vxplex -o rm dis.


Displaying Plex Information

To display detailed information about all plexes, use the following command:

vxprint -lp
To display detailed information about a specific plex, use the following command:

vxprint -l plex_name
Listing plexes helps identify free plexes that can be used for building volumes. Using the vxprint utility with the plex (-p) option lists information about all plexes; the -t option prints a single line of information about the plex. To list free plexes, use the following command:

vxprint -pt

Changing Plex Attributes

The comment field and the putil and tutil fields are used by the utilities after plex creation. putil attributes are maintained on reboot; tutil fields are temporary and are not retained on reboot. Both putil and tutil have three uses and are numbered according to those uses. These fields can be modified as needed. VxVM uses the utility fields marked putil0 and tutil0. Other VERITAS products use those marked putil1 and tutil1; those marked putil2 and tutil2 are user fields. Table 1 details the uses for the putil and tutil fields.

Table 1 The putil[n] and tutil[n] Fields
FieldDescription
putil0 This utility field is reserved for use by VxVM

utilities and is retained on reboot.

putil1This utility field is reserved for use by high-level utilities. such as the Visual Administrator interface. This field is retained on reboot.
putil2This utility field is reserved for use by the system administrator or site-specific applications. This field is retained on reboot.
tutil0This utility field is reserved for use by VxVM utilities and is cleared on reboot.
tutil1This utility field is reserved for use by high-level utilities. such as the Visual Administrator interface. This field is cleared on reboot.
tutil2 This utility field is reserved for use by the system administrator or site-specific applications. This field is cleared on reboot.

To change plex attributes, use the following command:

vxedit set field=value ... plex_name ...
The command:

vxedit set comment="my plex" tutil2="u" user="admin" vol01-02
uses vxedit to set the following attributes:

To prevent a particular plex from being associated with a volume, set the putil0 field to a non-null string, as specified in the following command:

vxedit set putil0="DO-NOT-USE" vol01-02

Changing Plex Status: Detaching and Attaching Plexes

Once a volume has been created and placed online (ENABLED), VxVM provides mechanisms by which plexes can be temporarily disconnected from the volume. This is useful, for example, when the hardware on which the plex resides needs repair or when a volume has been left unstartable and a source plex for the volume revive must be chosen manually.

Resolving a disk or system failure includes taking a volume offline and attaching and detaching its plexes. The two commands used to accomplish disk failure resolution are vxmend and vxplex.

To take a plex OFFLINE so that repair or maintenance can be performed on the physical disk containing that plex's subdisks, use the following command:

vxmend off plex_name ..
If a disk drive suffered a head crash, the system administrator should put all plexes that have associated subdisks represented on the affected drive OFFLINE. For example, if plexes vol01-02 and vol02-02 had subdisks on a drive to be repaired, use the following command:

vxmend off vol01-02 vol02-02
This command places vol01-02 and vol02-02 in the OFFLINE state, and they remain in that state until explicitly changed.

Detaching Plexes

To temporarily detach one plex in a mirrored volume, use the following command:

vxplex det plex_name  
For example, the command line to temporarily detach a plex labeled vol01-02 and place it in maintenance mode looks like this:

vxplex det vol01-02
This command temporarily detaches the plex, but maintains the association between the plex and its volume; however, the plex will not be used for I/O. A plex detached with the preceding command will be recovered on a system reboot. The plex state is set to STALE, so that if a vxvol start command is run on the appropriate volume (for example, on system reboot), the contents of the plex will be recovered and it will be made ACTIVE.

When the plex is ready to return as an active part of its volume, follow this procedure:

	vxvol start volume_name
If it is unstartable, set one of the plexes to CLEAN using the command:

	vxmend fix clean plex_name
and then start the volume.

	vxplex att volume_name plex_name ...
As with returning an OFFLINE plex to ACTIVE, this command recovers the contents of the plex(es), then sets the plex state to ACTIVE.

Attaching Plexes

When the disk has been repaired or replaced and is again ready for use, the plexes must be put back online (plex state set to ACTIVE).

If the volume is currently ENABLED, use the following command:

vxplex att volume_name plex_name ...
For example, the command for a plex labeled vol01-02 on a volume labeled vol01 looks like this:

vxplex att vol01 vol01-02
This starts to recover the contents of the plex and, after the revive is complete, sets the plex utility state to ACTIVE.

If the volume is not in use (not ENABLED), use the following command:

vxmend on plex_name
For example, the command for a plex labeled vol01-02 looks like this:

vxmend on vol01-02
In this case, the state of vol01-02 is set to STALE, so that when the volume is next started, the data on the plex will be revived from the other plex, and incorporated into the volume with its state set to ACTIVE.

If it becomes necessary to manually change the state of a plex, refer to "Volume Recovery." See the vxmake(1M) and vxmend(1M) manual pages for more information about these commands.

Moving Plexes

Moving a plex copies the data content from the original plex onto a new plex. In order for a move operation to be successful, the following criteria must be met:

The size of the plex has several important implications. If the new plex is smaller, or more sparse, than the original plex, an incomplete copy of the data on the original plex results. If this is the desired action, then the -o force option is required. If the new plex is longer, or less sparse, than the original plex, the data that exists on the original plex will be copied onto the new plex. Any area that was not on the original plex, but is represented on the new plex, will be filled from other complete plex associated with the same volume. If the new plex is longer than the volume itself, then the remaining area of the new plex above the size of the volume will not be initialized and remains unused.

The command to move data from one plex to another is:

vxplex mv original_plex new_plex 

Copying Plexes

This operation copies the contents of a volume onto a specified plex. The volume to be copied must not be enabled. The plex must not be associated with any other volume. To copy a plex, the following command is used:

vxplex cp volume_name new_plex
After the copy operation is complete, new_plex will not be associated with the specified volume volume_name. The plex contains a complete copy of the volume data. The plex that is being copied should be the same size or larger than the volume; otherwise, an incomplete copy of the data results. For the same reason, new_plex also should not be sparse.

Volume Operations

A volume is a collection of up to 32 plexes and appears as a block device in the /dev/vx/dsk directory and a character device in the /dev/vx/rdsk directory. A volume can be used as a partition device.

Volume operations include:

The following sections describe how to perform common volume operations. In some cases, either vxassist or one or more other commands can be used to accomplish the same task and both approaches are described. Detailed descriptions about the commands used to perform volume operations are contained in the VxVM manual pages.


Note: This section provides information on general volume operations. For details on operations specific to RAID-5 volumes, refer to RAID-5 Volume Operations."


Creating Volumes

Volumes can be created with either vxassist or vxmake.

The length of a new volume can be specified in sectors, kilobytes, megabytes, or gigabytes. The unit of measure is indicated by adding the appropriate suffix to the length (s, m, k, or g). If no unit is specified, sectors are assumed.

vxassist

The vxassist command can be used to create volumes with default settings or with user-specified attributes. vxassist automatically creates and attempts to enable new volumes. If the volume fails to be enabled, vxassist will attempt to remove it and release the space used to allocate that volume.

To create a simple volume using vxassist and its default settings, use the following command:

vxassist make volume_name length
For example, the following command creates a volume named voldef with a length of 10 megabytes on any available disk(s):

vxassist make voldef 10m
Additional parameters can be specified to vxassist to reflect the new volume's attributes. Refer to the vxassist(1M) manual page for details.

The following example illustrates how to create a volume named volzebra that is striped across disk03 and disk04, has the fsgen usage type, and is 10 megabytes long:

vxassist -Ufsgen make volzebra 10m layout=stripe disk03 disk04

vxmake

To create a volume with an attached plex using vxmake, use the following command:

vxmake -Uusage_type vol volume_name len=length plex=plex_name,... 
If you do not specify a length, the volume length will equal the length of the plex to which it is attached. You can select a length (less than or equal to the length of the plex) by specifying a length with the len= parameter. You can also create a volume without attaching a plex to it. You do this by omitting the plex= parameter. In this case, you must specify a length. The volume you create will not be available for use until you attach a plex to it using vxplex att.

Examples of commands for creating an fsgen-type volume called vol01 are:

vxmake -Ufsgen vol vol01 len=100000
or

vxmake vol vol01 use_type=fsgen plex=vol01-01,vol01-2 
The usage type for a volume can be specified as either -Ufsgen or use_type=fsgen. If a length is not specified or associated plexes are not identified, the length will be 0.

Instead of specifying parameters on the command line, you can use a vxmake description file to create a volume, as well as associated subdisks and plexes, by using the following command:

vxmake -d description_file
For detailed information about how to use vxmake, and an example of the vxmake description files, refer to "Creating VxVM Objects With vxmake" in this chapter or the vxmake(1M) manual page.

Initializing Volumes

During normal system operation, volume and plex states will be affected by system failures, shutdowns, and possible I/O failures. When a volume is first created, it is necessary to initialize the state of its one or more plexes according to what the state of the data is on each plex. Normally, if the user has created the volume using vxassist or one of the other higher-level interfaces, the state of the plexes will be properly set. However, when vxmake has been used to create a volume, the states of its plexes must be set manually before the volume can be made available for use through the vxvol start command. The command for setting the state of a volume's plexes is:

vxvol init state volume_name [plex_name]
where the state variable determines what the initialization does and what condition the volume and plexes will have after the volume has been initialized.

The most common form of manual initialization is setting the state of the volume to CLEAN. The following examples show how to do this for mirrored and non-mirrored volumes. In the simplest case, in which a volume has been created containing only one plex (mirror), the state of the plex is set to CLEAN. This is because there is no need for any synchronization of the data on the disk. Since there is only one plex in the volume, it is not necessary to specify the plex_name argument. The command to set the state of this volume to CLEAN is:

vxvol init clean volume_name

Note: The rest of this initialization section applies only to mirrored volumes (not RAID-5 volumes, which are discussed later).


Under more complicated circumstances, where a newly-created volume vol01 has multiple mirrors associated with it, one of the plexes must be chosen to which the other plexes are synchronized. For instance, if plex vol01-02 has been created over disk space that contained data that needed to be accessed through the volume after it is made available, then the following command would ensure that the data is synchronized out to the other plexes when the volume is started:

vxvol init clean vol01 vol01-02
This command will set the state of vol01-02 to CLEAN and the remainder of the plexes to STALE, so that they will be properly synchronized at the time the volume is made available. The administrator may avoid the initial synchronization of the volume in order to save time, but only when certain that none of the plexes contain data that will be the final contents of the volume. Under such a situation, it is possible to temporarily initialize the state of the volume so that the data can be loaded without having to perform a synchronization first. The command to do this is:

vxvol init enable volume_name
This enables the volume and all its plexes, but leaves the plex utility states set to EMPTY. After the entire volume's contents have been restored, both mirrors contain exactly the same data and will not need to be synchronized using the vxvol start operation. Such a volume (for example, home1) could be initialized for use and started at the same time using the following command:

vxvol init active home1

CAUTION! It is strongly recommended that data on each of the mirrors be exactly the same under these circumstances. Otherwise, the system may corrupt the data on both mirrors and, depending on the use for the volume, could crash the system. If you are not sure that the data is identical, then use the vxvol init clean method.


Sometimes it is necessary to remove all existing data from disks before new data is loaded. In this case, you can initialize every byte of the volume to zero by entering:

vxvol init zero volume_name

Estimating Maximum Volume Size

The vxassist command can provide information on the largest possible size for a volume that can currently be created with a given set of attributes. vxassist can also provide similar information for how much an existing volume can be extended under the current circumstances.

To determine the largest possible size for the volume to be created, use the command:

vxassist maxsize attributes...
For example, to determine the maximum size for a new RAID-5 volume on available disks, enter:

vxassist maxsize layout=raid5
This does not actually create the volume, but returns output such as the following (in sectors, by default):

Maximum volume size: 376832 (184Mb)
If, however, a volume with the specified attributes cannot be created, an error similar to the following might result:

vxvm:vxassist: ERROR: No volume can be created within the given
                      constraints
To determine how much an existing volume can grow, use the command:

vxassist maxgrow volume_name
For example, the command:

vxassist maxgrow raidvol
does not actually resize the volume, but results in output similar to the following:

Volume raidvol can be extended by 366592 to 1677312 (819Mb)
Notice that this output indicates both the amount by which the volume can be increased and the total size to which the volume can grow (in sectors, by default).

Removing Volumes

Removing a volume is the same as removing a physical disk partition on a system without VxVM. If the volume has been used, and critical data may still reside on the disk space defined by that volume, the system administrator should make a backup of the data, and ensure that this data is retained.

To remove a volume, use the following command:

vxedit rm volume_name

or

vxedit -rf rm volume_name

The -r option indicates recursive removal, which means the removal of all plexes associated with the volume and all subdisks associated with those plexes. The -f option forces removal, and is necessary if the volume is enabled.


CAUTION! The -r option of vxedit removes multiple objects. Exercise caution when using it.


Displaying Volume Information

It is possible to list information related to volumes under VxVM control. This information includes the name of the volume, its usage type, state, length, user and group IDs, and mode.

To list information on all volumes, use the following command:

vxprint -vt

To limit the display of vxprint to a single object, specify the object name after the vxprint command.

For example, to display detailed information about a specific volume, use the following command:

vxprint -l volume_name

If no volume is specified, detailed information is given for all volumes by using the following command:

vxprint -vl

To display the plexes for a volume labeled vol01, use the following command:

vxprint -vt vol01 

This command displays the volume, its plexes, and the subdisks in those plexes.

You can also display the plexes for vol01 with the command:

vxprint -e 'assoc= "vol01"'

Changing Volume Attributes

Volume attributes such as read policy, error policies, ownership, permissions, and the values in the comment and utility fields for existing volumes can be changed. These attributes are typically changed when the volume's use or users' needs change.

There are two VxVM commands associated with setting volume attributes:

Use vxedit as follows:

vxedit set field=value0 ... volume_name ...

Use vxvol as follows:

vxvol set field=value0 ... volume_name ...

Table 2 details which attributes can be set by each command.

Table 2 Setting Volume Attributes
CommandAttributeDescription
vxeditcommentthe comment field
tutil0, tutil1, tutil2

putil0, putil1, putil2

descriptive string of volume contents
fstypestring indicating file system type
writebackboolean (on/off) specifying read error correction mode
userowner of volume
groupgroup of volume
modepermission mode for volume
vxvollennumeric length of volume
logtype(drl/undef) specifier of Dirty Region Logging mode for volume
loglenlength of the Dirty Region Logging log
startoptsoptions to be executed to the
vxvol start operation


Note: Setting volume device permissions and ownership using the chgrp, chown, and chmod commands is ineffective. It is necessary to use vxedit set to modify these values.


For example, to change the owner of the group to susan and the permissions to read/write for owner, group, and other:

vxedit set user=susan group=staff mode=0666 vol01

Resizing Volumes

Resizing a volume is an instance of changing volume attributes that can be handled via either vxassist or vxvol.

The new size of a volume can be specified in sectors, kilobytes, megabytes, or gigabytes. The unit of measure is indicated by adding the appropriate suffix to the length (s, k, m, or g). If no unit is specified, sectors are assumed.

The vxresize command can be used to resize a volume containing a file system. Although other commands can be used to resize volumes containing file systems, vxresize offers the advantage of automatically resizing the file system as well as the volume. For details on how to use vxresize, see the vxresize(1M) manual page. Note that only vxfs and ufs file systems can be resized with vxresize.

vxassist

vxassist can resize a volume in any of the following ways:

growto--increase volume to specified length

growby--increase volume by specified amount

shrinkto--reduce volume to specified length

shrinkby--reduce volume by specified amount


CAUTION! If the volume contains a file system, do not shrink the volume below the size of the file system, as doing so could result in unrecoverable data loss. If you have a vxfs file system, you can shrink the file system before shrinking the volume.


If the volume is increased in size, vxassist automatically seeks out available disk space.

To increase a volume to a specified length, use the command:

vxassist growto volume_name new_length

To increase a volume by a certain amount, use the command:

vxassist growby volume_name length_change 

To reduce a volume to a specified length, use the command:

vxassist shrinkto volume_name new_length 

To reduce a volume by a certain amount, use the command:

vxassist shrinkby volume_name length_change

vxvol

To change the length of a volume using vxvol set, use the following command:

vxvol set len=value ... volume_name ...

For example, to change the length to 100000 sectors, use the following command:

vxvol set len=100000 vol01


Note: The vxvol set len command cannot increase the size of a volume unless the needed space is available in the plexes of the volume.

When a volume's size is reduced using the vxvol set len command, the freed space is not released into the free space pool.


Changing Volume Read Policy

VxVM offers the choice of the following read policies:

The read policy can be changed from round to prefer (or vice versa), or to a different preferred plex. The vxvol rdpol command sets the read policy for a volume.

To set the read policy to round, use the following command:

vxvol rdpol round volume_name

For example, the command line to set the read policy for volume vol01 to a round-robin read looks like this:

vxvol rdpol round vol01

To set the read policy to prefer, use the following command:

vxvol rdpol prefer volume_name  preferred_plex_name

For example, the command line to set the policy for vol01 to read preferentially from the plex vol01-02 looks like this:

vxvol rdpol prefer vol01 vol01-02

To set the read policy to select, use the following command:

vxvol rdpol select volume_name

Starting and Stopping Volumes

Like mounting and unmounting a file system, starting and stopping a volume affects its availability to the user. Starting a volume changes its state and makes it available for use. Stopping a volume makes it unavailable.

Starting a volume changes the volume state from DISABLED or DETACHED to ENABLED. The success of this operation depends on the ability to enable a volume. If a volume cannot be enabled, it remains in its current state. To start a volume, use the following command:

vxrecover -s volume_name ...

To start all DISABLED volumes, use the following command:

vxrecover -s

Stopping a volume changes the volume state from ENABLED or DETACHED to DISABLED. If the command cannot stop it, the volume remains in its current state. To stop a volume, use the following command:

vxvol stop volume_name ...

For example, the command line to stop a volume labeled vol01 looks like this:

vxvol stop vol01

To stop all ENABLED volumes, use the following command:

vxvol stopall

If all mirrors of the volume become STALE, put the volume in maintenance mode so that the plexes can be looked at while the volume is DETACHED and determine which plex to use for reviving the others. To place a volume in maintenance mode, use the following command:

vxvol maint volume_name

To assist in choosing the revival source plex, list the unstarted volume and displays its plexes.

To take plex vol01-02 offline, use the following command:

vxmend off vol01-02

The vxmend utility can change the state of an OFFLINE plex of a DISABLED volume to STALE, after which a vxvol start on the volume would revive the plex. To put a plex labeled vol01-02 in the STALE state, use the following command:

vxmend on vol01-02

Listing Unstartable Volumes

An unstartable volume is likely to be incorrectly configured or has other errors or conditions that prevent it from being started. To display unstartable volumes, use the command vxinfo, which displays information on the accessibility and usability of one or more volumes:

vxinfo [volume_name]

Mirroring Existing Volumes

A mirror (plex) can be added to an existing volume. This can be done with the vxassist command as follows:

vxassist mirror volume_name

For example:

vxassist mirror voltest

creates a mirror of the volume voltest.

Another way to mirror an existing volume is by first creating a plex and then associating it with a volume, using the following commands:

vxmake plex plex_name sd=subdisk_name ... 
vxplex att volume_name plex_name 

Volume Recovery

If a system crash or an I/O error corrupts one or more plexes of a volume and no plex is CLEAN or ACTIVE, mark one of the plexes CLEAN and instruct the system to use that plex as the source for reviving the others.

To place a plex in a CLEAN state, use the following command:

vxmend fix clean plex_name

For example, the command to place the plex labeled vol01-02 in the CLEAN state looks like this:

vxmend fix clean vol01-02

For information about how to use vxmend, refer to the vxmend(1M) manual page. For general information on recovery, refer to Appendix B, "Recovery."

RAID-5 Volume Operations

This section discusses the construction and operation of RAID-5 volumes. For more information on RAID-5 volumes, refer to Chapter 1.

RAID-5 and mirrored volumes both maintain redundancy of the data within a volume; mirrored volumes do so by keeping multiple, complete copies of the data in different plexes, whereas RAID-5 volumes keep one copy of the data and calculated parity in the same plex.

Both mirrored and RAID-5 volumes have the ability to do logging to minimize recovery time. Mirrored volumes use dirty region logs to keep a map of which regions of the volume are currently being written, while RAID-5 volumes use RAID-5 logs to keep a copy of the data and parity currently being written.

RAID-5 Volume Layout

Like a mirrored volume, a RAID-5 volume is made up of one or more plexes, each of which consists of one or more subdisks. Unlike mirrored volumes, not all plexes in a RAID-5 volume serve to keep a mirror copy of the volume data. A RAID-5 volume can have two types of plexes:

RAID-5 Plexes

RAID-5 volumes keep both the data and parity information in a single RAID-5 plex. A RAID-5 plex is made up of subdisks arranged in columns, similar to the striping model. Figure 29 shows a RAID-5 plex and the output of vxprint that would be associated with it:

Figure 29 vxprint Output for a RAID-5 Plex

PL 	NAME	VOLUME	KSTATE	STATE	LENGTH	LAYOUT	NCOL/WID	MODE
SD 	NAME	PLEX	DISK	DISKOFFS	LENGTH	[COL/]OFF	DEVICE	MODE

pl 	rvol-01	rvol	ENABLED	ACTIVE	20480	RAID 	3/16	RW
sd 	disk00-00	rvol-01	disk00	0	10240	0/0	c1b0t4d1	ENA
sd 	disk01-00	rvol-01	disk01	0	10240	1/0	c1b0t2d1	ENA
sd 	disk02-00	rvol-01	disk02	0	10240	2/0	c1b0t3d1	ENA

The plex line shows that the plex layout is RAID and that it has three columns and a stripe unit size of 16 sectors. Each subdisk line shows the column in the plex and offset in the column in which it is located.

RAID-5 Logs

Each RAID-5 volume can have at most one RAID-5 plex where the data and parity will be stored. Any other plexes associated with the volume are used to log information about data and parity being written to the volume. These plexes are referred to as RAID-5 log plexes or RAID-5 logs.

RAID-5 logs can be concatenated or striped plexes, and each RAID-5 log associated with a RAID-5 volume will have a complete copy of the logging information for the volume. It is suggested that you have a minimum of two RAID-5 log plexes for each RAID-5 volume. These log plexes should be located on different disks. Having two RAID-5 log plexes for each RAID-5 volume protects against the loss of logging information due to the failure of a single disk.

In order to support enough concurrent access to the RAID-5 array, the log should be several times the stripe size of the RAID-5 plex.

RAID-5 log plexes can be differentiated from the RAID-5 plex of a RAID-5 volume by examining vxprint output; the STATE field for a log plex is marked as LOG. Figure 30 shows the vxprint output for a RAID-5 volume.

Figure 30 vxprint Output for a RAID-5 Volume

V	NAME	USETYPE	KSTATE	STATE	LENGTH	READPOL	PREFPLEX
PL	NAME	VOLUME	KSTATE	STATE	LENGTH	LAYOUT	NCOL/WID	MODE
SD	NAME	PLEX	DISK	DISKOFFS	LENGTH	[COL/]OFF	DEVICE	MODE

v	r5vol	raid5	ENABLED	ACTIVE	20480	RAID	-	
pl	r5vol-01	r5vol	ENABLED	ACTIVE	20480	RAID	3/16	RW
sd	disk00-00	r5vol-01	disk00	0	10240	0/0	c1b0t4d1	ENA
sd	disk01-00	r5vol-01	disk01	0	10240	1/0	c1b0t2d1	ENA
sd	disk02-00	r5vol-01	disk02	0	10240	2/0	c1b0t3d1	ENA
pl	r5vol-l1	r5vol	ENABLED	LOG	1024	CONCAT	-	RW
sd	disk03-01	r5vol-l1	disk00	0	1024	0	c1b0t3d0	ENA
pl	r5vol-l2	r5vol	ENABLED	LOG	1024	CONCAT	-	RW
sd	disk04-01	r5vol-l2	disk02	0	1024	0	c1b0t1d1	ENA

The RAID-5 volume (r5vol) can be identified as a RAID-5 volume by its read policy being RAID. It has one RAID-5 plex (r5vol-01), similar to the one described earlier. It has two RAID-5 logs in the plexes r5vol-l1 and r5vol-l2; these are identified by the state field being LOG and the fact that they are associated with a RAID-5 volume and have a layout that is not RAID.

Creating RAID-5 Volumes

RAID-5 volumes can be created using either vxassist (the recommended approach) or vxmake. Both approaches are discussed in this section.

A RAID-5 volume contains a RAID-5 plex that consists of two or more subdisks located on two or more physical disks. Only one RAID-5 plex can exist per volume.

vxassist

Create a RAID-5 volume using vxassist as follows:

vxassist make volume_name length layout=raid5

For example, to create a 10M RAID-5 volume named volraid, enter:

vxassist make volraid 10m layout=raid5

This creates a RAID-5 volume with the default stripe unit size on the default number of disks.

vxmake

The creation of RAID-5 volumes through the vxmake command is similar to creation of other volumes (see "Creating Volumes" and the vxmake(1M) manual page for details on creating other volumes). The creation of subdisks for use in a RAID-5 volume is the same as creating any subdisk.

The procedure for creating a RAID-5 plex for a RAID-5 volume is similar to that for creating striped plexes, except that the layout attribute should be set to raid5. Subdisks can be implicitly associated in the same way as with striped plexes. Thus, a four-column RAID-5 plex with a stripe unit size of 32 sectors can be created from four existing subdisks as follows:

vxmake plex raidplex layout=raid5 stwidth=32 \
sd=disk00-01,disk01-00,disk02-00,disk03-00

Note that because four subdisks were specified without any specification of columns, vxmake assumes a four-column RAID-5 plex and places one subdisk in each column. This is the same behavior exhibited when creating striped plexes. If the subdisks are to be created later, the following command could be used to create the plex:

vxmake plex raidplex layout=raid5 ncolumn=4 stwidth=32


Note: If no subdisks are specified, the ncolumn attribute must be specified. Subdisks can later be filled in the plex by using vxsd assoc (see "Manipulating RAID-5 Subdisks").


To create a three-column RAID-5 plex using six subdisks, the following command could be used:

vxmake plex raidplex layout=raid5 stwidth=32 \
sd=disk00-00:0,disk01-00:1,disk02-00:2,disk03-00:0, \
disk04-00:1,disk05-00:2

This would stack subdisks disk00-00 and disk03-00 consecutively in column 0, subdisks disk01-00 and disk04-00 consecutively in column 1, and subdisks disk02-00 and disk05-00 in column 2. Offsets can also be specified to create sparse RAID-5 plexes, as for striped plexes.

Since log plexes are simply plexes without a RAID-5 layout, they can be created normally.

To create a RAID-5 volume with vxmake, simply specify the usage type to be RAID-5, as in:

vxmake -Uraid5 vol raidvol

RAID-5 plexes and RAID-5 log plexes can be associated implicitly:

vxmake -Uraid5 vol raidvol plex=raidplex,raidlog1, raidlog2

Initializing RAID-5 Volumes

A RAID-5 volume needs to be initialized if it was created by vxmake and has not yet been initialized or if it has somehow been set to an uninitialized state.

A RAID-5 volume can be initialized through vxvol in either of the following ways:

vxvol init zero volume_name

or

vxvol start volume_name

vxvol init zero writes zeroes to any RAID-5 log plexes and to the entire length of the volume; it then leaves the volume in the ACTIVE state.

vxvol start recovers parity by XORing together corresponding data stripe units in all other columns. Although it is slower than a vxvol init zero operation, vxvol start makes the RAID-5 volume immediately available.

Failures and RAID-5 Volumes

Failures, when taken in the context of volume management, come in two varieties: system failures and disk failures. A system failure means that the system has abruptly ceased to operate, such as due to an operating system panic or power failure. Disk failures imply that the data on some number of disks has become unavailable due to some sort of failure in the system (such as a head crash, electronics failure on disk, or disk controller failure).

System Failures

RAID-5 volumes are designed to remain available in the face of disk failures with a minimum of disk space overhead. However, many implementations of RAID-5 can become vulnerable to data loss after a system failure. This occurs because a system failure causes the data and parity in the RAID-5 volume to become unsynchronized because the disposition of writes that were outstanding at the time of the failure cannot be determined. If this occurs while a RAID-5 volume is being accessed, the volume is described as having stale parity. When this occurs, the parity must be reconstructed by reading all the non-parity columns within each stripe, recalculating the parity, and writing out the parity stripe unit in the stripe. This must be done for every stripe in the volume, so it can take a long time to complete.


CAUTION! While this resynchronization is going on, any failure of a disk within the array will cause the data in the volume to be lost. This only applies to RAID-5 volumes without log plexes.


Having the array vulnerable in this way is undesirable. Besides the vulnerability to failure, the resynchronization process can tax the system resources and slow down system operation.

RAID-5 logs reduce the possible damage that can be caused by system failures. Because they maintain a copy of the data being written at the time of the failure, the process of resynchronization consists of simply reading that data and parity from the logs and writing it to the appropriate areas of the RAID-5 volume. This greatly reduces the amount of time needed for a resynchronization of data and parity. It also means that the volume never becomes truly stale because the data and parity for all stripes in the volume is known at all times, so the failure of a single disk cannot result in the loss of the data within the volume.

Disk Failures

Disk failures can cause the data on a disk to become unavailable. In terms of a RAID-5 volume, this would mean that a subdisk becomes unavailable. This can occur due to an uncorrectable I/O error during a write to the disk (which causes the subdisk to be detached from the array) or due to a disk being unavailable when the system is booted (such as from a cabling problem or having a drive powered down). When this occurs, the subdisk cannot be used to hold data and is considered stale and detached. If the underlying disk becomes available or is replaced, the subdisk will still be considered stale and will not be used.

If an attempt is made to read data contained on a stale subdisk, the data is reconstructed from data from all other stripe units in the stripe; this operation is called a reconstructing-read. This is a significantly more expensive operation than simply reading the data, resulting in degraded read performance; thus, when a RAID-5 volume has stale subdisks, it is considered to be in degraded mode.

A RAID-5 volume in degraded mode can be recognized as such from the output of vxprint, as shown in Figure 31.

Figure 31 vxprint Output for a Degraded RAID-5 Volume

V	NAME	USETYPE	KSTATE	STATE	LENGTH	READPOL	PREFPLEX	
PL	NAME	VOLUME	KSTATE	STATE	LENGTH	LAYOUT	NCOL/WID	MODE
SD	NAME	PLEX	DISK	DISKOFFS	LENGTH	[COL/]OFF	DEVICE	MODE
v	r5vol	RAID-5	ENABLED	DEGRADED	20480	RAID	-	
pl	r5vol-01	r5vol	ENABLED	ACTIVE	20480	RAID	3/16	RW
sd	disk00-00	r5vol-01	disk00	0	10240	0/0	c1b0t4d1	
sd	disk01-00	r5vol-01	disk01	0	10240	1/0	c1b0t2d1	dS
sd	disk02-00	r5vol-01	disk02	0	10240	2/0	c1b0t3d1	-
pl	r5vol-l1	r5vol	ENABLED	LOG	1024	CONCAT	-	RW
sd	disk03-01	r5vol-l1	disk00	10240	1024	0	c1b0t3d0	-
pl	r5vol-l2	r5vol	ENABLED	LOG	1024	CONCAT	-	RW
sd	disk04-01	r5vol-l2	disk02	10240	1024	0	c1b0t1d1	-	  

The volume r5vol is in degraded mode, as shown by the STATE, which is listed as DEGRADED. The failed subdisk is disk01-00, as shown by the flags in the last column -- the d indicates that the subdisk is detached and the S indicates that the subdisk contents are stale.

It is also possible that a disk containing a RAID-5 log could experience a failure. This has no direct effect on the operation of the volume; however, the loss of all RAID-5 logs on a volume makes the volume vulnerable to a complete failure, as described earlier. In the output of vxprint -ht, failure within a RAID-5 log plex is indicated by the plex state being BADLOG, as shown in Figure 32, where the RAID-5 log plex r5vol-l1 has failed.

Figure 32 vxprint Output for a RAID-5 Volume with Failed Log Plex

V	NAME	USETYPE	KSTATE	STATE	LENGTH	READPOL	PREFPLEX	
PL	NAME	VOLUME	KSTATE	STATE	LENGTH	LAYOUT	NCOL/WID	MODE
SD	NAME	PLEX	DISK	DISKOFFS	LENGTH	[COL/]OFF	DEVICE	MODE
v	r5vol	RAID-5	ENABLED	ACTIVE	20480	RAID	-	
pl	r5vol-01	r5vol	ENABLED	ACTIVE	20480	RAID	3/16	RW 
sd	disk00-00	r5vol-01	disk00	0	10240	0/0	c1b0t4d1	ENA
sd	disk01-00	r5vol-01	disk01	0	10240	1/0	c1b0t2d1	dS
sd	disk02-00	r5vol-01	disk02	0	10240	2/0	c1b0t3d1	ENA
pl	r5vol-l1	r5vol	DISABLED	BADLOG	1024	CONCAT	-	RW
sd	disk03-01	r5vol-l1	disk00	10240	1024	0	c1b0t3d0	ENA
pl	r5vol-l2	r5vol	ENABLED	LOG	1024	CONCAT	-	RW
sd	disk04-01	r5vol-l2	disk02	10240	1024	0	c1b0t1d1	ENA

RAID-5 Recovery

The following are the types of recovery typically needed for RAID-5 volumes:

These types of recoveries are discussed in the sections that follow. Parity resynchronization and stale subdisk recovery are typically performed when the RAID-5 volume is started, shortly after the system boots, or by calling the vxrecover command. For more information on starting RAID-5 volumes, see "Starting RAID-5 Volumes."

If hot-relocation is enabled at the time of a disk failure, system administrator intervention is not required (unless there is no suitable disk space available for relocation). Hot-relocation will be triggered by the failure and the system administrator will be notified of the failure by electronic mail. Hot-relocation will automatically attempt to relocate the subdisks of a failing RAID-5 plex. After any relocation takes place, the hot-relocation daemon (vxrelocd) will also initiate a parity resynchronization. In the case of a failing RAID-5 log plex, relocation will only occur if the log plex is mirrored; vxrelocd will then initiate a mirror resynchronization to recreate the RAID-5 log plex. If hot-relocation is disabled at the time of a failure, the system administrator may need to initiate a resynchronization or recovery.

Parity Resynchronization

In most circumstances, a RAID-5 array will not have stale parity. Stale parity should only occur after all RAID-5 log plexes for the RAID-5 volume have failed, and then only if there is a system failure. Furthermore, even if a RAID-5 volume has stale parity, it is usually taken care of as part of the volume start process. However, if a volume without valid RAID-5 logs is started and the process is killed before the volume is resynchronized, the result will be an active volume with stale parity. This can be confirmed by checking the volume state displayed in the output of the vxprint -ht command, as shown in Figure 33.

Figure 33 vxprint Output for a Stale RAID-5 Volume

V	NAME	USETYPE	KSTATE	STATE	LENGTH	READPOL	PREFPLEX	
PL	NAME	VOLUME	KSTATE	STATE	LENGTH	LAYOUT	NCOL/WID	MODE
SD	NAME	PLEX	DISK	DISKOFFS	LENGTH	[COL/]OFF	DEVICE	MODE
v	r5vol	RAID-5	ENABLED	NEEDSYNC	20480	RAID	-	
pl	r5vol-01	r5vol	ENABLED	ACTIVE	20480	RAID	3/16	RW
sd	disk00-00	r5vol-01	disk00	0	10240	0/0	c1b0t4d1	ENA
sd	disk01-00	r5vol-01	disk01	0	10240	1/0	c1b0t2d1	ENA
sd	disk02-00	r5vol-01	disk02	0	10240	2/0	c1b0t3d1	ENA

This output lists the volume state as NEEDSYNC, indicating that the parity needs to be resynchronized. The state could also have been SYNC, indicating that a synchronization was attempted at start time and that a synchronization process should be doing the synchronization. If no such process exists or if the volume is in the NEEDSYNC state, a synchronization can be manually started using the resync keyword for the vxvol command. For example, to resynchronize the RAID-5 volume in Figure 33, the following command would be used:

vxvol resync r5vol

Parity is regenerated by issuing VOL_R5_RESYNC ioctls to the RAID-5 volume. The resynchronization process starts at the beginning of the RAID-5 volume and resynchronizes a region equal to the number of sectors specified by the
-o iosize option or, if -o iosize is not specified, the default maximum I/O size. The resync operation then moves onto the next region until the entire length of the RAID-5 volume has been resynchronized.

For larger volumes, parity regeneration can take a significant amount of time and it is possible that the system could be shut down or crash before the operation is completed. Unless the progress of parity regeneration is somehow kept across reboots, the process would have to start all over again. To avoid this situation, parity regeneration is checkpointed, meaning that the offset up to which the parity has been regenerated is saved in the configuration database. The -o checkpt=size option controls how often the checkpoint is saved; if not specified, it defaults to the default checkpoint size. Since saving the checkpoint offset requires a transaction, making the checkpoint size too small can significantly extend the time required to regenerate parity. After a system reboot, a RAID-5 volume that has a checkpoint offset smaller than the volume length will start a parity resynchronization at the checkpoint offset.

Subdisk Recovery

Like parity resynchronization, stale subdisk recovery is usually done at volume start time. However, it is possible that the process doing the recovery may get killed, or that the volume was started with an option to prevent subdisk recovery. It's also possible that the disk on which the subdisk resides was replaced without any recovery operations being performed. In any case, a subdisk recovery can be achieved by using the recover keyword of the vxvol command. For example, to recover the stale subdisk in the RAID-5 volume shown in Figure 31, the following command would be used:

vxvol recover r5vol disk01-00

If a RAID-5 volume has multiple stale subdisks to be caught up all at once, calling vxvol recover with only the name of the volume will achieve this:

vxvol recover r5vol

Log Plex Recovery

RAID-5 log plexes may become detached due to disk failures, as shown in Figure 32. These RAID-5 logs can be reattached by using the att keyword for the vxplex command. To reattach the failed RAID-5 log plex shown in Figure 32, the following command should be issued:

vxplex att r5vol r5vol-l1

Miscellaneous RAID-5 Operations

Many operations exist for manipulating RAID-5 volumes and associated objects. These operations are usually performed by other commands such as vxassist and vxrecover as part of larger operations, such as evacuating disks, etc. These command line operations should not be necessary for casual usage of the Volume Manager.

Manipulating RAID-5 Logs

RAID-5 logs are represented as plexes of RAID-5 volumes and are manipulated using the vxplex command. A RAID-5 log can be added using vxplex att:

vxplex att r5vol r5log

The attach operation can only proceed if the new log's size is large enough to hold an entire stripe's worth of data. If the RAID-5 volume already has logs, the new log length is the minimum of each individual log length. This is because the new log is a mirror of the old logs.

If the RAID-5 volume is not enabled, the new log is marked as BADLOG and will be enabled when the volume is started, though its contents will be ignored. If the RAID-5 volume is enabled and has other enabled RAID-5 logs, the new log will have its contents synchronized with the other logs via ATOMIC_COPY ioctls. If the RAID-5 volume currently has no enabled logs, the new log is zeroed before it is enabled.

Log plexes can be removed from a volume using the vxplex dis command:

vxplex dis r5log3

If removing the log would leave the volume with less than two valid logs, a warning will be printed and the operation will not be allowed to continue. The operation must be forced by using the -o force option.

Manipulating RAID-5 Subdisks

Like other subdisks, subdisks of the RAID-5 plex of a RAID-5 volume are manipulated using the vxsd command. Association is done using the assoc keyword in the same manner as for striped plexes. For example, to add subdisks at the end of each column of the RAID-5 volume in Figure 30, the following command would be used:

vxsd assoc r5vol-01 disk10-01:0 disk11-01:1 disk12-01:2

If a subdisk is filling a "hole" in the plex (i.e., some portion of the volume's logical address space is mapped by the subdisk), the subdisk will be considered stale. If the RAID-5 volume is enabled, the association operation will regenerate the data that belongs on the subdisk using VOL_R5_RECOVER ioctls; otherwise, it is simply marked as stale and will be recovered when the volume is started.

Subdisks can be removed from the RAID-5 plex by using vxsd dis:

vxsd dis disk10-01


CAUTION! If the subdisk maps a portion of the RAID-5 volume's address space, this places the volume in DEGRADED mode. If this is the case, the dis operation will print a warning and must be forced using the -o force option. Additionally, if removing the subdisk would make the RAID-5 volume unusable (because another subdisk in the same stripe is unusable or missing) and the volume is not DISABLED and empty, this operation is not allowed.


Subdisks can be moved to change the disks which a RAID-5 volume occupies by using vxsd mv. For example, if disk03 needed to be evacuated and disk22 has enough room by using two portions of its space, the following command could be used:

vxsd mv disk03-01 disk22-01 disk22-02

While this command is similar to that for striped plexes, the actual mechanics of the operation are not. In order to do RAID-5 subdisk moves, the old subdisk is removed from the RAID-5 plex and replaced by the new subdisks, which are marked as stale and then recovered using VOL_R5_RECOVER operations either by vxsd or (if the volume is not active) when the volume is started. This means that the RAID-5 volume is degraded for the duration of the operation. Another failure in the stripes involved in the move will make the volume unusable. The RAID-5 volume could also become invalid if the parity of the volume were to become stale.

To avoid these situations, the vxsd utility will not allow a subdisk move if:

Only the third case can be overridden using the -o force option.

Subdisks of RAID-5 volumes can also be split and joined by using vxsd split and vxsd join. These operations work the same as for mirrored volumes.

Starting RAID-5 Volumes

When a RAID-5 volume is started, it can be in one of many states. After a normal system shutdown, the volume should be clean and require no recovery. However, if the volume was not closed (or was not unmounted before a crash), it may require some type of recovery when it is started before it can be made available. This section outlines actions that should be taken under certain circumstances.

Under normal circumstances, volumes are started automatically after a reboot and any recovery takes place automatically or is done via the vxrecover command.

Unstartable RAID-5 Volumes

A RAID-5 volume is unusable if some part of the RAID-5 plex does not map the volume length. In other words, the RAID-5 plex cannot be sparse in relation to the RAID-5 volume length. The RAID-5 plex does not map a region where two subdisks have failed within a stripe, either because they are stale or because they are built on a failed disk. When this occurs, the vxvol start command will return the following error message:

vxvm:vxvol: ERROR: Volume r5vol is not startable; RAID-5 plex does
not map entire volume length.

At this point, the contents of the RAID-5 volume are unusable.

Another possible way that a RAID-5 volume can become unstartable is if the parity is stale and a subdisk becomes detached or stale. This occurs because within the stripes that contain the bad subdisk, the parity stripe unit is invalid (because the parity is stale) and the stripe unit on the bad subdisk is also invalid. This situation is shown in Figure 34, which illustrates a RAID-5 volume that has become invalid due to stale parity and a failed subdisk.

Figure 34 Invalid RAID-5 Volume

This example shows four stripes in the RAID-5 array. All parity is stale and subdisk disk05-00 has failed. This makes stripes X and Y unusable because two failures have occurred within those stripes.

This qualifies as two failures within a stripe and prevents the use of the volume. In this case, the output from vxvol start would be:

vxvm:vxvol: ERROR: Volume r5vol is not startable; some subdisks  are
unusable and the parity is stale.

This situation can be avoided by always using RAID-5 log plexes in RAID-5 volumes, preferably two or more. RAID-5 log plexes prevent the parity within the volume from becoming stale, thus preventing this situation (see "System Failures" for details).

Forcibly Starting RAID-5 Volumes

It is possible that the user may want to start a volume despite the fact that subdisks are marked as stale. For example, if a stopped volume has stale parity and no RAID-5 logs and a disk becomes detached and then reattached, the subdisk is considered stale even though the data is not out of date (because the volume was in use when the subdisk was unavailable) and the RAID-5 volume is considered invalid. The best way to prevent this case is to always have multiple valid RAID-5 logs associated with the array. However, this may not always be possible.

To start a RAID-5 volume despite the presence of stale subdisks, the -f option can be used with the vxvol start command. This will cause all stale subdisks to be marked as non-stale before the start operation begins its evaluation of the validity of the RAID-5 volume and what is needed to start it. Alternately, individual subdisks can be marked as non-stale by using the command vxmend fix unstale subdisk.

Recovery When Starting RAID-5 Volumes

Several operations may be necessary to fully restore the contents of a RAID-5 volume and make it usable. Whenever a volume is started, any RAID-5 log plexes are zeroed before the volume is started; this is done to prevent random data from being interpreted as a log entry and corrupting the volume contents. Beyond RAID-5 log zeroing, it is also possible that some subdisks may need to be recovered, or that the parity may need to be resynchronized (if RAID-5 logs have failed).

The following steps are taken when a RAID-5 volume is started:

1. If the RAID-5 volume was not cleanly shut down, it is checked for valid RAID-5 log plexes.


The volume is not made available while the parity is resynchronized because any subdisk failures during this period would make the volume unusable. This can be overridden by using the -o unsafe start option with vxvol.


CAUTION! The -o unsafe start option is considered dangerous, as it can make the contents of the volume unusable. It is therefore not recommended.


If any stale subdisks exist, the RAID-5 volume is unusable.

2. Any existing logging plexes are zeroed and enabled. If all logs fail during this process, the start process is aborted.

3. If no stale subdisks exist or those that exist are recoverable, the volume is put in the ENABLED kernel state and the volume state is set to ACTIVE. The volume is now started.

4. If some subdisks are stale and need recovery, and if valid logs exist, the volume is enabled by placing it in the ENABLED kernel state and the volume is available for use during the subdisk recovery. Otherwise, the volume kernel state is set to DETACHED and it is not available during subdisk recovery.

This is done because if the system were to crash or the volume was ungracefully stopped while it was activevp, the parity would become stale, making the volume unusable. If this is undesirable, the volume can be started with the -o unsafe start option.


CAUTION! The -o unsafe start option is considered dangerous, as it can make the contents of the volume unusable. It is therefore not recommended.


5. The volume state is set to RECOVER and stale subdisks are restored. As the data on each subdisk becomes valid, the subdisk is marked as no longer stale.

If any subdisk recovery fails and there are no valid logs, the volume start is aborted because the subdisk would remain stale and a system crash would make the RAID-5 volume unusable. This can also be overridden by using the -o unsafe start option.


CAUTION! The -o unsafe start option is considered dangerous, as it can make the contents of the volume unusable. It is therefore not recommended.


If the volume has valid logs, subdisk recovery failures are noted but do not stop the start procedure.

6. When all subdisks have been recovered, the volume is placed in the ENABLED kernel state and marked as ACTIVE. It is now started.

Changing RAID-5 Volume Attributes

Several attributes of RAID-5 volumes can be changed by the user. For RAID-5 volumes, the volume length and RAID-5 log length can be manipulated using the vxvol set command. To change the length of a RAID-5 volume, the following command can be used:

vxvol set len=10240 r5vol

The length of a volume cannot exceed the mapped region (called the contiguous length, or contiglen) of the RAID-5 plex. In other words, the length cannot be extended so as to make the volume unusable. If the RAID-5 volume is active and the length is being shortened, the operation must be forced using the -o force usage type option; this is done to prevent yanking space away from applications using the volume.

The length of the RAID-5 logs can also be changed using vxvol set, as in

vxvol set loglen=2M r5vol

Remember that RAID-5 log plexes are only valid if they map the entire length of the RAID-5 volume's log length. If increasing the log length would make any of the RAID-5 logs invalid, the operation will not be allowed. Also, if the volume is not active and is dirty (i.e., was not shut down cleanly), the log length cannot be changed. This avoids the loss of any of the log contents (if the log length is decreased) or the introduction of random data into the logs (if the log length is being increased).

Performing Online Backup

VxVM provides the ability to perform snapshot backups of volume devices. This capability is provided through vxassist and other utilities. There are various possible procedures for doing backups, depending upon the requirements for integrity of the volume contents. These procedures have the same starting requirement: a mirror that is large enough to store the complete contents of the volume. The plex can be larger than necessary, but if a plex that is too small is used, an incomplete copy results.

The recommended approach to volume backup involves the use of the vxassist utility. The vxassist procedure is convenient and relatively simple. The vxassist snapstart, snapwait, and snapshot operations provide a way to do online backup of volumes with minimal interruption of data change and access activity.

The snapstart operation creates a write-only backup plex which gets attached to and synchronized with the volume. When synchronized with the volume, the backup plex is ready to be used as a snapshot mirror. The end of the update procedure is signified by the new snapshot mirror changing its state to SNAPDONE. This change can be tracked by the vxassist snapwait operation, which waits until at least one of the mirrors changes its state to SNAPDONE. If the attach process fails, the snapshot mirror is removed and its space is released.

Once the snapshot mirror is synchronized, it continues being updated until it is detached. The system administrator can then select a convenient time at which to create a snapshot volume as an image of the existing volume. The system administrator can also ask users to refrain from using the system during the brief time required to perform the snapshot (typically less than a minute). The amount of time involved in creating the snapshot mirror is long and indefinite in contrast to the brief amount of time that it takes to create the snapshot volume.

The online backup procedure is completed by running a vxassist snapshot command on a volume with a SNAPDONE mirror. This operation detaches the finished snapshot (which becomes a normal mirror), creates a new normal volume and attaches the snapshot mirror to it. The snapshot then becomes a normal, functioning mirror and the state of the snapshot is set to ACTIVE.

If the snapshot procedure is interrupted, the snapshot mirror is automatically removed when the volume is started.

Use the following steps to perform a complete vxassist backup:


Note: This procedure does not apply to RAID-5.


1. Create a snapshot mirror for a volume as follows:

	vxassist snapstart volume_name

2. When the snapstart operation is complete and the mirror is in a SNAPDONE state, select a convenient time to complete the snapshot operation. Inform users of the upcoming snapshot and ask them to save files and refrain from using the system briefly during that time.

3. Create a snapshot volume that reflects the original volume as follows:

	vxassist snapshot volume_name temp_volume_name 

4. Use fsck (or some utility appropriate to the application running on the volume) to clean the temporary volume's contents. For example:

	fsck -y /dev/vx/rdsk/temp_volume_name

5. Copy the temporary volume to tape, or to some other appropriate backup media.

6. Remove the new volume as follows:

	vxedit -rf rm temp_volume_name


[Next] [Previous] [Top] [Contents] [Index]