vxdisk list
c1t0d0s2 sliced -- error
c1t0d0d2 sliced -- error
-- root-disk rootdg removed was:c1t0d0s2
1. Remove c1t0d0s2 entries from vxvm control
vxdisk -f rm c1t0d0s2
2.Remove the disk c1t0d0s2 using luxadm
luxadm remove_device c1t0d0s2
luxadm remove_device /dev/rdsk/c1t0d0s2
Pull the disk out as per the luxadm instrunctions.
3.Run command "devfsadm -C"
4. Run command "vxdctl enable"
5. luxadm -e offline /dev/dsk/c1t0d0s2
6. Run command "devfsadm -C"
7. Run command "vxdctl enable"
9.You need to use "luxadm insert_device" to replace the disk failed device.
[Once the disk have been replaced, use "vxdctl enable" and "vxdiskadm" option 5 after syncing with the remaining mirror.]
Result:
#vxdisk list
Now both the O/S device tree and VXVM are in a clean state corresponding to disk c1t0d0s2.
VXVM: Resolving duplicate disk/device entries in "vxdisk list" or vxdisksetup
Friday, May 6, 2011
at
8:13 AM
Posted In
VXVM
|
0
comments
|
HBA cards in Solaris 8, 9 and 10
Tuesday, May 3, 2011
at
4:57 AM
bash-2.03# luxadm probe
No Network Array enclosures found in /dev/es
Logical Path:/dev/rdsk/c5t50060E800475D109d6s2
Node WWN:50070e800475e108 Device Type:Disk device
Logical Path:/dev/rdsk/c5t50060E800475D109d7s2
HBA card WWN
# prtconf -vp | grep wwn
port-wwn: 2100001b.3202f94b
node-wwn: 2000001b.3202f94b
port-wwn: 210000e0.8b90e795
node-wwn: 200000e0.8b90e795
#prtconf -vp | more
For Solaris 8 and 9:
Run the following script to determine the WWNs of the HBAs that are currently being utilized:
#!/bin/sh for i in `cfgadm |grep fc-fabric|awk ‘{print $1}’`;
do
dev=”`cfgadm -lv $i|grep devices |awk ‘{print $NF}’`” wwn= \
“`luxadm -e dump_map $dev |grep ‘Host Bus’|awk ‘{print $4}’`”
echo “$i: $wwn” done
To show link status of card
bash-2.03# luxadm -e port
Found path to 2 HBA ports
/devices/ssm@0,0/pci@18,700000/SUNW,qlc@2/fp@0,0:devctl CONNECTED
/devices/ssm@0,0/pci@19,700000/SUNW,qlc@2/fp@0,0:devctl CONNECTED
To see the WWN’s (using address given to you from previous commands),
it is the last one that specifies it is a HBA, so the port WWN here is 50070e800475e108
bash-2.03# luxadm -e dump_map /devices/ssm@0,0/pci@18,700000/SUNW,qlc@2/fp@0,0:devctl
Pos Port_ID Hard_Addr Port WWN Node WWN Type
0 642113 0 50070e800475e108 50070e800475e108 0×0 (Disk device)
1 643f13 0 550070e800475e108 50070e800475e108 0×0 (Disk device)
2 643913 0 2100001b3205e828 2000001b3205e828 0x1f (Unknown Type,Host Bus Adapter)
SAN Foundation Software versions display as such
bash-2.03# modinfo | grep SunFC
38 102bcd25 209b8 150 1 fcp (SunFC FCP v20070703-1.98)
39 102d4071 855c - 1 fctl (SunFC Transport v20070703-1.41)
42 102ead69 164e0 149 1 fp (SunFC Port v20070703-1.60)
44 10300a79 cd574 153 1 qlc (SunFC Qlogic FCA v20070212-2.19)
To show Sun/Qlogic HBA’s
bash-2.03# luxadm qlgc
Found Path to 2 FC100/P, ISP2200, ISP23xx Devices
Opening Device: /devices/ssm@0,0/pci@18,700000/SUNW,qlc@2/fp@0,0:devctl
Detected FCode Version: ISP2312 Host Adapter fcode version 1.16 11/15/06
Complete
To show all vendor HBA’s
bash-2.03# luxadm fcode_download -p
Found Path to 0 FC/S Cards
Complete
Found Path to 0 FC100/S Cards
Complete
Found Path to 2 FC100/P, ISP2200, ISP23xx Devices
Opening Device: /devices/ssm@0,0/pci@18,700000/SUNW,qlc@2/fp@0,0:devctl
Detected FCode Version: ISP2312 Host Adapter fcode version 1.16 11/15/06
Opening Device: /devices/ssm@0,0/pci@19,700000/SUNW,qlc@2/fp@0,0:devctl
Detected FCode Version: ISP2312 Host Adapter fcode version 1.16 11/15/06
Complete
Found Path to 0 JNI1560 Devices.
Complete
Found Path to 0 Emulex Devices.
Complete
#fcinfo hba-port
HBA Port WWN: 10000000c98e6a99
OS Device Name: /dev/cfg/c1
Manufacturer: Emulex
Model: LPe11000-S
Firmware Version: 2.82a4 (Z3D2.82A4)
FCode/BIOS Version: Boot:5.02a1 Fcode:1.50a9
Serial Number: 0999VM0-09320010LF
Driver Name: emlxs
Driver Version: 2.50o (2010.01.08.09.45)
Type: unknown
State: offline
Supported Speeds: 1Gb 2Gb 4Gb
Current Speed: not established
Node WWN: 20000000c98e6a99
HBA Port WWN: 10000000c98e6a86
OS Device Name: /dev/cfg/c2
Manufacturer: Emulex
Model: LPe11000-S
Firmware Version: 2.82a4 (Z3D2.82A4)
FCode/BIOS Version: Boot:5.02a1 Fcode:1.50a9
Serial Number: 0999VM0-09320010LK
Driver Name: emlxs
Driver Version: 2.50o (2010.01.08.09.45)
Type: unknown
State: offline
Supported Speeds: 1Gb 2Gb 4Gb
Current Speed: not established
Node WWN: 20000000c98e6a86
No Network Array enclosures found in /dev/es
Logical Path:/dev/rdsk/c5t50060E800475D109d6s2
Node WWN:50070e800475e108 Device Type:Disk device
Logical Path:/dev/rdsk/c5t50060E800475D109d7s2
HBA card WWN
# prtconf -vp | grep wwn
port-wwn: 2100001b.3202f94b
node-wwn: 2000001b.3202f94b
port-wwn: 210000e0.8b90e795
node-wwn: 200000e0.8b90e795
#prtconf -vp | more
For Solaris 8 and 9:
Run the following script to determine the WWNs of the HBAs that are currently being utilized:
#!/bin/sh for i in `cfgadm |grep fc-fabric|awk ‘{print $1}’`;
do
dev=”`cfgadm -lv $i|grep devices |awk ‘{print $NF}’`” wwn= \
“`luxadm -e dump_map $dev |grep ‘Host Bus’|awk ‘{print $4}’`”
echo “$i: $wwn” done
To show link status of card
bash-2.03# luxadm -e port
Found path to 2 HBA ports
/devices/ssm@0,0/pci@18,700000/SUNW,qlc@2/fp@0,0:devctl CONNECTED
/devices/ssm@0,0/pci@19,700000/SUNW,qlc@2/fp@0,0:devctl CONNECTED
To see the WWN’s (using address given to you from previous commands),
it is the last one that specifies it is a HBA, so the port WWN here is 50070e800475e108
bash-2.03# luxadm -e dump_map /devices/ssm@0,0/pci@18,700000/SUNW,qlc@2/fp@0,0:devctl
Pos Port_ID Hard_Addr Port WWN Node WWN Type
0 642113 0 50070e800475e108 50070e800475e108 0×0 (Disk device)
1 643f13 0 550070e800475e108 50070e800475e108 0×0 (Disk device)
2 643913 0 2100001b3205e828 2000001b3205e828 0x1f (Unknown Type,Host Bus Adapter)
SAN Foundation Software versions display as such
bash-2.03# modinfo | grep SunFC
38 102bcd25 209b8 150 1 fcp (SunFC FCP v20070703-1.98)
39 102d4071 855c - 1 fctl (SunFC Transport v20070703-1.41)
42 102ead69 164e0 149 1 fp (SunFC Port v20070703-1.60)
44 10300a79 cd574 153 1 qlc (SunFC Qlogic FCA v20070212-2.19)
To show Sun/Qlogic HBA’s
bash-2.03# luxadm qlgc
Found Path to 2 FC100/P, ISP2200, ISP23xx Devices
Opening Device: /devices/ssm@0,0/pci@18,700000/SUNW,qlc@2/fp@0,0:devctl
Detected FCode Version: ISP2312 Host Adapter fcode version 1.16 11/15/06
Complete
To show all vendor HBA’s
bash-2.03# luxadm fcode_download -p
Found Path to 0 FC/S Cards
Complete
Found Path to 0 FC100/S Cards
Complete
Found Path to 2 FC100/P, ISP2200, ISP23xx Devices
Opening Device: /devices/ssm@0,0/pci@18,700000/SUNW,qlc@2/fp@0,0:devctl
Detected FCode Version: ISP2312 Host Adapter fcode version 1.16 11/15/06
Opening Device: /devices/ssm@0,0/pci@19,700000/SUNW,qlc@2/fp@0,0:devctl
Detected FCode Version: ISP2312 Host Adapter fcode version 1.16 11/15/06
Complete
Found Path to 0 JNI1560 Devices.
Complete
Found Path to 0 Emulex Devices.
Complete
#fcinfo hba-port
HBA Port WWN: 10000000c98e6a99
OS Device Name: /dev/cfg/c1
Manufacturer: Emulex
Model: LPe11000-S
Firmware Version: 2.82a4 (Z3D2.82A4)
FCode/BIOS Version: Boot:5.02a1 Fcode:1.50a9
Serial Number: 0999VM0-09320010LF
Driver Name: emlxs
Driver Version: 2.50o (2010.01.08.09.45)
Type: unknown
State: offline
Supported Speeds: 1Gb 2Gb 4Gb
Current Speed: not established
Node WWN: 20000000c98e6a99
HBA Port WWN: 10000000c98e6a86
OS Device Name: /dev/cfg/c2
Manufacturer: Emulex
Model: LPe11000-S
Firmware Version: 2.82a4 (Z3D2.82A4)
FCode/BIOS Version: Boot:5.02a1 Fcode:1.50a9
Serial Number: 0999VM0-09320010LK
Driver Name: emlxs
Driver Version: 2.50o (2010.01.08.09.45)
Type: unknown
State: offline
Supported Speeds: 1Gb 2Gb 4Gb
Current Speed: not established
Node WWN: 20000000c98e6a86
Posted In
HBA
|
0
comments
|
zfs
Thursday, April 14, 2011
at
12:10 AM
Adding the disk to the existing pool
zpool add eqptpool mirror c5t600508B4000909E00000C00007760000d0 c5t600508B4000900A70000800007820000d0
zpool add eqptpool mirror c5t600508B4000909E00000C00007760000d0 c5t600508B4000900A70000800007820000d0
Posted In
ZFS
|
0
comments
|
SUN Cluster Quick Reference commands
Saturday, April 9, 2011
at
3:06 AM
Shut down a resource group:
scswitch -F -g [RESOURCE_GROUP_NAME]
Bring up a resource group:
scswitch -Z -g [RESOURCE_GROUP_NAME]
Move failover resource group to node_name:
scswitch -z -g [RESOURCE_GROUP_NAME] -h [NODE_NAME]
Restart resource group:
scswitch -R -h [NODE_NAME] -g [RESOURCE_GROUP_NAME]
Evacuate all resources from node_name:
scswitch -S -h [NODE_NAME]
Disable resource:
scswitch -n -j [RESOURCE]
Enable resource:
scswitch -e -j [RESOURCE]
Clear STOP_FAILED on resource:
scswitch -c -j [RESOURCE] -h [NODE_NAME] -f STOP_FAILED
Disable resource’s fault monitor:
scswitch -n -M -j [RESOURCE]
Enable resource’s fault monitor:
scswitch -e -M -j [RESOURCE]
Enable resource’s fault monitor:
scswitch -e -M -j [RESOURCE]
Lists currently configured DID’s:
scdidadm -L
Put a new device under cluster control:
scgdevs
Displays status of the cluster, resources, resource groups, etc.:
scstat
Display useful setup info about cluster nodes, cluster transport, disksets, etc.:
scconf -p -v
scswitch -F -g [RESOURCE_GROUP_NAME]
Bring up a resource group:
scswitch -Z -g [RESOURCE_GROUP_NAME]
Move failover resource group to node_name:
scswitch -z -g [RESOURCE_GROUP_NAME] -h [NODE_NAME]
Restart resource group:
scswitch -R -h [NODE_NAME] -g [RESOURCE_GROUP_NAME]
Evacuate all resources from node_name:
scswitch -S -h [NODE_NAME]
Disable resource:
scswitch -n -j [RESOURCE]
Enable resource:
scswitch -e -j [RESOURCE]
Clear STOP_FAILED on resource:
scswitch -c -j [RESOURCE] -h [NODE_NAME] -f STOP_FAILED
Disable resource’s fault monitor:
scswitch -n -M -j [RESOURCE]
Enable resource’s fault monitor:
scswitch -e -M -j [RESOURCE]
Enable resource’s fault monitor:
scswitch -e -M -j [RESOURCE]
Lists currently configured DID’s:
scdidadm -L
Put a new device under cluster control:
scgdevs
Displays status of the cluster, resources, resource groups, etc.:
scstat
Display useful setup info about cluster nodes, cluster transport, disksets, etc.:
scconf -p -v
Posted In
Sun Cluster
|
0
comments
|
No output shows for df -h and /etc/mnttab file empty
Friday, March 25, 2011
at
10:47 AM
No output for the below command
df -h
more /etc/mnttab
mount the mnttab file
mount -F mntfs mnttab /etc/mnttab
Now try
df -h
more /etc/mnttab
df -h
more /etc/mnttab
mount the mnttab file
mount -F mntfs mnttab /etc/mnttab
Now try
df -h
more /etc/mnttab
Posted In
Trouble Shooting
|
1 comments
|
Running Linux applications in Solaris Linux branded zones
Saturday, February 19, 2011
at
8:40 PM
While playing around with the latest version of Nevada this week, I decided to see how well Linux branded zones work. In case your not following the Sun development efforts, Linux branded zones allow you to run Linux ELF executables unmodified on Solaris hosts. This is pretty interesting, and I definitely wanted to take this technology for a test drive. After reading through the documentation in the brandz community, I BFU’ed my Nevada machine to the latest nightly build, and installed the packages listed on the brandz download page. Since brandz currently only supports CentOS 3.0 – 3.7 and the Linux 2.4 kernel series, I first had to download the three CentoS 3.7 iso images (branded zones currently don’t support CentOS 3.8 without some hacking):
$ cd /home/matty/CentOS
$ wget http://www.gtlib.gatech.edu/pub/centos/3.7/isos/i386/CentOS-3.7-i386-bin1of3.iso
$ wget http://www.gtlib.gatech.edu/pub/centos/3.7/isos/i386/CentOS-3.7-i386-bin2of3.iso
$ wget http://www.gtlib.gatech.edu/pub/centos/3.7/isos/i386/CentOS-3.7-i386-bin3of3.iso
After I retrieved the ISO images, I needed to create a branded zone. Creating Linux branded zones is a piece of cake, and is accomplished by running the zonecfg utility with the “-z” option and a name to assign to your zone, and then specifying one or more parameters inside the zone configuration shell. Here is the configuration I used with my test zone:
$ zonecfg -z centostest
centostest: No such zone configured
Use 'create' to begin configuring a new zone.
zonecfg:centostest> create -t SUNWlx
zonecfg:centostest> add net
zonecfg:centostest:net> set physical=ni0
zonecfg:centostest:net> set address=192.168.1.25
zonecfg:centostest:net> end
zonecfg:centostest> set zonepath=/zones/centostest
zonecfg:centostest> set autoboot=true
zonecfg:centostest> verify
zonecfg:centostest> commit
zonecfg:centostest> exit
This zone configuration is pretty basic. It contains one network interface (when you boot the zone, a virtual interface is configured on that interface with the address passed to the address attribute), a location to store the zone data, and it is configured to automatically boot when the system is bootstrapped. Next I needed to install the CentOS binaries in the zone. To install the CentOS 3.7 binaries in the new zone I created, I ran the zoneadm utility with the ‘install’ option, and passed the directory with the CentOS ISO images as an argument:
$ zoneadm -z centostest install -v -d /home/matty/CentOS
Verbose output mode enabled.
Installing zone "centostest" at root "/zones/centostest"
Attempting ISO-based install from directory:
"/home/matty/CentOS"
Checking possible ISO
"/home/matty/CentOS/CentOS-3.7-i386-bin1of3.iso"...
added as lofi device "/dev/lofi/1"
Attempting mount of device "/dev/lofi/1"
on directory "/tmp/lxisos/iso.1"... succeeded.
Checking possible ISO
"/home/matty/CentOS/CentOS-3.7-i386-bin2of3.iso"...
added as lofi device "/dev/lofi/2"
Attempting mount of device "/dev/lofi/2"
on directory "/tmp/lxisos/iso.2"... succeeded.
Checking possible ISO
"/home/matty/CentOS/CentOS-3.7-i386-bin3of3.iso"...
added as lofi device "/dev/lofi/3"
Attempting mount of device "/dev/lofi/3"
on directory "/tmp/lxisos/iso.3"... succeeded.
Checking for distro "/usr/lib/brand/lx/distros/centos35.distro"...
Checking iso file mounted at "/tmp/lxisos/iso.1"...
read discinfo file "/tmp/lxisos/iso.1/.discinfo"
ISO "/tmp/lxisos/iso.1": Serial "1144177644.47"
Release "CentOS [Disc Set 1144177644.47]" Disc 1
Checking iso file mounted at "/tmp/lxisos/iso.2"...
read discinfo file "/tmp/lxisos/iso.2/.discinfo"
ISO "/tmp/lxisos/iso.2": Serial "1144177644.47"
Release "CentOS [Disc Set 1144177644.47]" Disc 2
Checking iso file mounted at "/tmp/lxisos/iso.3"...
read discinfo file "/tmp/lxisos/iso.3/.discinfo"
ISO "/tmp/lxisos/iso.3": Serial "1144177644.47"
Release "CentOS [Disc Set 1144177644.47]" Disc 3
Checking for distro "/usr/lib/brand/lx/distros/centos36.distro"...
Checking iso file mounted at "/tmp/lxisos/iso.1"...
read discinfo file "/tmp/lxisos/iso.1/.discinfo"
ISO "/tmp/lxisos/iso.1": Serial "1144177644.47"
Release "CentOS [Disc Set 1144177644.47]" Disc 1
Checking iso file mounted at "/tmp/lxisos/iso.2"...
read discinfo file "/tmp/lxisos/iso.2/.discinfo"
ISO "/tmp/lxisos/iso.2": Serial "1144177644.47"
Release "CentOS [Disc Set 1144177644.47]" Disc 2
Checking iso file mounted at "/tmp/lxisos/iso.3"...
read discinfo file "/tmp/lxisos/iso.3/.discinfo"
ISO "/tmp/lxisos/iso.3": Serial "1144177644.47"
Release "CentOS [Disc Set 1144177644.47]" Disc 3
Checking for distro "/usr/lib/brand/lx/distros/centos37.distro"...
Checking iso file mounted at "/tmp/lxisos/iso.1"...
read discinfo file "/tmp/lxisos/iso.1/.discinfo"
ISO "/tmp/lxisos/iso.1": Serial "1144177644.47"
Release "CentOS [Disc Set 1144177644.47]" Disc 1
Added ISO "/tmp/lxisos/iso.1" as disc 1
Checking iso file mounted at "/tmp/lxisos/iso.2"...
read discinfo file "/tmp/lxisos/iso.2/.discinfo"
ISO "/tmp/lxisos/iso.2": Serial "1144177644.47"
Release "CentOS [Disc Set 1144177644.47]" Disc 2
Added ISO "/tmp/lxisos/iso.2" as disc 2
Checking iso file mounted at "/tmp/lxisos/iso.3"...
read discinfo file "/tmp/lxisos/iso.3/.discinfo"
ISO "/tmp/lxisos/iso.3": Serial "1144177644.47"
Release "CentOS [Disc Set 1144177644.47]" Disc 3
Added ISO "/tmp/lxisos/iso.3" as disc 3
Installing distribution 'CentOS [Disc Set 1144177644.47]'...
Installing cluster 'desktop'
Installing zone miniroot.
Installing miniroot from ISO image 1 (of 3)
RPM source directory: "/tmp/lxisos/iso.1/RedHat/RPMS"
Attempting to expand 30 RPM names...
Installing RPM "SysVinit-2.85-4.4.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "basesystem-8.0-2.centos.0.noarch.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "bash-2.05b-41.5.centos.0.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "beecrypt-3.0.1-0.20030630.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "bzip2-libs-1.0.2-11.EL3.4.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "coreutils-4.5.3-28.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "elfutils-0.94-1.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "elfutils-libelf-0.94-1.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "filesystem-2.2.1-3.centos.1.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "glibc-2.3.2-95.39.i586.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "glibc-common-2.3.2-95.39.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "gpm-1.19.3-27.2.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "initscripts-7.31.30.EL-1.centos.1.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "iptables-1.2.8-12.3.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "iptables-ipv6-1.2.8-12.3.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "kernel-utils-2.4-8.37.14.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "laus-libs-0.1-70RHEL3.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "libacl-2.2.3-1.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "libattr-2.2.0-1.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "libgcc-3.2.3-54.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "libtermcap-2.0.8-35.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "ncurses-5.3-9.4.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "pam-0.75-67.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "popt-1.8.2-24_nonptl.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "rpm-4.2.3-24_nonptl.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "rpm-libs-4.2.3-24_nonptl.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "setup-2.5.27-1.noarch.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "termcap-11.0.1-17.1.noarch.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "zlib-1.1.4-8.1.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "centos-release-3-7.1.i386.rpm" to miniroot at
"/zones/centostest"...
Setting up the initial lx brand environment.
System configuration modifications complete!
Duplicating miniroot; this may take a few minutes...
Booting zone miniroot...
Miniroot zone setup complete.
Installing zone 'centostest' from ISO image 1.
RPM source directory: "/zones/centostest/root/iso/RedHat/RPMS"
Attempting to expand 667 RPM names...
Installing 433 RPM packages; this may take several minutes...
Preparing... ##################################################
libgcc ##################################################
setup ##################################################
filesystem ##################################################
hwdata ##################################################
redhat-menus ##################################################
mailcap ##################################################
XFree86-libs-data ##################################################
basesystem ##################################################
gnome-mime-data ##################################################
[.....]
After the brandz installer finished installing the CentOS 3.7 RPMs, I used the zoneadm ‘boot’ option to start the zone:
$ zoneadm -z centostest boot
To view the console output while the zone was booting, I immediately fired up the zlogin utility to console into the new Linux branded zone, and ran a few commands to see what the environment looked like after the zone was booted:
$ zlogin -C centostest
[Connected to zone 'centostest' console] [ OK ]
Activating swap partitions: [ OK ]
Checking filesystems [ OK ]
Mounting local filesystems: [ OK ]
Enabling swap space: [ OK ]
modprobe: Can't open dependencies file /lib/modules/2.4.21/modules.dep (No such file or directory)
INIT: Entering runlevel: 3
Entering non-interactive startup
Starting sysstat: [ OK ]
Starting system logger: [ OK ]
Starting kernel logger: [ OK ]
Starting automount: No Mountpoints Defined[ OK ]
Starting cups: [ OK ]
Starting sshd:[ OK ]
Starting crond: [ OK ]
Starting atd: [ OK ]
Rotating KDC list [ OK ]
CentOS release 3.7 (Final)
Kernel 2.4.21 on an i686
centostest login: root
$ uname -a
Linux centos 2.4.21 BrandZ fake linux i686 i686 i386 GNU/Linux
$ cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 6
model name : Intel Celeron(r)
stepping : 5
cpu MHz : 1662.136
cache size : 2048 KB
fdiv_bug : no
hlt_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 10
flags : fpu pse tsc msr mce cx8 sep mtrr pge cmov mmx fxsr sse sse2 ss
Yum works swell in a branded zone, and most of the tools you typically use work out of the box. Linux branded zones are wicked cool, and I can see tons of uses for them. Some folks are dead set on running Linux instead of Solaris, which means they can’t take advantage of things like ZFS, FMA and DTrace. If you need to better understand your application and the way it interacts with the system, or if you want to take advantage of the stability the Solaris kernel brings to production system, you can fire up a branded zone and run your application transparently on a Solaris system.
$ cd /home/matty/CentOS
$ wget http://www.gtlib.gatech.edu/pub/centos/3.7/isos/i386/CentOS-3.7-i386-bin1of3.iso
$ wget http://www.gtlib.gatech.edu/pub/centos/3.7/isos/i386/CentOS-3.7-i386-bin2of3.iso
$ wget http://www.gtlib.gatech.edu/pub/centos/3.7/isos/i386/CentOS-3.7-i386-bin3of3.iso
After I retrieved the ISO images, I needed to create a branded zone. Creating Linux branded zones is a piece of cake, and is accomplished by running the zonecfg utility with the “-z” option and a name to assign to your zone, and then specifying one or more parameters inside the zone configuration shell. Here is the configuration I used with my test zone:
$ zonecfg -z centostest
centostest: No such zone configured
Use 'create' to begin configuring a new zone.
zonecfg:centostest> create -t SUNWlx
zonecfg:centostest> add net
zonecfg:centostest:net> set physical=ni0
zonecfg:centostest:net> set address=192.168.1.25
zonecfg:centostest:net> end
zonecfg:centostest> set zonepath=/zones/centostest
zonecfg:centostest> set autoboot=true
zonecfg:centostest> verify
zonecfg:centostest> commit
zonecfg:centostest> exit
This zone configuration is pretty basic. It contains one network interface (when you boot the zone, a virtual interface is configured on that interface with the address passed to the address attribute), a location to store the zone data, and it is configured to automatically boot when the system is bootstrapped. Next I needed to install the CentOS binaries in the zone. To install the CentOS 3.7 binaries in the new zone I created, I ran the zoneadm utility with the ‘install’ option, and passed the directory with the CentOS ISO images as an argument:
$ zoneadm -z centostest install -v -d /home/matty/CentOS
Verbose output mode enabled.
Installing zone "centostest" at root "/zones/centostest"
Attempting ISO-based install from directory:
"/home/matty/CentOS"
Checking possible ISO
"/home/matty/CentOS/CentOS-3.7-i386-bin1of3.iso"...
added as lofi device "/dev/lofi/1"
Attempting mount of device "/dev/lofi/1"
on directory "/tmp/lxisos/iso.1"... succeeded.
Checking possible ISO
"/home/matty/CentOS/CentOS-3.7-i386-bin2of3.iso"...
added as lofi device "/dev/lofi/2"
Attempting mount of device "/dev/lofi/2"
on directory "/tmp/lxisos/iso.2"... succeeded.
Checking possible ISO
"/home/matty/CentOS/CentOS-3.7-i386-bin3of3.iso"...
added as lofi device "/dev/lofi/3"
Attempting mount of device "/dev/lofi/3"
on directory "/tmp/lxisos/iso.3"... succeeded.
Checking for distro "/usr/lib/brand/lx/distros/centos35.distro"...
Checking iso file mounted at "/tmp/lxisos/iso.1"...
read discinfo file "/tmp/lxisos/iso.1/.discinfo"
ISO "/tmp/lxisos/iso.1": Serial "1144177644.47"
Release "CentOS [Disc Set 1144177644.47]" Disc 1
Checking iso file mounted at "/tmp/lxisos/iso.2"...
read discinfo file "/tmp/lxisos/iso.2/.discinfo"
ISO "/tmp/lxisos/iso.2": Serial "1144177644.47"
Release "CentOS [Disc Set 1144177644.47]" Disc 2
Checking iso file mounted at "/tmp/lxisos/iso.3"...
read discinfo file "/tmp/lxisos/iso.3/.discinfo"
ISO "/tmp/lxisos/iso.3": Serial "1144177644.47"
Release "CentOS [Disc Set 1144177644.47]" Disc 3
Checking for distro "/usr/lib/brand/lx/distros/centos36.distro"...
Checking iso file mounted at "/tmp/lxisos/iso.1"...
read discinfo file "/tmp/lxisos/iso.1/.discinfo"
ISO "/tmp/lxisos/iso.1": Serial "1144177644.47"
Release "CentOS [Disc Set 1144177644.47]" Disc 1
Checking iso file mounted at "/tmp/lxisos/iso.2"...
read discinfo file "/tmp/lxisos/iso.2/.discinfo"
ISO "/tmp/lxisos/iso.2": Serial "1144177644.47"
Release "CentOS [Disc Set 1144177644.47]" Disc 2
Checking iso file mounted at "/tmp/lxisos/iso.3"...
read discinfo file "/tmp/lxisos/iso.3/.discinfo"
ISO "/tmp/lxisos/iso.3": Serial "1144177644.47"
Release "CentOS [Disc Set 1144177644.47]" Disc 3
Checking for distro "/usr/lib/brand/lx/distros/centos37.distro"...
Checking iso file mounted at "/tmp/lxisos/iso.1"...
read discinfo file "/tmp/lxisos/iso.1/.discinfo"
ISO "/tmp/lxisos/iso.1": Serial "1144177644.47"
Release "CentOS [Disc Set 1144177644.47]" Disc 1
Added ISO "/tmp/lxisos/iso.1" as disc 1
Checking iso file mounted at "/tmp/lxisos/iso.2"...
read discinfo file "/tmp/lxisos/iso.2/.discinfo"
ISO "/tmp/lxisos/iso.2": Serial "1144177644.47"
Release "CentOS [Disc Set 1144177644.47]" Disc 2
Added ISO "/tmp/lxisos/iso.2" as disc 2
Checking iso file mounted at "/tmp/lxisos/iso.3"...
read discinfo file "/tmp/lxisos/iso.3/.discinfo"
ISO "/tmp/lxisos/iso.3": Serial "1144177644.47"
Release "CentOS [Disc Set 1144177644.47]" Disc 3
Added ISO "/tmp/lxisos/iso.3" as disc 3
Installing distribution 'CentOS [Disc Set 1144177644.47]'...
Installing cluster 'desktop'
Installing zone miniroot.
Installing miniroot from ISO image 1 (of 3)
RPM source directory: "/tmp/lxisos/iso.1/RedHat/RPMS"
Attempting to expand 30 RPM names...
Installing RPM "SysVinit-2.85-4.4.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "basesystem-8.0-2.centos.0.noarch.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "bash-2.05b-41.5.centos.0.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "beecrypt-3.0.1-0.20030630.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "bzip2-libs-1.0.2-11.EL3.4.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "coreutils-4.5.3-28.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "elfutils-0.94-1.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "elfutils-libelf-0.94-1.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "filesystem-2.2.1-3.centos.1.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "glibc-2.3.2-95.39.i586.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "glibc-common-2.3.2-95.39.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "gpm-1.19.3-27.2.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "initscripts-7.31.30.EL-1.centos.1.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "iptables-1.2.8-12.3.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "iptables-ipv6-1.2.8-12.3.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "kernel-utils-2.4-8.37.14.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "laus-libs-0.1-70RHEL3.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "libacl-2.2.3-1.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "libattr-2.2.0-1.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "libgcc-3.2.3-54.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "libtermcap-2.0.8-35.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "ncurses-5.3-9.4.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "pam-0.75-67.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "popt-1.8.2-24_nonptl.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "rpm-4.2.3-24_nonptl.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "rpm-libs-4.2.3-24_nonptl.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "setup-2.5.27-1.noarch.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "termcap-11.0.1-17.1.noarch.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "zlib-1.1.4-8.1.i386.rpm" to miniroot at
"/zones/centostest"...
Installing RPM "centos-release-3-7.1.i386.rpm" to miniroot at
"/zones/centostest"...
Setting up the initial lx brand environment.
System configuration modifications complete!
Duplicating miniroot; this may take a few minutes...
Booting zone miniroot...
Miniroot zone setup complete.
Installing zone 'centostest' from ISO image 1.
RPM source directory: "/zones/centostest/root/iso/RedHat/RPMS"
Attempting to expand 667 RPM names...
Installing 433 RPM packages; this may take several minutes...
Preparing... ##################################################
libgcc ##################################################
setup ##################################################
filesystem ##################################################
hwdata ##################################################
redhat-menus ##################################################
mailcap ##################################################
XFree86-libs-data ##################################################
basesystem ##################################################
gnome-mime-data ##################################################
[.....]
After the brandz installer finished installing the CentOS 3.7 RPMs, I used the zoneadm ‘boot’ option to start the zone:
$ zoneadm -z centostest boot
To view the console output while the zone was booting, I immediately fired up the zlogin utility to console into the new Linux branded zone, and ran a few commands to see what the environment looked like after the zone was booted:
$ zlogin -C centostest
[Connected to zone 'centostest' console] [ OK ]
Activating swap partitions: [ OK ]
Checking filesystems [ OK ]
Mounting local filesystems: [ OK ]
Enabling swap space: [ OK ]
modprobe: Can't open dependencies file /lib/modules/2.4.21/modules.dep (No such file or directory)
INIT: Entering runlevel: 3
Entering non-interactive startup
Starting sysstat: [ OK ]
Starting system logger: [ OK ]
Starting kernel logger: [ OK ]
Starting automount: No Mountpoints Defined[ OK ]
Starting cups: [ OK ]
Starting sshd:[ OK ]
Starting crond: [ OK ]
Starting atd: [ OK ]
Rotating KDC list [ OK ]
CentOS release 3.7 (Final)
Kernel 2.4.21 on an i686
centostest login: root
$ uname -a
Linux centos 2.4.21 BrandZ fake linux i686 i686 i386 GNU/Linux
$ cat /proc/cpuinfo
processor : 0
vendor_id : GenuineIntel
cpu family : 6
model : 6
model name : Intel Celeron(r)
stepping : 5
cpu MHz : 1662.136
cache size : 2048 KB
fdiv_bug : no
hlt_bug : no
coma_bug : no
fpu : yes
fpu_exception : yes
cpuid level : 10
flags : fpu pse tsc msr mce cx8 sep mtrr pge cmov mmx fxsr sse sse2 ss
Yum works swell in a branded zone, and most of the tools you typically use work out of the box. Linux branded zones are wicked cool, and I can see tons of uses for them. Some folks are dead set on running Linux instead of Solaris, which means they can’t take advantage of things like ZFS, FMA and DTrace. If you need to better understand your application and the way it interacts with the system, or if you want to take advantage of the stability the Solaris kernel brings to production system, you can fire up a branded zone and run your application transparently on a Solaris system.
Posted In
Zone
|
0
comments
|
Build a Guest Domain
at
8:30 PM
A guest domain is made up of the following components:
CPU
MAU (Cryptographic Thread)
Memory
Networking
Storage
The control domain will partition CPU threads as VCPU's for the guest domain. Each CPU core has an MAU for cryptographic processing. Only one logical domain using the CPU threads in a core can have control over this thread. So it's important to decide if your guest domain will require one. Memory is partitioned in 8K segments. Networking is handled by connecting a virtual network interface to a virtual switch in one of the service domains. In our example, I configured each physical interface as a separate virtual switch in the control/service (a.k.a. primary) domain. Storage can come from a wide variety of sources:
Local Disk
SAN LUN
Virtual Disk Image File
Disk Slice
ZFS Volume
The T2000 for example has four drive bay that could be used, but obviously that doesn't leave us with a lot of flexibility or space. SAN storage can be used with greater flexibility since it's remote and can easily be migrated or replicated. It's possible to create a sparse file and use it as a virtual disk. This has the advantage of being stored on local, SAN, or even NAS. The fact that files can be used opens the door for very flexible options. Using a disk slice is also possible, but it can not be used for jumpstart installation. One could create ZFS volumes and use them as storage for logical domains as well. However, it can not be used for jumpstart installation. However, it makes for easy allocation of storage for applications. You can even take SAN LUN's and create a ZFS pool and export it into a logical domain. For our example, I'll use two virtual disk image files created on a ZFS file system and use SVM mirroring:) The following will be configured:
4 x VCPU's
1 x MAU
4GB's RAM
2 x 10GB Virtual Disk Image Files
2 x Network Ports
# ldm add-domain ldom1
# ldm add-vcpu 4 ldom1
# ldm add-mau 1 ldom1
# ldm add-memory 4G ldom1
# mkfile 10g /ldoms/vdisk1_10gb.img
# mkfile 10g /ldoms/vdisk2_10gb.img
# ldm add-vdiskserverdevice /ldoms/vdisk1_10gb.img vdisk1@primary-vds0
# ldm add-vdiskserverdevice /ldoms/vdisk2_10gb.img vdisk2@primary-vds0
# ldm add-vdisk vdisk1 vdisk1@primary-vds0 ldom1
# ldm add-vdisk vdisk2 vdisk2@primary-vds0 ldom1
# ldm add-vnet vnet0 primary-vsw0 ldom1
# ldm add-vnet vnet1 primary-vsw2 ldom1
# ldm set-variable auto-boot\?=false ldom1
# ldm set-variable local-mac-address\?=true ldom1
# ldm set-variable boot-device=/virtual-devices@100/channel-devices@200/disk@0 ldom1
# ldm bind-domain ldom1
# ldm start ldom1
So with the above commands we allocated the vcpu's, mau, and the memory. Then created the virtual disk images files, added them as virtual disk devices to the primary domain's VDS service, and finally added them as virtual disks to the guest domain. Then attached virtual networks, set auto-boot to false in the OBP(yes that's right each logical domain gets its own OBP), set local-mac-address to true, and set the default boot device. Finally we've bound the configuration and started the guest domain. So what do we get?
# ldm list-bindings ldom1
Name: ldom1
State: active
Flags: transition
OS:
Util: 0.2%
Uptime: 1d 6h 43m
Vcpu: 4
vid pid util strand
0 4 0.7% 100%
1 5 0.1% 100%
2 6 0.1% 100%
3 7 0.0% 100%
Mau: 1
mau cpuset (4, 5, 6, 7)
Memory: 4G
real-addr phys-addr size
0x4800000 0x104800000 4G
Vars: auto-boot?=false
boot-device=/virtual-devices@100/channel-devices@200/disk@0
local-mac-address?=true
Vldcc: vldcc0 [Domain Services]
service: primary-vldc0 @ primary
[LDC: 0x0]
Vnet: vnet0 [LDC: 0x2]
mac-addr=0:14:4f:fb:c4:ef
service: primary-vsw0 @ primary
[LDC: 0x1]
Vnet: vnet1 [LDC: 0xd]
mac-addr=0:14:4f:fb:24:b6
service: primary-vsw2 @ primary
[LDC: 0xc]
Vdisk: vdisk1 vdisk1@primary-vds0
service: primary-vds0 @ primary
[LDC: 0x17]
Vdisk: vdisk2 vdisk2@primary-vds0
service: primary-vds0 @ primary
[LDC: 0x18]
Vcons: [via LDC:25]
ldom1@primary-vcc0 [port:5000]
As you can see, everything that's been previously configured is listed. Some important things to note are the MAC addresses for the network interfaces (which are assigned automatically) and the Vcons port for the console. So now we can jumpstart our domain:
# telnet localhost 5000
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
Connecting to console "ldom1" in group "ldom1" ....
Press ~? for control options ..
Sun Fire T200, No Keyboard
Copyright 2007 Sun Microsystems, Inc. All rights reserved.
OpenBoot 4.26.0.build_07, 4096 MB memory available, Serial #66831599.
Ethernet address 0:14:4f:fb:c4:ef, Host ID: 83fbc4ef.
{0} ok show-nets
a) /virtual-devices@100/channel-devices@200/network@1
b) /virtual-devices@100/channel-devices@200/network@0
q) NO SELECTION
Enter Selection, q to quit: a
/virtual-devices@100/channel-devices@200/network@1 has been selected.
Type ^Y ( Control-Y ) to insert it in the command line.
e.g. ok nvalias mydev ^Y
for creating devalias mydev for /virtual-devices@100/channel-devices@200/network@1
{0} ok boot /virtual-devices@100/channel-devices@200/network@1 - install
Boot device: /virtual-devices@100/channel-devices@200/network@1 File and args:
- install
Requesting Internet Address for 0:14:4f:fb:24:b6
Requesting Internet Address for 0:14:4f:fb:24:b6
Requesting Internet Address for 0:14:4f:fb:24:b6
Requesting Internet Address for 0:14:4f:fb:24:b6
Requesting Internet Address for 0:14:4f:fb:24:b6
Requesting Internet Address for 0:14:4f:fb:24:b6
Requesting Internet Address for 0:14:4f:fb:24:b6
SunOS Release 5.10 Version Generic_118833-33 64-bit
Copyright 1983-2006 Sun Microsystems, Inc. All rights reserved.
Use is subject to license terms.
WARNING: machine_descrip_update: new MD has the same generation (1) as the old MD
whoami: no domain name
Configuring devices.
Using RPC Bootparams for network configuration information.
Attempting to configure interface vnet1...
Configured interface vnet1
Attempting to configure interface vnet0...
Skipped interface vnet0
Setting up Java. Please wait...
Extracting windowing system. Please wait...
Beginning system identification...
Searching for configuration file(s)...
...
So after the guest domain is finished jumpstarting, we can take a look around.
# psrinfo -vp
The physical processor has 4 virtual processors (0-3)
UltraSPARC-T1 (cpuid 0 clock 1000 MHz)
# psrinfo -v
Status of virtual processor 0 as of: 04/05/2007 22:17:04
on-line since 04/05/2007 22:16:15.
The sparcv9 processor operates at 1000 MHz,
and has a sparcv9 floating point processor.
Status of virtual processor 1 as of: 04/05/2007 22:17:04
on-line since 04/05/2007 22:16:16.
The sparcv9 processor operates at 1000 MHz,
and has a sparcv9 floating point processor.
Status of virtual processor 2 as of: 04/05/2007 22:17:04
on-line since 04/05/2007 22:16:16.
The sparcv9 processor operates at 1000 MHz,
and has a sparcv9 floating point processor.
Status of virtual processor 3 as of: 04/05/2007 22:17:04
on-line since 04/05/2007 22:16:16.
The sparcv9 processor operates at 1000 MHz,
and has a sparcv9 floating point processor.
# prtdiag -v
System Configuration: Sun Microsystems sun4v Sun Fire T200
Memory size: 4096 Megabytes
========================= CPUs ===============================================
CPU CPU
Location CPU Freq Implementation Mask
------------ ----- -------- ------------------- -----
MB/CMP0/P0 0 1000 MHz SUNW,UltraSPARC-T1
MB/CMP0/P1 1 1000 MHz SUNW,UltraSPARC-T1
MB/CMP0/P2 2 1000 MHz SUNW,UltraSPARC-T1
MB/CMP0/P3 3 1000 MHz SUNW,UltraSPARC-T1
========================= IO Configuration =========================
IO
Location Type Slot Path Name Model
----------- ----- ---- --------------------------------------------- ------------------------- ---------
========================= HW Revisions =======================================
System PROM revisions:
----------------------
OBP 4.26.0.build_07 2007/02/14 19:20
IO ASIC revisions:
------------------
Location Path Device Revision
-------------------- ---------------------------------------- ------------------------------ ---------
# df -h
Filesystem size used avail capacity Mounted on
/dev/md/dsk/d0 7.8G 2.2G 5.5G 30% /
/devices 0K 0K 0K 0% /devices
ctfs 0K 0K 0K 0% /system/contract
proc 0K 0K 0K 0% /proc
mnttab 0K 0K 0K 0% /etc/mnttab
swap 5.1G 1.1M 5.1G 1% /etc/svc/volatile
objfs 0K 0K 0K 0% /system/object
/platform/SUNW,Sun-Fire-T200/lib/libc_psr/libc_psr_hwcap1.so.1
7.8G 2.2G 5.5G 30% /platform/sun4v/lib/libc_psr.so.1
/platform/SUNW,Sun-Fire-T200/lib/sparcv9/libc_psr/libc_psr_hwcap1.so.1
7.8G 2.2G 5.5G 30% /platform/sun4v/lib/sparcv9/libc_psr.so.1
fd 0K 0K 0K 0% /dev/fd
swap 1.6G 0K 1.6G 0% /tmp
swap 5.1G 32K 5.1G 1% /var/run
# metastat
d1: Mirror
Submirror 0: d11
State: Okay
Submirror 1: d21
State: Okay
Pass: 1
Read option: roundrobin (default)
Write option: parallel (default)
Size: 4194600 blocks (2.0 GB)
d11: Submirror of d1
State: Okay
Size: 4194600 blocks (2.0 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c0d0s1 0 No Okay No
d21: Submirror of d1
State: Okay
Size: 4194600 blocks (2.0 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c0d1s1 0 No Okay No
d0: Mirror
Submirror 0: d10
State: Okay
Submirror 1: d20
State: Okay
Pass: 1
Read option: roundrobin (default)
Write option: parallel (default)
Size: 16644000 blocks (7.9 GB)
d10: Submirror of d0
State: Okay
Size: 16644000 blocks (7.9 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c0d0s0 0 No Okay No
d20: Submirror of d0
State: Okay
Size: 16644000 blocks (7.9 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c0d1s0 0 No Okay No
Device Relocation Information:
Device Reloc Device ID
c0d1 No -
c0d0 No -
# ifconfig -a
lo0: flags=2001000849 mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
vnet0: flags=9040843 mtu 1500 index 2
inet 192.168.1.2 netmask ffffff00 broadcast 192.168.1.255
groupname ipmp1
ether 0:14:4f:fb:c4:ef
vnet0:1: flags=1000843 mtu 1500 index 2
inet 192.168.1.1 netmask ffffff00 broadcast 192.168.1.255
vnet1: flags=9040843 mtu 1500 index 3
inet 192.168.1.3 netmask ffffff00 broadcast 192.168.1.255
groupname ipmp1
ether 0:14:4f:fb:24:b6
# uptime
10:20pm up 5 min(s), 1 user, load average: 0.02, 0.11, 0.06
So now we have a guest domain running Solaris 10 Update 3, with SVM mirrored boot drives that are really sparse files, IPMP on virtual NICs, four CPU's, 4GB's RAM, etc
CPU
MAU (Cryptographic Thread)
Memory
Networking
Storage
The control domain will partition CPU threads as VCPU's for the guest domain. Each CPU core has an MAU for cryptographic processing. Only one logical domain using the CPU threads in a core can have control over this thread. So it's important to decide if your guest domain will require one. Memory is partitioned in 8K segments. Networking is handled by connecting a virtual network interface to a virtual switch in one of the service domains. In our example, I configured each physical interface as a separate virtual switch in the control/service (a.k.a. primary) domain. Storage can come from a wide variety of sources:
Local Disk
SAN LUN
Virtual Disk Image File
Disk Slice
ZFS Volume
The T2000 for example has four drive bay that could be used, but obviously that doesn't leave us with a lot of flexibility or space. SAN storage can be used with greater flexibility since it's remote and can easily be migrated or replicated. It's possible to create a sparse file and use it as a virtual disk. This has the advantage of being stored on local, SAN, or even NAS. The fact that files can be used opens the door for very flexible options. Using a disk slice is also possible, but it can not be used for jumpstart installation. One could create ZFS volumes and use them as storage for logical domains as well. However, it can not be used for jumpstart installation. However, it makes for easy allocation of storage for applications. You can even take SAN LUN's and create a ZFS pool and export it into a logical domain. For our example, I'll use two virtual disk image files created on a ZFS file system and use SVM mirroring:) The following will be configured:
4 x VCPU's
1 x MAU
4GB's RAM
2 x 10GB Virtual Disk Image Files
2 x Network Ports
# ldm add-domain ldom1
# ldm add-vcpu 4 ldom1
# ldm add-mau 1 ldom1
# ldm add-memory 4G ldom1
# mkfile 10g /ldoms/vdisk1_10gb.img
# mkfile 10g /ldoms/vdisk2_10gb.img
# ldm add-vdiskserverdevice /ldoms/vdisk1_10gb.img vdisk1@primary-vds0
# ldm add-vdiskserverdevice /ldoms/vdisk2_10gb.img vdisk2@primary-vds0
# ldm add-vdisk vdisk1 vdisk1@primary-vds0 ldom1
# ldm add-vdisk vdisk2 vdisk2@primary-vds0 ldom1
# ldm add-vnet vnet0 primary-vsw0 ldom1
# ldm add-vnet vnet1 primary-vsw2 ldom1
# ldm set-variable auto-boot\?=false ldom1
# ldm set-variable local-mac-address\?=true ldom1
# ldm set-variable boot-device=/virtual-devices@100/channel-devices@200/disk@0 ldom1
# ldm bind-domain ldom1
# ldm start ldom1
So with the above commands we allocated the vcpu's, mau, and the memory. Then created the virtual disk images files, added them as virtual disk devices to the primary domain's VDS service, and finally added them as virtual disks to the guest domain. Then attached virtual networks, set auto-boot to false in the OBP(yes that's right each logical domain gets its own OBP), set local-mac-address to true, and set the default boot device. Finally we've bound the configuration and started the guest domain. So what do we get?
# ldm list-bindings ldom1
Name: ldom1
State: active
Flags: transition
OS:
Util: 0.2%
Uptime: 1d 6h 43m
Vcpu: 4
vid pid util strand
0 4 0.7% 100%
1 5 0.1% 100%
2 6 0.1% 100%
3 7 0.0% 100%
Mau: 1
mau cpuset (4, 5, 6, 7)
Memory: 4G
real-addr phys-addr size
0x4800000 0x104800000 4G
Vars: auto-boot?=false
boot-device=/virtual-devices@100/channel-devices@200/disk@0
local-mac-address?=true
Vldcc: vldcc0 [Domain Services]
service: primary-vldc0 @ primary
[LDC: 0x0]
Vnet: vnet0 [LDC: 0x2]
mac-addr=0:14:4f:fb:c4:ef
service: primary-vsw0 @ primary
[LDC: 0x1]
Vnet: vnet1 [LDC: 0xd]
mac-addr=0:14:4f:fb:24:b6
service: primary-vsw2 @ primary
[LDC: 0xc]
Vdisk: vdisk1 vdisk1@primary-vds0
service: primary-vds0 @ primary
[LDC: 0x17]
Vdisk: vdisk2 vdisk2@primary-vds0
service: primary-vds0 @ primary
[LDC: 0x18]
Vcons: [via LDC:25]
ldom1@primary-vcc0 [port:5000]
As you can see, everything that's been previously configured is listed. Some important things to note are the MAC addresses for the network interfaces (which are assigned automatically) and the Vcons port for the console. So now we can jumpstart our domain:
# telnet localhost 5000
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
Connecting to console "ldom1" in group "ldom1" ....
Press ~? for control options ..
Sun Fire T200, No Keyboard
Copyright 2007 Sun Microsystems, Inc. All rights reserved.
OpenBoot 4.26.0.build_07, 4096 MB memory available, Serial #66831599.
Ethernet address 0:14:4f:fb:c4:ef, Host ID: 83fbc4ef.
{0} ok show-nets
a) /virtual-devices@100/channel-devices@200/network@1
b) /virtual-devices@100/channel-devices@200/network@0
q) NO SELECTION
Enter Selection, q to quit: a
/virtual-devices@100/channel-devices@200/network@1 has been selected.
Type ^Y ( Control-Y ) to insert it in the command line.
e.g. ok nvalias mydev ^Y
for creating devalias mydev for /virtual-devices@100/channel-devices@200/network@1
{0} ok boot /virtual-devices@100/channel-devices@200/network@1 - install
Boot device: /virtual-devices@100/channel-devices@200/network@1 File and args:
- install
Requesting Internet Address for 0:14:4f:fb:24:b6
Requesting Internet Address for 0:14:4f:fb:24:b6
Requesting Internet Address for 0:14:4f:fb:24:b6
Requesting Internet Address for 0:14:4f:fb:24:b6
Requesting Internet Address for 0:14:4f:fb:24:b6
Requesting Internet Address for 0:14:4f:fb:24:b6
Requesting Internet Address for 0:14:4f:fb:24:b6
SunOS Release 5.10 Version Generic_118833-33 64-bit
Copyright 1983-2006 Sun Microsystems, Inc. All rights reserved.
Use is subject to license terms.
WARNING: machine_descrip_update: new MD has the same generation (1) as the old MD
whoami: no domain name
Configuring devices.
Using RPC Bootparams for network configuration information.
Attempting to configure interface vnet1...
Configured interface vnet1
Attempting to configure interface vnet0...
Skipped interface vnet0
Setting up Java. Please wait...
Extracting windowing system. Please wait...
Beginning system identification...
Searching for configuration file(s)...
...
So after the guest domain is finished jumpstarting, we can take a look around.
# psrinfo -vp
The physical processor has 4 virtual processors (0-3)
UltraSPARC-T1 (cpuid 0 clock 1000 MHz)
# psrinfo -v
Status of virtual processor 0 as of: 04/05/2007 22:17:04
on-line since 04/05/2007 22:16:15.
The sparcv9 processor operates at 1000 MHz,
and has a sparcv9 floating point processor.
Status of virtual processor 1 as of: 04/05/2007 22:17:04
on-line since 04/05/2007 22:16:16.
The sparcv9 processor operates at 1000 MHz,
and has a sparcv9 floating point processor.
Status of virtual processor 2 as of: 04/05/2007 22:17:04
on-line since 04/05/2007 22:16:16.
The sparcv9 processor operates at 1000 MHz,
and has a sparcv9 floating point processor.
Status of virtual processor 3 as of: 04/05/2007 22:17:04
on-line since 04/05/2007 22:16:16.
The sparcv9 processor operates at 1000 MHz,
and has a sparcv9 floating point processor.
# prtdiag -v
System Configuration: Sun Microsystems sun4v Sun Fire T200
Memory size: 4096 Megabytes
========================= CPUs ===============================================
CPU CPU
Location CPU Freq Implementation Mask
------------ ----- -------- ------------------- -----
MB/CMP0/P0 0 1000 MHz SUNW,UltraSPARC-T1
MB/CMP0/P1 1 1000 MHz SUNW,UltraSPARC-T1
MB/CMP0/P2 2 1000 MHz SUNW,UltraSPARC-T1
MB/CMP0/P3 3 1000 MHz SUNW,UltraSPARC-T1
========================= IO Configuration =========================
IO
Location Type Slot Path Name Model
----------- ----- ---- --------------------------------------------- ------------------------- ---------
========================= HW Revisions =======================================
System PROM revisions:
----------------------
OBP 4.26.0.build_07 2007/02/14 19:20
IO ASIC revisions:
------------------
Location Path Device Revision
-------------------- ---------------------------------------- ------------------------------ ---------
# df -h
Filesystem size used avail capacity Mounted on
/dev/md/dsk/d0 7.8G 2.2G 5.5G 30% /
/devices 0K 0K 0K 0% /devices
ctfs 0K 0K 0K 0% /system/contract
proc 0K 0K 0K 0% /proc
mnttab 0K 0K 0K 0% /etc/mnttab
swap 5.1G 1.1M 5.1G 1% /etc/svc/volatile
objfs 0K 0K 0K 0% /system/object
/platform/SUNW,Sun-Fire-T200/lib/libc_psr/libc_psr_hwcap1.so.1
7.8G 2.2G 5.5G 30% /platform/sun4v/lib/libc_psr.so.1
/platform/SUNW,Sun-Fire-T200/lib/sparcv9/libc_psr/libc_psr_hwcap1.so.1
7.8G 2.2G 5.5G 30% /platform/sun4v/lib/sparcv9/libc_psr.so.1
fd 0K 0K 0K 0% /dev/fd
swap 1.6G 0K 1.6G 0% /tmp
swap 5.1G 32K 5.1G 1% /var/run
# metastat
d1: Mirror
Submirror 0: d11
State: Okay
Submirror 1: d21
State: Okay
Pass: 1
Read option: roundrobin (default)
Write option: parallel (default)
Size: 4194600 blocks (2.0 GB)
d11: Submirror of d1
State: Okay
Size: 4194600 blocks (2.0 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c0d0s1 0 No Okay No
d21: Submirror of d1
State: Okay
Size: 4194600 blocks (2.0 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c0d1s1 0 No Okay No
d0: Mirror
Submirror 0: d10
State: Okay
Submirror 1: d20
State: Okay
Pass: 1
Read option: roundrobin (default)
Write option: parallel (default)
Size: 16644000 blocks (7.9 GB)
d10: Submirror of d0
State: Okay
Size: 16644000 blocks (7.9 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c0d0s0 0 No Okay No
d20: Submirror of d0
State: Okay
Size: 16644000 blocks (7.9 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c0d1s0 0 No Okay No
Device Relocation Information:
Device Reloc Device ID
c0d1 No -
c0d0 No -
# ifconfig -a
lo0: flags=2001000849 mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
vnet0: flags=9040843 mtu 1500 index 2
inet 192.168.1.2 netmask ffffff00 broadcast 192.168.1.255
groupname ipmp1
ether 0:14:4f:fb:c4:ef
vnet0:1: flags=1000843 mtu 1500 index 2
inet 192.168.1.1 netmask ffffff00 broadcast 192.168.1.255
vnet1: flags=9040843 mtu 1500 index 3
inet 192.168.1.3 netmask ffffff00 broadcast 192.168.1.255
groupname ipmp1
ether 0:14:4f:fb:24:b6
# uptime
10:20pm up 5 min(s), 1 user, load average: 0.02, 0.11, 0.06
So now we have a guest domain running Solaris 10 Update 3, with SVM mirrored boot drives that are really sparse files, IPMP on virtual NICs, four CPU's, 4GB's RAM, etc
Posted In
LDOM
|
0
comments
|
Subscribe to:
Posts (Atom)
