Build a Guest Domain

Saturday, February 19, 2011 at 8:30 PM
A guest domain is made up of the following components:


CPU

MAU (Cryptographic Thread)
Memory
Networking
Storage
The control domain will partition CPU threads as VCPU's for the guest domain. Each CPU core has an MAU for cryptographic processing. Only one logical domain using the CPU threads in a core can have control over this thread. So it's important to decide if your guest domain will require one. Memory is partitioned in 8K segments. Networking is handled by connecting a virtual network interface to a virtual switch in one of the service domains. In our example, I configured each physical interface as a separate virtual switch in the control/service (a.k.a. primary) domain. Storage can come from a wide variety of sources:


Local Disk

SAN LUN
Virtual Disk Image File
Disk Slice
ZFS Volume
The T2000 for example has four drive bay that could be used, but obviously that doesn't leave us with a lot of flexibility or space. SAN storage can be used with greater flexibility since it's remote and can easily be migrated or replicated. It's possible to create a sparse file and use it as a virtual disk. This has the advantage of being stored on local, SAN, or even NAS. The fact that files can be used opens the door for very flexible options. Using a disk slice is also possible, but it can not be used for jumpstart installation. One could create ZFS volumes and use them as storage for logical domains as well. However, it can not be used for jumpstart installation. However, it makes for easy allocation of storage for applications. You can even take SAN LUN's and create a ZFS pool and export it into a logical domain. For our example, I'll use two virtual disk image files created on a ZFS file system and use SVM mirroring:) The following will be configured:


4 x VCPU's
1 x MAU
4GB's RAM

2 x 10GB Virtual Disk Image Files
2 x Network Ports


# ldm add-domain ldom1
# ldm add-vcpu 4 ldom1
# ldm add-mau 1 ldom1
# ldm add-memory 4G ldom1
# mkfile 10g /ldoms/vdisk1_10gb.img
# mkfile 10g /ldoms/vdisk2_10gb.img
# ldm add-vdiskserverdevice /ldoms/vdisk1_10gb.img vdisk1@primary-vds0
# ldm add-vdiskserverdevice /ldoms/vdisk2_10gb.img vdisk2@primary-vds0
# ldm add-vdisk vdisk1 vdisk1@primary-vds0 ldom1
# ldm add-vdisk vdisk2 vdisk2@primary-vds0 ldom1
# ldm add-vnet vnet0 primary-vsw0 ldom1
# ldm add-vnet vnet1 primary-vsw2 ldom1
# ldm set-variable auto-boot\?=false ldom1
# ldm set-variable local-mac-address\?=true ldom1
# ldm set-variable boot-device=/virtual-devices@100/channel-devices@200/disk@0 ldom1
# ldm bind-domain ldom1
# ldm start ldom1


So with the above commands we allocated the vcpu's, mau, and the memory. Then created the virtual disk images files, added them as virtual disk devices to the primary domain's VDS service, and finally added them as virtual disks to the guest domain. Then attached virtual networks, set auto-boot to false in the OBP(yes that's right each logical domain gets its own OBP), set local-mac-address to true, and set the default boot device. Finally we've bound the configuration and started the guest domain. So what do we get?


# ldm list-bindings ldom1
Name: ldom1
State: active
Flags: transition
OS:
Util: 0.2%
Uptime: 1d 6h 43m
Vcpu: 4
vid pid util strand
0 4 0.7% 100%
1 5 0.1% 100%
2 6 0.1% 100%
3 7 0.0% 100%
Mau: 1
mau cpuset (4, 5, 6, 7)
Memory: 4G
real-addr phys-addr size
0x4800000 0x104800000 4G
Vars: auto-boot?=false
boot-device=/virtual-devices@100/channel-devices@200/disk@0
local-mac-address?=true
Vldcc: vldcc0 [Domain Services]
service: primary-vldc0 @ primary
[LDC: 0x0]
Vnet: vnet0 [LDC: 0x2]
mac-addr=0:14:4f:fb:c4:ef
service: primary-vsw0 @ primary
[LDC: 0x1]
Vnet: vnet1 [LDC: 0xd]
mac-addr=0:14:4f:fb:24:b6
service: primary-vsw2 @ primary
[LDC: 0xc]
Vdisk: vdisk1 vdisk1@primary-vds0
service: primary-vds0 @ primary
[LDC: 0x17]
Vdisk: vdisk2 vdisk2@primary-vds0
service: primary-vds0 @ primary
[LDC: 0x18]
Vcons: [via LDC:25]
ldom1@primary-vcc0 [port:5000]


As you can see, everything that's been previously configured is listed. Some important things to note are the MAC addresses for the network interfaces (which are assigned automatically) and the Vcons port for the console. So now we can jumpstart our domain:


# telnet localhost 5000
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.

Connecting to console "ldom1" in group "ldom1" ....
Press ~? for control options ..

Sun Fire T200, No Keyboard
Copyright 2007 Sun Microsystems, Inc. All rights reserved.
OpenBoot 4.26.0.build_07, 4096 MB memory available, Serial #66831599.
Ethernet address 0:14:4f:fb:c4:ef, Host ID: 83fbc4ef.



{0} ok show-nets
a) /virtual-devices@100/channel-devices@200/network@1
b) /virtual-devices@100/channel-devices@200/network@0
q) NO SELECTION
Enter Selection, q to quit: a
/virtual-devices@100/channel-devices@200/network@1 has been selected.
Type ^Y ( Control-Y ) to insert it in the command line.
e.g. ok nvalias mydev ^Y
for creating devalias mydev for /virtual-devices@100/channel-devices@200/network@1
{0} ok boot /virtual-devices@100/channel-devices@200/network@1 - install
Boot device: /virtual-devices@100/channel-devices@200/network@1 File and args:
- install
Requesting Internet Address for 0:14:4f:fb:24:b6
Requesting Internet Address for 0:14:4f:fb:24:b6
Requesting Internet Address for 0:14:4f:fb:24:b6
Requesting Internet Address for 0:14:4f:fb:24:b6
Requesting Internet Address for 0:14:4f:fb:24:b6
Requesting Internet Address for 0:14:4f:fb:24:b6
Requesting Internet Address for 0:14:4f:fb:24:b6
SunOS Release 5.10 Version Generic_118833-33 64-bit
Copyright 1983-2006 Sun Microsystems, Inc. All rights reserved.
Use is subject to license terms.
WARNING: machine_descrip_update: new MD has the same generation (1) as the old MD
whoami: no domain name
Configuring devices.
Using RPC Bootparams for network configuration information.
Attempting to configure interface vnet1...
Configured interface vnet1
Attempting to configure interface vnet0...
Skipped interface vnet0
Setting up Java. Please wait...
Extracting windowing system. Please wait...
Beginning system identification...
Searching for configuration file(s)...
...
So after the guest domain is finished jumpstarting, we can take a look around.


# psrinfo -vp
The physical processor has 4 virtual processors (0-3)
UltraSPARC-T1 (cpuid 0 clock 1000 MHz)
# psrinfo -v
Status of virtual processor 0 as of: 04/05/2007 22:17:04
on-line since 04/05/2007 22:16:15.
The sparcv9 processor operates at 1000 MHz,
and has a sparcv9 floating point processor.
Status of virtual processor 1 as of: 04/05/2007 22:17:04
on-line since 04/05/2007 22:16:16.
The sparcv9 processor operates at 1000 MHz,
and has a sparcv9 floating point processor.
Status of virtual processor 2 as of: 04/05/2007 22:17:04
on-line since 04/05/2007 22:16:16.
The sparcv9 processor operates at 1000 MHz,
and has a sparcv9 floating point processor.
Status of virtual processor 3 as of: 04/05/2007 22:17:04
on-line since 04/05/2007 22:16:16.
The sparcv9 processor operates at 1000 MHz,
and has a sparcv9 floating point processor.
# prtdiag -v
System Configuration: Sun Microsystems sun4v Sun Fire T200
Memory size: 4096 Megabytes

========================= CPUs ===============================================

CPU CPU
Location CPU Freq Implementation Mask
------------ ----- -------- ------------------- -----
MB/CMP0/P0 0 1000 MHz SUNW,UltraSPARC-T1
MB/CMP0/P1 1 1000 MHz SUNW,UltraSPARC-T1
MB/CMP0/P2 2 1000 MHz SUNW,UltraSPARC-T1
MB/CMP0/P3 3 1000 MHz SUNW,UltraSPARC-T1


========================= IO Configuration =========================

IO
Location Type Slot Path Name Model
----------- ----- ---- --------------------------------------------- ------------------------- ---------

========================= HW Revisions =======================================

System PROM revisions:
----------------------
OBP 4.26.0.build_07 2007/02/14 19:20

IO ASIC revisions:
------------------
Location Path Device Revision
-------------------- ---------------------------------------- ------------------------------ ---------
# df -h
Filesystem size used avail capacity Mounted on
/dev/md/dsk/d0 7.8G 2.2G 5.5G 30% /
/devices 0K 0K 0K 0% /devices
ctfs 0K 0K 0K 0% /system/contract
proc 0K 0K 0K 0% /proc
mnttab 0K 0K 0K 0% /etc/mnttab
swap 5.1G 1.1M 5.1G 1% /etc/svc/volatile
objfs 0K 0K 0K 0% /system/object
/platform/SUNW,Sun-Fire-T200/lib/libc_psr/libc_psr_hwcap1.so.1
7.8G 2.2G 5.5G 30% /platform/sun4v/lib/libc_psr.so.1
/platform/SUNW,Sun-Fire-T200/lib/sparcv9/libc_psr/libc_psr_hwcap1.so.1
7.8G 2.2G 5.5G 30% /platform/sun4v/lib/sparcv9/libc_psr.so.1
fd 0K 0K 0K 0% /dev/fd
swap 1.6G 0K 1.6G 0% /tmp
swap 5.1G 32K 5.1G 1% /var/run
# metastat
d1: Mirror
Submirror 0: d11
State: Okay
Submirror 1: d21
State: Okay
Pass: 1
Read option: roundrobin (default)
Write option: parallel (default)
Size: 4194600 blocks (2.0 GB)

d11: Submirror of d1
State: Okay
Size: 4194600 blocks (2.0 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c0d0s1 0 No Okay No


d21: Submirror of d1
State: Okay
Size: 4194600 blocks (2.0 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c0d1s1 0 No Okay No


d0: Mirror
Submirror 0: d10
State: Okay
Submirror 1: d20
State: Okay
Pass: 1
Read option: roundrobin (default)
Write option: parallel (default)
Size: 16644000 blocks (7.9 GB)

d10: Submirror of d0
State: Okay
Size: 16644000 blocks (7.9 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c0d0s0 0 No Okay No


d20: Submirror of d0
State: Okay
Size: 16644000 blocks (7.9 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c0d1s0 0 No Okay No


Device Relocation Information:
Device Reloc Device ID
c0d1 No -
c0d0 No -
# ifconfig -a
lo0: flags=2001000849 mtu 8232 index 1
inet 127.0.0.1 netmask ff000000
vnet0: flags=9040843 mtu 1500 index 2
inet 192.168.1.2 netmask ffffff00 broadcast 192.168.1.255
groupname ipmp1
ether 0:14:4f:fb:c4:ef
vnet0:1: flags=1000843 mtu 1500 index 2
inet 192.168.1.1 netmask ffffff00 broadcast 192.168.1.255
vnet1: flags=9040843 mtu 1500 index 3
inet 192.168.1.3 netmask ffffff00 broadcast 192.168.1.255
groupname ipmp1
ether 0:14:4f:fb:24:b6
# uptime
10:20pm up 5 min(s), 1 user, load average: 0.02, 0.11, 0.06

So now we have a guest domain running Solaris 10 Update 3, with SVM mirrored boot drives that are really sparse files, IPMP on virtual NICs, four CPU's, 4GB's RAM, etc

0 comments

Post a Comment

Solaris | Powered by Blogger | Entries (RSS) | Comments (RSS) | Designed by MB Web Design | XML Coded By Cahayabiru.com