Cloning a virtual machine or user as it is called in z/VM terms from within another guest/user running Linux can be used in a lot of different scenario’s. Especially when the people managing the z/VM platform and virtual machines are less experienced with z/VM, it could be good to let them create a clone without the need to access a 3270 terminal. This scenario can also be used to let users do some self service and provide them with new instances without intervention.
This post will describe how to create a new z/VM user and provide it with the data from another instance, customize it and prepare it to be bootable. All of this should be done from a Linux guest running on the sam z/VM environment without the need to open a terminal or have any deep knowledge of z/VM.
For this explanation, I was using a guest (or user as it should be called in z/VM terms) is running Linux for zSeries (zLinux as it once was) on a z/VM host. I was using SLES11 running on z/VM 6.3. The system from where I will execute all steps is called the cloner. As a source for the new VM, I will use the volume that was the result of a minimal installation of SLES11 (Suse Linux Enterprise System). This source needs to be created once and can be used as much as you want to generate different clones.
Connect the cloner to the FCP-device of the z/VM machine
This first step is to connect the cloner to the FCP-devices on the z/VM host. In my example, I’m using different devices to connect to the LUN devices. Device 6022 will be used to connect to the source LUN (the result of the minimal install) and FCP device 6023 will be used to connect to the target LUN (the disk that will be used by the new VM).
Attaching devices requires the necessary privileges on the z/VM environment. To be sure that we can communicate from the guest (user) to the host (CMS), we need to make sure that kernel module vmcp is loaded in the cloner:
cloner:/ # modprobe vmcp
After loading vmcp, we can send commands to the z/VM host and attach two FCP-devices to our cloner. Cloner is the hostname of the Linux VM and also the z/VM user’s name.
cloner :/ # vmcp att 6022 cloner cloner :/ # vmcp att 6023 cloner
After attaching the FCP devices, we need to put them online and we can check if the device correspondends with the correct WWPN’s of the SAN:
cloner:/ # echo "1">/sys/bus/ccw/drivers/zfcp/0.0.6022/online cloner:/ # echo "1">/sys/bus/ccw/drivers/zfcp/0.0.6023/online cloner:/ # lszfcp -H -P 0.0.6022 host3 0.0.6023 host2 0.0.800f host0 0.0.800f/0x50050768023595b2 rport-0:0-0 0.0.800f/0x50050768023595b1 rport-0:0-1 0.0.6023/0x50050768021595b1 rport-2:0-0 0.0.6023/0x50050768021595b2 rport-2:0-1 0.0.6022/0x50050768021595b2 rport-3:0-0 0.0.6022/0x50050768021595b1 rport-3:0-1
As you can see, the cloner was already connected to FCP device 8000F and now we can also use devices 6022 and 6023 to connect the cloner to the source and target LUN’s.
Connect the cloner to the source and target LUN
Now that we have access to a few FCP-devices, we can connect our cloner to the source and target LUN. The source LUN contains a minimal SLES install (installed with the GUI and bootable from it’s own device, configured with a fixed IP). The target LUN is a a new LUN of the same size.
- LUN 0x003c000000000000 is the source for our clone
- LUN 0x003d000000000000 is the target for our clone
Connect to the LUN using the previously connected FCP adapters:
cloner:/ # echo "0x003c000000000000">/sys/bus/ccw/drivers/zfcp/0.0.6022/0x50050768021595b1/unit_add cloner:/ # echo "0x003d000000000000">/sys/bus/ccw/drivers/zfcp/0.0.6023/0x50050768021595b1/unit_add cloner:/ # lszfcp -D 0.0.6022/0x50050768021595b1/0x003c000000000000 3:0:1:60 0.0.6023/0x50050768021595b1/0x003d000000000000 2:0:0:61 0.0.800f/0x50050768023595b2/0x003b000000000000 0:0:0:59
In the above output we can see that the LUN’s are connected to the system including the disk of the cloner itself.
To use the LUN’s on Linux, we need to know the device name in /dev. We can find these with lsscsi:
cloner:/ # lsscsi [0:0:0:59] disk IBM 2145 0000 /dev/sda [2:0:0:61] disk IBM 2145 0000 /dev/sdd [3:0:1:60] disk IBM 2145 0000 /dev/sdc
Clone the data from source to target
In the previous step, we connected the source and target LUN to the cloner and got to know the device names that represent those LUN’s. The next step is to clone the data from our source LUN to the target LUN:
cloner:/ # dd if=/dev/sdc of=/dev/sdd bs=4096 conv=notrunc,noerror 2621440+0 records in 2621440+0 records out 10737418240 bytes (11 GB) copied, 55.4229 s, 194 MB/s
After cloning the data, we need to inform the cloner that we have a new partition configuration since we cloned the partitions from the source LUN to the target LUN
cloner:~ # partprobe
Customize the data of the clone
In the previous step we cloned the filesystem of another Linux instance so we need to customize the data on the cloned filesystem in order to make it bootable from another FCP device and LUN and to not interfere with the original system.
To customize the cloned filesystem, we’ll mount the cloned filesystem on /mnt/clone and adjust some files:
cloner:/ # mkdir /mnt/clone cloner:/ # mount /dev/sdd2 /mnt/clone
Change the network configuration and hostname:
cloner:/ # vi /mnt/clone/etc/sysconfig/network/ifcfg-eth0 cloner:/ # vi /mnt/clone/etc/HOSTNAME
Change the IPL configuration to include the correct FCP-device and new LUN. The /dev/disk/by-path/ccw- corresponds to the portname and LUN which can be found with lszfcp -D for FCP-device 6023.
cloner:~ # vi /mnt/clone/etc/zipl.conf
[defaultboot] default=SLES11_SP3 [SLES11_SP3] image = /boot/image-3.0.76-0.11-default target = /boot/zipl ramdisk = /boot/initrd-3.0.76-0.11-default,0x2000000 parameters = "root=/dev/disk/by-path/ccw-0.0.6023-zfcp-0x50050768021595b1:0x003d000000000000-part2 hvc_iucv=8 TERM=dumb resume=/dev/disk/by-path/ccw-0.0.6023-zfcp-0x50050768021595b1:0x003d000000000000-part2"
Now all file adjustments have been done and we need to generate a new initrd and write a new bootsector that includes the correct FCP-device and LUN:
Let’s chroot into the cloned filesystem:
cloner:/ # mount --bind /proc /mnt/clone/proc cloner:/ # mount --bin /dev /mnt/clone/dev cloner:/ # chroot /mnt/clone
Re-create the initrd:
cloner:/ # mkinitrd Kernel image: /boot/image-3.0.76-0.11-default Initrd image: /boot/initrd-3.0.76-0.11-default Root device: /dev/disk/by-id/scsi-360050768028087ffd800000000000041-part2 (/dev/sdd2) (mounted on / as ext3) Resume device: /dev/system/swap Kernel Modules: dm-mod dm-log dm-region-hash dm-mirror dm-snapshot scsi_mod scsi_dh scsi_dh_hp_sw scsi_dh_alua scsi_dh_emc scsi_dh_rdac mbcache jbd ext3 scsi_tgt scsi_transport_fc qdio zfcp linear crc-t10dif sd_mod Features: dm block zfcp lvm2 resume.userspace resume.kernel 41247 blocks
Re-create the bootsector using zipl:
cloner:/ # zipl -V Using config file '/etc/zipl.conf' Target device information Device..........................: 08:30 Partition.......................: 08:32 Device name.....................: sdd Device driver name..............: sd Type............................: disk partition Disk layout.....................: SCSI disk layout Geometry - start................: 1525760 File system block size..........: 4096 Physical block size.............: 512 Device size in physical blocks..: 19445760 Building bootmap in '/boot/zipl' Adding IPL section 'SLES11_SP3' (default) initial ramdisk...: /boot/initrd-3.0.76-0.11-default kernel image......: /boot/image-3.0.76-0.11-default kernel parmline...: 'root=/dev/disk/by-path/ccw-0.0.6023-zfcp-0x50050768021595b1:0x003d000000000000-part2 hvc_iucv=8 TERM=dumb resume=/dev/disk/by-path/ccw-0.0.6023-zfcp-0x50050768021595b1:0x003d000000000000-part2' component address: kernel image....: 0x00010000-0x008a6dff parmline........: 0x008a7000-0x008a71ff initial ramdisk.: 0x02000000-0x028237ff internal loader.: 0x0000a000-0x0000a3ff Preparing boot device: sdd. Detected SCSI PCBIOS disk layout. Writing SCSI master boot record. Syncing disks... Done.
At this stage, the filesystem is prepared and should be bootable from the changed LUN and FCP-device.
Let’s leave the chroot and cleanup mounts and devices:
cloner:/ # exit cloner:~ # umount /mnt/clone/dev cloner:~ # umount /mnt/clone/proc cloner:~ # umount /mnt/clone cloner:/ # echo "0">/sys/bus/ccw/drivers/zfcp/0.0.6023/online cloner:/ # vmcp det 6023 cloner
Create a new z/VM user
The next thing to do is to create a new user (or VM) on the z/VM environment. In a previous post, I described how you can access CMS-files from a Linux guest running on z/VM. This technique will be used to create the user and a PROFILE.EXEC for that new user. More information here: Read and write a CMS minidisk on a z/VM system from a user/guest running Linux.
First, we need to load the kernel module fuse to be able to access the CMS-files:
cloner:~ # modprobe fuse
The next steps are to connect to minidisk 191 owned by user cloner, which is running our current Linux system. On the minidisk, we need to write a file that contains the configuration of the new z/VM user. Then, we’ll use dirmaint to create that new user on the z/VM machine. The new user will be called newvm.
Put minidisk 191 online and mount it:
cloner:~ # chccwdev -e 191 Setting device 0.0.0191 online Done cloner:~ # lsdasd 191 Bus-ID Status Name Device Type BlkSz Size Blocks ============================================================================== 0.0.0191 active dasda 94:0 FBA 512 0MB 1000 cloner:~ # mkdir /mnt/mdisk cloner:~ # cmsfs-fuse --ascii /dev/dasda /mnt/mdisk
Create a new, empty file and set it’s attributes to FB 80:
cloner:~ # touch /mnt/mdisk/NEWVM.DIRECT cloner:~ # setfattr -n user.record_format -v F /mnt/mdisk/NEWVM.DIRECT cloner:~ # setfattr -n user.record_lrecl -v 80 /mnt/mdisk/NEWVM.DIRECT
Create a temporary file and add the configuration for the new user to the NEVWM.DIRECT file. Make sure that the configuration contains the correct device numbers for your configuration. Device 6023 corresponds to the FCP-adapter which I put in the zipl configuration and which will be used from the newly generated bootloader:
USER NEWVM P4SSWD 512M 1G G COMMAND define storage 512M standby 512M COMMAND SET RUN ON CPU 00 BASE CPU 01 CPU 02 CPU 03 IPL CMS PARM AUTOCR CONSOLE 0009 3215 NICDEF 7000 TYPE QDIO LAN SYSTEM VSW1 SPOOL 000C 2540 READER * SPOOL 000D 2540 PUNCH A SPOOL 000E 1403 A LINK MAINT 0190 0190 RR LINK MAINT 019D 019D RR LINK MAINT 019E 019E RR AMDISK 191 FB-512 AUTOG 1000 USER M BLKSIZE 4096 DEDICATE 6023 6023
I created the file in /tmp/newvm.direct so it needs to be copied to the cloner’s 191 minidisk. It’s important to not directly edit the file on the minidisk but copy it after changing. See the post about writing to minidisks from Linux to find out more about this.
cloner:~ # cp /tmp/newvm.direct /mnt/mdisk/NEWVM.DIRECT
The next step is to use the newly created file to actually create the user using dirmaint. The rest of the user creation get’s handled by dirmaint (that’s what it’s made for):
cloner :~ # vmcp smsg vmutil dirm add NEWVM
At this time, the new z/VM user gets created. This is an asynchronous proces so it’s hard to tell when the creation is completed.
Create a PROFILE.EXEC for the new z/VM user
Once the user has been created, we need to create a 191-minidisk and a PROFILE.EXEC for the new user containing the correct boot-device in order to easily IPL that new user.
First link the 191-minidisk of the new user NEWVM to 291 of the cloner:
cloner:~ # vmcp link NEWVM 191 291 m
Put minidisk 291 online in the cloner and mount it:
cloner:~ # chccwdev -e 291 Setting device 0.0.0291 online Done cloner:~ # lsdasd 291 Bus-ID Status Name Device Type BlkSz Size Blocks ============================================================================== 0.0.0291 active dasdb 97:0 FBA 512 0MB 1000 cloner:~ # mkdir /mnt/mdisk2 cloner:~ # cmsfs-fuse --ascii /dev/dasdb /mnt/mdisk2
Create a new PROFILE.EXEC on the 291 minidisk and set the correct attributes for the file:
cloner:~ # touch /mnt/mdisk2/PROFILE.EXEC cloner:~ # setfattr -n user.record_format -v F /mnt/mdisk2/PROFILE.EXEC cloner:~ # setfattr -n user.record_lrecl -v 80 /mnt/mdisk2/PROFILE.EXEC
Create a temporary file and write the following to the file. The LOADDEV PORTNAME and LUN corresponds to the portname which can be found with lszfcp -D for FCP-device 6023.
/***************************/ /* LINUX PROFILE EXEC */ /***************************/ ADDRESS COMMAND 'CP TERMINAL MODE VM' 'CP SPOOL CONSOLE * START' 'CP SET PF11 RETRIEVE FORWARD' 'CP SET PF12 RETRIEVE BACKWARD' 'CP SET PF23 RETRIEVE FORWARD' 'CP SET PF24 RETRIEVE BACKWARD' 'CP LINK TCPMAINT 592 592 RR' 'ACCESS 592 E' 'CP SET LOADDEV PORTNAME 50050768 021595b1 LUN 003d0000 00000000' '============================' 'ENTER IPL 6023 TO BOOT LINUX' '============================'
Now copy the temporary file containing the user’s PROFILE.EXEC, which we just created, to the mounted 291 minidisk:
cloner:~ # cp /tmp/newvm.profexec /mnt/mdisk2/PROFILE.EXEC
After these steps, the system is ready to boot but first we should cleanup our devices and mounts:
cloner:~ # fusermount -u /mnt/mdisk cloner:~ # fusermount -u /mnt/mdisk2 cloner:~ # chccwdev -d 191 cloner:~ # chccwdev -d 291 cloner:~ # vmcp det 291
This should be all it takes to clone a system. You could repeat these steps multiple times, using a different FPC device, different LUN and different IP address. After the clone, logon to the newly created user with the password written in the NEWVM.DIRECT file and enter IPL 6023 to start the clone. With a little bit of trying, this should be possible from the cloner as well.