Create a new virtual machine in Vsphere with Python, Pysphere and the VMWare API

The VMWare API is very extensive and allows you to do almost all operations that are possible with VMWare using API calls. In order to be able to easily create and deploy new virtual machines, it can be a good idea to standardize and create VM’s using a (Python) script that calls the API. In this post, I will give some examples on how to easily create a new VM using Pysphere and the VMWare API.

In order to create a new VM on a fast and standardized way, I figured it would be a good idea to create a script that calls the VMWare API and executes the actions that I normally do manually via the GUI. For example to enable hot add cpu and memory. The script uses pysphere, documentation can be found here: https://code.google.com/p/pysphere/ and the VMWare API, documentation can be found here: http://pubs.vmware.com/vsphere-55/index.jsp#com.vmware.wssdk.apiref.doc/right-pane.html

Preparation to run the script

In order to use pysphere, we need to install the Pysphere library. The easiest way is to do this with PIP:

[jensd@cen ~]$ sudo yum install python-pip
...
Complete !
[jensd@cen ~]$ sudo pip install pysphere
Downloading/unpacking pysphere
  Downloading pysphere-0.1.8.zip (538kB): 538kB downloaded
  Running setup.py egg_info for package pysphere

Installing collected packages: pysphere
  Running setup.py install for pysphere

Successfully installed pysphere
Cleaning up...

In order to simplify my scripts that were using Pysphere, I created a file that can be included into other scripts. The file contains function definitions related to Pysphere. Create a file vm_include.py and copy the following contents to the file or download it here: http://jensd.be/download/vm_include.py

#!/usr/bin/python
from pysphere import VIServer, VIProperty, MORTypes
from pysphere.resources import VimService_services as VI
from pysphere.vi_task import VITask

def connectToHost(host,host_user,host_pw):
    #create server object
    s=VIServer()
    #connect to the host
    try:
        s.connect(host,host_user,host_pw)
        return s
    except VIApiException, err:
        print "Cannot connect to host: "+host+" error message: "+err

def createGuest(host_con,guest_dc,guest_host,guest_name,guest_ver,guest_mem,guest_cpu,guest_iso,guest_os,guest_disk_gb,guest_ds,guest_network,guest_enterbios):
    #get dc MOR from list
    dc_list=[k for k,v in host_con.get_datacenters().items() if v==guest_dc]
    if dc_list:
        dc_mor=dc_list[0]
    else:
        host_con.disconnect()
        return "Cannot find dc: "+guest_dc
    dc_props=VIProperty(host_con, dc_mor)
    #get vmFolder
    vmf_mor = dc_props.vmFolder._obj
    #get hostfolder MOR
    hf_mor=dc_props.hostFolder._obj
    #get computer resources MORs
    cr_mors=host_con._retrieve_properties_traversal(property_names=['name','host'],from_node=hf_mor,obj_type='ComputeResource')
    #get host MOR
    try:
        host_mor=[k for k,v in host_con.get_hosts().items() if v==guest_host][0]
    except IndexError, e:
        host_con.disconnect()
        return "Cannot find host: "+guest_host
    #get computer resource MOR for host
    cr_mor=None
    for cr in cr_mors:
        if cr_mor:
            break
        for p in cr.PropSet:
            if p.Name=="host":
                for h in p.Val.get_element_ManagedObjectReference():
                    if h==host_mor:
                         cr_mor=cr.Obj
                         break
                if cr_mor:
                    break
    cr_props=VIProperty(host_con,cr_mor)
    #get resource pool MOR
    rp_mor=cr_props.resourcePool._obj

    #build guest properties
    #get config target
    request=VI.QueryConfigTargetRequestMsg()
    _this=request.new__this(cr_props.environmentBrowser._obj)
    _this.set_attribute_type(cr_props.environmentBrowser._obj.get_attribute_type())
    request.set_element__this(_this)
    h=request.new_host(host_mor)
    h.set_attribute_type(host_mor.get_attribute_type())
    request.set_element_host(h)
    config_target=host_con._proxy.QueryConfigTarget(request)._returnval
    #get default devices
    request=VI.QueryConfigOptionRequestMsg()
    _this=request.new__this(cr_props.environmentBrowser._obj)
    _this.set_attribute_type(cr_props.environmentBrowser._obj.get_attribute_type())
    request.set_element__this(_this)
    h=request.new_host(host_mor)
    h.set_attribute_type(host_mor.get_attribute_type())
    request.set_element_host(h)
    config_option=host_con._proxy.QueryConfigOption(request)._returnval
    defaul_devs=config_option.DefaultDevice
    #get network names
    if guest_network:
        net_name=guest_network
    else:
        for net in config_target.Network:
            if net.Network.Accessible:
                net_name = net.Network.Name
    #get ds
    ds_target = None
    for d in config_target.Datastore:
        if d.Datastore.Accessible and (guest_ds and d.Datastore.Name==guest_ds) or (not guest_ds):
            ds_target=d.Datastore.Datastore
            guest_ds=d.Datastore.Name
            break
    if not ds_target:
        host_con.disconnect()
        return "Cannot find datastore: "+guest_ds
    ds_vol_name="[%s]" % guest_ds

    #create task request
    create_vm_request=VI.CreateVM_TaskRequestMsg()
    config=create_vm_request.new_config()
    #set location of vmx
    vm_files=config.new_files()
    vm_files.set_element_vmPathName(ds_vol_name)
    config.set_element_files(vm_files)
    if guest_enterbios:
        #set boot parameters
        vmboot=config.new_bootOptions()
        vmboot.set_element_enterBIOSSetup(True)
        config.set_element_bootOptions(vmboot)
    #set general parameters
    config.set_element_version(guest_ver)
    config.set_element_name(guest_name)
    config.set_element_memoryMB(guest_mem)
    config.set_element_memoryHotAddEnabled(True)
    config.set_element_numCPUs(guest_cpu)
    config.set_element_guestId(guest_os)
    config.set_element_cpuHotAddEnabled(True)

    #create devices
    devices = []
    #add controller to devices
    disk_ctrl_key=1
    scsi_ctrl_spec=config.new_deviceChange()
    scsi_ctrl_spec.set_element_operation('add')
    scsi_ctrl=VI.ns0.ParaVirtualSCSIController_Def("scsi_ctrl").pyclass()
    scsi_ctrl.set_element_busNumber(0)
    scsi_ctrl.set_element_key(disk_ctrl_key)
    scsi_ctrl.set_element_sharedBus("noSharing")
    scsi_ctrl_spec.set_element_device(scsi_ctrl)
    devices.append(scsi_ctrl_spec)
    #find ide controller
    ide_ctlr = None
    for dev in defaul_devs:
        if dev.typecode.type[1] == "VirtualIDEController":
            ide_ctlr = dev
    #add cdrom
    if ide_ctlr:
        cd_spec = config.new_deviceChange()
        cd_spec.set_element_operation('add')
        cd_ctrl = VI.ns0.VirtualCdrom_Def("cd_ctrl").pyclass()
        cd_device_backing =VI.ns0.VirtualCdromIsoBackingInfo_Def("cd_device_backing").pyclass()
        ds_ref = cd_device_backing.new_datastore(ds_target)
        ds_ref.set_attribute_type(ds_target.get_attribute_type())
        cd_device_backing.set_element_datastore(ds_ref)
        cd_device_backing.set_element_fileName("%s %s" % (ds_vol_name,guest_iso))
        cd_ctrl.set_element_backing(cd_device_backing)
        cd_ctrl.set_element_key(20)
        cd_ctrl.set_element_controllerKey(ide_ctlr.get_element_key())
        cd_ctrl.set_element_unitNumber(0)
        cd_spec.set_element_device(cd_ctrl)
        devices.append(cd_spec)
    #add disk
    disk_spec=config.new_deviceChange()
    disk_spec.set_element_fileOperation("create")
    disk_spec.set_element_operation("add")
    disk_ctlr=VI.ns0.VirtualDisk_Def("disk_ctlr").pyclass()
    disk_backing=VI.ns0.VirtualDiskFlatVer2BackingInfo_Def("disk_backing").pyclass()
    disk_backing.set_element_fileName(ds_vol_name)
    disk_backing.set_element_diskMode("persistent")
    disk_ctlr.set_element_key(0)
    disk_ctlr.set_element_controllerKey(disk_ctrl_key)
    disk_ctlr.set_element_unitNumber(0)
    disk_ctlr.set_element_backing(disk_backing)
    guest_disk_size=guest_disk_gb*1024*1024
    disk_ctlr.set_element_capacityInKB(guest_disk_size)
    disk_spec.set_element_device(disk_ctlr)
    devices.append(disk_spec)
    #add a network controller
    nic_spec = config.new_deviceChange()
    if net_name:
        nic_spec.set_element_operation("add")
        nic_ctlr = VI.ns0.VirtualVmxnet3_Def("nic_ctlr").pyclass()
        nic_backing = VI.ns0.VirtualEthernetCardNetworkBackingInfo_Def("nic_backing").pyclass()
        nic_backing.set_element_deviceName(net_name)
        nic_ctlr.set_element_addressType("generated")
        nic_ctlr.set_element_backing(nic_backing)
        nic_ctlr.set_element_key(4)
        nic_spec.set_element_device(nic_ctlr)
        devices.append(nic_spec)

    #create vm request
    config.set_element_deviceChange(devices)
    create_vm_request.set_element_config(config)
    new_vmf_mor=create_vm_request.new__this(vmf_mor)
    new_vmf_mor.set_attribute_type(vmf_mor.get_attribute_type())
    new_rp_mor=create_vm_request.new_pool(rp_mor)
    new_rp_mor.set_attribute_type(rp_mor.get_attribute_type())
    new_host_mor=create_vm_request.new_host(host_mor)
    new_host_mor.set_attribute_type(host_mor.get_attribute_type())
    create_vm_request.set_element__this(new_vmf_mor)
    create_vm_request.set_element_pool(new_rp_mor)
    create_vm_request.set_element_host(new_host_mor)

    #finally actually create the guest :)
    task_mor=host_con._proxy.CreateVM_Task(create_vm_request)._returnval
    task=VITask(task_mor,host_con)
    task.wait_for_state([task.STATE_SUCCESS,task.STATE_ERROR])

    if task.get_state()==task.STATE_ERROR:
        return "Cannot create guest: "+task.get_error_message()
    else:
        return "Succesfully created guest: "+guest_name

def getMac(host_con,guest_name):
    vm=host_con.get_vm_by_name(guest_name)
    net = vm.get_property('net', from_cache=False)
    if net:
        for interface in net:
            mac = interface.get('mac_address', None)
            if mac:
                return mac

    for v in vm.get_property("devices").values():
        if v.get('macAddress'):
            return v.get('macAddress')

def powerOnGuest(host_con,guest_name):
    vm=host_con.get_vm_by_name(guest_name)
    vm.power_on()

Test the connection to Vsphere

Now we can start with a basic Pysphere example that will try to connect to a Vsphere cluster and list the type of connection.

Create host-info.py with the following contents:

#!/usr/bin/python
import vm_include

def main():
    #change these to match your installation
    host="yourvspherehost"
    user="root"
    pw="vmware"

    #connect to the host
    hostcon=vm_include.connectToHost(host,user,pw)

    #list server type
    print "Type:",hostcon.get_server_type()

    #disconnect from the host
    hostcon.disconnect()

if __name__ == '__main__':
        main()

Edit the host, user and password on line 6, 7 and 8 to match your installation.

The above script should return the type of the host (ESX or Vsphere) and the names of the VM’s that are running on it:

[jensd@cen ~]$ chmod +x host-info.py
[jensd@cen ~]$ ./host-info.py
Type: VMware vCenter Server

Create a new VM

The next step is to create a script that will actually add a VM to this host. To do so, copy and edit the contents of the following example to create-vm.py:

#!/usr/bin/python
import vm_include

def main():
    #connection properties
    #change these to match your installation
    host="yourvspherehost"
    host_user="root"
    host_pw="vmware"

    #properties of the new VM:
    guest_name="newvm"          #name of the VM
    guest_mem=1024              #memory in MB
    guest_cpu=1                 #number of virtual CPU
    guest_space=2               #space in GB
    datastore="non-ssd"         #name of the datastore
    esx_host="10.0.0.1"         #specific host in the cluster
    guest_dc="testdc"           #datacenter name
    guest_ver="vmx-08"          #version of VMX (v8 is editable via the client)
    guest_iso=""                #iso to mount (from datastore)
    guest_os="rhel6Guest"       #guest-template
    guest_network="VM Network"  #network-name
    guest_enterbios=False

    #connect to the host
    host_con=vm_include.connectToHost(host,host_user,host_pw)

    #create the new VM
    res=vm_include.createGuest(host_con,guest_dc,esx_host,guest_name,guest_ver,guest_mem,guest_cpu,guest_iso,guest_os,guest_space,datastore,guest_network,guest_enterbios)
    print "Result:",res

    #start the new VM
    vm_include.powerOnGuest(host_con,guest_name)

    #disconnect from host
    host_con.disconnect()

if __name__ == '__main__':
        main()

Edit the host, user and password on line 7, 8 and 9 to match your installation.

When executing the script, the new VM should be created as we defined and after creating, it should be started.

[jensd@cen ~]$ chmod +x create-vm.py
[jensd@cen ~]$ ./create-vm.py
Result: Succesfully created guest: newvm

When looking at the inventory in the vSphere client, we can see that these action were performed as we expected:

As you can see, it’s very easy to get started to create VM’s from a script language like Python. This example on itself isn’t very ground-breaking, it’s what you do in combination with the script that matters.

For example, in order to create multiple new VM’s, I combine the above with a script that takes a YAML file as input. The input contains a list of specifications for the new machines. When executed, the script creates the VM’s, creates an entry in DNS for them, generates a root password and stores it in our password manager, gets the MAC of the newly created VM and PXE-boots the new VM with a kickstart file. As a result, it’s possible to deploy 100’s of VM’s including OS-installation and DNS-entry in minutes without any further action and even more important, without mistakes :)

More information about the scripts can be found on GitHub: https://github.com/jensdepuydt/python-hypervisor-api

38 thoughts on “Create a new virtual machine in Vsphere with Python, Pysphere and the VMWare API

  1. Hi Jens,
    Thanks for the script for creating vm using Pysphere API. It helped a lot.

    Since I am a newbie to Python & Pysphere API, I need help from you.
    I want to achieve build automation same way as you have mentioned in this post “Create a new virtual machine in Vsphere with Python, Pysphere and the VMWare API” (http://jensd.be/?p=370).

    I want combine the above script with a script that takes a YAML file as input. The input contains a list of specifications for the new machines. When executed, the script creates the VM’s, gets the MAC of the newly created VM and PXE-boots the new VM with a kickstart file.

    Can you provide me detailed steps to achieve this ?
    Thanks in advance.
    Parimal

  2. esx_host=”127.0.0.1″ … it says it cannot find it
    I’m guessing the esx_host is the machine on which I’m building the VM on?
    I also tried setting to the same IP that I’m connecting to.
    Thanks.

    • The esx_host is a specific host in the vSphere cluster.
      If you have a cluster that has 3 hosts, you need to determine on which of those 3 the VM needs to be deployed. it can be a name or IP.

  3. Thank you for this. I think you have a typo in the vm_include.py script:

    def createGuest(host_con,guest_dc,guest_host,guest_name,guest_ver,guest_mem,guest_cpu,guest_iso,guest_os,guest_disk_gb,guest_ds,guest_networkg,guest_enterbios):
    #get dc MOR from list

    The error doesn’t exist in the code on this page, but does exist in the code in this link here – http://jensd.be/download/vm_include.py

    typo is guest_networkg

    *You receive errors in the VM creation script regarding that variable not being globally defined.

  4. Hi Jens,
    I try to change the format of the disk to by replacing the line disk_backing=VI.ns0.VirtualDiskFlatVer2BackingInfo_Def(“disk_backing”).pyclass() by
    disk_backing=VI.ns0.VirtualDiskSparseVer2BackingInfo_Def(“disk_backing”).pyclass()

    But I have the error: Result: Cannot create guest: The device or operation specified at index ‘2’ is not supported for the existing virtual machine platform.

    Do you have any suggestion ?
    Thanks in advance
    Roch

    • Hi Jens,
      I found my error.
      To enable a disk with a thin provisionning type, I’ve just need to set the thinProvisioned( property, not to change to a VirtualDiskSparseVer2BackingInfo_Def disk that I think is used by VMware workstation.

      So, it works perfectly with
      disk_backing=VI.ns0.VirtualDiskFlatVer2BackingInfo_Def(“disk_backing”).pyclass()
      disk_backing.set_element_thinProvisioned(“True”)

  5. Hi,

    How can installation of OS on the newly created VM be handled through Python?

    Thanks,
    Prajakta

    • Hi,

      What I did at the time in my Python script was the following:
      – Create the VM using the scripts in this post
      – Once the VM is created, fetch back the MAC-address of the virtual NIC
      – Generate a root password
      – Generate a kickstart file (fill in some values in a template)
      – Generate a PXE-boot file supplying the kickstart file and name it with the MAC-address
      – Start the VM and let it PXE-boot (default in ESX)

      The VM will fetch the PXE-boot file and start the installation using the parameters supplied in the kickstart. Once installed, the boot order will prevent the VM to boot from PXE (you could also delete it afterwards).

      For the rest of the deployment you can continue with your favourite CM-tool (Ansible/Puppet/Chef/…)

    • you have to put hostname as ESXI hostname or IP address.
      Keep in mind datacenter name should be “ha-datacenter” like below:
      #!/usr/bin/python
      import vm_include

      def main():
      #connection properties
      #change these to match your installation
      host=”yourvspherehost”
      host_user=”root”
      host_pw=”vmware”

      #properties of the new VM:
      guest_name=”newvm” #name of the VM
      guest_mem=1024 #memory in MB
      guest_cpu=1 #number of virtual CPU
      guest_space=2 #space in GB
      datastore=”non-ssd” #name of the datastore
      esx_host=”10.0.0.1″ #specific host in the cluster
      guest_dc=”ha-datacenter” #datacenter name
      guest_ver=”vmx-08″ #version of VMX (v8 is editable via the client)
      guest_iso=”” #iso to mount (from datastore)
      guest_os=”rhel6Guest” #guest-template
      guest_network=”VM Network” #network-name
      guest_enterbios=False

      #connect to the host
      host_con=vm_include.connectToHost(host,host_user,host_pw)

      #create the new VM
      res=vm_include.createGuest(host_con,guest_dc,esx_host,guest_name,guest_ver,guest_mem,guest_cpu,guest_iso,guest_os,guest_space,datastore,guest_network,guest_enterbios)
      print “Result:”,res

      #start the new VM
      vm_include.powerOnGuest(host_con,guest_name)

      #disconnect from host
      host_con.disconnect()

      if __name__ == ‘__main__’:
      main()

  6. I am trying to understand about vm_include.py where does this needs to be defined at since it is already created and or what do I need to modify to be able to call this

    • Hi,

      It just needs to be there. It’s called from the other scripts by “import vm_include”. No need to modify it.

    • I’m sorry but I think your question is quite unrelated to the contents of the post. You can get all info on VM’s using the VMWare API in a similar way as described here.

  7. I want to create vm without the vcente. what should i do? I use your guest_ds but isn’t successful.

    • Sorry, haven’t tested the script without vCenter. I would need to take some time to have a look at this.

    • Hi, either you forgot to import the correct libraries or pySphere isn’t installed. This is where you should have a look at.

      • Hello I am facing th same error, I am running the script on ubuntu. Is it ohk? or i needed to run over centos.? I can see that pyshere library is installed which other libraries are needed?
        Please help,
        Thanks,

        • Hello, I have solved it with the help of a friend, below I leave the code for those who need it. Basically the problem is that the certificate is self-signed and should be ignored. Just add lines 7 and 11

          #!/usr/bin/python
          # Created on 29/12/2014 – Jens Depuydt – http://www.jensd.be

          from pysphere import VIServer, VIProperty, MORTypes
          from pysphere.resources import VimService_services as VI
          from pysphere.vi_task import VITask
          import ssl

          def connectToHost(host,host_user,host_pw):
          #create server object
          default_context = ssl._create_default_https_context
          s=VIServer()
          #connect to the host
          try:
          ssl._create_default_https_context = ssl._create_unverified_context
          s.connect(host,host_user,host_pw)
          return s
          except VIApiException, err:
          print “Cannot connect to host: “+host+” error message: “+err

  8. hi
    i install your scripts but i have problem
    i change host to *.*
    but the error display
    //////////////////////////////////////////////////////
    Cannot find host: 10.0.0.1
    /////////////////////////////////////////////////////
    Please help me
    Tnx

    • You will need to change the script to reflect to your environment. 10.0.0.1 is in the example, the name/IP of a specific host in the cluster. Please adjust to what is valid for your host/cluster.

  9. Hi,I try it,but the error says:
    build vm’s result:Cannot create guest: The operation is not supported on the object.

    I don’t know why,I use the vcenter 5.1,maybe the version cause the error?

  10. Hello,
    I am getting below error while executing vm_create.py

    vm=host_con.get_vm_by_name(guest_name)
    File “build/bdist.linux-x86_64/egg/pysphere/vi_server.py”, line 304, in get_vm_by_name
    pysphere.resources.vi_exception.VIException: [Not Connected]: Must call ‘connect’ before invoking this method

    Please help me to resolve this error.

    Thanks,
    Shikha

  11. I am getting following error ->
    Note that My Host IP and esx_host ip’s are same because on esx_host only I want to create new vm.
    Error —->
    Result: Cannot find host: 172.16.213.222
    Traceback (most recent call last):
    File “./create-vm.py”, line 39, in
    main()
    File “./create-vm.py”, line 33, in main
    vm_include.powerOnGuest(host_con,guest_name)
    File “/root/training/python_training/my_practice/net/vm_include.py”, line 213, in powerOnGuest
    vm=host_con.get_vm_by_name(guest_name)
    File “/usr/lib/python2.7/site-packages/pysphere/vi_server.py”, line 304, in get_vm_by_name
    FaultTypes.NOT_CONNECTED)
    pysphere.resources.vi_exception.VIException: [Not Connected]: Must call ‘connect’ before invoking this method

  12. Hi,

    I’m hitting the following error. Please help me in resolving this error.
    Traceback (most recent call last):
    File “C:/Users/VinuMalu’s/PycharmProjects/Malathi/sphere.py”, line 81, in
    cg.create_guest_os()
    File “C:/Users/VinuMalu’s/PycharmProjects/Malathi/sphere.py”, line 65, in create_guest_os
    for p in cr.PropSet:
    File “C:\Python27\lib\site-packages\pysphere\ZSI\generate\pyclass.py”, line 160, in get
    return getattr(self, what().aname)
    AttributeError: ‘DynamicData_Holder’ object has no attribute ‘_propSet’

Leave a Reply to testing Cancel reply

Your email address will not be published. Required fields are marked *