Use of Diva hardware with XEN PVM

XEN allows access to PCI and PCIe hardware from PVM guest (using PCI backend/frontend) drivers. This is well tested and works with Diva hardware (tested with XEN 3.2, reported previous versions and XEN 3.3 works too).

This is no differences in performance if using Diva hardware from PVM guest.

Important notice:

Virtual machines are known to be well isolated from the system (hypervisor, Domain 0, other guests) and this is supposed that crash of one VM guest affects this guest only.
This is not true if you access PCI hardware from PVM guest. PCI hardware can use DMA (Diva hardware uses DMA) and after PVM is terminated DMA on hardware remains active and accessing (writing) memory which free or assigned to other domain now.
The solution is to modify XEN hypervisor to erase master bit in PCI configuration space after PVM exit detected but before assigned to PVM memory is freed.
In the future this problem will be probably resolved using IOMMU/VT-D.

Before continue with set up procedure, please check for following originated by XEN limitations:

• XEN can not share interrupts between assigned to different domains devices. Please ensure that Diva board you plan to use with PVM guest does not shares interrupt with other used by Domain 0 or by other domains devices
• XEN does not allows to use DAC (Dual Address Cycle, PLCI) and Long Address Format (PCIe) from PVM guest. In case all memory below 4GByte is allocated PVM will fail to allocate DMA descriptors and Diva hardware will not start. To prevent this problem please start PVM which accesses PCI hardware as first and Start Diva hardware at PVM start
• XEN does not allows use of MSI (Message signaling interrupt) by PVM guest
• XEN does not allows to use AER (Advanced Error Reporting, PCIe) by PVM guest

Now the set up procedure:

Domain 0:
modprobe pciback
cd /sys/bus/pci/drivers/pciback
# Show information about Diva hardware
lspci
07:01.0 Network controller: Eicon Networks Corporation Diva Server 4PRI (rev 01)
07:02.0 Network controller: Eicon Networks Corporation Diva Server 4PRI (rev 01)
# Assign both Diva boards to PCI backend driver
echo -n "0000:07:01.0" > new_slot
echo -n "0000:07:01.0" > bind
echo -n "0000:07:02.0" > new_slot
echo -n "0000:07:02.0" > bind
# Allow access to configuration space.
# Diva drivers use VPD (Vital Product Data) PCI configuration space area
# Probably bettter to use xend-pci-permissive.sxp to allow access to VPD
# are only.
echo -n "0000:07:01.0" > permissive
echo -n "0000:07:02.0" > permissive

Now add to PVM guest configuration:
pci = [ '07:01.0', '07:02.0' ]

Finally start PVM, install, configure and start Diva drivers