VNC support in QEMU CVS

Fabrice just committed it this afternoon. I broke the Win32 build (whoops) but Fabrice quickly fixed it.

VNC in QEmu

As I mentioned in a previous post, I have been developing a VNC patch for QEmu. QEmu is a really great platform to explore different ideas as it's entirely in userspace and therefore is somewhat sane to debug. The two ideas I've been examining are both accelerations for VNC and so far have shown promising results.

The first is what I'll refer to as subtiling. A fundamental VNC optimization is to only transmit the portions of the screen that change to the client. The easiest way to do that is to keep track of which portions of the screen change and then transmit those portions during your normal update event. In a virtualized environment, it's a bit harder to keep track of which regions change because it's just memory being written to. This means you usually end up with oversized dirty regions. This tends to result in 1-2 orders of magnitude more data being marked dirty than there really is. To eliminate this, it becomes necessary to maintain a copy of the client framebuffer (that is, the framebuffer that you think the client has). When it comes time to transmit an update, instead of blindly transmitting dirty regions, I've implemented a "subtiling" algorithm which compares against our copy of the client framebuffer to determine which regions have really changed.

The second optimization I've implemented is video-to-video blit acceleration. The basic idea is that most modern VGA hardware (and more important, the Cirrus card that QEmu emulates) implements 2d acceleration routines. In the case of the Cirrus card, this is a fast copy between video memory (optionally using one of a number of ROP operators). With the right filtering and a little magic, we can convert this video-to-video copy into a VNC CopyRect operation. I suspected that this would be a useful optimization but I had no idea how much of a difference it would make. I was surprised to learn just how many places a modern operating system makes use of accelerated copying. When you scroll a web page in internet explorer, Windows uses 2d acceleration. The result is an extremely usable interface that is awfully pleasant to use even over the network. In fact, even with a poor encoding, the interface is at least as responsive on my local LAN as a real VNC server.

Tonight I got a chance to talk to Fabrice about this patch and he seemed to be happy with it. In fact, it looks like I've got a shot of getting it in for the next release. The code is quite ugly right now but hopefully I'll have something that's much cleaner by the end of the week. Then I can start looking at reusing it for Xen so I can finally get rid of the libvncserver dependency for the VFB.

Xen, VMware, and upstream merge

There's been a whole lot of articles about Xen, VMware, and the on going upstream merge effort. I just want to point out a couple things. The first is that a number of the articles are a bit misleading/speculative. The second is that there is much more common ground than the articles seem to suggest.

There are only a very small number of areas where VMI and Xen differ. Most of these differences are superficial. One thing to keep in mind is that neither set of patches is complete at all. Neither support SMP (fully), writable page tables, or management mode stuff. Both Zach and Chris tried to make this very clear in the initial posts.

There's a lot of work to do and a lot of hard problems to solve. In any case, even though neither set of patches is really complete, they are still way to big. Far too big to go into Linux all at once. Upstream merge is going to require a large number of small changes to incrementally add support for virtualization into Linux. That's what's going to be interesting over the next year. Once we figure out how to break things up, we'll start doing it.

The whole VMI vs. Xen thing is overrated. This is really just about what's the best interface for Linux. The actual underlying hypervisor doesn't really matter.

Ambiguoity in RFB (VNC) specification

In the RFB protocol specification (the protocol used by VNC), the best non-compressed protocol is by far Hextile. Hextile tile's the screen and provides a mechanism to cheaply enough solid tiles, or tiles that contain subrects. One of the optimizations is that each tile can have a background and foreground color and if they aren't specified, they are implicitly assumed to be the previous tiles color.

There is a special tile mode called SubrectsColoured (or SubrectsColored for us Yankees). In this mode, each sub rectangle has its own color specified before the subrect coordinates. What the specification doesn't make clear, is that this color is treated to be a "foreground" color such that if the next tile relies on an implicit foreground color, the foreground color should be the very last colored subrectangle's color.

I wasted a good part of a day on this and most certainly much longer when I was previously working on a client. I'm going to try and request that the spec be clarified to make this very explicit.

3.0.2 is *finally* out

After a number of problems (a couple of last minute regressions and an outbreak of the flu), we've finally cut a 3.0.2 release. Major features include SVM (Pacifica) support, a fresh 2.6.16 Linux guest kernel, XML-RPC support in the userspace tools, and lots of bug fixes and enhancements.

You can download 3.0.2 from bit torrent here.

Absolute Mouse Support in Xorg

I've been reading bugzilla entries and Xorg code and finally have a pretty good idea of the general support for absolute USB mice. Windows apparently supports absolute mice natively (although both axis' have to be absolute or relative). I don't know how Windows interprets these coordinates (whether it converts them to relative movement or actually as absolute positions).

The Linux kernel supports absolute mice just fine. Of course, the Xorg I'm running on my system doesn't. Fortunately, there's a bugzilla entry against the evdev driver and initial support for absolute mouse events is present in Xorg CVS. To me, this means that it is a viable solution to this problem. It's at least as good of a solution as writing a custom Xorg input driver for Xen. It has the additional benefit of not requiring me to maintain a new X driver and I like that very much :-)

USB HID--The perfect match for virtualization?

An interesting discussion has emerged on qemu-devel about a topic dear to my heart. Anyone who has done anything with input device emulation knows that mice are a big pain to virtualize as PS/2 only supports relative input coordinates. This requires the all-to-familar "input grabbing" mode that things like QEmu and VMware both require since the emulator has no idea where the mouse actually is within the guest (and therefore doesn't know when the mouse "leaves" the window).

The most accepted solution to this is to paravirtualize the input driver. This is the approach VMware takes and is the approach I'm currently taking in Xen. Another approach that I've explored is emulating a drawing tablet. These devices use absolute coordinates (as you want exactly what you draw to appear on the screen). I even implemented a proof-of-concept Wacom emulator for QEmu. Alas, these devices tend to be serial-based so you get no automatic probing on guest install. This means users have to manually configure the devices which is a no-go for a large set of our target user-base.

Someone on qemu-devel pointed out that the USB HID specification allows devices to be either relative or absolute. A compliant HID driver would therefore Just Work. Best of all, USB tends to be probed automatically so it satisfies that requirement. This got me thinking about how useful HID could be in general. Besides keyboards and mice, the HID also specifies things like USB speakers. This may be a practical way of having cross-platform plug-n-play sound. Plus, with things like USB over IP, there's an awful lot of potential for remoting these things.