Ajax terminals

I recently ran across AjaxTerm. It's pretty neat. It got me rather excited as I think it would be a really cool part of a xen Web-based management console.

I was rather curious how it worked with respect to updates. Clearly, when you send console data it requires an HTTP request. That's unavoidable. However, how do you detect when it's time to receive console data? There's no such thing as a reverse HTTP connection so that's out of the question. A naive solution would be to poll the server. That seems rather wasteful though. More importantly, there is an inevitable lag depending on how tightly you loop.

That got me thinking about an alternative approach this evening. What if you started a request (asynchronous of course) and the server simply didn't respond right away. Even if the browser eventually times out the connection, as long as it lasts for a pretty good portion of time, you could use this as a way to wait (without polling) for new data to arrive. So far, some experiments have shown promise. We'll see whether this results in a noticeable effect on latency.

Turning on optimization gives better warnings?

This could be one of those things that everyone knows except for me, but I learned a very hard lesson today. We had a code review internally for a bit of code I've been doing. A very embarrassing bug turned up that was the result of a poorly tested last minute query-replace.

The crazy thing was that it *should* have been caught by the compiler as an uninitialized variable. I had -Wall in my CFLAGS. Sure enough though, no matter how blatant I made the uninitialized usage, the compiler said nothing.

I looked in the info pages and discovered that uninitialized values are only reported if -O is present. -O does the necessary bookkeeping to allow data flow analysis which is clearly required for reporting of uninitialized values. I typically don't enable -O but I guess most projects I work on do. I will certainly be using -O from now on though.

Some interesting stats on sensitive instruction usage

One surprising thing about x86 virtualization is that while there is a small number of instructions that require special effort (around 17 depending on what you count), there are an even smaller number that make 95% of all the calls to these instructions.

These four instructions are cli, sti, pushf, popf (which probably isn't all that surprising). An interesting result of this is that you only really have to focus on making these instructions fast. This is an important conclusion when discussing the performance trade-offs of supporting a transparently paravirtualizable kernel.

Rusty Russell posted some stats that confirm this here.

Xen Becoming a Microkernel?

Keith Adams has posted an entry about his thoughts on microkernels. I completely agree with his argument. He makes a curious point though at the end of the article:

Xen has been carefully, and somewhat silently transmogrifying itself into a microkernel.

What I find curious about this statement is that I think the inverse is actually true. Historically, starting in the 2.0.x series, Xen owes quite a bit of it's design to the Nemesis exokernel. A lot of the things that seem out of place in a hypervisor (event channels and usage of the nomenclature of domains) are actually vestiges of Nemesis.

Xen is not becoming a microkernel, rather it was an exokernel and is becoming more monolithic. This leads to a more interesting topic of debate (that I think Keith is partially alluding to). Do the same arguments about about microkernel vs. monolithic kernel apply to hypervisors?

I've always thought an interesting patch to Linux would be one that allowed the kernel to exist in 64mb. However, even this may be unnecessary. As VT/SVM hardware improves over time, and the cost of a switching address spaces during an exit decreases (thanks to tagged TLBs), it really stops mattering.

It's an interesting thing to think about...

Some thoughts on the Xen Virtual Framebuffer

I have about a day of free time before my family comes in for graduation. I've figured I'd use some of this to do a little work on the Xen framebuffer. Unfortunately, it's very low on my priority list so I've not gotten very much time at all to work on it.

The one required task I have to complete is to bring the current code up to date and to make it use XenStore. That's pretty straight forward.

Another thing I'd like to do is finally address some of the nagging performance problems I've seen. The first fix is to implement a shadow framebuffer to reduce the size of the update regions. Instead of doing a full shadow (like I did in QEmu), I'm thinking of instead just allocating a page when a portion of the framebuffer is write faulted. I'll copy the original contents of the page to the allocated page and then when it comes time to invalidate, I'll have something kosher to compare to.

I also want to implement the dirty bitmap. I like that approach much better because it lets dom0 decide how to break the region into rectangles.

Analysis of RealVNC Vulnerability

As many of you may have saw, Slashdot ran a story about a flaw found in RealVNC 4.1.1 that allowed an attacker to gain access to a password protected VNC server without any credentials.

By the time I saw this article, the site was already Slashdotted so I wasn't able to get a packet trace. The disclosure of the flaw was a bit troubling as there was no explanation of how it worked. Moreover, it sounded like it was stumbled upon by accident. My first thought was buffer overflow or integer overflow but it takes a very carefully crafted packet to exploit such flaws. Plus, VNC doesn't really have many variable length strings and certainly doesn't have any in the right parts of the protocol to explain this vulnerability.

Then it dawned on me while on campus the other day. Starting in version 3.7 of the RFB protocol, the security type negotiation was changed. It used to be that the server just sent back whatever security type it wanted to use. Common security types are rfbAuthNone (no password) and rfbAuthVnc (triple-des challenge/response). The new security type negotiation allowed the server to send back a list of security types letting the client choose which one it wanted to use.

Obviously, this makes supporting new security types more backwards compatible. A really silly mistake to make in the server would be to not check to make sure that the security type the client chooses is actually part of the original list the server supports. I looked at the RealVNC source code and sure enough, they weren't checking. What happened here is that the person who found this hole must have had a bug in his client where it sent the wrong security type (one that wasn't in the list).

This brings up a pretty interesting question. Why wasn't there a test case for this? This seems like a rather obvious thing to check for. Fortunately, this is really limited to RealVNC 4.1.x as all other VNC codebases branched off before the rewrite for 4.1.x and this problem was introduced in that rewrite. If you're using RealVNC, and you do not want password-less access to your machine, you should visit RealVNC and pick up the new version that fixes this problem.

QEMU 0.8.1 is out

Fabrice posted the following today:

QEMU version 0.8.1 is available at http://bellard.org/qemu/download.html.

version 0.8.1:

  - USB tablet support (Brad Campbell, Anthony Liguori)
  - win32 host serial support (Kazu)
  - PC speaker support (Joachim Henke)
  - IDE LBA48 support (Jens Axboe)
  - SSE3 support
  - Solaris port (Ben Taylor)
  - Preliminary SH4 target (Samuel Tardieu)
  - VNC server (Anthony Liguori)
  - slirp fixes (Ed Swierk et al.)
  - USB fixes
  - ARM Versatile Platform Baseboard emulation (Paul Brook)