Summer of Code Midterm Update (Tuesday, July 10, 2007: 04:19 am)
As we reach mid-summer, I think now is a good time to provide a more detailed update on my summer of code project, vlosuts. For anyone not familiar (or for anyone forgetful) the following is a brief summary of my project proposal. In order to assist in distribution upgrades, vlosuts (Virtual Live Operating System Upgrade Test Suite) aims at performing upgrades of live systems (between releases and/or repositories) while logging any errors encountered. By running on a live system (virtualized via QEMU/KVM/Xen/other) problems that may only be encountered with services upgrading can be identified early. The full proposal can be found at: http://code.google.com/soc/2007/debian/appinfo.html?csaid=24C7766EC4271F15
I am happy to say that vlosuts is successfully in alpha, with most of the desired features already implemented. Based on my initial project propasal timeline, I'm well on schedule.
Some of the features that were proposed included the ability to run the upgrade with packages selected by priority, from a list, and as mirrored from an existing system (packages can even have a specific version specified, though of course the repository used must have that version available). Additionally, when mirroring this package list, configuration files can optionally saved as well. This allows testers to (as nearly as possible) reproduce a specific system. These options have all been implemented among others. I have also put in support for running with QEMU, KVM, and Xen backends.
For the remainder of the summer, my goal will be adding additional features and tackling a few bugs that have come up. In particular I'm trying to take advantage of QEMU's ability to emulate various architectures. However, the stability of user-mode emulation has been uneven for me (segmentation faults whenever any program is launched), which is required in order to cross bootstrap. Once this is worked out, one should be able to test upgrades on any of QEMU's supported architectures.
Additionally, vlosuts is not yet available as a package, which is clearly something that needs to be fixed soon. I'll be spending some time in the next couple weeks packaging it and making sure it sets up correctly.
I think that about sums up the current state of things. It's been a good experience thus far, and hopefully the rest of the summer continues to go well.
A vlosuts update (Tuesday, June 26, 2007: 06:43 am)
I'd say vlosuts is just about alpha at this point. It perfectly manages building an image and then upgrading (at least on the systems I've tested it on) and I haven't seen any major bugs. The main feature I'd like to get working is support for a Xen backend. This shouldn't be too difficult, it's just taking me a bit of effort to get Xen working. It would've been easy enough, but I just couldn't get the network configured. I followed guide after guide, and finally figured out the problem was that my wireless card didn't support bridging. Which explains why I'd never managed to get that to work in the past either. So now I'm reconfiguring my Xen setup with NAT, which should give me a system good enough for testing. After that I'll start fixing some of the less important bugs and adding some features here and there.
Foreign chrooting (Tuesday, June 19, 2007: 06:12 am)
As part of testing upgrades, I'm trying to build disk images for foreign architectures. debootstrap already has the first-stage second-stage framework for this, but of course you have to be able to boot a system in the target architecture to finish the second stage. I've tried a few ways of accomplishing this.
The first is extracting a kernel from a package and using qemu-system to boot that. Trouble there is that AFAIK there aren't prebuilt initrds and so the root image can't be mounted.
The second option occurred to me that I could use user mode emulation to do the necessary second-stage bootstrapping. You'd still need to be able to chroot into the system, though, and so I tried this out by statically building qemu binaries. This didn't seem to compile too cleanly, but after some searching I discovered that what I was trying to do had already been done over here. Seems like a pretty good solution. Only thing is if I go with this method, it would require those packages as prerequisities which aren't in the Debian repository.
If anyone has any additional suggestions on bootstrapping for a foreign architecture, let me know.
GSoC, vlosuts, and other acronyms (Tuesday, May 29, 2007: 02:16 am)
Summer of Code officially started today, and as such it was time to start hacking out some code for it. For the past couple of weeks I've been casually looking over an upgrade testing suite for Ubuntu (website, bzr repo). It's fairly sophisticated and does a lot of things that I'd like to bring to Debian. However, after considering it for awhile, I decided that I would prefer to start on the Debian version from scratch. This decision came about for a few reasons:
The Ubuntu version focuses on upgrading between releases, whereas I would like to possibly upgrade between entirely separate repositories. While this could fairly easily be added as a configuration parameter, it was still a point of consideration.
I would like to write the scripts for building the image as a seperate module. This would aid in the future goal of snapshotting and rebuilding systems based on a list of installed packages, versions, and config files.
Until recently the Ubuntu upgrade tester used chroot and had no virtualization option. In addition to having this, the aforementioned process of building the initial image independent of backend should make the resulting scripts easily pluggable into any virtualization option.
At any rate, I will probably end up reusing many pieces of the code from that tester.
GSoC 2007: Project is Underway (Wednesday, May 16, 2007: 01:38 am)
I've gotten started on setup for my Google Summer of Code project (original abstract for anyone unfamiliar), originally titled "Automated Upgrade Testing Using QEMU." In thinking about the direction I'd like to take the project, I've created an Alioth project with the title "Virtual Live OS Upgrade Testing Suite" (short name vlosuts). As discussed briefly in the GSoC application comments, the system should have a plugable backend so that one could use QEMU, Xen, etc. as preferred, and so I dropped QEMU from the title in favor of the word Virtual.
The project itself is still in a planning phase, but Ubuntu has a similar package using chroot which may serve a good basis for what I'm doing. I will be making periodic updates of my progress via this blog, but status and mailing lists should also be available at the Alioth project: http://vlosuts.alioth.debian.org/.
