Glendix has been making the rounds on the web lately, with coverage from OSNews, Reddit, and even a Russian site, Linux.org.ru. It’s really motivating to see some buzz around the project, giving me the extra boost needed to push the completion of a usable release!

Some of the major obstacles that have to be crossed before making a beta-quality release are the completion of critical synthetic filesystems – especially /net and /draw. We also have to work out the kinks with per-process namespaces and union mounts. I don’t know if we’ll be able to get Rio running as a WM before an official release; if not, Plan9Port’s Rio and WMII are good candidates.

The biggest criticism of Glendix seems to be the reasoning that Plan9 user-space tools are somehow superior to their GNU counterparts, and several people have asked us to substantiate our claims. At this point, however, I don’t think that it is really important, or even relevant. Even if Plan9 user-space tools aren’t somehow better – I think it is generally a refreshing idea to see Linux combined with a user-space other than GNU.

We’re at FOSS.IN this year, where a large number of kernel developers have gathered; and I hope to get some of their valuable input on the problems Glendix currently faces, and maybe even write some code to solve them

Thanks for all the community support and critical comments, they are all very vital feedback for the project and are much appreciated!

It’s been a long night, but Weave 0.2 is finally out! Grab it while it’s hot

The last week was mostly spent sprinting towards this release. We’ve fixed over 75 bugs, the major features being: streamlined first-run experience; passwords, tabs and form history synchronization; switch to NSS-based component instead of openssl; better encryption (AES and RSA instead of DES3); and general performance improvements. For a more detailed overview of the changes, check out the release notes.

Ubuntu users: You can’t use the Firefox that the package manager installs, you need to get a stock build here and use that instead. (This is because Ubuntu doesn’t include some parts of Firefox that Weave depends on).

I’m really excited to be a part of this, because I’m very new to the feeling of being part of a team that writes software that a ton of people use. Within minutes of the release, we started getting feedback from users; which is really awesome, and I doubt I can get this kind of experience elsewhere.

I’m also a little nervous, because I got to work on some of the server-side code that Weave depends on, and I’ve actually never written code that is going to be tested by thousands of people. My only hope is that all goes well and that I didn’t break something. My best moment was when I wiped all the old accounts to start afresh (that means all 0.2 users have to register again, irrespective of whether they had an account before. Weave should automatically popup a registration wizard when you install it). All in all, it’s been a wonderful experience so far, and I definitely look forward to more releases working at the Labs!

On a side note, the Labs upgraded to a really snazzy looking website today, going hand-in-hand with the 0.2 release.

So, I’ll be heading to bed now, and check if the world hasn’t ended tomorrow morning. Enjoy the new Weave release!

Looks like everyone’s doing one of these around the blogosphere lately, so I’m joining in the fun:

[theghost ~]$ uname -a
Darwin theghost.local 9.2.2 Darwin Kernel Version 9.2.2: Tue Mar  4 21:17:34 PST 2008;
root:xnu-1228.4.31~1/RELEASE_I386 i386

[theghost ~]$ history|awk '{a[$2]++ } END{for(i in a){print a[i] " " i}}'|sort -rn|head
118 ls
81 cd
61 hg
39 exit
29 vi
24 ssh
24 mate
23 grep
19 rm
9 wget

And for the Linux virtual machine:

anant@tg-nix ~ $ uname -a
Linux tg-nix 2.6.24-gentoo-r1 #32 SMP Sun Apr 13 09:15:20 IST 2008
i686 Genuine Intel(R) CPU T2600 @ 2.16GHz GenuineIntel GNU/Linux

anant@tg-nix ~ $ history|awk '{a[$2]++ } END{for(i in a){print a[i] " " i}}'|sort -rn|head
142 ls
88 cd
83 sudo
48 vi
33 emerge
30 exit
8 rm
8 mv
7 startx
7 cmake

I’m going to leave it for you to figure out what mate’ is

The first step in the Glendix project was to write a binary loader for the Plan 9 a.out format. Linux has a clean interface for registering new binary format handlers from a module. Basically, you define a structure of type linux_binfmt and call register_binfmt during initialization of the module. Now all that’s left to do is implement the three functions that you pointed to in your structure: load_binary, load_shlib and core_dump.

Luckily for me, all Plan 9 executables are statically linked so I can just leave load_shlib as NULL. core_dump is also not that important during the development stages, although the final product must definitely implement it. To get a feel of what I needed to do in load_binary, I decided to take a peek into some of the other binary format handlers. I tried to comprehend the code for ELF with not much luck. I then turned to UTLK, which helped me understand what was going on. I highly recommend the book for anyone interested in kernel programming.

Anyway, here is when I found out that all ELF executables have sections that are actually page aligned! That means every ELF executable contains a bunch of zeroes after the TEXT section, so that the DATA section starts at the next page address. That’s how the executable is supposed to be laid out in memory, but I had no idea someone would actually think of doing it in the file. I guess they have their reasons, all the binary format loader does is mmap the file. Maybe for ELF2 they could put in zeros for the BSS section in the file too

Plan 9 executables on the other hand, are just normal files with no padding. This gives me a headache because I can no longer use mmap. Recall that all addressees passed to mmap have to be page-aligned. But the DATA section in Plan 9’s a.out will start at a non-page-aligned address most of the time.

One of the first things I tried to do was to mmap the file into a high address, copy portions into the appropriate locations and then free the mapping. That didn’t work so well because:

• memcpy works only on physical addresses. Logical addresses from the virtual process address space can’t be easily translated to physical ones because Linux delays physical memory allocation for as long as possible. Now we know why all the loaders use mmap, it is fundamental to the “Linux way” of memory management.
• There is no generic copy_in_user implementation. There are specific ones that use assembly code for PPC, SPARC and even x86_64, but none for x86. The alternative was to use copy_from_user to move data into kernel addresses and then bring them back using copy_to_user. That didn’t work out well either – copy_to_user kept failing for some reason.

I ended up writing a userspace program called ‘pad’ that page-aligns a Plan 9 a.out executable. The loader just mmap’s the file, like all other loaders. The solution is suboptimal, if someone knows a clean way of doing all of this in kernel-space, I’ll be grateful for the help. The ultimate goal is to run Plan 9 executables on Linux, unmodified.

The code for the loader and the pad program can be found on git here.

I’ve been contemplating switching to a virtual private server for quite some time. The cheapest one I could find was SliceHost. While their services seem to be great, $20 a month was too steep a price. A few weeks ago, I stumbled upon Gandi.

Gandi is a known name in the domain registrar space, but they recently launched a “beta” version of their Xen-based hosting service. Because the service is beta, they offer a rock-bottom price of $7 per month for 256 MB RAM, 1/64th the processing power of a Quad Dual-Core AMD, and ~500GB of monthly traffic! Talk about tempting, I wasted no time in signing up

It’s been 3 weeks now, and I must say that their service is absolutely fantastic. My server was up in no more than 10 minutes (that includes the time taken for payment). I chose a Debian-based machine (Gentoo isn’t on their list yet :(). As soon as I logged in, I got to work – I needed to setup my web and mail server. Here’s what I did first:

$ apt-get update
$ apt-get upgrade

Now to setup the web server with PHP 5:

$ apt-get install php5 php5-cli
The following NEW packages will be installed:
apache2-mpm-prefork apache2-utils apache2.2-common libapache2-mod-php5
libapr1 libaprutil1 libexpat1 libpcre3 libpq4 libsqlite3-0 php5 php5-cli php5-common ucf

Edit the default configuration files to your liking and get the default web site running:

$ vi /etc/apache2/sites-available/default
$ /etc/init.d/apache2 restart

Now, onto the mail server. I chose the postfix, procmail, saslauthd, spamassassin and dovecot combo:

$ apt-get install postfix procmail sasl2-bin libsasl2-modules
The following NEW packages will be installed:
postfix procmail sasl2-bin ssl-cert

You’ll be asked a few questions, but we’re going to be reconfiguring anyway so it doesn’t matter what you say:

$ dpkg-reconfigure postfix
General type of configuration?
Internet Site
Where should mail for root go
someuser
Mail name?
mail.example.com
Other destinations to accept mail for? (blank for none)
server.example.com, mail.example.com, example.com, localhost
Force synchronous updates on mail queue?
No
Local networks?
127.0.0.0/8
Use procmail for local delivery?
Yes
Mailbox size limit
0
Local address extension character?
+
Internet protocols to use?
ipv4

To get postfix to play along with saslauthd, we need to make some changes to the postfix configuration. You can edit /etc/postfix/main.cf directly, or use postconf:

$ postconf -e 'smtpd_sasl_local_domain ='
$ postconf -e 'smtpd_sasl_auth_enable = yes'
$ postconf -e 'smtpd_sasl_security_options = noanonymous'
$ postconf -e 'broken_sasl_auth_clients = yes'
$ postconf -e 'smtpd_recipient_restrictions = permit_sasl_authenticated,permit_mynetworks,reject_unauth_destination'
$ postconf -e 'inet_interfaces = all'
$ echo 'pwcheck_method: saslauthd' >> /etc/postfix/sasl/smtpd.conf
$ echo 'mech_list: plain login' >> /etc/postfix/sasl/smtpd.conf

Since postfix on Debian runs in a chroot, you need to make sure it can access saslauthd:

$ mkdir -p /var/spool/postfix/var/run/saslauthd
$ vi /etc/default/saslauthd #Edit to your liking, I use /etc/passwd as my auth source
$ /etc/init.d/postfix restart
$ /etc/init.d/saslauthd start

The most important thing while setting up a mail server is to test at every interval so you can know where a problem came from, if one comes at all. I tested whether postfix + saslauthd were working file using telnet. For the PLAIN authentication type, you can find your auth string by determining the Base64 encoding of the string “\0username\0password”. Here’s a transcript of the telnet session (‘<’ is data sent from you to the server – meaning you have to type it and press the return key, ‘>’ is data sent to your computer from the server):

$ telnet mail.example.com 25
> Trying 10.10.10.10
> Connected to mail.example.com.
> Escape character is '^]'.
> 220 mail.example.com ESMTP Postfix (Debian/GNU)
< auth plain AHVzZXJuYW1lAHBhc3N3b3Jk
> 235 2.0.0 Authentication successful
< quit
> 221 2.0.0 Bye
> Connection closed by foreign host.

When you see “Authentication successful”, you can be sure that saslauthd is working fine with postfix. Now, edit the .procmailrc file in your home directory as required. I love procmail because it lets me do all sorts of preprocessing on my mail (liking moving spam to /dev/null and arranging mails into the proper IMAP folders).

I also use greylisting, a technique used to block 99% of incoming spam (while spamassassin catches the rest 1%). This comes at a price though, whenever someone genuine sends you mail for the first time, it may take upto 20 minutes for it to reach your inbox. I’m not going to discuss greylisting in detail here, but I think 20 minutes is a fair compromise to keep my inbox spam-free:

$ apt-get install postgrey
The following NEW packages will be installed:
libberkeleydb-perl libdigest-hmac-perl libdigest-sha1-perl libio-multiplex-perl
libnet-cidr-perl libnet-dns-perl libnet-ip-perl libnet-server-perl postgrey
$ vi /etc/postfix/main.cf
# Add the line
smtpd_recipient_restrictions=check_policy_service inet:127.0.0.1:60000
# to the file

Time for another test to see if postgrey is doing its work. You also might want to send yourself a test mail to see if your system is working.

$ tail /var/log/mail.log

Now, for spamassassin:

$ apt-get install spamassassin spamc
The following NEW packages will be installed:
libarchive-tar-perl libcompress-zlib-perl libhtml-parser-perl libhtml-tagset-perl
libhtml-tree-perl libio-zlib-perl libsocket6-perl liburi-perl libwww-perl spamassassin spamc

You need to create a user so you can run spamassassin securely:

$ groupadd spamd
$ useradd -g spamd -s /sbin/nologin -d /var/lib/spamassassin spamd
$ mkdir /var/lib/spamassassin
$ chown spamd:spamd /var/lib/spamassassin

Now edit the spamassassin configuration:

$ vi /etc/default/spamassassin
ENABLED=1
OPTIONS="--create-prefs --max-children 5 --username spamd --helper-home-dir /var/lib/spamassassin -s /var/log/spamd.log"
$ vi /etc/spamassassin/local.cf
$ /etc/init.d/spamassassin start
# usual check
$ tail /var/log/spamd.log

Get postfix to start using spamassassin:

$ vi /etc/postfix/master.cf
# Append this to the first line:
-o content_filter=spamassassin
# and then add this at the end:
spamd unix - n n - - pipe
user=spamd argv=/usr/bin/spamc -f -e /usr/sbin/sendmail -oi -f
${sender} ${recipient}
$ /etc/init.d/postfix restart

Time for a final test!

telnet mail.example.com 25
> Trying 10.10.10.10...
> Connected to mail.example.com.
> Escape character is '^]'.
> 220 mail.example.com ESMTP Postfix (Debian/GNU)
< ehlo example.net
> 250-mail.example.com
> 250-PIPELINING
> 250-SIZE 10240000
> 250-VRFY
> 250-ETRN
> 250-STARTTLS
> 250-AUTH LOGIN PLAIN
> 250-AUTH=LOGIN PLAIN
> 250-ENHANCEDSTATUSCODES
> 250-8BITMIME
> 250 DSN
< auth plain AHVzZXJuYW1lAHBhc3N3b3Jk
> 235 2.0.0 Authentication successful
< mail from: test@example.net
> 250 2.1.0 Ok
< rcpt to: test@example.com
> 250 2.1.5 Ok
< data
> 354 End data with .
< Subject: Test Mail
< Let's see if you get this test mail!
< .
> 250 2.0.0 Ok: queued as 944E12E3EB
< quit
> 221 2.0.0 Bye
> Connection closed by foreign host.

Now how do you check if you actually got the mail? You can ssh into the server and use something like mutt, but a long term solution would be to setup an IMAP server so you can connect with your favorite mail client:

$ apt-get install dovecot-imapd
The following NEW packages will be installed:
dovecot-common dovecot-imapd libmysqlclient15off mysql-common
$ vi /etc/dovecot/dovecot.conf

The most important portions in the configuration file are the protocols, mail_location and auth sections. Once again, I chose to authenticate against /etc/passwd. Start the server and check for any warnings or errors:

$ /etc/init.d/dovecot start
$ tail /var/log/dovecot.log

Now, I also wanted to transfer all my mail from an old server to the new one. I came across a nifty utility called imapsync to do that for me:

$ apt-get install imapsync
The following NEW packages will be installed:
imapsync libio-socket-ssl-perl libmail-imapclient-perl libnet-ssleay-perl
libparse-recdescent-perl libterm-readkey-perl
$ imapsync --help

Phew!

Plan9 is an operating system that is designed to make up for all the mistakes that UNIX made. And it’s quite different from any of the current UNIX clones, be it any of the BSDs or Linux. To ease the migration, there’s a project called Plan9Port; which is essentially a port of several Plan9 utilities to the POSIX platform.

I recently made an ebuild for Plan9Port, also called Plan from User Space. Just ‘emerge plan9port’ and spread the Plan9 love!

Speaking of which, my Summer of Code project for Plan9 is progressing well. More details on my SoC blog, or the aggregator at KiX. Laters!

ZOMG. Parallels‘ latest build brings along with it some nifty features. Which ultimately leads your desktop to look something like this:

No, that’s not some fabricated image made in GIMP. It’s for real; you can now run all the big 3 OSes side by side, just as if they’ve known each other for years

Here’s how you do it:

• Get yourself a powerful Intel Mac with 2Gigs of RAM (okay you may not really need this, but it’s always a good idea)
• Buy a copy of the latest Parallels Desktop. More than worth it!
• Install your favourite Linux Distro and Windows on two Parallels VM’s
• Or, if you already have BootCamp and want to use the native Windows already installed, Parallels allows you to boot from that too! I already had a native Windows running on Boot Camp, and I was up and running in no time with the useful guide you can find here
• Fire up the Linux VM, start the X server bundled with OS X. SSH to your Linux VM with X forwarding enabled, and start your favourite desktop: I chose XFCE, but Gnome/KDE will run equally well
• Fire up your Windows VM and enable the “Coherence” mode
• Start using Windows/OS X/Linux applications side by side, all within the comfort of your OS X desktop
• If you’re really insane create virtual machines for FreeBSD and Solaris too…

This has got to be the most coolest thing I’ve ever done with a computer

(In the screenshot: Windows Start menu just below the Apple menu; XFCE-Terminal on Linux; Safari on OS X; IE on Windows; XFCE-Panel (on Linux) just to the right of the OS X Dock. Windows applications even have their own Dock items; All Linux apps will be shown as only one X app though)

The batteries of my Macbook Pro suddenly swelled up today. I haven’t a clue in the world as to why it had to happen all of a sudden today but it did.

Maybe because I gave a hand at programming on the Mac for the first time today. We now have an installer for PHP-GTK. Intel Mac users only. Was also having a look at Metisse; sounds great. Anyone for a HCI research project?