Christmas came early this year.

Today, Google announced their new open source systems programming language: Go. I’m super excited about this, we all have been wondering what Rob Pike has been upto since he joined the big G, and now we know. Not just that, but Ken Thomson, Robert Griesemer, Ian Taylor and Russ Cox were all involved in the project, with Ken doing what he does best, writing compilers in lightning speed If that isn’t a list of heavyweight respectable computer scientists, I don’t know what is!

I think Go is poised to be the dominant systems programming language of the future. Go has nailed almost every aspect of a systems language, though some would say I’m biased. Go has been strongly influenced by Oberon, CSP languages like Limbo, and the standard libraries have tantalizing similarities to Plan 9. We’ve had Limbo and Plan 9 for a while now (more than a decade), but this is where my real love for Google begins to bubble, they took something awesome but unpopular and gave it a push to the masses. There are very few companies in the world who would attract the talent to do this, and even fewer who would open source the results. The attention Go has been getting is just mind blowing. Pike had been doing amazing work at Bell-Labs for quite a while, but none of it even got an inkling of the publicity Go is currently getting.

Google was what Pike needed to prove Utah2000 wrong.

I know one thing for sure, I’ll definitely be using my Plan 9 virtual machine a lot less; now that I can write clean concurrent programs that don’t make my head hurt, both in Linux and OS X. And GCC, I’m not shedding any tears while I bid you goodbye.

On another note, Google also announced today that they’ll be sponsoring free WiFi at a whole bunch of US airports this holiday season. For all its faults, Google definitely seems to be doing the right thing. For how long, it remains to be seen, but so far I’d say their track record has been better than excellent.

UPDATE: John Gruber points out that “judging from the copyright statements, [Go is] not an official Google project”. Could this be a result of the famous 20% time scheme?

The mere presence of systems like OpenID, Facebook Connect and a host of other identity services on the web today is attestation to the fact.

Authentication should be a feature of the protocol, not something that relies on hacks like cookies. 99% of websites today rely on cookies for authentication for their websites, besides offering custom registration and login pages. This means the browser, as the user’s agent, has no clue of what is going on. A user is forced to manually track myriads of accounts, remember passwords for each of them, and remember what personal information each of them holds. Sure, part of the problem is solved by using password managers (like the one in-built into Firefox, or external programs like 1Password), but even these programs rely on heuristic algorithms to determine if something looks like a login credential or not. There’s no explicit way for web pages to tell your browser: “This is a login form, please fill in details of the user’s identity here” or “These pages are privileged, please give me the user’s identity”. Why is that?

Actually, there is such a mechanism: HTTP based Authentication has been a feature present since HTTP/1.0, but only 1% of sites actually use it. The reason for that is purely cosmetic, most browsers display a very bland modal dialog when it encounters a page that requires HTTP Auth, and sites are unable to customize that interaction. So, the technically right way to do things sucks from a user experience perspective, and websites started adopting alternate means. Someone discovered they could use cookies to store session information on the client, and the whole situation exploded ever since. As a programmer, I feel very sad when I think about the fact that instead of fixing the problem in HTTP/1.1, web-based authentication took the route it did and led to the mess we are in today.

However, I must also state that HTTP authentication doesn’t solve the entire problem – there is still the issue of users having to create an account for every site they want to be part of. This is because there existed no protocols to federate and provide decentralized authentication. That is, until OpenID and OAuth came about. Now we’re at this exciting juncture, and the browser is in a unique position to use these tools together to provide the user with an experience that is secure and easy to use. Every architect will agree that it is indeed a fun challenge to use the state of identity on the web today and make it into something awesome.

This is precisely what the Mozilla Labs team has been thinking about for a while now. Sometime ago, we added support for automagic one-click OpenID logins to Weave. We plan to spin that “feature” out into it’s own extension and build on it, something we call “Weave Identity“, part of the broader “Open Identity” initiative by the Labs. “Weave Sync“, the original extension, will just focus on the synchronization parts so we can tackle these two different problems separately.

So, how exactly are we planning on doing this? Take a look at an initial version of a document describing an in-browser “Account Manager“. We’ve also put up a WEP (which expands to Weave Enhancement Proposal, by the way) describing the raw form of a specification for automatic actions on websites, such as user registration or password changes.

Keep in mind that all of this is in its very early stages (pre-alpha); but that also means it’s a great opportunity for the community to get involved! What are your thoughts on Open Identity? Use the discussion tab on any of those Wiki pages, start a thread on the Mozilla Labs group, or simply leave a comment on this blog entry, and chip in – we’d love to hear from you!

The grand Summer of Code Mentor Summit of 2009 concluded last week and I had the fantastic opportunity of being able to attend on behalf of Gentoo, Plan 9 and Mozilla. What follows is some indication of how awesome the summit was:

(Photo courtesy of warthog from Etherboot)

I met so many folks I’d only interacted with online so far (the classic nickname-to-face matching), but even better was the opportunity to meet folks powering open source projects from so many diverse backgrounds. I met many of my personal rockstars, and learned about a bunch of open source projects I’d never heard of

Also, one of the things that is only possible at an event like the summit was the ability to get a whole bunch of non-linux operating system groups in one room. We had a great discussion, and it resulted in the creation of the “rosetta-os” special interest group. Look for more activity on the common device drivers for non-linux operating systems front soon!

Other sessions worthy of special mention were Open Source Security, Recruiting and Retaining Awesome People, Advanced Trolling (yes, you read that right), and of course the always welcoming Casablanca where I spent most of my time. We discussed everything from our SoC experiences to the Afro Celt Sound System in that room, always full of creative energy and warmth.

After 4 years of participating in the Summer of Code, I am super happy to have finally met the faces behind the program. Every single person I met over the course of last weekend was friendly, intelligent and just generally awesome; that sort of thing doesn’t happen by chance. I feel warm and fuzzy inside to think that I’m actually a part of the revolution that is free and open source software, three cheers to everyone that made it possible!

I’m back from the EU MozCamp in Prague and we all had a great time! Check out the slides from my talks: Labs Overview and Weave in Depth.

A few people at the MozCamp were interested in Weave’s use of cryptography to protect the user’s data and privacy. Although the specs for the Weave server are available, it may take someone new a while to wrap their head around the whole scheme. I’m going to attempt explaining what crypto operations we do and why we do it in this blog post.

First, let’s get some basic definitions out of the way. Symmetric cryptography means you have one key that can perform both encryption and decryption, and they are complementary operations. For Weave, we use AES with a 256 bit key, and we use it in a mode that requires an ‘initialization vector’ for every decryption. Asymmetric cryptography means there’s a pair of keys (usually called ‘public’ and ‘private’ keys). A piece of text “encrypted” by one key can only be “decrypted” by the other key. Here, we use RSA with a 2048 bit private key.

So, when a user first signs up for Weave using the wizard on their computer, we generate a (random) pair of public and private keys. Next, we use the user’s passphrase to create a symmetric key. This is done using a pretty standard algorithm known as PBKDF2 (short for “Password Key Derivation Function”). The PBKDF2 algorithm requires a ’salt’ value which is also stored on the server. Now that we have a symmetric key, we use it to encrypt the user’s private key and upload it along with the public key to the server. Note that the passphrase is never sent to the server, so if the user’s password ever gets compromised all the attacker can get is their encrypted private key, which really isn’t of much use (especially given that the key is 2048 bits long).

Whenever a particular “engine” is to be synchronized (an engine could be Tabs, Bookmarks, History etc.) we generate a random symmetric key for that engine. This key is then encrypted using the user’s public key (now, one can only retrieve the original symmetric key with the corresponding private key) and uploaded as being associated with a particular engine. All entries (the ‘ciphertext’ property in a “Weave Basic Object”) in that engine are encrypted with the symmetric key that was generated for it.

To make things clear, let’s enumerate the steps we would take to decrypt a single tab object for user ‘foo’:

1. Find the user’s cluster by making a GET request to https://services.mozilla.com/user/1/foo/node/weave. It returns https://sj-weave06.services.mozilla.com/.
2. Fetch the user’s encrypted private key and public key from https://sj-weave06.services.mozilla.com/0.5/foo/storage/keys/privkey and https://sj-weave06.services.mozilla.com/0.5/foo/storage/keys/pubkey respectively. The user’s password is required to access these JSON objects.
3. Ask the user for their passphrase and generate a 256 bit symmetric key from it using PBKDF2 and the ’salt’ found in the privkey object.
4. Use the generated symmetric key and the initialization vector found in the ‘iv’ property of the privkey object to decrypt the user’s private key.
5. Fetch the user’s encrypted tab objects from https://sj-weave06.services.mozilla.com/0.5/foo/storage/tabs/?full=1.
6. Fetch the corresponding symmetric key (the URL is also listed in the “encryption” property of every WBO), in this case https://sj-weave06.services.mozilla.com/0.5/foo/storage/crypto/tabs.
7. Decrypt the symmetric key with the user’s private key.
8. Use the decrypted symmetric key to decrypt any WBO from the tabs collection with the initialization vector found in the ‘bulkIV’ property of the tabs symmetric key WBO.
9. Profit.

A word about the formats in which the keys are actually stored in. All values are Base64. For symmetric keys, the key is stored as-is. For asymmetric keys, I wish we used a standard format like PKCS#12, but we don’t. It’s still ASN.1 though, in some format NSS exports private keys in. You need to do a bit of ASN.1 parsing to figure out the values you’re interested in.

Fortunately, I’ve already figured out most of the details for you – check out my Javascript or PHP implementations of the crypto elements required to decrypt Weave Basic Objects.

Finally, a quick note about why we do all this. Sharing is now reasonably easy, if you want to share your bookmarks with someone, you just need to encrypt the corresponding symmetric key with their public key and they’re good to go. Also, each WBO has it’s own ‘encryption’ property so this can be as granular as needed. Secondly, the passphrase is never stored anywhere (except possibly on the user’s computer) so the server never sees anything other than encrypted blobs of Base64′ed text. Along with making HTTPS mandatory, we think this is a pretty secure way of protecting the user’s data.

If you have other encryption schemes that might fit into Weave’s use cases please let us know! (We’ve already been looking at interesting developments in this area such as Tahoe). I’d also love to hear from you if you have any questions on our current cryptography scheme. We’re constantly trying to improve the security and efficiency of our system so these details are only valid until we change our scheme

Now, go write that third-party Weave client, you have no excuse not to!

I’m off to the beautiful city of Prague, or “Praha” as it is known locally, for the European MozCamp of 2009. Memories from the MozCamp last year are still fresh, and I’m definitely looking forward to this one!

On Friday, we’re going to be hosting a Labs Hackathon on Jetpack. This is your chance to get to know more about the framework that’s so easy to use that your mom could write an extension with it. Maybe not your Grandma though, you do need to know a bit of Javascript The hack session will last as long into the night as needed for you folks to come up with amazing ideas for Jetpacks and implement them. Drew Willcoxon from the Firefox team and I will be on hand all day to help you, so feel free to come and poke us. Oh, I almost forgot to mention that there’s Free Pizza involved.

On Saturday, I’ll be giving a talk on Weave. With 0.7 just released, we’ll be taking a look at our current state, what’s in store for the future, and maybe a few cool demos. We’re also especially interested in engaging with addon developers to see what Weave can do to make it easier for them to add sync functionality to their addons.

Be there!

Last night, Joseph Smarr from Plaxo was our guest speaker and he talked about how the “web is going social”, and how the “social web is going open”. We discussed all the elements that make up the social web today: identity providers, social web providers and content aggregators, and how each of  them are leveraging open standards and protocols such as OpenID and OAuth to create better experiences for their users. Check out his slides here.

This talk was a nice prelude to some interesting discussion about the role that the browser can play in handling the user’s data and identity on their behalf. Very relevant to this was also the recent experimentation by Weave on identity in the browser, and Myk gave us a demo of the auto-sign-in features.

Labs Night is also a chance for everybody to talk about cool stuff they’ve been working on, so Brandon gave us an update on what’s new in Ubiquity 0.5. There’s some really neat stuff in there: Ubiquity is possibly one of the first pieces of software that perform truly internationalized natural language parsing (0.5 rolls out with support for Japanese and Danish). Do check out this blog post for a detailed discussion of the features in 0.5.

I followed with an update on some of the work I’ve been doing with Jetpack – namely providing the capability for “jetpacks” to record audio. The code to enable this is checked into the repository, but you’ll have to wait until a release later this month if you’re not feeling brave enough to build the extension from source to play around with it. I was especially interested to know the kinds of applications that might be possible with this capability, so you if you have any ideas, I’d love to hear them. Myk also gave us a demo of the new streamlined way of subscribing to feeds using Snowl, check out this release announcement for more details on what’s new with the message reader you know you want to use!

Paul Tarjan from the Searchmonkey team at Yahoo! gave us some really cool demos demonstrating Searchmonkey Objects and YQL. I’m especially excited about YQL because it can make some of the back-end ubiquity code really simple and efficient. Incidentally, the Bing team was here at Mozilla just a couple of days ago and they also demoed some features similar to Searchmonkey Objects, albeit restricted to video and snippets of data for now.

Search is starting to feel exciting again, a sentiment similar to one we feel in the browser space today. There’s a lot of innovation in the area outside of the big daddy, and it is indeed heartening to see that major players in the web are beginning to recognize the importance of openness and competition

Labs Nights are monthly events, so we look forward to seeing you sometime in July to discuss more cool stuff that everyone’s been working on!

It’s that time of the year. Google is, yet again, sponsoring students to write some awesome open source code this summer. If you’re a student, and you’d like to make some money contributing to some of the most well known and exciting open source software projects out there, you’d be missing out on a lot by not applying.

If you’re wondering about what the best way to get started is, check out this great advice page. All projects have also been tagged by programming language and field in this delicious profile. You can also search for ideas here.

I’m going to be mentoring for Mozilla, Glendix (under the Plan 9 Umbrella) and Gentoo this year. Get in touch if you’re interested in any of those ideas

The window for applications opens in a few hours. Good luck everyone!

As a developer, I have to say that FOSS.IN/08 is possibly the most productive conference I’ve been to until now! In just 5 days, I’ve got more things done than I have in the last 5 months

Let’s start with the Beacon workout: Nandeep joined us via VoIP and we got started almost immediately, thanks to the dynamic nature and small size of our project – we didn’t have any infrastructural trouble as a few other C/C++ projects with huge codebases and complex build systems did. We had a list of 6 tasks in mind, and we managed to complete 3 of them. Salil Kothadia got started with writing a PDO data backend, and promptly submitted the patch to us next day. Thanks Salil, hope you continue to contribute to the development of Beacon (thereby increasing the development team size by 25%)!

I also attended Philip’s workout on porting HTML::Template to Javascript. As mentioned on the Wiki page, we mostly worked on the design during the first half or so, and then moved on to writing a skeleton for the whole framework. I think this is an extremely interesting project, and am very happy to be associated with its birth. Hope we can continue the momentum and work until it is finished.

Perhaps the biggest take-away from the conference for me was the ability to give a lightning talk about Glendix, with several kernel hackers present in the audience. Christoph then kindly offered to review some of the patches during the workout. Even the possibility of Plan 9 binary emulation being considered for inclusion into the main kernel tree is amazing, let alone the fact that I got the guidance of an experienced kernel hacker for a good 2 hours! I think the effort was largely successful – I now have a better idea of what I need to do in order to get a kernel patch in order, and also got a few hints as to how I can implement the missing bits.

My primary focus at the conference was to give a talk on Mozilla Labs and Innovation. I think I managed to stir up a decent amount of interest in the various Labs initiatives. I covered the different ways in which members of the community can contribute, specifically focussing on Weave, Ubiquity and the Concept Series. We even covered how easy it is to actually write an Ubiquity command. I now look forward to increased participation by the Indian Mozilla community in Labs projects. Don’t forget to thank Mary for all the goodies!

All this, apart from regular conference happenings like catching up with old friends, making new ones and free swag (great mugs and t-shirts this time around) makes FOSS.IN/08 one of the most successful conferences I’ve been to so far! I can’t wait for the 2009 edition

Glendix has been making the rounds on the web lately, with coverage from OSNews, Reddit, and even a Russian site, Linux.org.ru. It’s really motivating to see some buzz around the project, giving me the extra boost needed to push the completion of a usable release!

Some of the major obstacles that have to be crossed before making a beta-quality release are the completion of critical synthetic filesystems – especially /net and /draw. We also have to work out the kinks with per-process namespaces and union mounts. I don’t know if we’ll be able to get Rio running as a WM before an official release; if not, Plan9Port’s Rio and WMII are good candidates.

The biggest criticism of Glendix seems to be the reasoning that Plan9 user-space tools are somehow superior to their GNU counterparts, and several people have asked us to substantiate our claims. At this point, however, I don’t think that it is really important, or even relevant. Even if Plan9 user-space tools aren’t somehow better – I think it is generally a refreshing idea to see Linux combined with a user-space other than GNU.

We’re at FOSS.IN this year, where a large number of kernel developers have gathered; and I hope to get some of their valuable input on the problems Glendix currently faces, and maybe even write some code to solve them

Thanks for all the community support and critical comments, they are all very vital feedback for the project and are much appreciated!

Continuing with my travel spree, I made a trip to Volos, Greece and back for the 3rd International workshop on Plan 9. I was to make short presentation on Glendix, a paper on which was selected for the workshop.

Being a brown single guy in his early twenties, traveling around Europe is not exactly fun. Just saying. I got picked out not once, not twice, but THREE times for “random passport and security checks”. Once in Munich on my way to Volos, once on the streets of Athens, and finally on my way back at the Frankfurt airport. Not that I’m complaning, they were just doing their job; but really, they need to get better at profiling.

Athens is a really nice city, though it reminds me of India: crowded trains, chaotic traffic and sketchy bus stations. I knew most of the Greek symbols, thanks to high school Math courses, but pronouncing them wasn’t easy. Thankfully, the people at the counters in the Airport, Bus and Metro stations knew English. After a 5 hour bus ride, I reached Volos in the wee hours of Thursday. After around 3 hours of sleep and about 30 minutes of slide preparation, I was set for my talk.

Considering I was at a Plan 9 conference, talking about integrating it into Linux, my talk was very well received. Certainly beyond my general expectations: I got some really excellent questions, comments and general observations, and most importantly, a lot of help on the current issues that Glendix faces. All the other talks during the conference were extremely interesting as well, I was particularly fascinated by the concept of “Upperware”, the Inferno port to Nindento DS, and the Mrph morphological analyzer. Do check out the entire conference proceedings.

It was great to finally meet all the Plan 9 and Bell Labs folks in-person, especially: Sape Mullender, Charles Forsyth, Bruce Ellis; not to mention the IRC regulars uriel, quintile, sqweek and fgb!

The return trip was a bit more scenic, thanks to it being afternoon. After spening the night in Athens, I was back in Amsterdam the next day. More adventures followed, but that’s for another blog post.

Nothing like a trip to IWP9 to humble you!

P.S. Cool Glenda goodies for sale at Cafepress