Christmas came early this year.

Today, Google announced their new open source systems programming language: Go. I’m super excited about this, we all have been wondering what Rob Pike has been upto since he joined the big G, and now we know. Not just that, but Ken Thomson, Robert Griesemer, Ian Taylor and Russ Cox were all involved in the project, with Ken doing what he does best, writing compilers in lightning speed If that isn’t a list of heavyweight respectable computer scientists, I don’t know what is!

I think Go is poised to be the dominant systems programming language of the future. Go has nailed almost every aspect of a systems language, though some would say I’m biased. Go has been strongly influenced by Oberon, CSP languages like Limbo, and the standard libraries have tantalizing similarities to Plan 9. We’ve had Limbo and Plan 9 for a while now (more than a decade), but this is where my real love for Google begins to bubble, they took something awesome but unpopular and gave it a push to the masses. There are very few companies in the world who would attract the talent to do this, and even fewer who would open source the results. The attention Go has been getting is just mind blowing. Pike had been doing amazing work at Bell-Labs for quite a while, but none of it even got an inkling of the publicity Go is currently getting.

Google was what Pike needed to prove Utah2000 wrong.

I know one thing for sure, I’ll definitely be using my Plan 9 virtual machine a lot less; now that I can write clean concurrent programs that don’t make my head hurt, both in Linux and OS X. And GCC, I’m not shedding any tears while I bid you goodbye.

On another note, Google also announced today that they’ll be sponsoring free WiFi at a whole bunch of US airports this holiday season. For all its faults, Google definitely seems to be doing the right thing. For how long, it remains to be seen, but so far I’d say their track record has been better than excellent.

UPDATE: John Gruber points out that “judging from the copyright statements, [Go is] not an official Google project”. Could this be a result of the famous 20% time scheme?

The mere presence of systems like OpenID, Facebook Connect and a host of other identity services on the web today is attestation to the fact.

Authentication should be a feature of the protocol, not something that relies on hacks like cookies. 99% of websites today rely on cookies for authentication for their websites, besides offering custom registration and login pages. This means the browser, as the user’s agent, has no clue of what is going on. A user is forced to manually track myriads of accounts, remember passwords for each of them, and remember what personal information each of them holds. Sure, part of the problem is solved by using password managers (like the one in-built into Firefox, or external programs like 1Password), but even these programs rely on heuristic algorithms to determine if something looks like a login credential or not. There’s no explicit way for web pages to tell your browser: “This is a login form, please fill in details of the user’s identity here” or “These pages are privileged, please give me the user’s identity”. Why is that?

Actually, there is such a mechanism: HTTP based Authentication has been a feature present since HTTP/1.0, but only 1% of sites actually use it. The reason for that is purely cosmetic, most browsers display a very bland modal dialog when it encounters a page that requires HTTP Auth, and sites are unable to customize that interaction. So, the technically right way to do things sucks from a user experience perspective, and websites started adopting alternate means. Someone discovered they could use cookies to store session information on the client, and the whole situation exploded ever since. As a programmer, I feel very sad when I think about the fact that instead of fixing the problem in HTTP/1.1, web-based authentication took the route it did and led to the mess we are in today.

However, I must also state that HTTP authentication doesn’t solve the entire problem – there is still the issue of users having to create an account for every site they want to be part of. This is because there existed no protocols to federate and provide decentralized authentication. That is, until OpenID and OAuth came about. Now we’re at this exciting juncture, and the browser is in a unique position to use these tools together to provide the user with an experience that is secure and easy to use. Every architect will agree that it is indeed a fun challenge to use the state of identity on the web today and make it into something awesome.

This is precisely what the Mozilla Labs team has been thinking about for a while now. Sometime ago, we added support for automagic one-click OpenID logins to Weave. We plan to spin that “feature” out into it’s own extension and build on it, something we call “Weave Identity“, part of the broader “Open Identity” initiative by the Labs. “Weave Sync“, the original extension, will just focus on the synchronization parts so we can tackle these two different problems separately.

So, how exactly are we planning on doing this? Take a look at an initial version of a document describing an in-browser “Account Manager“. We’ve also put up a WEP (which expands to Weave Enhancement Proposal, by the way) describing the raw form of a specification for automatic actions on websites, such as user registration or password changes.

Keep in mind that all of this is in its very early stages (pre-alpha); but that also means it’s a great opportunity for the community to get involved! What are your thoughts on Open Identity? Use the discussion tab on any of those Wiki pages, start a thread on the Mozilla Labs group, or simply leave a comment on this blog entry, and chip in – we’d love to hear from you!

The grand Summer of Code Mentor Summit of 2009 concluded last week and I had the fantastic opportunity of being able to attend on behalf of Gentoo, Plan 9 and Mozilla. What follows is some indication of how awesome the summit was:

(Photo courtesy of warthog from Etherboot)

I met so many folks I’d only interacted with online so far (the classic nickname-to-face matching), but even better was the opportunity to meet folks powering open source projects from so many diverse backgrounds. I met many of my personal rockstars, and learned about a bunch of open source projects I’d never heard of

Also, one of the things that is only possible at an event like the summit was the ability to get a whole bunch of non-linux operating system groups in one room. We had a great discussion, and it resulted in the creation of the “rosetta-os” special interest group. Look for more activity on the common device drivers for non-linux operating systems front soon!

Other sessions worthy of special mention were Open Source Security, Recruiting and Retaining Awesome People, Advanced Trolling (yes, you read that right), and of course the always welcoming Casablanca where I spent most of my time. We discussed everything from our SoC experiences to the Afro Celt Sound System in that room, always full of creative energy and warmth.

After 4 years of participating in the Summer of Code, I am super happy to have finally met the faces behind the program. Every single person I met over the course of last weekend was friendly, intelligent and just generally awesome; that sort of thing doesn’t happen by chance. I feel warm and fuzzy inside to think that I’m actually a part of the revolution that is free and open source software, three cheers to everyone that made it possible!

I usually never pay attention to the auto complete suggestions offered by Google, until now, when I found they can be quite amusing. What’s even more fun though, is repeating a search across different country specific sites, to get a idea of what the people of that nation are most worried about. Here’s a sampling, let’s start with India:

#1 certainly explains India’s growing population. We’re also quite obsessive about learning proper English (Outsour Singh is desperately looking to land that call center job) and hacking Orkut accounts. Now, for the Netherlands:

I guess the one take-away from this is that the English speaking Dutch population (which is quite large, mind you) are mostly looking for more info on some romantic comedy from Hollywood. I was also curious about the results for the USA:

Hmm, why are there so many Americans wanting to learn to “tie a tie”? “How to solve a Rubix cube” is about the only intellectual entry to appear on the suggestion list among all three countries, until you realize that it’s actually spelled “Rubik’s”. I wouldn’t be surprised if “Rubix” makes the dictionary soon.

The common theme for all countries seems to be: learning to kiss. Indians are confused between “losing weight” and “reducing weight”, which also explains why everyone wants to get better at English. Some Indians also want to gain weight, a term which is most definitely absent from American searches. Our Dutch friends have no interest in either, I completely understand why; they maintain a very healthy lifestyle by cycling all over the place. The Americans have apparently mastered the art of downloading videos from Youtube, while the Indians and Dutch are still learning the ropes. American women first want to learn to get pregnant and then quickly want to get rid of the resulting stretch marks, while Indian ladies don’t bother with the latter.

The geeks out there will notice the UI improvements on the US version of Google over the other two. I think I’ll stop drawing inferences now

Try your own fun searches to see what auto-suggest has in store! Suggested starting point: “How to use”…

I’m back from the EU MozCamp in Prague and we all had a great time! Check out the slides from my talks: Labs Overview and Weave in Depth.

A few people at the MozCamp were interested in Weave’s use of cryptography to protect the user’s data and privacy. Although the specs for the Weave server are available, it may take someone new a while to wrap their head around the whole scheme. I’m going to attempt explaining what crypto operations we do and why we do it in this blog post.

First, let’s get some basic definitions out of the way. Symmetric cryptography means you have one key that can perform both encryption and decryption, and they are complementary operations. For Weave, we use AES with a 256 bit key, and we use it in a mode that requires an ‘initialization vector’ for every decryption. Asymmetric cryptography means there’s a pair of keys (usually called ‘public’ and ‘private’ keys). A piece of text “encrypted” by one key can only be “decrypted” by the other key. Here, we use RSA with a 2048 bit private key.

So, when a user first signs up for Weave using the wizard on their computer, we generate a (random) pair of public and private keys. Next, we use the user’s passphrase to create a symmetric key. This is done using a pretty standard algorithm known as PBKDF2 (short for “Password Key Derivation Function”). The PBKDF2 algorithm requires a ’salt’ value which is also stored on the server. Now that we have a symmetric key, we use it to encrypt the user’s private key and upload it along with the public key to the server. Note that the passphrase is never sent to the server, so if the user’s password ever gets compromised all the attacker can get is their encrypted private key, which really isn’t of much use (especially given that the key is 2048 bits long).

Whenever a particular “engine” is to be synchronized (an engine could be Tabs, Bookmarks, History etc.) we generate a random symmetric key for that engine. This key is then encrypted using the user’s public key (now, one can only retrieve the original symmetric key with the corresponding private key) and uploaded as being associated with a particular engine. All entries (the ‘ciphertext’ property in a “Weave Basic Object”) in that engine are encrypted with the symmetric key that was generated for it.

To make things clear, let’s enumerate the steps we would take to decrypt a single tab object for user ‘foo’:

1. Find the user’s cluster by making a GET request to https://services.mozilla.com/user/1/foo/node/weave. It returns https://sj-weave06.services.mozilla.com/.
2. Fetch the user’s encrypted private key and public key from https://sj-weave06.services.mozilla.com/0.5/foo/storage/keys/privkey and https://sj-weave06.services.mozilla.com/0.5/foo/storage/keys/pubkey respectively. The user’s password is required to access these JSON objects.
3. Ask the user for their passphrase and generate a 256 bit symmetric key from it using PBKDF2 and the ’salt’ found in the privkey object.
4. Use the generated symmetric key and the initialization vector found in the ‘iv’ property of the privkey object to decrypt the user’s private key.
5. Fetch the user’s encrypted tab objects from https://sj-weave06.services.mozilla.com/0.5/foo/storage/tabs/?full=1.
6. Fetch the corresponding symmetric key (the URL is also listed in the “encryption” property of every WBO), in this case https://sj-weave06.services.mozilla.com/0.5/foo/storage/crypto/tabs.
7. Decrypt the symmetric key with the user’s private key.
8. Use the decrypted symmetric key to decrypt any WBO from the tabs collection with the initialization vector found in the ‘bulkIV’ property of the tabs symmetric key WBO.
9. Profit.

A word about the formats in which the keys are actually stored in. All values are Base64. For symmetric keys, the key is stored as-is. For asymmetric keys, I wish we used a standard format like PKCS#12, but we don’t. It’s still ASN.1 though, in some format NSS exports private keys in. You need to do a bit of ASN.1 parsing to figure out the values you’re interested in.

Fortunately, I’ve already figured out most of the details for you – check out my Javascript or PHP implementations of the crypto elements required to decrypt Weave Basic Objects.

Finally, a quick note about why we do all this. Sharing is now reasonably easy, if you want to share your bookmarks with someone, you just need to encrypt the corresponding symmetric key with their public key and they’re good to go. Also, each WBO has it’s own ‘encryption’ property so this can be as granular as needed. Secondly, the passphrase is never stored anywhere (except possibly on the user’s computer) so the server never sees anything other than encrypted blobs of Base64′ed text. Along with making HTTPS mandatory, we think this is a pretty secure way of protecting the user’s data.

If you have other encryption schemes that might fit into Weave’s use cases please let us know! (We’ve already been looking at interesting developments in this area such as Tahoe). I’d also love to hear from you if you have any questions on our current cryptography scheme. We’re constantly trying to improve the security and efficiency of our system so these details are only valid until we change our scheme

Now, go write that third-party Weave client, you have no excuse not to!

I’m off to the beautiful city of Prague, or “Praha” as it is known locally, for the European MozCamp of 2009. Memories from the MozCamp last year are still fresh, and I’m definitely looking forward to this one!

On Friday, we’re going to be hosting a Labs Hackathon on Jetpack. This is your chance to get to know more about the framework that’s so easy to use that your mom could write an extension with it. Maybe not your Grandma though, you do need to know a bit of Javascript The hack session will last as long into the night as needed for you folks to come up with amazing ideas for Jetpacks and implement them. Drew Willcoxon from the Firefox team and I will be on hand all day to help you, so feel free to come and poke us. Oh, I almost forgot to mention that there’s Free Pizza involved.

On Saturday, I’ll be giving a talk on Weave. With 0.7 just released, we’ll be taking a look at our current state, what’s in store for the future, and maybe a few cool demos. We’re also especially interested in engaging with addon developers to see what Weave can do to make it easier for them to add sync functionality to their addons.

Be there!

My last day at Mozilla this summer was last Thursday. I didn’t take a lot of pictures this summer, because, you know, I took a lot last time around. Also, this strategy turned out pretty well because now there are more pictures of me floating around on the tubes! After a longish trans-atlantic flight, I’m back in Amsterdam now resuming work on my Master’s (because hacking on Minix is awesome).

No other internship has been ever so satisfying: over the summer, I worked on a wide range of mini-projects which allowed me to exercise skills ranging from systems to application level programming. I even did a bit of work in the mobile space (turns out programming in limited memory and processing speed is a *lot* different).

One such project that I’m especially excited about is support for video recording in the browser. Yes, there is even a canvas-based live preview of your webcam feed, in addition to Ogg/Theora encoding support! Combined with the audio recording support I wrote sometime ago, some really cool applications are now possible. Skype-like dialer in the browser? Why not?! (*hint* anyone is free to send in a patch for multiplexing the audio and video, they’re currently two separate Vorbis and Theora streams *hint*).

We also had 3 major releases for Weave during the summer: 0.4, 0.5 and 0.6. The last one was especially big, given the completely new, HTML based UI (big kudos to thunder for pulling it off!) and a bunch of other performance fixes. Also, the web UI I wrote last year underwent so many great changes by the wonderful folks at Glaxstar. Now we’re putting up a community design challenge to revamp the UI so we can ship the thing! (*hint* if you’re good at UI design you should participate in the challenge *hint*).

There’s so many more cool things I worked on that I’d like to talk about, but perhaps they deserve a separate blog post. Soon… (I keep promising myself that I should blog more often, it never works).

To add the already good times, my two students in the Summer of Code this year passed with flying colors. Yay!

Very cool.

Last night, Joseph Smarr from Plaxo was our guest speaker and he talked about how the “web is going social”, and how the “social web is going open”. We discussed all the elements that make up the social web today: identity providers, social web providers and content aggregators, and how each of  them are leveraging open standards and protocols such as OpenID and OAuth to create better experiences for their users. Check out his slides here.

This talk was a nice prelude to some interesting discussion about the role that the browser can play in handling the user’s data and identity on their behalf. Very relevant to this was also the recent experimentation by Weave on identity in the browser, and Myk gave us a demo of the auto-sign-in features.

Labs Night is also a chance for everybody to talk about cool stuff they’ve been working on, so Brandon gave us an update on what’s new in Ubiquity 0.5. There’s some really neat stuff in there: Ubiquity is possibly one of the first pieces of software that perform truly internationalized natural language parsing (0.5 rolls out with support for Japanese and Danish). Do check out this blog post for a detailed discussion of the features in 0.5.

I followed with an update on some of the work I’ve been doing with Jetpack – namely providing the capability for “jetpacks” to record audio. The code to enable this is checked into the repository, but you’ll have to wait until a release later this month if you’re not feeling brave enough to build the extension from source to play around with it. I was especially interested to know the kinds of applications that might be possible with this capability, so you if you have any ideas, I’d love to hear them. Myk also gave us a demo of the new streamlined way of subscribing to feeds using Snowl, check out this release announcement for more details on what’s new with the message reader you know you want to use!

Paul Tarjan from the Searchmonkey team at Yahoo! gave us some really cool demos demonstrating Searchmonkey Objects and YQL. I’m especially excited about YQL because it can make some of the back-end ubiquity code really simple and efficient. Incidentally, the Bing team was here at Mozilla just a couple of days ago and they also demoed some features similar to Searchmonkey Objects, albeit restricted to video and snippets of data for now.

Search is starting to feel exciting again, a sentiment similar to one we feel in the browser space today. There’s a lot of innovation in the area outside of the big daddy, and it is indeed heartening to see that major players in the web are beginning to recognize the importance of openness and competition

Labs Nights are monthly events, so we look forward to seeing you sometime in July to discuss more cool stuff that everyone’s been working on!

It’s been 3 weeks since I started my (second) summer internship at Mozilla Labs, and needless to say it’s been a blast! I’m continuing my work on Weave, besides helping out with the gamut of experiments that are currently running at the Labs. Weave is going to see some major strides forward in the near future, as we now have our very own Product Manager (Welcome, Ragavan!) in addition to the awesome Mike Connor joining the team

Within the first week of my arrival here, Mozilla made the move to the new office, which is possibly the sweetest workplace I’ve ever seen in my life. Check out selected pictures here.

There’s been the usual slew of intern activities, including, but not limited to: Canoeing, Movie nights, Birthday celebrations, Music discovery, and even a few dungeon runs on WoW

Look for more posts on labsy stuff in the near future. Peace!