The magical code push finally happened! Time to send PownceFS into the Internet. PownceFS is a Fuse filesystem that mounts your friends’ files from Pownce as a local filesystem. Given a mountpoint, it fakes a directory for each of your friends and puts all the files they’ve uploaded inside. Files are cached locally for a while so you can repeatedly access things without spending your life waiting on yet another API call.

I can ls -l Mike Malone now!

rcrowley@banzai:~/powncefs$ ./powncefs.py mnt
rcrowley@banzai:~/powncefs$ ls -l mnt/
total 0
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 ajvchuk
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 andreiz
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 Andrew
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 chaddickerson
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 cowsandmilk
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 eston
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 evilcindy
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 flawedartist
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 halletecco
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 iamcal
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 ianloic
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 karaemurphy
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 katherinerose1224
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 katiejane
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 laughingsquid
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 leahculver
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 mmalone
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 philfreo
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 phil_halley
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 rcrowley
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 tychay
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 waferbaby
rcrowley@banzai:~/powncefs$ ls -l mnt/mmalone/
total 41429
-r--r--r-- 1 rcrowley users    59381 1969-12-31 16:00 loveletter.jpg
-r--r--r-- 1 rcrowley users   291326 1969-12-31 16:00 mule.png
-r--r--r-- 1 rcrowley users     3830 1969-12-31 16:00 powncer.zip
-r--r--r-- 1 rcrowley users    75202 1969-12-31 16:00 sxsw_badge.jpg
-r--r--r-- 1 rcrowley users 41992791 1969-12-31 16:00 TWiT0134H.mp3
rcrowley@banzai:~/powncefs$ fusermount -u mnt
rcrowley@banzai:~/powncefs$

How can I get some, too?

On Linux it’s simply a matter of installing Fuse, python-fuse and the Python OAuth library.

$ sudo apt-get install python-fuse
$ cd /usr/lib/python2.5/site-packages/
$ sudo svn co http://oauth.googlecode.com/svn/code/python/oauth

On a Mac I think it’s pretty easy but my Python’s wonky. Methinks the dependencies work themselves out — can someone verify?

$ sudo port install fuse-bindings-python
$ cd <wherever the hell Python is>
$ sudo svn co http://oauth.googlecode.com/svn/code/python/oauth

powncefs-0.1.tar.bz2 or http://svn.rcrowley.org/svn/powncefs/

Nerd out with me

The “architecture” (if you can really call something with 2 files an architecture) goes something like this: when you run the script a PownceFS object is created. The first thing it does is authenticate with Pownce and pull down your friends list to create it’s directory structure. Whenever necessary it makes an additional API call to Pownce to fetch the list of files for a particular user (say, if you ls their directory). API calls and files are cached for 45 minutes (safely under the one hour TTL of the S3 URLs returned by the API).

The root of the filesystem is a PownceFS.Base object that contains a dictionary of usernames to PownceFS.User objects. Each PownceFS.User contains a dictionary of filenames to PownceFS.File objects (look, Ma, inheritance!).

The main PownceFS object implements system call equivalents like getattr and readdir but defers the thinking to the tree of PownceFS.Base and friends. Each of those objects has a getattr function handling all of the filesystem bookkeeping, get and put methods for manipulating children and a fetch method for performing any API calls that need to happen. PownceFS.Files have an additional read method that reads the locally-cached copy of its file.

The point of that long-winded explanation is that the “architecture” is useful for absolutely any web service that might be a useful filesystem. The api.py file handles all of the OAuth-y stuff and leaves powncefs.py fairly generic. Hack!

Again, http://svn.rcrowley.org/svn/powncefs/.

Future plans

The most obvious enhancement is to allow file uploads, which I’m planning on adding soon. Otherwise I need to find a few minutes to loop the API calls to return more than 100 files for those Power Users. Details, details.

The majority of my (awesomest ever) day job at Flickr is written in Javascript. I do write some C++ code for some of the trickier parts, but wc tells me more that only about a tenth of my code is C++.

That tenth doesn’t really do it for me, though. It isn’t dirty and dangerous. Because of the Netscape Portable Runtime, it’s actually rather tame. See, I spent college SEGFAULTing at least once a day. In my nearly three months at Flickr, I’ve managed to do so only twice.

Fear not — I have a way to get my fix. I’m intrigued by FUSE and started to think about the design of a Flickr filesystem. About 14 seconds after that I discovered Manish Rai Jain’s flickrfs project. It is more complicated than I would have designed, but because of that it’s incredibly powerful. Flickr isn’t out as a possible project for a FUSE filesystem but I’d rather not step on his toes. Onward.

The problem is, I’m short on imagination right now. I think the most powerful aspect of making non-filesystem data available as a filesystem is the huge set of possibilities that open up when you start chaining standard UNIX commands.

Do any C-shy webdev types want anything else out of their filesystem?