Crowley Code! 
 (Take 12)

PownceFS 2008/03/22

The magical code push finally happened!  Time to send PownceFS into the Internet.  PownceFS is a Fuse filesystem that mounts your friends' files from Pownce as a local filesystem.  Given a mountpoint, it fakes a directory for each of your friends and puts all the files they've uploaded inside.  Files are cached locally for a while so you can repeatedly access things without spending your life waiting on yet another API call.

I can ls -l Mike Malone now!

rcrowley@banzai:~/powncefs$ ./powncefs.py mnt
rcrowley@banzai:~/powncefs$ ls -l mnt/
total 0
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 ajvchuk
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 andreiz
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 Andrew
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 chaddickerson
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 cowsandmilk
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 eston
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 evilcindy
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 flawedartist
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 halletecco
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 iamcal
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 ianloic
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 karaemurphy
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 katherinerose1224
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 katiejane
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 laughingsquid
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 leahculver
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 mmalone
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 philfreo
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 phil_halley
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 rcrowley
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 tychay
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 waferbaby
rcrowley@banzai:~/powncefs$ ls -l mnt/mmalone/
total 41429
-r--r--r-- 1 rcrowley users    59381 1969-12-31 16:00 loveletter.jpg
-r--r--r-- 1 rcrowley users   291326 1969-12-31 16:00 mule.png
-r--r--r-- 1 rcrowley users     3830 1969-12-31 16:00 powncer.zip
-r--r--r-- 1 rcrowley users    75202 1969-12-31 16:00 sxsw_badge.jpg
-r--r--r-- 1 rcrowley users 41992791 1969-12-31 16:00 TWiT0134H.mp3
rcrowley@banzai:~/powncefs$ fusermount -u mnt
rcrowley@banzai:~/powncefs$

How can I get some, too?

On Linux it's simply a matter of installing Fuse, python-fuse and the Python OAuth library.

$ sudo apt-get install python-fuse
$ cd /usr/lib/python2.5/site-packages/
$ sudo svn co http://oauth.googlecode.com/svn/code/python/oauth

On a Mac I think it's pretty easy but my Python's wonky.  Methinks the dependencies work themselves out — can someone verify?

$ sudo port install fuse-bindings-python
$ cd <wherever the hell Python is>
$ sudo svn co http://oauth.googlecode.com/svn/code/python/oauth

powncefs-0.1.tar.bz2 or http://svn.rcrowley.org/svn/powncefs/ PownceFS on GitHub

Nerd out with me

The "architecture" (if you can really call something with 2 files an architecture) goes something like this: when you run the script a PownceFS object is created.  The first thing it does is authenticate with Pownce and pull down your friends list to create it's directory structure.  Whenever necessary it makes an additional API call to Pownce to fetch the list of files for a particular user (say, if you ls their directory).  API calls and files are cached for 45 minutes (safely under the one hour TTL of the S3 URLs returned by the API).

The root of the filesystem is a PownceFS.Base object that contains a dictionary of usernames to PownceFS.User objects.  Each PownceFS.User contains a dictionary of filenames to PownceFS.File objects (look, Ma, inheritance!).

The main PownceFS object implements system call equivalents like getattr and readdir but defers the thinking to the tree of PownceFS.Base and friends.  Each of those objects has a getattr function handling all of the filesystem bookkeeping, get and put methods for manipulating children and a fetch method for performing any API calls that need to happen.  PownceFS.Files have an additional read method that reads the locally-cached copy of its file.

The point of that long-winded explanation is that the "architecture" is useful for absolutely any web service that might be a useful filesystem.  The api.py file handles all of the OAuth-y stuff and leaves powncefs.py fairly generic.  Hack!

Again, http://svn.rcrowley.org/svn/powncefs/ PownceFS on GitHub.

Future plans

The most obvious enhancement is to allow file uploads, which I'm planning on adding soon.  Otherwise I need to find a few minutes to loop the API calls to return more than 100 files for those Power Users.  Details, details.

Comments (27)

  1. Neat addition.

    So if you make a nice GUI version of this, is it similar to the DropBox concept? (http://www.getdropbox.com/). Because that could be pretty lucrative for Pownce.

    Unfortunately command line fun won't capture the masses.

    — Dimitry — 2008/03/22 1:20 pm

  2. The beauty is that I wouldn't have to "make a nice GUI," because it'd just work.  Right now it's like a read-only folder in Finder.  Soon it'll be a read-write folder in Finder.

    — Richard Crowley — 2008/03/22 1:56 pm

  3. Now if only I could get it to work in OS X :(

    Damn fink.

    — Mike Malone — 2008/03/22 11:17 pm

  4. Plus++.

    This is rad ..  I love how the "REST is like a filesystem" connection is starting to sprout legs.

    — Michal Migurski — 2008/03/22 11:18 pm

  5. [...  PownceFS, the Pownce File System. Per-buddy subdirectories containing their uploaded files. [...]

    — Hendrik Mans » PownceFS — 2008/03/23 3:22 am

  6. [...  PownceFS — Richard Crowley’s blog Richard pownceFS mountolasa (tags: fuse pownce api python web programming social macfuse richardcrowley filesystem) [...]

    — links for 2008-03-23@kobak pont org — 2008/03/23 2:19 pm

  7. [...  Richard Crowley’s blog. I found there three cool hacks, specially the last two. The first is PownceFS and implementation of your Pownce friends archives as a filesystem. Systems like this and like [...]

    — Alcides Fonseca - Geekish Links — 2008/03/23 3:35 pm

  8. [...  PownceFS Uses FUSE to set up a directory with all the files your friends have uploaded on Pownce. Nifty. Also OAuth, which is showing up all over these days. (tags: fuse oauth pownce python) [...]

    — links for 2008-03-24 | SOCIALMOB — 2008/03/23 4:35 pm

  9. I might have to cave in and try this Pownce thing..  this looks sexy. Does this mean I can have easier access to your porn stash?

    — Mike Panchenko — 2008/03/23 8:45 pm

  10. Hey Richard, I gave installing PownceFS on a fairly clean 10.5 setup and documented it on my blog.  I was able to get past the MacPorts problems (there were problems first w/ my having a prior MacFUSE installation, then w/ libfuse's lack of CF linking) and got far enough to get past any fatal errors, but haven't been able to actually list or read files.

    — leonard — 2008/03/24 1:38 am

  11. [...  PownceFS (Richard Crowley) [...]

    — Dew Drop - March 22, 2008 | Alvin Ashcraft's Morning Dew — 2008/03/24 4:08 am

  12. [...  (Source: PownceFS — Richard Crowley’s blog) [...]

    — PownceFS by Richard Crowley | Hi, I’m Colin Devroe. — 2008/03/24 5:36 am

  13. That is way too rad.  I have this working in ubuntu, I'll have to try OS X when I get home.

    — K. Adam Christensen — 2008/03/24 9:19 am

  14. Err, just to note you also need python-json installed

    — Xipietotec — 2008/03/24 5:33 pm

  15. Richard, this is awesome. I so want this but I'm not quite there yet. Blame it on my linux ignorance..

    I'm using Ubuntu gutsy. I'm sooo close.

    I added svn, python-fuse and python-json.

    Now I got a permission error. I tried using sudo and got it to run on /mnt withtout the permission error. Now that folder is kind of screwed. I've been using Linux for a while, but this mountpoint stuff is throwing me off.

    Here's a sample. Any ubuntu folks have comments?

    mrjabba@ionia:~/dev/python/powncefs/trunk$ sudo python ./powncefs.py /mnt fuse: mountpoint is not empty fuse: if you are sure this is safe, use the 'nonempty' mount option Traceback (most recent call last): File "./powncefs.py", line 324, in main() File "./powncefs.py", line 321, in main fs.main() File "./powncefs.py", line 275, in main return Fuse.main(self, *a, **kw) File "/usr/lib/python2.5/site-packages/fuse.py", line 713, in main main(**d) fuse.FuseError: filesystem initialization failed mrjabba@ionia:~/dev/python/powncefs/trunk$

    I'll blog this and work on it in time :-) Also, I'm kind of new to Python as well.

    K

    — Kevin H — 2008/03/25 4:38 pm

  16. @Kevin: may I suggest getting rid of that slash in front of mnt?  In Richard's example, he's in a folder just off his home directory called ~/powncefs.  He then mounts the Fuse filesystem into a directory called mnt, so it is located at ~/powncefs/mnt.  What you're doing is trying to put it in /mnt , which isn't a good idea.  If you put it in something like /mnt/powncefs, that might be acceptable, but still not a good idea.

    — David Hall — 2008/03/25 5:40 pm

  17. Also, I should note that powncefs has the +x bit when it's checked out, so you don't need that python, and when you put it in a directory you control instead of one that is owned by root, you don't need to do sudo.  And you don't need to be sudo to look at it in things like your file browser.  Just don't put it in /mnt is essentially what all that means...

    — David Hall — 2008/03/25 5:43 pm

  18. What DHall said.

    — Richard Crowley — 2008/03/25 7:49 pm

  19. Um add a "Comments" link to the bottom of your posts. I'm far too lazy to scroll back up to the top of the page to click on the title - and you know this!

    — Mike Panchenko — 2008/03/26 6:30 am

  20. Thanks for the help guys. I obviously misread the docs the 1st time. So, you're just creating a folder. I thought "mnt" was significant.

    OK. Got it. However, I had to modify my permissions on fuse and fusermount. I followed the advice on the FUSE FAQ: http://fuse.sourceforge.net/wiki/index.php/FAQ

    I ran it again and it worked. My friend's folders show up now. Wow. This is so cool.

    Thanks SO much!

    Kevin

    — Kevin H — 2008/03/27 5:03 pm

  21. I managed to run powncefs and mount properly.

    but...

    ls: cannot open directory .  Invalid argument

    Is the error I get when trying to enter the mounted directory. Why is that?

    — some dude — 2008/04/03 7:34 am

  22. [...  pownce, powncefs, python Well, the first version of PownceFS didn’t turn out as well as I had hoped. After I put it out in the world, Leonard ran with it [...]

    — PownceFS, round two — Richard Crowley’s blog — 2008/06/04 10:03 pm

  23. [...  PownceFS - Fuse filesystem for Pownce files. [...]

    — Where I Can Get Desktop Client Of Pownce.com? « TECH IDEA — 2008/06/07 4:22 am

Richard Crowley?  Kentuckian engineer who cooks and eats in between bicycling and beering.

I blog mostly about programming and databases.  Browse by month or tag.

To blame for...


© 2009 Richard Crowley.  Managed by Bashpress.