crowley code!

Archive for March, 2008

3/22 PownceFS (21)

fuse, pownce, python

The magical code push finally happened! Time to send PownceFS into the Internet. PownceFS is a Fuse filesystem that mounts your friends’ files from Pownce as a local filesystem. Given a mountpoint, it fakes a directory for each of your friends and puts all the files they’ve uploaded inside. Files are cached locally for a while so you can repeatedly access things without spending your life waiting on yet another API call.

I can ls -l Mike Malone now!

rcrowley@banzai:~/powncefs$ ./powncefs.py mnt
rcrowley@banzai:~/powncefs$ ls -l mnt/
total 0
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 ajvchuk
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 andreiz
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 Andrew
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 chaddickerson
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 cowsandmilk
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 eston
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 evilcindy
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 flawedartist
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 halletecco
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 iamcal
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 ianloic
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 karaemurphy
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 katherinerose1224
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 katiejane
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 laughingsquid
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 leahculver
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 mmalone
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 philfreo
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 phil_halley
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 rcrowley
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 tychay
dr-xr-xr-x 2 rcrowley users 0 1969-12-31 16:00 waferbaby
rcrowley@banzai:~/powncefs$ ls -l mnt/mmalone/
total 41429
-r--r--r-- 1 rcrowley users    59381 1969-12-31 16:00 loveletter.jpg
-r--r--r-- 1 rcrowley users   291326 1969-12-31 16:00 mule.png
-r--r--r-- 1 rcrowley users     3830 1969-12-31 16:00 powncer.zip
-r--r--r-- 1 rcrowley users    75202 1969-12-31 16:00 sxsw_badge.jpg
-r--r--r-- 1 rcrowley users 41992791 1969-12-31 16:00 TWiT0134H.mp3
rcrowley@banzai:~/powncefs$ fusermount -u mnt
rcrowley@banzai:~/powncefs$

How can I get some, too?

On Linux it’s simply a matter of installing Fuse, python-fuse and the Python OAuth library.

$ sudo apt-get install python-fuse
$ cd /usr/lib/python2.5/site-packages/
$ sudo svn co http://oauth.googlecode.com/svn/code/python/oauth

On a Mac I think it’s pretty easy but my Python’s wonky. Methinks the dependencies work themselves out — can someone verify?

$ sudo port install fuse-bindings-python
$ cd <wherever the hell Python is>
$ sudo svn co http://oauth.googlecode.com/svn/code/python/oauth

powncefs-0.1.tar.bz2 or http://svn.rcrowley.org/svn/powncefs/

Nerd out with me

The “architecture” (if you can really call something with 2 files an architecture) goes something like this: when you run the script a PownceFS object is created. The first thing it does is authenticate with Pownce and pull down your friends list to create it’s directory structure. Whenever necessary it makes an additional API call to Pownce to fetch the list of files for a particular user (say, if you ls their directory). API calls and files are cached for 45 minutes (safely under the one hour TTL of the S3 URLs returned by the API).

The root of the filesystem is a PownceFS.Base object that contains a dictionary of usernames to PownceFS.User objects. Each PownceFS.User contains a dictionary of filenames to PownceFS.File objects (look, Ma, inheritance!).

The main PownceFS object implements system call equivalents like getattr and readdir but defers the thinking to the tree of PownceFS.Base and friends. Each of those objects has a getattr function handling all of the filesystem bookkeeping, get and put methods for manipulating children and a fetch method for performing any API calls that need to happen. PownceFS.Files have an additional read method that reads the locally-cached copy of its file.

The point of that long-winded explanation is that the “architecture” is useful for absolutely any web service that might be a useful filesystem. The api.py file handles all of the OAuth-y stuff and leaves powncefs.py fairly generic. Hack!

Again, http://svn.rcrowley.org/svn/powncefs/.

Future plans

The most obvious enhancement is to allow file uploads, which I’m planning on adding soon. Otherwise I need to find a few minutes to loop the API calls to return more than 100 files for those Power Users. Details, details.

3/19 Oh yeah, that’s right, I went to SXSW (1)

conference, sxsw

I wrote this about a week ago and then forgot about it. Best course of action is to just post it.

The hangover has ended, time for some reflection.

There were so many people this year that it expanded into new parts of the Austin Convention CenterCave System I didn’t know existed. The increased attendance was most noticeable at the parties where I frequently had to request favors to gain entry. Thanks, pals.

The panel schedule this year was, to say the least, weird. There were days on end that contained nothing of interest to me, even given my temporarily broadened interests (more on that later). Emotional design? Gaming? Continued beating of the Internet video dead horse? No thanks across the board. Where was Khoi Vinh with a followup to his excellent grid design presentation from last year? Where were Mark Boulton and Richard Rutter with more typography? Hell, where were the Rails folks with at least something technical? Answer: they were all at the same time on Monday afternoon. I heard someone say they put all of the good panels in parallel to force people to choose so they wouldn’t have one one ginormous panel in each block with empty houses everywhere else. Slightly logical and incredibly annoying.

A quick list of favorites: the Kevin Rose led and Cal Henderson profaned panel on scalability; John Resig and company’s panel on JavaScript libraries (Alex Russell and Andrew Dupont were hilarious); and icanhascheezburger. In other words, I didn’t learn a great deal.

I’ve been kicking around this idea in my head that started as my explanation for what I hope to get out of South-By: SXSW is a necessary yearly dose of design for engineers. With as much fun as the “How to Rawk…” panels were to watch (again, learning absolutely nothing), it seems like a panel along those lines would float pretty well. Thoughts?

Anyway I’ve finally gotten back to the real worldInternet and been hacking up some good sauce. More on that whenever I damn well please.

3/3 Functional programming in C (3)

c

I’ve been meaning to write this post for some time to document how I avoid the giant-if-statement-with-strcmp() problempattern when writing C programs.

The solution to every problem in C is to use pointers. Keeping with that lemma, it’s possible, beneficial and even fun to make pointers to functions do your bidding without the pile of if or switch statements typical of most student programming projects. It’s really close enough that I started calling it functional programming years ago even though this is still a far cry from Lisp/Python/JavaScript/whatever. It doesn’t matter because C is still awesome.

Use the STL, Luke

I’m cheating a bit when I say functional programming in C. The use case I’m going to present is a C++ sample but there’s no reason why you can’t hash your own table in straight C99. I just don’t care to myself. The example uses the STL map class to map strings to functions, making it trivially easy to build your own shell-like app that takes some argument and dispatches a function to make it so. The map provides the nice dispatch-y interface but there’s no reason you couldn’t do something like this in plain C:

void foo(int i) {
    printf("You called foo(%d)!\n", i);
}

void bar(void(*)(int) f) {
    (*f)(2);
}

Anyone fluent in a scripting language that isn’t PHP will likely understand exactly what’s going on here. foo is a simple function that does nothing interesting. bar is a function that takes as its argument a pointer to a function. That function can’t return anything and must take a single integer argument. Many will call foul here saying that the strong type restrictions here make this sort of “functional programming” rather useless. I say nay and cite curvr.cc as my example. In any language, a function lacking any expectations for its arguments will be rather tedious. C doesn’t give you this as a way to shoot yourself in the foot (there are so many other ways, however).

Function pointer declaration

The simplest way I know to explain the difficult syntax of function pointer declaration is an analogy to something easy like a character and a pointer to a character.

char c : char * p :: void foo(int i) : void (*)(int)

The first half declares a character, c and a pointer, p that can point to c. Similarly, the second half declares a void function foo that takes an integer argument and a pointer to a void function that takes an integer argument.

Now for the useful part

Using this pattern for creating function pointers we can now define a few functions and map them in a useful way. This is again taken from my curvr.cc file (more info about curvr).

// First a few function prototypes to map
int curve(Magick::Image & img);
int bigcurve(Magick::Image & img);
int anticurve(Magick::Image & img);

// Then the map part
int main(int argc, char * * argv) {
    // Magically figure out a string called process

    // Setup the map using the address operator (&)
    std::map<std::string, int (*)(Magick::Image &)> processes;
    processes["curve"] = &curve;
    processes["bigcurve"] = &bigcurve;
    processes["anticurve"] = &anticurve;

    // Call a function from the map
    (*processes[process])(img);

}

Your code will be shorter now. Go forth and hack.