REPORT: Buckets - Full Coverage
see also: py-wsclustr, testing py-wsclustr and selflesh's photostream
This blog post is full of links.
#py-wclustrThis blog post is full of links.
#pandaUntitled Intimacies
I've been threatening to write an extra-long and extra-twisty blog post ever since PaperCamp that would tie a whole bunch of related ideas together (like a set of headphones knotted in your pocket). I can't even remember what half of them are, or how they fit together, now and the other half continue to be bogged down in messy coding and packaging details. So, I figure I might as well try to kick the problem in the untitled intimacy with the your ~/bin directory
post and see what happens. Also, Myles complained about not having anything to read.
A few years ago, the now defunct oreillynet.com website ran a series of articles called something like your ~/bin directory
. The idea was to get people to write about the quick and dirty scripts they wrote for themselves to solve the variety of problems and hurdles they encountered every day. They often weren't very elegant but they worked — they just fucking did it — at least for that person. It was a lovely series because it was an especially confortable window on which to sit and see how a person solved a problem and, just as importantly, which issues they simply chose to side-step.
Everything is easier when you're just writing for yourself, right?
I was mentioning all this to Seth one day, during a meeting when I'd showed him the Untitled Intimacies Twitter-posts-on-a-map pictures I'd been creating and uploading to Flickr. These are selected Twitter messages that have been geotagged and plotted on a stylized map with a big honking pinwin containing a cropped image of Twitter's own big honking pinwin, generated using a rasterized version of the site's HTML. I could have also used the API to draw the text of a post but I like the idea of including the actual markup specific to a time and in place both the user and site's history.
Layers of context.
You can draw a pretty straight line from the early Net::Flickr::Geo maps to the pinwins and then from history box to history box to history box to these and then back again to delicious maps and, carrying on the grand tradition of one name stupider than the next, bucket
maps. There's a lot to talk about in all of that; I'm pretty sure that's at least one end to the thread with...no end, that I mentioned above. These days, it mostly just comes out in the form of drunken exhaustion, angry ranting and borderline character assasinations.
But that's a story for another day.
What I was talking about, that day, was trying to do a your ~/bin directory
style post about the code that generates the Twitter maps because I figured someone else might want to geotag
their Twitter posts and I thought it would be nice to share. In that way that you want to share where you don't want any of the burden of maintaining the code or making it work anywhere but your own setup.
Enter GitHub.
I come slowly to revision control systems in part because they go in and out of fashion at roughly the same rate as boy-boy and girly-girl bands. I believe that some, like Git, can have real advantages over others or have been tailored to fit a specific class of projects but they are not the ones I work on. My needs are pretty pedestrian and while there may be perfectly good reasons that git commit does both exactly what it sounds like and not at all what you'd normally expect, given the semantics of all the version control systems that came before it, the disconnect has always made me a little wary of making the effort.
There's also the part where it seems like every single introductory HOWTO on the subject of Git fails the Hello World
test by requiring what feels like four times as many steps to do the most basic of things, like this:
cvs checkout cd ; touch bar cvs add bar cvs commit -mur mum uses svnbar
Git feels like the Java of version control which is fine since Java's good at a lot of things but generally seems to come at the cost of making simple things a nuisance. But hey, it also seems ideal for your ~/bin directory
style projects! That's what Paul Hammond did when he released his MiniMuni webapp. He just slapped it up on GitHub and said:
As the about page says, if you live exactly 6 minutes from Sunset Tunnel East Portal, 8 minutes from Duboce and Church, and 10 minutes from Church Station you may find it useful too.
Within a day, two or three people had found it useful enough to fork and to tweak to work within whatever (n) minute walk from the MUNI they lived near. I like this. I like this because it's fast and cheap and no one asked Paul to make his code any more complicated than it needed it to be for him or fall in to the rabbit hole of abstraction; first another MUNI stop, then another transportation system, then the moon.
Simon Wistow is not wrong in his takedown of the Gitastic habit of forking first and asking questions later but I do think there is a place for the sort of rapid cloning of simple projects described above and GitHub seems to make it easier than most.
Which is what I chose to hear Seth say to me that day, even if he didn't. So, here it is:
http://github.com/straup/untitled-intimacies/
It's mostly written in Perl and will require you to install stuff using the scary
CPAN. It currently only works on a Mac because it uses webkit2png and requires PyObjC. It shells out to Python. Twice. If you have a custom background you will need to patch the crop_tweet.py script which has about as much grace as a land mine. Depending on how your Mac is set up it shells out to Python twice, to two different versions of Python; if you can get PyObjC to build out of MacPorts, more power to you. It effectively shells out three times if you count the call to the ModestMaps ws-pinwin server running on localhost (and which is not started automatically).
Actually, if your Twitter stream is not public it will probably work on a (not a Mac) because the part where you could pass your (Twitter) credentials in a GET request suddenly stopped working the other day and webkit2png, as it is written, doesn't read from a cookies file so I modified the code to work with a screen grab.
# Usage (simple) $> perl ./map_post.pl -c your.cfg -l '45.123,-37.939' -u http://twitter.com/you/status/1234 # Use a screenshot instead of webkit2png (usually, because Twitter is doing # weird auth-y stuff they won't explain to anyone...) $> perl ./map_post.pl -c your.cfg -l '45.123,-37.939' -u /path/to/screenshot
I'd like to think that it would be as easy to fix as fetching a Twitter page using curl -c and rendering it as an image using webkit2png on a local file. But given how completely hosed Twitter's persistent cookies have been in the last month a more likely scenario will involve using WWW::Mechanize and pretending to actually log in to the site itself before writing an HTML file for the Twitter message in question to disk.
Update. Or like this which hasn't yet been plugged in to the rest of the code : http://github.com/.../fetch_tweet.pl. I love Perl.
But otherwise it just works. I have pictures, and everything, to prove it.
There are a lot of moving parts. Most are things that I use, or prefer to use, for a variety of related tasks: Python for maps and images, Perl for Flickr, Java for barcodes and so on. The important part in that is: preferred. I briefly considered rewriting map_post.pl in Python so that it could load the ModestMaps code natively but then realized that I had already written an HTTP wrapper in Perl, I would have to rewrite all the Flickr code in Python and mostly I just wanted something that worked now
and didn't really care about building a shiny tower of idealized beautiful
code. I can live with that.
All in all, I've probably forgotten some of grunt work in setting things up but, then again, these days almost everything installs out of one package manager or another so it's mostly a question of typing install x
over and over while you read the paper for a little while.
Welcome to my ~/bin directory.
This blog post is full of links.
#intimaciesNo training for trouble-shooting
I was asked to do a short presentation to the Flick team on the stuff I saw at ETech 2009. I don’t normally take notes at conferences, and I was only there for two days, so this was the best I could do on short notice. There is only one image.
It is also difficult to go to a conference like ETech and not confuse, or project, what you’re doing with what people are talking about so I didn’t bother trying to do otherwise. I’m not going to get in to what those connections are, or might be, because that would be kissing and telling so you’ll just have to find your own associations.
I would love to hear what other people would say using only these slides as the source material for their own talk. This one is taken from the last slide from Eric Paulos’ presentation which in turn was taken from a Chinese proverb.
It is poor form to talk about climate change, and personal tprints, and then have the nerve to say things like: If everyone lived like me, we’d need 42 planets. I have no idea if I said this or if Chris did...
I would love to be able to think of something other than iTunes when I look at an Airport Express but since the hardware and software is designed (read: locked) to prevent that, Mike’s point bears that much more truthiness.
As an aside (to ETech entirely) these people have been researching using both tags and explicit geo data, associated with Flickr photos, to infer points of interest in large cities. It turns out that the Apple store is the fifth most interesting “place” in New York City.
Tarikh’s talk is near the 36 minute mark. Meanwhile, it is comforting (I think) to know that someone can find a deeper meaning in Ashot passed out on the couch.
This is me projecting, again, and getting all weak in the knees at the idea of using Aaron’s flight pattern map tiles for Flickr photos that have been geotagged at airports.
This one seems really important to me. We don’t do enough of it in general and certainly not when we’re all hand-waving and making happy-talk about the future.
A collection of interfaces becomes a network; A collection of networks becomes a territory; A territory exposes interfaces.
Bruno Latour; I’m not sure that I buy this in practice but it is interesting protective gear for touching the possibility jelly with.
It’s like FireEagle for your sensor-world, complete with all the sticky questions about (near) real-time and historical data, privacy and two-way sync.
There's also a PDF version available here, for safe-keeping.
This blog post is full of links.
#etech09