Lantern Bash Configuration Script on OSX

September 4, 2011

I’ve been working for awhile on a piece of software called Lantern. It’s a new censorship circumvention and monitoring-prevention tool built atop the LittleShoot P2P networking layer and also atop LittleProxy, another little project I’ve completely neglected to mention! You’ll be finding out a lot more about Lantern soon, but sign up for the mailing list at if you want to be notified when you can start playing with it. It’ll be fun, and ultimately should help people all over the world who don’t enjoy the relatively unfettered internet most of us take for granted.

I was actually inspired to write this post, because I just added a little bash code snippet for auto-configuring proxies on OSX using networksetup, a pac file, and some nifty for looping to read lines (trickier than you might think!). I thought others might find it useful. Enjoy!

#!/usr/bin/env bash

mkdir ~/Library/Logs/Lantern

function log() {
echo "`date`: $@" >> ~/Library/Logs/Lantern/installer.log

log "Configuring network services"
while read s;
log "Configuring network: $s"
networksetup -setautoproxyurl "$s" file://localhost$HOME/.lantern/proxy.pac || log "Could not set auto proxy URL for $s"
networksetup -setautoproxystate "$s" "on" || log "Could not turn auto proxy on for $s"
log "Configured network: $s"
done < <(networksetup -listallnetworkservices | tail +2)
log "Done configuring network services!!"


GData Maven Deploy Script

July 10, 2010

I just wrote a quick script to deploy the latest GData versions to the LittleShoot maven repository, and I thought others might find it handy. GData doesn’t deploy to a maven repo, so you’ve got to do a little extra work. There are various solutions out there, but this is a little more straightforward. I’m jumping on board with Google’s latest 2.2.x alphas of their GData Java clients, as it looks like that’s where the future action will be. So this only applies to the 2.2.x alphas. The script simply takes the version argument, checks out that version from SVN, and deploys the jar to your repo.

You’ll want to swap out your repository for the LittleShoot repository listed at the end — both the URL and repository ID. If you want to just grab this script from the LittleShoot repository, it’s here.

#!/usr/bin/env bash

# This is a script for automatically deploying the GData jars to the LittleShoot
# maven repository.


function die() {
rm -rf $gdataVersion
echo $*
exit 1

me=`basename $0`

if [ $# -ne 1 ]
"Should be a single GData version argument, as in './$me 2.2.1-alpha'. For versions, see:"
exit 1

echo "Checking out GData version $gdataVersion"
svn co$gdataVersion || die "Could not checkout SVN"

pushd $gdataVersion/gdata || die "Could not move to gdata base dir"
ant dist || die "Could not build with ant"
cd build-bin/dist/ || die "Could not cd to dist directory"

mvn deploy:deploy-file -DartifactId=gdata -Dversion=$gdataVersion -Dfile=gdata-$gdataVersion.jar -Dpackaging=jar -DgeneratePom=true -Durl=scpexe:// -DrepositoryId=littleshoot || die "Could not deploy file"

echo "Cleaning up"
rm -rf $gdataVersion

Happy Birthday Gnutella!

March 20, 2010

OK, I’ve been severely neglecting the ol’ blog, I know. I’m breaking the silence to join Janko Roettgers and NewTeeVee in wishing Gnutella a Happy Birthday on the week of its 10 year anniversary. We had some great times at LimeWire and battling it out on the Gnutella Developer’s Forum back in the day.

One of my favorite moments had to be in the summer of 2000 when Vinnie Falco, creator of BearShare, and my fellow LimeWire developer and partner in crime in the old days, Chris Rohrs, would talk about Gnutella architecture decisions. Vinnie’s an Italian American who was living in Miami Beach at the time. Chris is an MIT grad who grew up one town away from me in Deerfield, MA, one of the oldest towns in the country (close to my beloved Greenfield), and about as culturally far away from Miami Beach as one can get in the US. Chris would give Vinnie some piece of advice about this or that, and Vinnie would respond with the perfect Miami Beach Italian American accent, “Hey, you’re talking to the Vin.” And that’s how the Gnutella network was built.

Chris has been tearing it up at Google for years now, and Vinnie probably owns some tiny island in the South Pacific where he buried a chest full of BearShare gold back in 2002.

Check out Janko’s more thorough article, the best and one of the few histories of Gnutella I’ve seen written down.

I’ll be writing more here soon — been hammering out so much code for so many different things (all feeding back in to LittleShoot), it’s been a struggle to find time to blog about them. Technology’s progressing so rapidly on so many fronts, it’s more fun to keep coding! Stay tuned for news about LittleProxy, a sweet little HTPP proxy I’ve been working on using Netty. LittleProxy will be a core component of a new system for circumventing censors around the world. Oh yeah, and lots of other stuff too.

MIME API from LittleShoot on Google App Engine

July 23, 2009

The other day I needed to add an HTML form to the LittleShoot site to upload files directly to Amazon S3. S3’s “browser-based uploads” allow you to send files directly into S3 using hidden form fields to sign the request as opposed to sending files through your server. Nice and efficient.

Everything was going along fine until I realized one glaring problem: how the heck could I set the Content-Type on the file? S3 requires you to do this in the initial request, in this case using another hidden form field, but how could I determine the MIME type at all? Cut past 30 minutes of digging around, and it turns out you can, of course do the following:

  1. Attach to the form submit event. In JQuery, this is just:
    $("#myForm").submit(function() {...});
  2. Lookup the MIME type in JavaScript (somehow – pay attention to this one)
  3. Dynamically add a hidden “Content-Type” form field just before the upload.

The only problem? To do this, you need to include a full JavaScript library to lookup all MIME types in the submit function above, another 40K+ of JavaScript code. That seems a little excessive for a call you’ll only make every once in awhile.

Now comes the beauty of Google App Engine (GAE). With App Engine, you can fire up a site in a matter of minutes that will scale to millions. With a little help from my favorite App Engine Patch, you can run it all on Django. The speed and ease of deployment are breathtaking: 1 hour to deploy a REST API for MIME lookups, all for free.

The result is  It’s a trivial, ridiculously simple API that basically just exposes the Python “mimetypes” module. It’s the simplest, silliest, and most trivial REST API I’ve ever seen or written, and that’s the whole point. Google App Engine annihilates the barrier to creating this type of API on the only two scales that really matter in this case: time and cost.

I predict we’ll see a lot of similarly trivial services continuing to pop up because they’re so easy. They seem almost stupid at first. Simple building blocks like these are the secret sauce in building any truly powerful platform, however, and with tools like GAE, the web is getting more powerful by leaps and bounds.

If you ever run into the obscure case where you need to lookup a MIME type via an API, check out mimeapi. I built it because it makes more sense than cramming another JavaScript library down my user’s pipes. Maybe it’ll help you too.

Google App Engine Full Text Search From App Engine Patch Team

July 1, 2009

Waldemar Kornewald and the rest of the Google App Engine Patch team have just released the first full text search implementation for Google App Engine (GAE).  LittleShoot has been using App Engine Patch since the early days of App Engine, and we can’t recommend it highly enough.  Its seamless Django integration has saved us countless hours, and features like its tool for combining and compressing all of your JavaScript and CSS using the YUI compressor are just stellar.

While we don’t have an immediate need for full text search over at LittleShoot, we can tell you the team behind GAE full text search is rock solid and battle tested.  Given the various limitations of the GAE datastore API, it’s also quite a technical feat.

If you need full text search, and you’re running on App Engine using python, GAE search will save you a great deal of pain.  Go get it.

Django and More on Google App Engine with App Engine Patch

November 16, 2008

I’ve recently had the chance to play with Waldemar Kornewald’s “Google App Engine Patch” and have come away very impressed. All LittleShoot Google App Engine (GAE) projects now run on it for a couple of simple reasons:

  1. Seamless Django integration (including 1.0.1)
  2. Thorough documentation
  3. Healthy open source development community with an excellent steward in Kornewald and frequent new releases

The Django integration got me first. Almost everything works, such as, Django authentication, Django testing (the original reason I switched), etc. There are also lots of other goodies in there, like support for boto’s SQS module. If you’re unfamiliar with it, boto SQS allows you to call Amazon’s Simple Queue Service (SQS) from Python. That’s a huge step in getting around GAE’s limitation on longer lived, CPU-intensive tasks. Just queue it up in SQS, and your GAE app will keep humming along fine — cloud integration at its finest.

To get started with App Engine Patch, download the zip file from the home page and work off the sample project. The download includes App Engine Patch itself as well as the sample project.  

With Django App Engine Helper not supporting Django 1.0 and seemingly inactive, App Engine Patch is a godsend and gets a huge thumbs up.

Great work Waldemar, and don’t forget to donate.

Voting Location Via Text Message

October 30, 2008

My buddies Chris Muscarella and Benjamin Stein over at Mobile Commons just released a simple service to find your voting place via text message.  Here’s all you have to do:

text pp then your street address and zip to 69866 (eg: pp 101 market st 94105)

That’ll give you the polling place for your address or the number for the Election Protection Coalition if your address isn’t in their system.  Their original post is here.

We’ll be studying this election for centuries.  Great job fellas.  I’m rooting for your servers holding up!