Foursquare API integration

Foursquare’s API changes quickly, so this post may be out of date before you get started.

However they offer up to the minute, user generated, location based venue data that makes it well worth the effort, especially if you’re cross referencing it with other location based data sources.

As with many social APIs, there are requests which need oauth (i.e. the end user opts-in) and those which don’t. This post gives an example of integrating with the public (non-oauth) data using ruby.

A quick example :

  def foursquare latlon
    apikey = get_my_foursquare_api_key # sign up as a developer, hardcode your key here
    apisecret = get_my_foursquare_api_secret  # ... and your secret key here
    @url = "https://api.foursquare.com/v2/venues/search?ll=#{latlon}&client_id=#{apikey}&client_secret=#{apisecret}"
    hsh = download(@url) # use Curl or some other method to get the json results
    results = {}
    results = hsh.response.groups.first.items if hsh.response && hsh.response.groups && hsh.response.groups.first && hsh.response.groups.first.items
    results
  end

Hopefully this is quite self explanatory. The result is an array of all nearby venues.

For example if you wanted to find popular venues near DC :

  x = foursquare("38.898717,-77.035974")
  pp x
  puts x.first.name # West Wing

---

Or, if you’d like to save some time and avoid this work altogether, we integrate and aggregate a range of interesting data, and deliver it through 1 simple API, so you don’t have to

Advertisement

Ruby curl with follow redirects

Just a technical FYI; this took a little digging to find in the documentation.

If you’re using Curl::Easy in ruby to download html (or results from our API), then FYI the default is to NOT follow redirects. If you want to follow redirects and download the page contents of the target, you’ll need to set the option option

easy.follow_location

to true.

Here’s a code snippet :

  def download_url url
    res = ""

    tries = 0
    begin 
      tries += 1
      easy = Curl::Easy.new
      easy.timeout = 30
      easy.follow_location = true
      easy.url = url
      easy.perform
      res = easy.body_str 
    rescue Exception => e  
      retry unless tries > 2
      puts "#{url} failed, returning empty string, #{e.message}"
    end

    ic = Iconv.new('UTF-8//IGNORE', 'UTF-8')
    res = ic.iconv(res + ' ')[0..-2]

    return res.downcase
  end

Consider EC2 auto config instead of Capistrano

We’re just at the tail end of shifting our infrastructure from a static VPS to Amazon EC2. Someone much smarter than I designed a server self configuration process. Here’s what we came up with.

The final product

Here’s all it takes for us to spin up a fully functioning web node now :

 
How it works :

1. Take existing AMI, create it with the default groups (web, worker, etc), and the default size
2. The AMI includes a startup script to do a git clone of the repository.
3. The startup script runs exactly 1 script within that repository, which can be used for anything. We use it for a bundle install, installing a couple of server dependencies that aren’t in the AMI (yet), mount the EBS block if it’s a db_role, assign the elasticip (or load balancer) if a web role
4. That script also launches god for each role, which is how we make sure our web servers, background processing tasks, and database engines are running

Does it work?

Sure does, we’re pretty happy with the final outcome. Now to deploy new code we just reboot all running instances. We switched from capistrano, so there are some pros & cons :

Benefits of auto-launch

• Less maintenance overhead : Deploy new server == Deploy new code. There is only one set of scripts to maintain. There is no way to end of with the “oops, I forgot to tell you I would always have to run rake blah blah on each node after reboot” problem
• Auto scaling : Using tools like scalr or amazon auto-scale, we can trigger new nodes to spin up or shut down without human intervention.
• Identical environments for test, deploy, staging, and production
• Everything is in git : Everything is a rake task or config script in the one repository / project for now. Bringing a new engineer on board is less painful than it used to be. As we grow, we may need to split this out so that the environment and application can be managed independently, but for now this is a positive

Problems

Our prior process had it’s advantages which, in hindsight, we undervalued :

• Capistrano deploy via symlink switching. We do our best for now and don’t reboot all servers at the same time, but it’s still riskier than the capistrano approach which deploys then switches the symlink at the last moment. We need to be careful not to suffer downtime during a deploy. We’re may end of using capistrano or our own process for this to deploy minor tweaks without a reboot (by dynamically querying all running instances to get the :roles).
• Capistrano rollback. We can’t easily rollback if a deploy was a bad idea. We love TDD so hope this isn’t a problem, but issues always slip through so we need to be careful to deploy via branches.
• Painful linux startup scripts. Writing code in a fully loaded ruby environment is easier than shell scripting, especially when the shell script is run at startup (without the environment configured). This took a while to get right. As we add complexity I hope this doesn’t slow us down.

Recommendation

If you want the best of heroku, but still need your own custom environment, then I’d look in to this. Above all else it created a discipline for us to feel comfortable that when demand spikes occur, we’ll be ready.