Andme-apide märkmed loengust

Weather

EMHI example http://www.ilmateenistus.ee/ilma_andmed/xml/observations.php

Foursquare

Example (replace MYTOKEN, you can make it on the docs site or by going to https://developer.foursquare.com/)

https://api.foursquare.com/v2/venues/search?ll=59.437,24.745&radius=1000&&oauth_token=MYTOKEN&v=20141001&limit=50&intent=browse

See foursquare dev docs

Google places

Example (replace MYKEY, you can make it on the docs site) https://maps.googleapis.com/maps/api/place/search/json?location=-33.8670522,151.1957362&radius=500&types=food&name=harbour&sensor=false&key=MYKEY

Twitter

Example url from old api, which is no longer functional: http://search.twitter.com/search.json?q=%40tallinn

See twitter dev docs and twitter search docs

Google search

Example (sign in to your Google account and read docs about how to create a custom search. Then create a new engine and put its id in place of MYENGINE below. Also replace MYKEY, you can make it on the docs site) https://www.googleapis.com/customsearch/v1?cx=MYENGINE&key=MYKEY&q=lectures

See web search docs and custom-search docs

See also old stackoverflow discussion on alternatives and https://serpapi.com/

Pricing:

• overview of some major search apis
• 3d party sales pitch overview of google search apis with features and prices
• maps pricing
• places api pricing deprecated? parts do not work?

Facebook graph api

https://developers.facebook.com/docs/graph-api

Quick example: https://graph.facebook.com/me?fields=id,name&access_token=MY_TOKEN

you can get MY_TOKEN from several places, for example: https://developers.facebook.com/tools/explorer?method=GET&path=me%3Ffields%3Did%2Cname&version=v2.1

Check also non-api scraping:

• http://www.pcworld.com/article/2056080/facebook-stalker-tool-uses-graph-search-for-powerful-data-mining.html
• https://github.com/milo2012/osintstalker

Python example for experimenting

#!/usr/bin/python

from math import *
import os
import urllib2
import json
import xml.dom.minidom

# NB! Replace YOURKEY and YOURTOKEN below with a key and token you make yourself
# on the corresponding api doc page

#url="https://api.foursquare.com/v2/venues/search?ll=59.437,24.745&radius=1000&&oauth_token=YOURTOKEN&v=20120103&limit=50&intent=browse"
#url="https://maps.googleapis.com/maps/api/place/search/json?location=-33.8670522,151.1957362&radius=500&sensor=false&key=YOURKEY"
#url="https://www.googleapis.com/customsearch/v1?key=YOURKEY&q=lectures"

def main():
  f=urllib2.urlopen(url)
  s=f.read()
  data=json.loads(s)
  #print data
  print json.dumps(data, sort_keys=True, indent=2)
  

def main2():
  req=urllib2.Request(url=url)
  rawresp=urllib2.urlopen(req)
  print "headers:\n"
  h=rawresp.info().headers
  for e in h: print e,
  s=rawresp.read()    
  data=json.loads(s)
  print "\ndata:\n"
  print json.dumps(data, sort_keys=True, indent=2)

  
main2()