Using Twitter API and Python Programming Language

Using Twitter API is fun and today, I am going to have some fun with you, hopefully, making good use of the free speech and the love of python programming. If the name John Boehner is alien to you, don’t worry; he is the speaker of the house in Washington. He likes crying – you could say he carries his cup of tears in his sleeves. Let us find out what people are saying about him on Twitter. Meanwhile, enjoy this:

john boehner crying 267x300 Using Twitter API and Python Programming Language

The Tools We Will Need.

Before we do anything, we will need a few things. If you do any programming in any language, you will find this easier than it sounds:

  • Install Python on your computer or find an online lab to run the code.
  • An internet connection to make the Twitter API calls.
  • Basic skills in Python programming language and a text editor of choice for example Sublime Text 2, Aptana, Eclipse etc.
  • And finally, curiosity above all else. Let us do it.

Now that we have that out of the way, let us move on to the next step. Fire up your text editor if you are interested in following along. I am going to use the interactive IDLE that comes with most Python installations.

Import The Necessary Module


#script to find out what people are saying

#about speaker John Boehner on Twitter

import requests

############################

#the twitter api that we will need to do the search

url = 'http://search.twitter.com/search.json?q=johnboehner'

As you might have noticed, am using the requests module instead of urllib2 which is too clumsy and not currently recommended. The former is pretty easy to use as you will soon see.

In order for us to make any searches on Twitter, we do so using twitter api shown above. I have appended the q(query) parameter to it and used johnboehner as the value. This request will query Twitter servers for tweets that have that name and return the results in json format.

 Make The Twitter API Call

Assuming you have imported the requests module, we can now make the api call to Twitter servers for the goodies.


#from earlier;

import requests

url = 'http://search.twitter.com/search.json?q=johnboehner'

#define a variable to store results and make the request

#passing in the above url - simple right?

results = requests.get(url)

#if we needed authentication to do this, we could have

#passed in more parameters to the requests.get() but we don't

Let us pause for a second here and see what has just happened. First we have to appreciate requests’ ease of use. Pretty straight-forward. We now have a load of data stored inside results variable and a twitter api call doesn’t come with pretty make-up on. We need to figure out a way to make that data useful and thanks to json, it is easier than seeing John Boehner cry.


#------------------------------------------

#-----add the above code here-------

results = requests.get(url)

#use json() to make things nicer here

better_results = results.json()

If you are not familiar with JSON(JavaScript Object Notation), you can read more by clicking here. As you might have seen in the above code, I called json() on the returned results and stored the final results in a new variable better_results.

Now The Fun Part Is Here

First of all, not all people on Twitter are nice. We have to agree on that and some of the things people say about the Speaker of the house aren’t pretty. It is just a rough world out there.

Since we now have easy-to-use data from our request, it is good to understand the structure. JSON should be ringing in your head right now. Accessing JSON data is as easy as using the bracket notation like this somedata['key']. Let us do it. Here is a typical structure of a search result from a Twitter api call:

  • We have stored our results in a better_results variable.
  • better_results['results'] - this data has a ‘results’ key  which is important to us.
  • better_results['results'][0] - As you might have guessed it, you get more than one tweet. An array is indexed from zero as usual. Basically it is an array of tweet objects.
  • better_results['results'][0]['text'] - This is the most important part of the results. The first exact text mentioning the speaker of the house. So to get the texts, you just go a little deeper. Let us do it, shall we?

The easiest way to loop through the texts would be a for loop for now but you can do whatever you want to get the same results.


#use a for loop to get "text" only from each tweet object

############################################

for post in better_results['results']:

print post['text'].encode('utf-8')

###########################################

############## end of the loop here###

That’s it. It is that easy. Now the long wait is over. Let us see what people are really saying about John Boehner on Twitter. Be warned, some of the words used might be hurtful.

The Final Results of Using Twitter API:

  • @johnboehner U keep saying that- for yrs now. What is it specifically u r doing? No sign of any work or progress on job programs.
  • @johnboehner not isolationist, but we need to become productive again to feel valuable again & will become BOTH again as ppl and country
  • @johnboehner Because of Republican economic policies. Surely you can’t believe your own BS.  #wherearethejobs? #orange #lyingliars
  • @johnboehner You are lying!  Do something to prove it! #lyingliars #orange
  • @johnboehner raise/enforce stiff import tariffs instead of domestic taxes & enable even smallest businesses to compete with foreign imports
  • RT @hpjohn1: @johnboehner #Benghazi #Extortion17 #IRS #AP
  • @johnboehner The scared Democrats in alliance with Senate of no should be voted out We tweeters need their names and districts #RETWEET
  • RT @crazylary51: #GOP #RNC #JohnBoehner Says Jobs Come First, But The #House Has Taken 1 Jobs Vote This Year http://t.co/EfUAUGum3G More lie’s by the GOP
  • @johnboehner #fairtax permanently ends political abuse of IRS as well! FREE people should not put up with government agency like the IRS
  • #GOP #RNC #JohnBoehner Says Jobs Come First, But The #House Has Taken 1 Jobs Vote This Year http://t.co/EfUAUGum3G More lie’s by the GOP
  • @johnboehner start promoting the trades, crafts, mfg & industrial vocations-let’s start MAKING & BUILDING things by hand again when feasible
  • @johnboehner the they ought to be passing #HR25 #fairtax bill as NOTHING would do more to create jobs and expand opportunity than that!
  • @johnboehner icon smile Using Twitter API and Python Programming Language Enough bullshit
  • @johnboehner No you’re not. You’re focusing on a bunch of bogus “scandals.” It didn’t work for the GOP in the ’90s and it won’t work now.
  • @JohnBoehner @RepPeteKing @ #TimeIsNow 4 Congress 2 B #MakingLifeWork 4 immigrant families! #RememberNovember #CIR http://t.co/BjSnHqUBhs

That my friends is just a single request. What do you get back? I bunch of tweets from angry people who, just like me and maybe you, think Washington is way out of control and run by lobbyists.

 Where to go next….

  • Encapsulate the script by wrapping everything in a method that takes an argument.
  • Use a variable as a value to the query parameter.
  • Now search for what people are saying about you on Twitter or your local mayor or your company.
  • Have fun while you can.

Thanks for stopping by. If you liked this post on how to make an api call using twitter api, please share it with your friends using the buttons below. If you have questions or comments, please ask them below. Thanks again and stay safe. Subscribe for more!

 

 

 

 

 

 

17 thoughts on “Using Twitter API and Python Programming Language

  1. Pingback: Programming Awesome » How To Use Python To See What People Are Saying About Speaker John Boehner

  2. Hi. I’m just in the initial stages of learning to program. I tried to do this but I get an import error that “no module named requests”

    Am I not using the right version of Python? Do you know why this might happen? Thanks.

    • There are two possible reasons for the errors you are getting:

      First, you probably don’t have the module installed and if that is true, please follow instructions on how to install it here:

      http://www.python-requests.org/en/latest/user/install/

      Remember you need to first install either easy_install or pip then use either one to install requests module in your python installation.

      The second possible cause might be path issues. Perhaps I could ask you this: where are you trying to use the module? On the interactive shell (IDLE) or a text editor or an IDE? If you have it installed(I doubt it), you can try importing it on the interactive shell first and seeing if you get any errors.

      I had a problem once where I had installed it but couldn’t use it in an IDE and yet I could use it on the shell.

      Please let me know if you still help with this problem and I can help you remotely to install all those tools and even test them out.

      It might also depend on the machine you are using; Windows or Mac?

      • Hi, Thanks for the reply. I didn’t realize you needed to install the requests module first. I’m really in the early stages of learning to program. I was able to install it without a problem.

        I tried running the script and it seemed to run without any errors but I didn’t see any output. Should the output print to the terminal/ interactive shell?

        I tried running this from IDLE and from the command line terminal but no luck.

        A bit about my system:
        Mac OS 10.8.4
        Also have IDLE for Python 2.7 and 3.3
        Using Sublime 2 as a text editor

        BTW, Great work on the blog so far.

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code class="" title="" data-url=""> <del datetime=""> <em> <i> <q cite=""> <strike> <strong> <pre class="" title="" data-url=""> <span class="" title="" data-url="">