A beginner’s guide to twitter brand sentiment analyser

download (38)

In this article, we will look at how to build a basic brand sentiment analyser.

We will be building an application which will fetch running stream from twitter filtered for specified brands, try to categorise the emotion in postive or negative bucket, and compare total sentiments summary across brands.

This application will be built in nodejs, using sentiment module to analyze the emotion behind the post.

Setup:

  1. Register on Twitter developer platform and create a new application.
  2. In “Keys and access token” section, note consumer key and consumer secret. We will use it to authenticate our requests later.
  3. Install nodejs, if not done already. There are plenty of guides online on installation part.
  4. Install express (command: sudo npm install express -g)
  5. Create a new folder and initialize express module by filling relevant information (command: express)
  6. Step 5 should create a package.json file in your work directory, Open it in your favourite editor and add following dependencies: debug, sentiment, twitter. Your file should look like this: 
  7. Run npm install to install dependencies.

Now lets dive into coding part

Here is gist of code that I came up with while building this application

Feel free to reach me out for any query related to above.

Image credits: Animal NewYork

Tweet Race between Modi and Rahul – A programmer’s Eye using Node/Redis

Tweet Race between Modi and Rahul – A programmer’s Eye using Node/Redis

download-300x163

In this article I will discuss about making a sample app which pulls out live stream of tweets around specific keywords, stores them in database or publish them on web page. Here we will be targeting Keywords like Rahul Gandhi and Narendra Modi and saving number of occurrences of each as Key-Value pair.

Technologies used: Node.js with Express framework, Redis as key value storage, Twitter API for collecting tweets.

Tutorial level: Beginner.

Setup

First thing first, Install Node.js and Express framework into your working directory using instructions given at their website. Next, we need to install Redis to store key value pairs. For Ubuntu users, run following set of instructions to install it.

wget http://download.redis.io/redis-stable.tar.gz
tar xvzf redis-stable.tar.gz
cd redis-stable
make

Assuming you have express installed in your current directory, it should have automatically created a package.json file, and directories like routes and views. Edit your package.json file in any text editor and update dependencies as below.

"dependencies": {
"express": "3.4.8",
"ntwitter": "0.5.x",
"jade": "*"
}

Now use Node Packet Manager to install the newly introduced ntwitter module. Run below command in terminal and let NPM do its job.

npm install

Redis configuration and setup

Now, As the setup part is complete, lets start the coding part.

If you are new to express framework, It provides basic MVC architecture to build your app with defined routes and views. We will be using these routes to develop our app. Firstly, we will add a new route (tweetapp) and as the application size is small, we will be writing our logic in route itself.

Add following code to routes/tweetapp.js (new file)

var twitter = require('ntwitter');
var redis = require('redis');

This will add redis and twitter module to our route. Next thing is to initialize the Redis client which can be done using

var client = redis.createClient();

Now all Redis related operations are available using this client. Simple Redis operations which we will be using here are (for exhaustive list of operations, please refer Redis Documentation)

  1. SET – To set a key value pair.
  2. GET – To retrieve a key value pair from Redis.
  3. MGET – To retrieve multiple values corresponding to keys passed as array
  4. INCR – Increment the value of given key by 1. (Note: This is only applicable to integer values)

Now coming back to our
tweeter app, To create a key, we first need to check if it already exists or not. This can be done via

client.exists();

It takes two arguments, Key to be searched and a callback function which will be called after look-up in redis is done. So our code will look like this.

client.exists('rahul', function(error, exists) {
  if(error) {
    console.log('ERROR: '+error);
  } 
  else if(!exists) {
    client.set('rahul', 0);
  };
});

It is checking for existence of a key named rahul in Redis and creating it in case it is not present with initial value as 0. Create a similar key for Modi.

Twitter API

Now as our Redis Key-Value pair is ready to accept entries, we will be moving to Twitter. Twitter Development API is open and powerful tool. We will be using it to collect all tweets on run time via streams. To connect with Twitter API, we need authentication tokens which can be taken from Twitter Dev plateform. Login to your twitter account and register for an app. Once you are done with filling required information, It will provide you four important parameters which will be required to pass authentication.

  1. Consumer key
  2. Consumer secret
  3. Access token
  4. Access token secret

Coming back to our app, we need to pass authentication tokens to twitter, which can be done as initializing our token variable,

var token = new twitter({
   consumer_key: 'QroiweVrj5CvfqSglv07Rg',
   consumer_secret: 'LdUxYEq8R01YhuzqKW6Kq1Evr2zdCtZt7CT8qbF6QI',
   access_token_key: '28957080-lzbIl9Wo9P0p3UGP66wTkKmJNJL30tm1QIhLCVbdn',
   access_token_secret: '9HakcRvSp4EC5G3ECCIDUTw9MbJlYW2Qd6omzGkgiASRO'
});

Now to start reading the stream we need to call stream function as below,

token.stream(
  'statuses/filter',
  { track: ['rahul', 'modi'] },
  function(stream) {
    stream.on('data', function(tweet) {
      console.log(tweet.text);
      if(typeof tweet.text!='undefined' && tweet.text.match('modi')) {
          client.incr('modi');
          //console.log('modi++');
      }
      if(typeof tweet.text!='undefined' && tweet.text.match('rahul')) {
          client.incr('rahul');
      }
    });
  }
);

This code simply reads a stream from twitter and tracks for keywords like rahul and modi, As a callback function, in case of text match this callback will be called with tweet as its parameter. This tweet parameter contains all information related to tweet among which we are interested in only its text. In case of a match with either keyword which we are interested in, we will increment its key value using Redis’s incr. As an additional log, above code is also logging all tweets parsed, in terminal.

redis.mget(["rahul", "modi"], function (error, count) {
  if (error !== null) {
    // handle error here
    console.log("ERROR: " + error);
  } else {
    var jsonObject = {
      "modi":count[1],
      "rahul": count[0]
    };
  // use res.json to return JSON objects instead of strings
  res.json(jsonObject); 
  }
});

This will return a Json response with both counters in response and we are done with our app.

Frontend JQuery & AJAX

In case you want to go ahead and show this on browser window, you can make AJAX calls in loop from browser via JQuery and show these values out there. Following code will do this basic version

var timeoutid;
clearTimeout(timeoutid);
timeoutid = setTimeout(function(){
    $.ajax({
        url: '/counttweets',
        success: function(response) {
                $("body").html("<p>Modi:"+response.modi+"</p>");
                $("body").append("<p>Rahul:"+response.rahul+ "</p>");
                main();
        }
    },5000);
});

 

Let me know in case you have any idea to improve this.

Happy Coding!!