Experiments with NoSQL databases- CouchDB
I started reading about NoSQL databases for a long time. Occasionally I used some NoSQL databases like Apache CouchDB and Apache Cassandra for some analytics purpose(Some minor projects) with Python. This time I just thought why can’t try something on Java + NoSQL. I created a small for project to play with. The idea of this project is: store Twitter search result to CouchDB. I used the following Operating System, Programming Languages and Libraries in this project.
Operating System : Fedora 16 (verne)
Programming Language : Java (JDK 1.6.0_29)
IDE : Eclipse 3.7.1
Apache CouchDB : 1.0.
External Libraries : Couchdb4J
Twitter4J
Apache Commons httpclient, logging, codec,commons,collections, beanutils
Jsonlib, ezmorph
Installing CouchDB To install CouchDB fire the terminal and type the command
$su -c 'yum -y install couchdb'
After succesful installation start the CoucbDB server by issuing the command in the terminal $su -c ‘/etc/init.d/couchdb start’
Now your CouchDB instance will be up and running. You can check this by opening CouchDB Futon in the broswer by navigating to http://localhost:5984/_utils/. If everything will fine you will see the Funton Interface.
Let’s start out project. First create a function to connect to the CouchDB instance,create and retrun a database with given name. If the database already exits it has to return the database.
/**
* @param strDBName
* @return dbCouchDB
*/
public static Database connectCouchDB(String strDBName) {
Database dbCouchDB = null;
Session dbCouchDBSession = new Session("localhost", 5984);
List<String> databases = dbCouchDBSession.getDatabaseNames();
if (databases.contains(strDBName)) {
dbCouchDB = dbCouchDBSession.getDatabase(strDBName);
} else {
dbCouchDBSession.createDatabase(strDBName);
dbCouchDB = dbCouchDBSession.getDatabase(strDBName);
}
return dbCouchDB;
}
Now we can create a function to search in Twitter Search and return the tweets.
/**
* @param strQuery
* @throws TwitterException
* @return queryResult
*/
public static QueryResult getTweets(String strQuery)
throws TwitterException {
Twitter twitter = new TwitterFactory().getInstance();
Query query = new Query(strQuery);
QueryResult queryResult = twitter.search(query);
return queryResult;
}
To insert the tweets to the CouchDB document collection(database) it has to be converted to a document. Lets create a function to convert individual tweets to CouchDB document.
/**
* @param tweet
* @return couchDocument
*/
@SuppressWarnings("deprecation")
public static Document tweetToCouchDocument(Tweet tweet) {
Document couchDocument = new Document();
couchDocument.setId(String.valueOf(tweet.getId()));
couchDocument.put("Tweet", tweet.getText().toString());
couchDocument.put("UserName", tweet.getFromUser().toString());
couchDocument.put("Time", tweet.getCreatedAt().toGMTString());
couchDocument.put("URL", tweet.getSource().toString());
return couchDocument;
}
Now we can try to write the Twitter Search results to the CouchDB document collection with the following function.
/**
* @param tweetQury
* @param dbName
* @throws TwitterException
*/
public static void writeTweetToCDB(String strTweetQury, String strdbName)
throws TwitterException {
QueryResult tweetResults = getTweets(strTweetQury);
Database dbInstance = connectCouchDB(strdbName);
dbInstance.getAllDocuments();
for (Tweet tweet : tweetResults.getTweets()) {
Document document = tweetToCouchDocument(tweet);
dbInstance.saveDocument(document);
}
}
Now it is time to execute our project. Add the following lines to the main() and run the project.
String query = "java";
String dbName = "javatweets";
System.out.println("Started");
writeTweetToCDB(query, dbName);
System.out.println("Finished");
That is all !!!!!! .
The entire code is available at my bitbucket repo
Happy Hacking !!!!!!!!
Migrated from my old blog jaganadhg.freeflux.net
NoSQL CouchDB Apache CouchDB