Posts

Revisiting charts: NodeJS, Twitter and Elasticsearch

Image
Some time ago, I developed a mashup using NodeJS and Couchbase to demonstrate how to use Couchbase to create stats and charts. Despite I still use Couchbase for many things in my day-to-day basis, it does not provide any comfortable way to perform free text search, so I moved to ElasticSearch for some projects. In this series of  posts I show how to create a very basic NodeJS application that gets tweets from Twitter stream, stores them into ElasticSearch and get some charts about them. First thing first, you have to install ElasticSearch. If you are using a Mac and Homebrew, just brew the following formula: $ brew install elasticsearch And then, run ElasticSearch as you prefer. Example: $/usr/local/bin/elasticsearch -f -D es.config=/usr/local/opt/elasticsearch/config/elasticsearch.yml After a few seconds, you will get something like this on you terminal: [2014-01-13 20:55:08,290][INFO ][node ] [Rhiannon] {0.20.2}[36723]: initializing ... [2014-01-13 20:55:08,309][INFO ][plugins ] [Rhi

Building Freeling Java API on Mac OS X

Next step on my Freeling adventure was to build the Java API. It comes together with Freeling, if you checked out the SVN repo , but the Makefile there is totally incorrect for the Mac. If you followed the instructions on my last post , you should change the provided Makefile to match these contents: Then, just run make make install And you are good to go.

Building Freeling in Mac OS X

I'm involved in a NLP project (the Natural Language Processing, not the Neuro-linguistic Programming thing) and I had to use Freeling . After some days of going back and forward with the installation instructions in the forums, I manage to build it. Here's my recipe. First, install  Macports . I tried brew , but it just does not work. You'll also need XCode command line tools from Apple. Then, open a Terminal and run: sudo port install autoconf sudo port install db46 sudo port install automake sudo port install libtool sudo port deactivate boost (optional, just in case you already have boost already installed in your system) sudo port install subversion (just in case you've never used SVN before) Edit /opt/local/etc/macports/sources.conf and add this line just before the "rsync" line file:///Users/Shared/dports mkdir /Users/Shared/dports cd /Users/Shared/dports Now install the specific version of boost by co-ing its revision (1.49, any other version will prev

Basic Couchbase querying for SQL people

Image
This is the second of a series of posts about Couchbase. For the first one, click here . Probably, you all know how to query a relational database. It is done using SQL, a declarative (non-imperative) language that is widely used to insert and retrieve information from databases. Taking a users table A typical SQL query should look like this: SELECT u.id, u.username, u.first_name, u.last_name, u.last_login FROM users u WHERE u.id = “1111”; But... How do I create a query similar to the one above in Couchbase? Well. The answer is: Creating a view CREATING A VIEW Download and install Couchbase server on your computer, and go through the set-up process. Then, you’ll need to create some documents. You can download these sample documents and use them in the following examples. Once you have created your documents, go to the VIEWS tab. Once there, you’ll see something like this page. As you have probably never created a view on your system, your views list will be empty. Let’s start by creat

Couchbase to Amazon S3 back-up script

On my last post, I was struggling with recovering all the documents from a Couchbase wacky installation after a system crash. It was an easy task to do, but I was very, very lucky to have the results of my "view all documents" view on a browser tab before the whole thing broke down. I'm publishing on a couple of scripts on github , based on what I used to recover my database. The exporter (exporter.py) can be used along with crontab to back-up your couchbase buckets to Amazon S3, while recover.py can be used to retrieve the data from S3 and restore it to a Couchbase installation. I found a small tweak, already documented on Couchbase issues , but I think it is related to python's library for Couchbase and not Couchbase itself. Enjoy. And let me know if you are using it. Update: Thanks to Francis Varga, from  www.crowdpark.com , I discovered a faster way to retrieve all documents, based on port 8092 and not on querying a view.

Emergency Recovery Script from Couchbase disaster

Today, I dealt with a totally broken installation of Couchbase 2.0.0 Developer Preview. I couldn't find why but, although views were returning the right results, when I tried to retrieve a single document, I was getting the error below: { "error": "badrpc", "reason": "{'EXIT',{{{badmatch,{error,closed}}, [{mc_client_binary,cmd_binary_vocal_recv,5}, {mc_client_binary,get_meta,3}, {ns_memcached,do_handle_call,3}, {gen_server,handle_msg,5}, {proc_lib,init_p_do_apply,3}]}, {gen_server,call, [{'ns_memcached-$data-default','ns...@127.0.0.1'}, {get_meta,<>,37}, 30000]}}}" } This is taken from Network Monitor panel on Chrome. On Coucbase admin console (:8091) the only message I got was "Unknown error". I tried to upgrade the couchbase version by downloading one of the latests builds, which is said to be more stable and have less problems that Developer Preview. But, this is what happened when I tried to instal

Creating an e-commerce platform with Couchbase 2.0

Image
For the last couple of months, I’ve been involved in the development of an e-commerce website named http://presive.com , focused on beautiful, unique products created by designers all over the world with a story behind. When I first met @swilera , @eoingalla and @cmgottschalk , the idea seemed so interesting to me that I got involved in the project really fast. And then, in a few weeks, we had the first release of presive.com. We created it using Python, Django, Satchmo and running on the top of a PostgreSQL database engine and it worked well for some weeks, until we started developing the first business-level report queries. The whole object-relational stack the product was based on wasn’t enough at all for our purposes, so a technological change was needed. And then, we moved to Couchbase. At first, I was quite skeptical on the power of NoSQL databases, but I soon changed my mind when I visit couchbase and read everything about it . So let’s start on the conceptual model (yes, I am