NoSQL Databases: A Brief Introduction

NoSQL databases have emerged as a serious alternative to the traditional relational database management systems (RDBMSs). This article, the first in a series on NoSQL, provides you a high-level overview of NoSQL databases, and a comparison of some of the leading offerings under this umbrella term. Subsequent articles will explore specific NoSQL products and their use cases in more detail.

The Challenge of Data Volumes with Relational Databases

First, the "why" of NoSQL. Consider the huge volumes of data that need to be stored in today's internet era. As a few examples:

• Twitter reported, in March 2013, that they had about 200 million active users, and about 400 million tweets per day.
• Facebook reported, in August 2012, that they had about 100 PB (PB is petabytes; a petabyte is a million GB, or a thousand KB) of data, with daily addition of about 500 TB (or half a petabyte) of data. This was a long time ago, and surely their data would be growing at a much faster rate today. Even if we assume only a slight increase in their rate of data growth, it is a safe bet to say that they would be storing, at the very least, 400 PB of data.
• Walmart has about a million customer transactions per hour.
• And not to forget the grand-daddy of data volumes: Google.

Can you imagine storing such volumes of data in a relational database? Even with a storage area network (SAN), it is difficult to imagine a storage system so vast. And how will you get the processing power to respond to hundreds of thousands of online users at any point of time with Facebook and Twitter, with almost no down time ever. There is a limit to scaling up. NoSQL databases opt for scaling out. In simple terms, what this means is: rather than adding more power to a server, add more servers to a cluster.

Relational databases don't do a very good job of scaling out. They have not been designed for distributing data across servers. Well, to some extent, you can do so but that introduces a lot of additional complexity.

Performance Challenges

Relational databases emphasize data "normalization", that is, avoiding duplication of data. This results in an increase in the number of tables, which often need to be "join"ed in order to present a report that collates data from various tables. This makes data easier to maintain, but there is always a  performance cost attached to the joins. For most OLTP systems, joins between a few tables is a routine thing, but when you have to deal with gigantic data volumes as described above, and a very large number of simultaneous users, these joins can result in unacceptably poor performance of the respective websites.

NoSQL as an Alternative to RDBMSs

NoSQL is an umbrella term that refers to a whole bunch of relatively newer database platforms that either don't use the relational model at all, or use very little of it. Some of the characteristics that are common to most of the NoSQL products, and which differentiate them from the RDBMS platforms, are discussed below.

• Data distribution: NoSQL platforms are usually distributed systems; they run on clusters of commodity servers, which may consist of hundreds or thousands of machines. Data gets automatically distributed to various servers based on some hashing mechanism.
• Data replication: In a cluster of commodity servers, at any point of time, one or more servers may be down. Data replication (automatically copying the same data to multiple servers) provides protection against data loss due to server failures.
• Schema-less, or flexible schema: In most NoSQL databases (like MongoDB and Neo4j), you don't provide a "table structure" upfront. You just start adding data to a database without any pre-defined structure. In some NoSQL databases (like Cassandra), you do specify a table structure, but you can add or drop columns on the fly, or even change the datatype of columns, if required.
• No joins and transactions: NoSQL databases compromise on various features of RDBMSs, including joins and ACID transaction, in order to deliver significantly higher performance, scalability, availability and flexibility of database structures.
• Lower Cost of Ownership: Most of the NoSQL databases have community editions that are free and open source. Organizations would typically pay for support and services.

Categories of NoSQL

As indicated earlier, NoSQL is not the name of any specific product. It is a generic term for a number of database products sharing most of the common traits discussed above. There are four major categories of NoSQL databases, briefly described below with one prominent example from each of the categories as the basis:

Column Family Databases, such as Apache Cassandra. These databases are usually the closest to the relational database systems. Using Cassandra, for example, you issue commands that would be familiar to SQL buffs, such as create table, insert into, update, select … from … where …, etc.

Cassandra delivers extremely high write performance (not only compared to RDBMS products, but also compared to other NoSQL databases), and a very high read performance as well. You can store collections within a table. For example, you can store a set of email ids, an ordered list of favourite books, a list of dependent family members as key-value pairs of dependent name and the relation, all within a single record.

Document Databases, such as MongoDB, a prominent user of MongoDB in India being Aadhaar project of the UIDAI. MongoDB is a schema-less database, and it stores data in the form of JSON documents. (JSON stands for JavaScript Object Notation, and is a lightweight alternative to XML). A document may be as complex as you need, with collections and sub-fields (called sub-documents).

For example, you may store a person's data consisting of fields like name (further broken into first name, last name), gender, date of birth, a list of qualifications (each qualification stating the name of the qualification, institute, years of study), an ordered list of languages known with proficiency levels in each of these languages. Somewhat like Apache Cassandra, but with a completely different syntax, and higher flexibility of database contents.

Graph Databases, such as Neo4j. Think of social or professional networking sites like Facebook and LinkedIn. There are persons (you, me and other), there are organizations where we work / have worked, there are institutes where we have studied. And there are connections or links between them: you like someone, you follow someone, you have worked in an organization / studied in an institute, you have interests and hobbies.

And now we want to traverse these links to meet a variety of needs such as:

• people you may know (perhaps because many of your friends know that person)
• products you may want to buy (because you clicked on ads for other, similar products)
• other people sharing common interests or hobbies with you

Graph databases are a good fit for things like: social networking sites, recommendation engines, fraud detection systems, network and data center management, etc.

Key-Value Databases, such as Riak. These are perhaps the simplest of all NoSQL databases. They essentially are just a key-value pair, where the "value" can be anything, such as an XML record, a JSON document, a picture, a movie, a text document, etc; the value being identified by a "key", which can also be pretty much anything.

For more information:

Neo4j: A NoSQL (Graph) Database - A One Page Introduction

Coming up soon:

An Introduction to Apache Cassandra (coming up soon)

Architecture of Apache Cassandra: How It Delivers Blazing Performance

An Introduction to MongoDB (coming up soon)

An Introduction to Neo4j (more detailed than the one linked above)

An Introduction to Riak

About the Author

Pradyumn Sharma is the CEO of Pragati Software Pvt Ltd, a leading IT training company based in Mumbai, India. Pradyumn has 30 years of experience in the IT industry. He regularly conducts training programs on NoSQL, Agile Methodologies, Application Architecture, Solution Architecture, Enterprise Architecture.