Welcome!

Linux Containers Authors: Liz McMillan, Elizabeth White, Pat Romanski, Yeshim Deniz, Flint Brenton

Related Topics: @DXWorldExpo, Linux Containers, Agile Computing, @CloudExpo, FinTech Journal

@DXWorldExpo: Blog Post

NoSQL: Filling the Gaps By @MapR | @CloudExpo [#Cloud #BigData]

Today's web based apps demand databases that are flexible, high performing, and easily scalable.

NoSQL: Filling the Gaps in Your Traditional Relational Database

With the many different characteristics of NoSQL databases available today, it's not always clear how to best categorize the different NoSQL offerings. Typically, though, NoSQL databases are labeled according to the associated data model, most commonly: key-value, wide-column, document, and graph. But more important than the differences between them are the reasons why they are growing in popularity as a whole. In general, NoSQL databases are meant to fill some of the capability gaps found in traditional relational database management systems (RDBMS).

Today's web-based applications are highly demanding, so the databases that hold their vast stores of information are not only expected to be very flexible in nature (supporting various data formats), but also to manage extreme performance and scaling. Enterprise architects have many great technology choices today, and it makes more sense than ever to spend the time to assess application requirements thoroughly before suggesting an appropriate database.

Initially promoted as a departure from, and improvement upon, the traditional RDBMS model, NoSQL is considered an independent and specialized database technology to support complex business needs. But NoSQL advantages don't necessarily represent an opportunity for outright replacement of RDBMSs. An enterprise architect's recommendation of RDBMS versus NoSQL comes down to the nature of the operational requirements, in other words, particular use cases. In some cases, an RDBMS is more appropriate, and sometimes quicker, for specific data management tasks compared to NoSQL, and the use of NoSQL could be problematic for the application as well as for the business. A look at some of the fundamental features of database technologies will lay the groundwork for a general understanding of how NoSQL and RDBMSs differ, and how the former can fill in the gaps of the latter.

RDBMS: transactional, consistent

RDBMSs have been around for decades and have become the foundation for many business applications today. They can guarantee good performance with a volume of thousands of transactions per second which, decades ago, was enough. They also support multi-update transactions, known by computer scientists as "ACID transactions." The ACID transaction principles--atomicity, consistency, isolation, and durability--ensure data integrity even when multiple data elements are updated together. But modern internet applications - especially those operating in real time, such as fraud detection, risk analysis, advertising, and gaming - involve millions of transactions per second, and typically don't need multi-update transactional guarantees. RDBMSs struggle to manage the sheer volume of these newer online transaction processing (OLTP) applications. RDBMS vendors are investing time, money, and effort into overcoming these issues, but for now the gap remains.

Another strength of RDBMSs is the implementation of SQL ("Structured Query Language") as the de-facto standard for data processing tasks like data query, data definition, data manipulation, etc. SQL, and thus RDBMSs, require a predefined tabular structure entailing rows and columns to ensure that data is stored in a consistent and expected format. The widespread knowledge of SQL along with the consistency of the data model make RDBMSs well-suited for business intelligence analysts. Any application that requires multi-element updates, such as those dealing with financial data, and that takes advantage of the power and ubiquity of SQL, should use an RDBMS.

NoSQL: scalable, flexible

Most NoSQL databases give up on multi-row ACID transaction capabilities to achieve cost-effective scalability and performance. For a certain class of use cases, this tradeoff is perfectly acceptable, especially when the data set has records that are independent of one another, so there is no requirement to make updates that require additional immediate updates. And with regard to the data model, NoSQL databases, in contrast with RDBMSs, use different formats to store data. This flexibility lets application developers store heterogeneous record formats in the same database, which is often needed for large-scale data sets loaded from numerous distinct sources. The most popular NoSQL data formats are key-value, wide-column, and document (like JSON). Among them, the key-value databases are the simplest, while wide-column databases provide the closest data model to RDBMS tables. NoSQL databases bypass some of the hard constraints used in RDBMS architectures to achieve data storage flexibility, scalability, and performance.

Use cases: NoSQL suitability over RDBMS

There are many applications for which the traditional ACID-driven RDBMS model is not the easiest or best option. Here's another look at what NoSQL can do:

  • A key-value database is ideal if your application requires only the storage of data for the purposes of a quick lookup. In this situation, an RDBMS is unnecessarily complex, and a key-value database will be more than sufficient to meet your application's needs. The value itself can be as simple or as complex as you require.
  • A wide-column database is ideal if your data is structured like an RDBMS table but the columns might be different across rows. This means that different types of records can be stored in the same table, and existing rows can be easily updated to accommodate new columns.
  • A document database is ideal if your data entails hierarchical objects. RDBMSs could accomplish this with the help of object relational mapping (ORM) tools, but the work is often complex, and it's not worth complicating matters when NoSQL solutions are available.
  • A graph database is ideal if your application deals with connected networks of entities or large trees. Graph databases are good at quickly identifying linkages between related entities.

Major influencing factors: scaling and performance

RDBMSs were not originally designed for horizontal scaling - in other words, adding more hardware servers to the mix -- so they get overwhelmed when the load and data increase beyond expectations. RDBMSs are not great at distributing data across servers, in the process known as "sharding." Either they don't have the ability to automatically shard, thus requiring application-level coding to distribute data, or the scale-out architecture is expensive.

Here, NoSQL solutions have an advantage. A NoSQL database does not have to break up records into distinct pieces across servers. They are always logically stored in a single place, thus simplifying the distribution process. And since there is no referential integrity required between these logical entities, NoSQL is easily able to perform automatic data sharding. Though NoSQL solutions have certain limitations when compared to the relational model, they were designed with the intention of providing high scalability. A NoSQL solution can scale horizontally over a distributed environment and support high availability.

Overall performance depends largely on the selection of appropriate technology (NoSQL or RDBMS) for a specific use case. Thus, if NoSQL is selected for the wrong use case, it can hurt the performance of the application. Apart from factors like network, caching, and disk I/O, performance is dictated by the data and its allocation across the distributed storage - and NoSQL is capable of handling large volumes of data in a clustered environment, boosting performance as a result.

It's important to remember that NoSQL is not a substitute for traditional RDBMSs, but an alternate solution to address a different set of use cases for which RDBMSs were not designed. Many NoSQL technologies exist on the market today, and if your NoSQL requirements include enterprise-grade reliability, high performance, and fast responsiveness, a production-proven NoSQL database worth investigating is MapR-DB by MapR Technologies.

More Stories By Dale Kim

Dale is Director of Industry Solutions at MapR. His technical and managerial experience includes work with relational databases, as well as non-relational data in the areas of search, content management, and NoSQL. Dale holds an MBA from Santa Clara University, and a BA in Computer Science from the UC Berkeley.

@ThingsExpo Stories
WebRTC is great technology to build your own communication tools. It will be even more exciting experience it with advanced devices, such as a 360 Camera, 360 microphone, and a depth sensor camera. In his session at @ThingsExpo, Masashi Ganeko, a manager at INFOCOM Corporation, introduced two experimental projects from his team and what they learned from them. "Shotoku Tamago" uses the robot audition software HARK to track speakers in 360 video of a remote party. "Virtual Teleport" uses a multip...
Business professionals no longer wonder if they'll migrate to the cloud; it's now a matter of when. The cloud environment has proved to be a major force in transitioning to an agile business model that enables quick decisions and fast implementation that solidify customer relationships. And when the cloud is combined with the power of cognitive computing, it drives innovation and transformation that achieves astounding competitive advantage.
Data is the fuel that drives the machine learning algorithmic engines and ultimately provides the business value. In his session at Cloud Expo, Ed Featherston, a director and senior enterprise architect at Collaborative Consulting, discussed the key considerations around quality, volume, timeliness, and pedigree that must be dealt with in order to properly fuel that engine.
IoT is rapidly becoming mainstream as more and more investments are made into the platforms and technology. As this movement continues to expand and gain momentum it creates a massive wall of noise that can be difficult to sift through. Unfortunately, this inevitably makes IoT less approachable for people to get started with and can hamper efforts to integrate this key technology into your own portfolio. There are so many connected products already in place today with many hundreds more on the h...
When shopping for a new data processing platform for IoT solutions, many development teams want to be able to test-drive options before making a choice. Yet when evaluating an IoT solution, it’s simply not feasible to do so at scale with physical devices. Building a sensor simulator is the next best choice; however, generating a realistic simulation at very high TPS with ease of configurability is a formidable challenge. When dealing with multiple application or transport protocols, you would be...
Detecting internal user threats in the Big Data eco-system is challenging and cumbersome. Many organizations monitor internal usage of the Big Data eco-system using a set of alerts. This is not a scalable process given the increase in the number of alerts with the accelerating growth in data volume and user base. Organizations are increasingly leveraging machine learning to monitor only those data elements that are sensitive and critical, autonomously establish monitoring policies, and to detect...
In his keynote at 18th Cloud Expo, Andrew Keys, Co-Founder of ConsenSys Enterprise, provided an overview of the evolution of the Internet and the Database and the future of their combination – the Blockchain. Andrew Keys is Co-Founder of ConsenSys Enterprise. He comes to ConsenSys Enterprise with capital markets, technology and entrepreneurial experience. Previously, he worked for UBS investment bank in equities analysis. Later, he was responsible for the creation and distribution of life settl...
In his session at @ThingsExpo, Dr. Robert Cohen, an economist and senior fellow at the Economic Strategy Institute, presented the findings of a series of six detailed case studies of how large corporations are implementing IoT. The session explored how IoT has improved their economic performance, had major impacts on business models and resulted in impressive ROIs. The companies covered span manufacturing and services firms. He also explored servicification, how manufacturing firms shift from se...
DevOpsSummit New York 2018, colocated with CloudEXPO | DXWorldEXPO New York 2018 will be held November 11-13, 2018, in New York City. Digital Transformation (DX) is a major focus with the introduction of DXWorldEXPO within the program. Successful transformation requires a laser focus on being data-driven and on using all the tools available that enable transformation if they plan to survive over the long term. A total of 88% of Fortune 500 companies from a generation ago are now out of bus...
The Jevons Paradox suggests that when technological advances increase efficiency of a resource, it results in an overall increase in consumption. Writing on the increased use of coal as a result of technological improvements, 19th-century economist William Stanley Jevons found that these improvements led to the development of new ways to utilize coal. In his session at 19th Cloud Expo, Mark Thiele, Chief Strategy Officer for Apcera, compared the Jevons Paradox to modern-day enterprise IT, examin...
IoT solutions exploit operational data generated by Internet-connected smart “things” for the purpose of gaining operational insight and producing “better outcomes” (for example, create new business models, eliminate unscheduled maintenance, etc.). The explosive proliferation of IoT solutions will result in an exponential growth in the volume of IoT data, precipitating significant Information Governance issues: who owns the IoT data, what are the rights/duties of IoT solutions adopters towards t...
Amazon started as an online bookseller 20 years ago. Since then, it has evolved into a technology juggernaut that has disrupted multiple markets and industries and touches many aspects of our lives. It is a relentless technology and business model innovator driving disruption throughout numerous ecosystems. Amazon’s AWS revenues alone are approaching $16B a year making it one of the largest IT companies in the world. With dominant offerings in Cloud, IoT, eCommerce, Big Data, AI, Digital Assista...
Organizations planning enterprise data center consolidation and modernization projects are faced with a challenging, costly reality. Requirements to deploy modern, cloud-native applications simultaneously with traditional client/server applications are almost impossible to achieve with hardware-centric enterprise infrastructure. Compute and network infrastructure are fast moving down a software-defined path, but storage has been a laggard. Until now.
Digital Transformation is much more than a buzzword. The radical shift to digital mechanisms for almost every process is evident across all industries and verticals. This is often especially true in financial services, where the legacy environment is many times unable to keep up with the rapidly shifting demands of the consumer. The constant pressure to provide complete, omnichannel delivery of customer-facing solutions to meet both regulatory and customer demands is putting enormous pressure on...
In his general session at 19th Cloud Expo, Manish Dixit, VP of Product and Engineering at Dice, discussed how Dice leverages data insights and tools to help both tech professionals and recruiters better understand how skills relate to each other and which skills are in high demand using interactive visualizations and salary indicator tools to maximize earning potential. Manish Dixit is VP of Product and Engineering at Dice. As the leader of the Product, Engineering and Data Sciences team at D...
DXWorldEXPO LLC announced today that All in Mobile, a mobile app development company from Poland, will exhibit at the 22nd International CloudEXPO | DXWorldEXPO. All In Mobile is a mobile app development company from Poland. Since 2014, they maintain passion for developing mobile applications for enterprises and startups worldwide.
"Akvelon is a software development company and we also provide consultancy services to folks who are looking to scale or accelerate their engineering roadmaps," explained Jeremiah Mothersell, Marketing Manager at Akvelon, in this SYS-CON.tv interview at 21st Cloud Expo, held Oct 31 – Nov 2, 2017, at the Santa Clara Convention Center in Santa Clara, CA.
IoT is at the core or many Digital Transformation initiatives with the goal of re-inventing a company's business model. We all agree that collecting relevant IoT data will result in massive amounts of data needing to be stored. However, with the rapid development of IoT devices and ongoing business model transformation, we are not able to predict the volume and growth of IoT data. And with the lack of IoT history, traditional methods of IT and infrastructure planning based on the past do not app...
DXWorldEXPO LLC announced today that the upcoming DXWorldEXPO | CloudEXPO New York event will feature 10 companies from Poland to participate at the "Poland Digital Transformation Pavilion" on November 12-13, 2018.
22nd International Cloud Expo, taking place June 5-7, 2018, at the Javits Center in New York City, NY, and co-located with the 1st DXWorld Expo will feature technical sessions from a rock star conference faculty and the leading industry players in the world. Cloud computing is now being embraced by a majority of enterprises of all sizes. Yesterday's debate about public vs. private has transformed into the reality of hybrid cloud: a recent survey shows that 74% of enterprises have a hybrid cloud ...