Posted by: Gagan Mehra
Big Data, Big Fast Data, BigMemory, in-memory, In-memory data management, Real-time analytics, Uncategorized
The in-memory data management space remains hot as more and more critical applications push the limits of traditional architectures. And seemingly every vendor and their dog (from mega vendor to startup) is feeding $100 bills into their marketing machine to claim they are the best and everyone else falls short. When I read the FUD, it makes me want to re-state the principles on which our Terracotta in-memory products were founded and evolve today. Not the principles of marketing noise, but the principles of in-memory design and customer success.
BigMemory is an in-memory data management product designed to support the data management needs of a majority of the Global 2000. Several thousand companies have selected the solution citing ease of use and quick ROI, among other things, as important factors in their decision.
The in-memory data management landscape is evolving quickly and this brings with it a great degree of change in the features and capabilities of products in this space – particularly so for the leading vendors. The Terracotta team is extremely excited about the next set of innovations we are releasing in the coming months and beyond – all adhering to our design principles. Before we start to announce these we wanted to share with our customers our key cornerstones of BigMemory on which they can continue to rely long into the future. These should be every enterprise’s top 5 must-haves in an in-memory data management solution, especially in situations where they want to build a flexible framework that can be utilized for multiple use cases and can scale as the business grows.
1. Ease of Use
BigMemory uses the Ehcache API to access and manage in-memory data. Ehcache is the de-facto java caching API that makes BigMemory integration easier and faster. Ehcache has been around for over ten years and is also offered as an open-source product. It is currently used by over 2.5 million developers who like it because it is:
- Easy to setup
- Uses simple API to integrate with existing applications, requiring minimal changes
- Able to scale vertically and horizontally without any performance degradation
- Offered as part of several other products such as Hibernate, Spring, and Grails
BigMemory inherits the above qualities and can be snapped in very quickly by Ehcache users. For first-time users, the product is very easy to deploy and is up and running within a couple of hours. Most customer implementations go live within a few weeks and the product “just works”, to quote a customer, when in production.
2. Guaranteed Performance
BigMemory offers guaranteed latency at the millisecond level regardless of the volume of data maintained in-memory. We have customers who started with 500GB of data in-memory and now have almost 20TB in-memory, and the product continues to perform at the millisecond level. This capability is a unique differentiator for BigMemory as our experience has shown that other in-memory products demonstrate degraded performance with increasing scale where as we guarantee predictable performance. Here are some performance metrics that prove this:
- One of our larger deployments has over 6TB in production on 16 servers with each server hosting 400GB of in-memory data. After extensive testing with several competitors, the customer selected us, as we were the only ones that could meet the SLA requirement of single digit millisecond.
- A recent customer deployment had 1.6TB of data with roughly 83,000 transactions per second – approximately 80% read transactions and 20% write transactions, the average latency was around 50ms. This customer also tested with 3.2TB of data and doubled the number of transactions to 170,000 per second but the latency continued to be around 50ms.
- Another customer deployed 1TB of data into BigMemory and experienced their transaction times go down from 30 seconds to 30 milliseconds. They also ran peak and endurance loads on BigMemory with data volumes exceeding 5TB and the latency continued to hover around 30 milliseconds. BigMemory was selected after the customer had run several similar tests with competitors, which were unsuccessful in delivering the desired results.
3. Versatile Offering
BigMemory has been successfully deployed in several different verticals – such as Financial Services, Retail, Gaming, Travel & Aviation, Government, Telecommunications, Utilities, Media and High Tech. The product is designed to be flexible, extensible and reliable to support a variety of use cases.
- A single instance of BigMemory can be deployed to support multiple applications. These applications can be all in the same programming language or can be a mix of .Net, C++, C# and Java. There have been several instances where the customer deployed the product for one use case and after observing the benefits, extended it to support other use cases.
- BigMemory can be deployed on premise, in the private cloud or the public cloud depending on the customer’s use case.
- We have several success stories to share though we do not rely on past successes to sell the product. Every customer is made comfortable before buying the product by implementing a proof-of-concept (POC) in the customer’s environment using customer data to meet, or even better, beat the success criteria defined for the POC. These POCs usually take a few days to implement once the environment setup is complete.
- BigMemory supports several mission-critical use cases since the product is designed for high-availability and fail-over. Several customers have deployed BigMemory in an environment that supports 99.999% or five 9′s of availability.
- Additionally, BigMemory is an integral part of Software AG’s market-leading suite of enterprise technologies that enable the digital enterprise through advanced process automation & integration, and real-time intelligent business operations. These include BigMemory as a bundled offering with webMethods Integration Server, OneData Master Data Management, Presto Real-Time Visual Analytics, Apama Complex Event Processing Platform and real-time messaging solution - Universal Messaging. This means that our customers can easily benefit from the broad family of products of which we are a part, to differentiate from their competitors, and better engage customers, partners and employees.
4. Lower Total Cost of Ownership (TCO)
There are several reasons why BigMemory has lower TCO compared to others.
- BigMemory is architected using Java and does not require any specialized skills to manage or operate.
- It is designed for high-availability and fail-over. BigMemory is already deployed in several mission-critical environments where even one second of downtime could result in millions of dollars of revenue impact.
- BigMemory uses very little RAM for its internal use making more available for use by the business resulting in better use of available RAM & lower costs.
- Several of our customers reduce their database footprint when they start using BigMemory resulting in lower DB license costs.
- And lastly, BigMemory comes with a monitoring dashboard that can be integrated with a customer’s existing monitoring dashboard by making simple web calls (RESTful integration). This makes it easy for the Operations team to, in most cases, monitor the application using only a part-time resource to manage our product compared to a large operational team.
5. Hardware Agnostic
BigMemory is a software solution that runs on any commodity server. There is no need to buy special hardware or to certify the product on different hardware devices. The product can be deployed on any hardware and uses the RAM available on different servers to create an in-memory cluster. Most customers use any spare hardware sitting in their data centers to start with the initial evaluation and deployment. This makes it a lot faster to get started with using the product.
We are committed to building a great product that exceeds our customers’ expectations, and I hope this post helps you understand BigMemory better. If you have any additional questions, we will be happy to discuss them. Please contact me at firstname.lastname@example.org.