Over the past 20 years, data analytics has grown from its infancy as a tool for sysadmins to a platform for practically any use case. Along the way, we’ve seen incredible innovation and progress in how these platforms have developed. We’ve also witnessed the exponential growth in data volumes and the, initially three, then four, and then five ‘Vs’ of big data (velocity, volume, value, variety and veracity).
But we have now reached a point where data analytics has become prohibitively expensive for many. When Tier 1 banks have to cautiously deliberate what data they really need to analyse, you know that things have become just a little too costly.
Time then for a new, more affordable way to do data analytics.
But how? We’ve been here before with promises of amazing new technologies. The cloud promised boundless scalability at an affordable price, but many have been bitten by escalating costs. But that doesn’t mean the cloud is the wrong choice; it was just the wrong choice for traditional data analytics platforms that were never designed to run in the cloud.
Enter the first, truly cloud-native data analytics platform, Axiom.
So, what makes Axiom’s approach different and affordable?
1. Storage
- For some time now, storage has been an expensive part of the infrastructure necessary to do data analytics, and whilst we enjoyed the leap in speed that SSD brought us, we didn’t enjoy the price tag that went along with it. Object storage, like AWS S3 (simple storage service), is very inexpensive but in the past didn’t lend itself to being used for analysis and was instead used as a storage medium for large volumes of data that only required infrequent access. Axiom figured out a way to use object storage for all of its requirements, which has led to significant cost savings. The key to making this work is the file system, which I’ll discuss below.
2. Compute
Typically, an army of compute instances has to run perpetually to service the data ingest and the users’ searches. This is expensive and goes against the cloud philosophy of embracing the fluctuations of dynamic workloads and switching off or turning down resources when demands allow, thus saving money and making cloud computing affordable. Axiom based its analysis engine on serverless functions, doing away with the costly compute infrastructure altogether. Again, this has resulted in a significant cost reduction.
3. File system
Finally, to make this all work, Axiom needed to figure out how to make object storage useful for search. To do this, they tried out some open-source file systems like Parquet but quickly concluded that nothing existed which could provide the speed and scale required for modern data analytics. So they wrote their own columnar file system format. This allows Axiom’s customers to leverage low-cost object storage for all their needs, whilst maintaining fast search response times.
Data onboarding is a breeze, and their variable schema means that you don’t need to worry about the format too much; just send your data to Axiom and you can use it immediately.
Today, Axiom is offering to store forever at no extra cost, and whilst I’m sceptical that this can scale indefinitely, the point is that retention is no longer an issue. Need to keep data for one year or ten years; no problem!
Get in touch to find out more about the first truly cloud-native data analytics platform. Call us on +44 330 128 9180 or email info@4datasolutions.com.