Once you’re aware of the downsides associated with data analysis platforms, you can take steps to counter them, as Ian Tinney explains.
Sprawling data architecture can mean insights are lost, often due to an inability to correlate data across disparate silos. The key to achieving these goals is a data analytics platform that improves data handling and provides better visibility.
A data analytics platform utilises data from numerous endpoints, allowing the business to monitor the data flow in real-time and use that to deliver operational insights. These platforms may feature hundreds of apps that contain report templates and configured dashboards specific to the technologies or the vertical being monitored.
The software typically uses a web-based interface and collects and indexes machine data from data sources such as websites, applications, sensors and systems. Agents are deployed at these data points and forward the data in its native format to the data analytics platform to be analysed to identify patterns, anomalies or exceptions. Alerts and thresholds can be set to enable effective, 24×7 monitoring and analysis of data. Disaster recovery techniques can be used to ensure precious data is protected.
But data analytics platforms can prove expensive over time. As the data mountain grows, so too do the storage and licensing costs, and given that data will only ever ramp up in volume, this can see the business saddled with spiralling costs. These rising costs create difficult budgetary choices that inhibit the adoption or growth of data analysis, leading to decreased business efficiency.
Consequently, investment in data analysis can seem like a double-edged sword. There are clear advantages in aggregating and analysing your data to deliver speedy insights, but there are also some very definite downsides. If you’re aware of the potential disadvantages, you can then take action to combat them. So let’s look at these in turn.
Pros
Real-time data. Draws data from a source in real-time, enabling it to be made accessible where and when needed and analysed instantly.
Range of apps available. Access to hundreds of apps that feature ready-made reports and dashboards built for specific technologies or use cases.
Ability to reduce data spend. Data storage and licence costs can be reduced by cutting down on the amount of data being processed.
Monitoring 24×7. Uses monitoring tools to identify patterns, anomalies and exceptions.
Alerts and thresholds. Criteria can be used to look for specific data events at data points.
Additional features. Fail-over and disaster recovery are built-in to ensure data is protected.
Cons
Size of investment. Initial investment can be high. The business will need to buy an enterprise license, pay physical or cloud storage costs.
Demonstrating ROI. It’s possible to generate ROI by simplifying management, streamlining data ingest, speeding up analysis and helping the business meet its compliance demands. However, if you don’t take these steps, it becomes difficult to continue to justify the investment as data management costs escalate.
Need to control spend. Storage and licence costs will increase over time, seeing costs grow exponentially, depending on the number of gigabytes being processed daily.
Vendor lock-in. As all data analysis is carried out within the platform’s applications, the business is tied to one provider. This means the business can’t send data to other processing environments and may find it difficult to utilise other best-of-breed, cloud-native technologies.
While there are clear upsides to using a data analytics platform, such as increased access to insights and versatility, these have to be considered against the downsides of escalating costs. Thankfully these can now be countered by applying technology to reduce the amount of data being processed and stored.
An observability pipeline tool can allow you to determine which data gets sent to which data analysis platform and in what format.It can also reduce the amount of data by filtering it in flight. This can substantially reduce storing and licensing costs. To see how organisations are using the Cribl LogStream solution to minimise cost, dependency and improve time to insight, take a look at our paper or contact us for a one-to-one demonstration.
Recent Comments