Cost Reduction: The Data Handling Edition 

Everyone who handles data experiences the same phenomenon, they find themselves enslaved to several very expensive services, most of which only provide a sliver of the features required. The reasons for this are apparent, without suitable alternatives companies are left between a rock and a hard place forking up the payments needed for well-branded data companies – while still needing to search for required capabilities elsewhere. 

With that said, I believe that every company should be able to effortlessly consume data at a reasonable cost to derive needed business insights. To do so means efficiency, while at the same time providing the competitive advantage all companies aspire for. But currently, I’ve noticed an absurd trend where companies are reducing their data engagement because they are terribly expensive, rather than investing more in creating insights from data to help them successfully navigate through the current period.  

5 Main Aspects to Consider for Reasonable Data Handling Costs:

1. The cost of the tools chosen relative to the benefit derived from them

Why sign a contract with a data company, even at a discounted rate, when you are only going to receive 50% of what is actually needed? This was the case for the CDO of a large American company that I sat down with a few weeks ago and he revealed that they had just signed a $90,000 deal – which was at a considerable discount.  What struck me the most was not just the outrageous price, but that this solution didn’t cover all of his company’s requirements; they were still searching for more products to cover the remaining 50%!

2. The many different services required for end-to-end data handling

Current services focus on partial aspects of the data handling process and require specific procurement and integration resources. However, in this day and age, organizations require solutions for all stages of the data handling process including; the extraction and sink phase of the data sources/targets, the transformation phase (sometimes carried out in the DWH itself, generating additional costs), data monitoring, data visualization, versioning, real-time data and more. Regardless of whether a company sticks with one service or not, they almost always seek out additional services to complete what is required as part of the data-handling task. This involves the investment of additional resources to overcome all of their data challenges, another costly venture in its entirety.

3. The pricing configuration of the various data service companies

Current pricing structures are based on the amount of data going through various platforms but, the truth is, there is no reason for this pricing method other than the pursuit of profit. The total amount of data being handled is only increasing, and costs are guaranteed to follow suit. Rather than bandwidth-based pricing models, companies should be paying for the value they generate from their data.

4.  ‘Hidden costs,’ the transition and integration resources required to work with chosen services

Additional costs are the “hidden” costs that are usually not taken into account in negotiations with the service companies.. These include the cost of integration and/or switching from existing tools or environments to new ones. Most of the current services in the market require the removal of legacy processes to introduce new and complex integrations. In turn, companies are forced to invest more engineering and development resources, expenses that were not considered from the get-go. 

5. New dependencies on service companies

A few years ago, we decided at Unit 8200 to reduce the amount of development personnel by adopting one of the data tools on the market. In order to meet the frequent changes in business-operational requirements, we wanted a solution that allowed us to make changes and additions ourselves. But what we discovered was that beyond the high cost (which of course was significantly higher than the cost of a developer in military service), we were forced to depend on the chosen vendor with each change taking a long time and involving separate costs – costs that were not included in the initial agreement.

The current period is characterized by a challenging financial reality. A reality in which complex business decisions need to be made, based on quality and timely data. In order to harness data and help companies succeed in the current period, we must bring financial relevance to data management. The main question is how do we do it? How do we deal with these cost challenges that our current and future data infrastructures present?

The Three Steps for the Ideal Data Handling Solution:

1. Transition from multiple services into one unified platform to perform the data handling process 

The industry requires a unified platform for data handling. This platform should be customizable, it should provide everyone who deals with data the ability to carry out any needed task. It should offer an end-to-end solution, in one place, for any data, with full flexibility and without the need to invest in different systems and services, integrations, and procurements.

2. Implement ease of use & connectivity, from “plug & play” to an existing infrastructure

The platform needs to allow companies to use only the services they require without the necessity to make changes to data infrastructures. It should have the ability to integrate into existing infrastructures easily and offer a solution to any problem as simple or as complex as it may be. It should help reduce current dependencies on data services and return control of data to data experts themselves. In this way, organizations will only pay for what they really need, will be able to make changes to their data tools easily without unnecessary investments, and will finally gain control over costs.

3. Ensure direct costs are reasonable and transparent

Choose a service that provides cost transparency with pricing based on the value derived from data not the amount of data passing through these services. Companies deserve transparency in their data expenses and in the pricing structures of data services themselves.  By changing pricing methods to be based on the quantity of data after data transformations have occurred, data costs will become much more reasonable and transparent.  

The Future of Data Handling Solutions

Do you think it is too good to be true? Or, that this is impossible? This mindset is one of the biggest issues halting the data world. By accepting our current situation as if it were fate, we stifle our ability to grow and prosper.  With the right combination of technology, paired with a modern approach to handling data, costs can be reduced while attaining the needed data handling systems we aspire for.  

Within our industry, it is believed that new technology entails big risks. In principle this was true for a long time, however, now that is simply not the case.  The ability to consume new technology, in parallel with existing systems, makes for a less risky approach by reducing migration requirements and allowing for new solutions to work side by side with legacy processes. 

The future of data handling is systems that provide the ability to experiment and grasp the full technological potential of your data while offering the significant cost savings needed. Whether in regards to younger companies or older data mammoths, rather than blindly following prominent and expensive brands known for their advanced data infrastructures, now is the time to explore suitable alternatives that provide more bang for your buck!  It is time to stop committing to unreasonable costs from the get-go,  start small with newer, customizable, easily implemented solutions, and scale according to your capacity needs. The industry is changing, stop conforming, and start customizing for cost savings now, and in the years to come – no matter how much your data needs grow.