Market research firm IDC predicts that all digital data created will reach 4 zettabytes in 2013. Gartner says that big data is moving from small, individual, and focused projects to an enterprise-wide architecture. All this requires a breakthrough when it comes to current approaches for leveraging big data.
We are clearly still in the earliest days of the big data movement. As such, companies are lacking the expertise and know-how to operationalize and monetize their data assets and are turning to services companies to help them bridge those gaps. Attempting to operationalize the assets can take months or even years and can be extremely expensive as well. Worst of all, the efforts may not help monetize big data directly. Thus, the current state of big data is great for service vendors, but not necessarily so great for big data buyers. Obviously, this is not a sustainable model, but it is the first step in the process.
Open source business intelligence firm Actuate recently announced the results of research that found 26 percent of companies with over $1bn in revenue are working on big data projects today, with a further 34 percent in the evaluation and planning phase. This observation points to the fact that with the services model, big data is still a luxury exclusively for very big companies.
Beginning in 2013, the market will be looking for innovation. Product-centric companies will begin to disrupt the patchwork of services-centric solutions that currently exist. The product-centric companies will deliver the speed of data-to-insight conversion with a compelling economic business case. The business case will include the time-value of data and the measure of its useful lifespan, which gets smaller as the data gets bigger. Hence, there is a compelling need for automation and a product-centric approach to big data.
Those businesses that start adopting the product-driven approach to big data early will have a significant advantage in their experience curve – leading to more enterprise-wide analytics usage. This will lead to competitive advantages in both the short and long-terms.
The biggest challenge will be to build a data-driven culture that will become a mandatory requirement for any business. The other challenge, for IT in particular, will be to keep up with the rapidly evolving suite of advanced processes to enable enterprise-wide utilization of data and collective insights.