In many organizations there is a drive to collect any available data, with the rationale that it may help the business. But no matter how an organization decides to store data, invariably much of it ends up packed away like an overfilled garage. And in a large enterprise, putting data away for storage typically means putting data away for good.
So it’s left to IT to curate and maintain the data. But without strong connections to the business users, data is even more likely to sit unused and it eventually becomes part of the clutter. And hoarding this data poses its own costs. Not only are there costs to storage and maintenance, but recent events are illuminating the liabilities of long term data storage.
When a business user requests data it becomes the task of IT to dig through the clutter and dust off the data in question. But is it clear that the necessary data even exists? Is the data the business user asked for is the right data? With high costs for storage and maintenance, along with the increasing risks of storing data, the modern enterprise needs a method for cleaning out the data junk pile.
So what exactly is in the garage? Getting through the clutter manually means querying for the data, querying to find the data, then querying to reveal the contents of the data. Data profiling is critical to moving and storing data, but profiling column by column is slow. And at enterprise scale, each of these queries can take hours. But what if there were a way to look across the entirety of the data and see a profile of every column?
Connecting the data warehouse to Emcien’s data discovery technology does just that. Data across the enterprise can be profiled automatically without resource-intensive queries or slow map-reduce jobs. Instead of digging through the data to look for that one thing, this means knowing where that one thing, and everything else, is at any time.
With automated data discovery, profiling is built in to the scan. Regardless of the storage type, Scanning gives you the full profile without querying the data column by column. That’s hours of work saved in an easy, repeatable process.