Big Data's Big Picture: Lots of Little Connectivity Issues

Almost by definition, Big Data must focus on the big picture. Unfortunately, as the volume of data involved in Big Data efforts grows, the resolution of that big picture might be getting a little grainier.

As our own Jesse Davis wrote back in 2012, “the days of complete visibility into Big Data are numbered – there is simply too much of it.” But volume is just one issue enterprises face. To get the most complete picture possible, you need to be able to connect, integrate and analyze as many different data sources as possible. And you need to do it securely and quickly.

Despite significant advances in the IT world, these are difficult goals to achieve. In today’s data-driven world, the abundance of rich data from applications and platforms requires access to increasingly complex data storage systems. As more data sources emerge—in both the cloud and the data center—businesses face even more pressure to constantly adapt and upgrade existing systems while deploying new ones. With all these options comes the headache of ensuring reliable and secure connections.

The most popular data connectivity options are especially light in performance. Davis wrote that “by choosing the ‘free’ options, architects are using drivers that have not been retooled for today’s business data volumes. And when working with Big Data and bulk loading, the use of such ‘free,’ but performance-limiting, data connectivity components can actually cost organizations more than they anticipate.”

In May, our Data Connections blog highlighted the seven signs of an impending data meltdown. They included poor data quality, data overload, not enough data, inflexible reporting, poor feedback cycles, stale data and an arms-length IT department. Any one of these is bad enough with everyday data. With Big Data, they’re killers.

At Progress, we’re working hard to bridge as many of the data connectivity gaps as possible. Learn more about our offerings at our Data Connectivity page.

* * * * *

Paul Nashawaty - is responsible for providing Product Marketing direction and leadership for the Progress Pacific platform, specifically for Service Providers, System Integrators, and Independent Software Vendors. Paul has extensive experience in defining product strategy and bringing product offerings to market.