Are You Up To The Challenge? Seizing Big Opportunities In Big Data

Many solution providers are still trying to pinpoint exactly where the opportunity lies with big data, but one thing is clear: It has captured the tech world's attention. The big data market, according to Gartner, drove $28 billion in worldwide IT spending in 2012. This year, that number is expected to grow to $34 billion, as more organizations invest in new technologies and services that are purpose-built for big data.

That spending, Gartner estimates, will be scattered across a number of IT landscapes. Demand for new social network analysis and content analytics tools will drive much of it -- nearly 45 percent, Gartner says -- while investments in application infrastructure, middleware and data integration technologies also are expected to soar.

Some solution providers are already breaking into this market, snapping up opportunities in helping organizations figure out how to apply big data, implementing new technologies and building custom applications. Diving into the big data market isn't for the faint of heart, however. Solution providers wanting a piece of the big data action will need to make some hefty investments in new skills and partnerships before reaping profits.

DEFINING USE CASES

Much of the buzz surrounding big data is there because it's new. So new, in fact, that many organizations don't really understand what the term means or how big data technologies -- be it the open-source Apache Hadoop framework, NoSQL databases or new analytics tools -- can drive value in their organizations.

Merv Adrian, an information management analyst at Gartner, said Gartner describes this point in a technology's life cycle -- where excitement gives way to confusion or ambiguity -- as the "Trough of Disillusionment."

"We are entering a very important, pivotal year for the [big data] market. We have tracked it in our 'Hype Cycle' research, and we have said it's up and moved over to what we call the 'Trough of Disillusionment,' which is what happens to early markets when people really start to flood it and don't understand it, but see that it's a bright shiny object," Adrian said. "They start to hit the ball, start to try to use it for things it's not suitable for, or run into skill set gaps. All of those things are utterly predictable, and they are all beginning to happen already."

For solution providers, Adrian said this phase represents a massive opportunity to swoop in and help organizations really understand what they can do with big data technologies. "At this point in the market, [solution providers] get to step in and say, 'We can help,' " Adrian said.

According to solution providers already active in the big data market, helping clients determine big data use cases -- particularly around which analytics tools they should use for big data and how they should use them -- is one of the biggest opportunities in the market right now.

Tom Johnstone, managing partner at Knowledgent, a New York-based consulting firm and solution provider specializing in the big data and information management space, said one of the biggest value-adds Knowledgent offers is helping clients understand the specific use cases they should be considering for big data, and then mapping those use cases to the best-fit technologies.

Knowledgent, which partners with a number of vendors in the big data space, including Alpine Data Labs, DataStax and IBM, hosts training sessions with clients hoping to embrace big data. Johnstone said the "first and probably most important" aim of those sessions is to help clients define a big data use case.

"Clients don't necessarily understand how they can leverage big data tools, technologies and methods to drive business value," Johnstone said.

Big data use cases vary largely by industry, which is partly why there's some confusion, he said. A financial services organization, for instance, might be interested in using big data for fraud detection, while clients in the life sciences industry could leverage big data technologies to streamline R&D initiatives. Retail outlets, meanwhile, might want to mine unstructured data from social media sites to create more targeted or personalized ads.

Pat Seitz, technical solution architect at World Wide Technology, a St. Louis-based systems integrator, agreed with Johnstone, noting that one of her biggest priorities right now in the big data space is helping customers define use cases and build out a broader big data strategy. "A lot of customers are starting to test the water here, but I think it's a lot like cloud, initiativewise. They are still at this point trying to get their heads wrapped around what it means specifically to their business," Seitz said. "So our key initiative at this point in time is just helping customers understand the use cases, the different technologies that are out there, and the impact this is going to have not just on their computing infrastructure, but on their people and their business processes as well."

NEW TECHNOLOGIES NEEDED

In addition to defining use cases, the technical complexities introduced by big data present an opportunity for solution providers. Some organizations, for instance, are looking to adopt new data management and infrastructure technologies to handle the "three V's" of big data -- the velocity, volume and variety of the unstructured data at their fingertips. Traditional relational databases aren't optimized to store and manage this new breed of unstructured data from sources such as mobile devices, social media sites or embedded sensors. So some businesses, as a result, are looking to next-generation technologies such as Hadoop, the open-source framework, for more scalable and distributed data storage, or NoSQL databases, which offer more flexibility and availability than traditional relational databases.

Some organizations are turning to these new storage and data management technologies as a replacement for their traditional databases, while others are looking to them as a complementary solution. Either way, solution providers have an opportunity to implement and then train their clients on this new breed of technology.

John Ross, an IT consultant and former CTO of Kittery, Maine-based solution provider GreenPages Technology Solutions, said the shift in database technologies brought about by big data is one of the earliest opportunities solution providers can jump on in the space.

"This is a good one right here, because [solution providers] have done this stuff in the past -- think about the Lotus Notes to Exchange migrations," Ross said. "Now what you are seeing is migrations from traditional SQL server or Oracle server to online, distributed databases. That is something they can do today and assist with today that will allow customers to be ready."

World Wide Technology's Seitz also believes the shift in data storage and management technologies being driven by big data represents one of the biggest opportunities for systems integrators. Specifically, she has noticed an uptick in demand for Hadoop distributions, such as those offered by Cloudera and Hortonworks, which wrap their own features around the open-source Hadoop to make it more enterprise-ready.

"I think Hadoop will continue to be the big player in this area, simply because it manages different data types so well," Seitz said. "We have [traditional] databases that handle structured data very well, and then Hadoop is really targeted toward all data types, and that includes video images, system logs, and all that unstructured and semi-structured data that traditional databases just can't handle."

Rich Harper, a colleague of Setiz's and fellow technical solutions architect at World Wide Technology, added that in addition to Hadoop distributions, big data is driving adoption of cloud-based storage systems and even high-performance computing (HPC) environments, which offer more flexibility and availability for handling large and unstructured volumes of data.

"We are seeing [cloud adoption] quite a bit, but we are also starting to see traction with the HPC aspects, where we can move large data sets in and out very, very quickly," Harper said. "We don't see it every day, but it's starting to pop up."

RISE OF PREDICTIVE ANALYTICS

While data management and storage represent a significant chunk of the channel opportunity being sparked by the big data trend, analytics -- and particularly predictive analytics -- is another sweet spot for solution providers.

Think Big Analytics, a purpose-built big data consultancy and services provider, recognized this trend early on and is now reaping the benefits. The Mountain View, Calif.-based company recently nabbed $3 million in investor funding, and told CRN it is seeing rapid growth in the big data space.

With an aim of helping clients "connect the dots" between big data and business value, Think Big emphasizes predictive analytics models, or systems that help businesses make proactive, rather than reactive, decisions.

"The real promise of big data is really to start automating a lot more of the decisions in the business -- it's not a supercharged business intelligence model, and it's not a model of creating dashboards on bigger data sets so people can make manual decisions," said Ron Bodkin, founder and CEO of Think Big. "It's about ultimately building predictive models that automate response and going from one-off decisions to a process of continually improving those decisions."

Bodkin said common predictive models include projecting customer churn, identifying upsell opportunities or even predicting mechanical failures by collecting data from objects such as embedded automobile sensors.

CREATING CUSTOM APPLICATIONS

Think Big partners with a number of companies in the big data analytics space, including Tableau Software and Pentaho. But Bodkin said one of Think Big's biggest value-adds is building custom applications on top of big data platforms from vendors such as Cloudera and DataStax.

"The app assembly process is really an important part of the value creation," said Bodkin. "There tends to be a rich amount of work on top. The best analogy is that DataStax or Cloudera are more like relational databases in the 1980s. You would buy an Oracle database and you would need to ... develop an app that worked against the database -- you didn't buy an app. There are only very nascent notions of packaged applications for big data."

Doug Guilbeau, managing partner and executive vice president of operations at Levementum, a Chandler, Ariz.-based solution provider, also said developing custom applications is one of the biggest opportunities for the channel when it comes to big data.

Levementum partners with GoodData, a maker of SaaS-based business intelligence and reporting tools, which it calls "Bashes," for the big data space. But Levementum also creates its own custom applications to run alongside those Bashes, Guilbeau said.

"We build basically finished applications that are tailored around the other products that we typically deploy," Guilbeau told CRN. "GoodData actually is even helping us construct and build those [apps], which I can then take to my customer base, resell it as a finished app, and then continue to offer customizing services around that standard solution."

These custom apps are often used as a complement to GoodData's lineup of Bashes, including its GoodSales Bash, which gives businesses real-time analytics related to their sales cycle, and GoodSupport, which helps organizations track customer satisfaction levels with their products.

Hortonworks, one of the earlier pioneers in the Hadoop distribution space, expects the process of building Hadoop-specific applications, or taking legacy applications such as those from Oracle or SAP and making them Hadoop-ready, to be a big money-maker for systems integrators and the channel. What's more, Hortonworks believes it will be the smaller, boutique-like systems integrator shops that will nab this opportunity, more so than the larger consultancy shops such as Accenture.

"Hadoop has just opened up an entirely new world of the kinds of applications you can now build," said Dave McJannet, vice president of strategic marketing at Hortonworks, citing applications for customer profiling and equipment failure prediction as examples. "There is a huge opportunity for system integrators to work with their customers to build new data-oriented applications with this new technology."

DATA SCIENCE SKILLS

As much as big data represents big opportunities for solution providers, it represents big challenges as well. In addition to familiarizing themselves with a new breed of technologies such as Hadoop, NoSQL and next-generation analytical models, solution providers that want to succeed in the big data world need to start broadening their skill sets. One of the first investments they should make, said Gartner's Adrian, is in data science. The term "data scientist" -- used to describe an emerging role in the IT world that combines traditional computer science with business decision-making and analytics -- has become almost as buzzworthy these days as big data itself.

In an article published last fall in the Harvard Business Review, Thomas H. Davenport and D.J. Patil dubbed the data scientist the "sexiest role of the 21st century." But while businesses are starting to recognize the importance of data scientists, it's hard to find them and hire them, the authors noted.

"If capitalizing on big data depends on hiring scarce data scientists, then the challenge for managers is to learn how to identify that talent, attract it to an enterprise, and make it productive," they wrote. "None of those tasks is as straightforward as it is with other, established organizational roles. Start with the fact that there are no university programs offering degrees in data science."

So for solution providers getting into the big data game, it's a smart move to start investing in data science training for their staff or hire some new employees who can play the part, said Gartner's Adrian. "The skills gap on the programming side -- meaning people who can write MapReduce and Java -- that's a real problem, and there are lot of boutique shops springing up to do things like outsourced data science," Adrian said, citing Think Big as an example.

Knowledgent's Johnstone said some of the company's clients have their own in-house data scientists, but added that Knowledgent has made it a priority to invest in training and hiring to bulk up its own data science practice.

"Clients do often look to us to fill that role for them and then develop their capabilities internally," Johnstone said. "One of the things we have done is make a large investment in both hiring people from the outside that have that skill set -- and that's an investment in recruiting because these people are harder to find -- as well as training our internal folks and sometimes repurposing them so they can fulfill that need on behalf of our clients."

NEW PARTNERSHIPS

Apart from data science, solution providers eyeing big data need to invest in new partnerships, said Gartner's Adrian. And most of these partnerships will need to be with newer, smaller vendors that have been built from the ground up with a focus on big data.

"[Solution providers] should pick a couple of key partners and build out those relationships, recognizing that the companies they are going to start partnering with may be small and immature, and just learning [the channel]," Adrian said. "It may take a little more effort on [solution providers'] part to elicit from their partners the kinds of services and support that they want, because those guys won't necessarily have built out their infrastructure yet. But they need to make some bets, pick some partners, build relationships and learn the new skills they are going to need, or they're going to be left behind."

Adrian noted that there are market incumbents -- namely, EMC, IBM and Microsoft -- that are aggressively plotting their course into the big data space and, of course, are considered veterans in the channel. But as newer companies such as Hortonworks, Cloudera and MapR build out their channels, Adrian said it's essential to pursue those opportunities as well.

Kevin Mullin, vice president of business development at A-TRAC, a Waltham, Mass.-based solution provider, said forging new partnerships is a must in the big data world. While A-TRAC partners with channel veterans Cisco, EMC and Microsoft, the company broke into the big data space about a year ago by making a point of forging relationships with younger, big data-specific vendors Cloudera and Zettaset, a maker of platforms for managing Hadoop compute clusters.

"In terms of having a full-scale solution, you really need to look at who is best of breed and bringing products to market that can really help customers weed their way through the complexity that is Hadoop and that is the big data space," Mullin said. "You have to look at both the hardware and the software, in terms of what are the [Hadoop] distributions that are out there, how you manage this, and how you put this all together."

Mullin said A-TRAC now offers its own homegrown big data products, in conjunction with Cloudera and Xtremeinsights. A-TRAC touts its Hattie product suite as delivering the "right combination" of hardware, software and advanced analytics to rapidly deploy and make sense out of Hadoop-based clusters.

Customers have been expressing an increasing interest in the big data space, Mullin said. Gartner's Adrian, for his part, said he expects the channel, in general, to start seeing an influx of big data-specific requests over the coming year.

"The eternal questions for the channel are who should I partner with and what are the trends that matter," Adrian told CRN. "The answer to the second question is that this thing matters. It's huge, and it's going to generate an enormous amount of opportunity for the channel."

PUBLISHED APRIL 8, 2013