In today’s fast-paced, consumer-driven world, major corporations, like Visa, MasterCard, Facebook and Google, are turning to big data technologies, such as Hadoop, to analyze the massive amounts of business transaction and application data.
In addition, the financial markets of the world’s trading centers are starting to tackle advanced analytics on billions of dollars of trade value on a daily basis, leveraging the scalable architectures now provided by the latest technology. When you combine these two trends, it’s easy to see why the hype surrounding big data seems to grow exponentially. Once organizations get access to the level of information big data promises, they’re armed with something that can help them surpass their rivals: a critical advantage during these harsh economic times. Simply put, big data is one of the biggest disrupters for the way companies have handled data in the past 20 years.
If enterprises don’t start planning now to adjust their organizational structures and procedures, to they will quickly be left behind. The ability to analyze this data has systematically changed the way IT departments should think about their role in data governance and delivery. Departments will need to rethink their roles within the organization; they must be involved to a much greater extent in the way BI solutions are placed within the company. IT departments will be required to support the business by delivering integrated, clean and secure data for their BI applications.
In this drive to attain the nirvana of answers from big data, where does self-service BI sit? The drivers for self-service BI are defined as:
- Constantly changing business needs.
- Inability to match the speed of business change.
- The need to be a more analytics-driven organization.
- Slow or untimely access to information.
Business teams and management now grasp what BI consultants and data management professionals have long preached: that data must be managed as an asset. As a result, data managers have been elevated to boardroom level and IT teams now use data to react to changing business needs more quickly. Therefore, today’s enterprise needs a new type of agile BI and analytics to keep pace with these ever-changing needs, a type of business intelligence that is truly self-service.
Due to new user demands of BI, typical BI relationships, such as business/IT alignment and the roles of traditional and next-generation BI technologies, are being turned upside down. In the past 10 years, only 10 percent of major business intelligence implementations have succeeded, mainly because BI implementation has not fully empowered information workers, who still depend mainly on IT for access to data that helps them stay ahead of competitors. In the modern, data-driven enterprise, organizations have no choice but to make business decisions quicker and more efficiently.
This need for speed, or “business intelligence right now” (BIRN), drives business users to demand that their BI tools enable them to gain insight on demand. However, this is beyond the scope of most major BI systems. With the rise of big data, this problem will be compounded by the business need to understand the value of fast-growing data stores and the need to empower teams to gain insight from data analysis. Most IT teams also struggle with maintaining data governance over new types of data. While the approach of seeking “one version of the truth” through master data management has long been the mantra for CIOs, I believe that the CIO and his department are going to have to accept some hard truths:
- They no longer control all the data within the enterprise.
- They can only provide a guarantee of data governance for those data stores they control.
- They will have to provide or support self-service BI for the business user.
Traditionally, BI has required enterprise-wide standards and a single version of the truth. I believe these will continue to be important, but they won’t be sought after with the focus they once were. Business users already demand more control – spending on the purchase of BI tools by various business units has increased by approximately 40 percent in the last four years – which indicates that the current BI approach, with its never-ending demands on IT, has become unsustainable. Business users have become dissatisfied with their sluggish access to the information to help them make well-educated decisions rapidly.
It’s likely that the roles of the business user and the data warehouse administrator will evolve dramatically, and organizations will need to balance the needs of end users and IT. Now, companies will need to provide all users with BI tools that are flexible and agile, offering up the best features of both enterprise BI and self-service BI. They also need to support the move of big data from silos on the periphery of the enterprise to a central position, guided by a certain level of data governance. While it won’t follow the traditional MDM route due to less flexibility, data governance will be required to ensure that the entire organization buys into the approach.
Successful companies will be the ones that embrace big data and all its subsequent challenges. A self-service option must be available so that users can connect to data sources, both inside and outside the enterprise, allowing them to analyze and visualize that data and collaborate with their colleagues.
Big data is on the horizon and approaching fast. Enterprises that don’t start planning now to adjust their organizational structures and procedures will be left behind.
Peter Evans is Business Intelligence product evangelist at Quest Software, and a recognized expert in business intelligence and data warehousing. Peter has more than 16 years of experience in the design, implementation, and delivery of complex business intelligence systems for a large number of major international companies. Click here to read his blog and follow him on Twitter: @evansbi