Businesspeople want to access their company's BI data easily, then combine it with information stored in a slew of nondatabase formats. Search engines can make available all sorts of information in a way business users find intuitive.
Business intelligence (BI) has become the software of choice for companies looking to assess their performance and garner vital insights from data across the enterprise. Providing quantitative data and tools to manipulate it, BI helps organizations produce analyses that support decision-making. Large and midsize companies have invested many millions of dollars in BI software implementations as they strive to give business users information that will improve the company's results.
Originally BI was intended only for technically astute business analysts, but now many organizations are deploying these solutions to workers in sales, marketing, the contact center, the supply chain, and other areas of operations. This broad deployment means that BI technologies must meet these new users' expectations about ease of use and responsiveness.
One issue for this now-broadened universe of users is the difficulty of locating, among the many diverse reports in the BI repository, information relevant to their current tasks. An organization's data stores typically will contain not only information on financials, but also data on inventory, customer service, production details, and much more. In addition, today's BI users often find it difficult to create queries. Even when they know which table in a database contains the information they want, they may not know the column names for the specific data elements. Using SQL or a vendor-specific programming language to build a query requires technical knowledge and skill that the average business user has neither the time nor the inclination to learn.
Concurrently, the Internet has changed people's expectations about information accessibility. When search engines can deliver useful results from Web sites worldwide in a few seconds, business users question why BI applications should demand that they know which database or report holds the information they want. Accustomed to getting results rapidly through technologies from Google, Yahoo, and others, the new breed of BI users want search interfaces like those they're already familiar with to locate what they need, promptly, regardless of where in the organization the data resides. Taking note of this change in attitude, many BI vendors have announced plans to incorporate search capabilities into their products. The question now is not whether BI and search will converge, but when and how.
Ventana Research foresees two primary results of this convergence. The first is that it will become easier for business users to find information that has already been generated in BI systems somewhere in the enterprise. In most organizations, different groups have created specific business reports or dashboards to make the information that they need easy to access and apply. But the company overall may lack methods that simplify enterprisewide access to such data; utilizing search as an interface to query systems across the enterprise can be such a method. The other result of a convergence of BI and search will be the integration of data stored in a range of file types. For example, business results from a BI system, information about customer requests or shipment status in an Excel spreadsheet, and news from the Internet about the company's competitors could all be merged automatically into one regularly updated report. Combining traditional BI content with the information stored outside of databases — “unstructured” data — will provide a richer context for decision-making.
Anyone who is evaluating business intelligence or business performance management (BPM) technology for his or her organization should consider these types of emerging capabilities, which can access and merge information from a variety of source files. To determine whether this capability is yet on organizations' radar screens, Ventana Research undertook industry benchmark research designed to identify trends and best practices in the combination of business intelligence and enterprise search software to improve business efforts. We based our analysis and research on input from participants in 322 qualified organizations, most of which are located in the United States; respondents represented all major industry groups and company sizes ranging from less than $100 million to more than $10 billion in annual revenue. Slightly more than half of research participants work in lines of business, while 48 percent have IT job titles.
Plans for Adoption And Deployment
The results of our research reveal that organizations are in the early stages of integrating BI and search. Just 30 percent of organizations indicated that their companies have already combined the two. Yet there is significant awareness of the potential synergy of search and BI technologies: More than half of organizations said they will deploy some type of search capability over the next two years, and one in five organizations plans to deploy combined BI-search applications for the first time within the next year.
As exhibit 1, below, indicates, many of the applications that organizations have deployed or will deploy in the next year combine BI and search to help users run ad hoc queries against their BI data; 19 percent of organizations said they already have deployed such an application, and 25 percent plan to do it within a year. After that, the two most common capabilities for which they want to combine BI and search are the ability to find reports previously run by someone else in the organization and the ability to see reports that contain particular information. Clearly, organizations want to take advantage of information that they already have close at hand but that is not easy to find, and they see integrating search with BI as a way to get to it more quickly. In each of these cases, integration of the technologies can accelerate access to specific business information and thus speed up the processes of making decisions and taking action.
In addition, companies view the integration of search and BI in strategic terms; a majority of organizations indicated they plan to deploy it first to business analysts (71 percent) and executives (57 percent). These also are the two groups most likely to already have access to the integrated technologies, followed by line-of-business owners, who were the top candidates for deployment in the next 12 months. While organizations' mainstream BI activities are focused on broader deployments to operational managers and frontline workers, only 16 percent of respondents rated extending BI to new users within their enterprise as the number-one benefit of combining BI technologies with search. It seems clear that in the early phases of this technology, it will be deployed primarily to the more elite users, just as BI was in the 1990s.
Once an organization has made the investment to provide a single application in which senior managers can search all business intelligence content, however, midlevel operations managers and frontline workers will stand to benefit from a second phase of deployment. Based on our benchmark findings, we advise organizations to think progressively and determine how to make BI more accessible by making the search interface available to all relevant knowledge workers.
Data Types and Content
One of the top features respondents cited as a requirement of their future BI systems was better search — or, in BI terms, querying. This will be delivered when BI software that integrates search technologies becomes the new interface for ad hoc data queries. Future business intelligence systems also will include, as part of the results delivered to business users, both semistructured data (such as XML and RSS feeds) and unstructured data such as free-form text. Eighty-four percent of respondents told us it is important to integrate these categories of data (see exhibit 2, below). But they aren't being integrated yet. Although 30 percent of organizations reported that they have already integrated BI and search, only 23 percent of all survey participants have extended that technology to unstructured data such as business documents or information on the Internet. Organizations also said it was valuable to be able to run ad hoc queries or perform analysis on textual data that could be stored in documents or applications such as call center or field service systems. However, at this point, most companies are using search only to find data in reports and BI repositories. This constraint indicates the lack of maturity in deployments to date.
Integrating BI data with related business information that's in an unstructured format remains a key step toward fulfilling the promise of business intelligence. To advance, organizations will need methods of integration that can draw upon the content and document management systems they already use. We advise them to identify their major content and document assets and then determine how to integrate these assets' data with other types of information.
A majority of the organizations participating in our benchmark research emphasized that the data type they most want to integrate with BI data is customer information. We found that the integration of BI systems with customer-related content and documents is critical in many deployments, as is gaining a better understanding of customers and activities related to them. Along the same lines, the metric that the most companies will use to measure the success of their BI-search integration efforts is higher customer satisfaction (41 percent); increased revenue (34 percent) ranked number three. These findings indicate that companies want to drive performance improvements by improving their understanding of the relationship between customer purchases and product attributes.
It should come as no surprise, then, that survey respondents placed priority on customer information when asked about their business case for adopting combined BI and search technology. While 83 percent said providing a single place to search for information was the most important capability of these systems, justifying adoption on the basis of technological efficiency alone does not meet most organizations' criteria for approving purchases. But using customer information to determine how to serve them better is an analytic activity that many organizations are willing to invest in.
Participants in our survey indicated that most of the integrated search and BI applications they have deployed or will deploy in the next year involve running search or ad hoc queries against data already in their BI systems. And most (59 percent) also told us that the highest-value benefit they will derive is being able to use search as an interface for BI, which will facilitate better querying. This reflects the fact that difficulty in using BI to perform ad hoc analyses is a chronic problem for users. The use of search as a query tool has the potential to change how organizations use BI. By simplifying the complexities of currently inflexible SQL-based interfaces, search technologies have the potential to extend ad hoc queries to business users companywide.
When we asked about the business results they expect to realize by integrating search with business intelligence (see exhibit 3, below), more than half of survey respondents rated making better-informed decisions as the most important benefit. This was true regardless of the respondent's organizational role. Those who selected this option also most often selected running ad hoc queries as their most important need.
Business intelligence is a key technology for enabling performance management, and combining search with it should help companies improve performance even more. The top two benefits of search integrated with BI, as shown above, are both key aspects of performance management. Providing fast access to needed information is critical for responding quickly to events, which in turn can improve performance. In fact, the five factors that our survey respondents selected most often as benefits of integrating BI and search are directly related to business performance.
Understanding performance and determining how to optimize it require relevant information. Most organizations expect integration of BI and search to improve customer satisfaction, lower costs, and increase revenue. Eighty-three percent of organizations said that bringing unstructured data into the BI environment is important for performance management processes. We advise organizations to examine the intersection of these technologies for ways to improve their integration and dissemination of corporate knowledge. Although the number-one search-related BI capability indicated in the research is providing a single point of access to information in the enterprise, that capability promotes understanding of performance but does not directly help improve it.
A Challenge for Technology Vendors
Our research found that the vendor landscape for combined search and BI technologies is populated by many suppliers. Organizations want search to encompass various BI systems; that will require suppliers to make available the indexes to their own content and to be able to process content from others. The idea of standardizing on one BI vendor is not always practical — nor did most participants in our survey identify it as a priority.
The penetration of search vendors into the business intelligence software market is still minimal. Most organizations in our research use Microsoft-related search technology. Interestingly, Google finished second, ahead of specialized suppliers. There has been substantial adoption of Google's search technology. Some organizations also may consider adopting the basic desktop and departmental elements of Google. For business intelligence, the majority of organizations use Microsoft, Oracle, Business Objects, or Cognos as their primary supplier. In text mining and analytics, SAS was the most prevalent provider in our research, leading others that also provide dedicated text analytics. This vendor diversity points to the reality of heterogeneous technology in the enterprise and the critical need to integrate information from disparate sources.
In addition, we found that organizations have concerns about whether the available technologies can carry out key tasks. For example, the respondent organizations that we rated most innovative in terms of their technology maturity expressed interest in the ability of integrated technologies to perform sophisticated text analyses that link broader information with traditional data found in BI systems. But at the same time they said they doubt whether integrated BI and search technology can do that. The significance of this concern is highlighted by our finding that the most important search components for integration (after indexing) are content management (66 percent) and document management (64 percent). Along similar lines, organizations' number-one concern about using search to find information in previously run BI reports was the integration of search and text-analytics technology.
Even the companies that we found to be the least ambitious technically — those that are focused on basic capabilities of an integrated search and BI environment, such as having a single place to search all content and dealing with words that can have multiple meanings — question how well BI products can accomplish those tasks. Across the board, then, we found data integration to be prospective customers' key concern; it must be addressed to ensure success in combined BI and search deployments.
Likewise, survey respondents expect integration of products from different vendors to be an issue. Despite their concern about integrating BI, text analytics, and search components, only 21 percent of organizations said that it is very important to get all their components from a single vendor (see exhibit 4, below). A larger proportion of respondents (29 percent) said that it is not very important or not at all important. This is just as well, because most organizations' requirements are so broad that a single technology vendor would have difficulty meeting them all. User organizations should expect to implement products from more than one or two vendors, due to the evolving state of this technology.
Although there are cost savings and other benefits to be realized through standardizing technology, effective search of data from disparate systems is more important. Search-related technology should be able to perform indexing across all related data in various BI systems and enterprise applications, as well as to handle data in its various forms elsewhere in the enterprise. Organizations that are considering standardizing on one solution for BI, text analytics, and search capabilities should be certain that their BI vendor's search functionality can access the complete variety of sources the organization wants to access through its BI system — or at least be sure the search capability exposes the metadata and indexes required to search each necessary data repository.
Organizations that are looking into the role of search where it intersects with BI also should look ahead. Already familiar to most users, search technology can help provide access to BI information for more people, in more roles, in the organization. Along with examining search as the potential next-generation query interface, companies should understand that it is leading us back to natural query languages. In fact, the fourth-most-important search-related component, identified in our research as important by 53 percent of organizations, is natural language processing, which today is transitioning from keyword searching to the use of fragments and sentences to find information. Part of being innovative is reaching new levels of efficiency, and search that's well-integrated with BI can deliver efficiency in querying, which can lead to better decision-making throughout the company.
Mark Smith is the CEO and executive vice president of research for Ventana Research, a research and advisory services firm focused on the performance management market. Ventana Research is a premium content partner of Business Finance and Business Performance Management (BPM) Magazine.