In today's highly competitive market for business performance management (BPM) systems and services, vendors are working overtime to make their mark. They're promising great things from these applications: better financial reporting and compliance; accurate, real-time information for all decision-makers; drastically improved forecasting; and precise, realistic budgets. The question for buyers is: How well are they delivering? How many of the promises are just marketing hype? And if the answer is more than "none," what do BPM software providers need to do about it?

Two separate surveys, neither sponsored by vendors, set out recently to answer just those questions. Late last year, BPM magazine contacted a select group of subscribers who had previously identified themselves as having invested in budgeting, planning, and financial reporting systems; a total of 329 responded to this first annual Solutions Satisfaction Survey. Simultaneously, consulting firm BPM Partners was soliciting responses to its 2005 BPM Pulse Survey from readers of BPM, Business Finance, and DM Review; it received 212 valid responses. Both studies came to the same conclusion: Performance management software vendors are not meeting users' expectations as well as they should.

Market Malaise

In the BPM study, as exhibit 1 below demonstrates, less than 20 percent of respondents said they are extremely satisfied with any of the following: their system's technology/functionality, its ease of use and integration, the cost to implement and maintain the product, or the vendor's customer service. And only with their BPM application's technology/functionality are as many as half either extremely or somewhat satisfied. A quarter of respondents are somewhat dissatisfied or dissatisfied with their system's functionality. Thirty-one percent are unhappy with their software's ease of use, 28 percent with the cost of the BPM solution, and 21 percent with the vendor's customer service.

Among respondents to the BPM Partners survey, only 21 percent said their BPM project has achieved more than 90 percent of the goals they established when they started the initiative. Seven percent said the project has achieved less than 50 percent of what they expected it to. This is not what the market needs to sustain the remarkable growth it's experienced in recent years. When a major purchase (more than half of BPM Pulse Survey respondents intended before implementation to spend more than $100,000 on the software, and 10 percent planned to spend more than $1 million) fully delivers on goals around one-fourth of the time, buyers will undoubtedly become wary of vendors' promises.

In addition, for many buyers the cost and time involved in implementing performance management solutions is far exceeding their expectations. According to BPM's Solutions Satisfaction Survey, 25 percent of purchasers spend more money on their performance management initiative than they expected to, and more than half spend more time implementing BPM than they planned. The highest proportion of respondents said both their time and financial investment met their expectations (35 percent), but the second-highest number said both the time required and the cost of the project exceeded their expectations (23 percent of total respondents). Among respondents whose financial investment exceeded their expectations, 90 percent said their time investment was also higher than expected. More surprisingly, 59 percent of people whose financial investment was below expectations said the implementation took more time than anticipated. It seems that buyers across the board are having trouble accurately predicting how much effort and spending a performance management software implementation will require.

BPM Partners reached the same conclusion. Thirty-six percent of respondents to the BPM Pulse Survey said the vendor selection process took longer than they expected. Thirty-one percent said they underestimated resource requirements when planning for their performance management implementation. And around one-fourth spent more than they were prepared to spend. Twenty-three percent said spending on software exceeded expectations, and 29 percent said spending on consulting assistance went over budget.

These results, when considered as a whole, suggest that BPM vendors and service providers are overpromising and underdelivering in terms of the software's benefits for a large proportion of buyers. At the same time, BPM vendors seem to be failing to present a clear picture for their prospects of the level of effort and spending that must go into a successful deployment. This finding is unfortunate. The performance management software market is still relatively young, and now is a good time for vendors to demonstrate these systems' usefulness by helping potential users set realistic expectations. Analysts attribute much of the growth of the past year to Sarbanes-Oxley compliance efforts. If BPM doesn't start delivering its promised benefits -- and more -- the spending spree may come to a screeching halt when companies feel they have compliance under control.

BPM's Better Than No BPM

Nevertheless, the BPM Solutions Satisfaction Survey makes clear that finance and IT professionals believe software from an established BPM vendor is preferable to the alternatives. Users whose systems were built in-house gave their BPM software's functionality extremely low marks. In-house systems ranked lower than any of the packaged software in the survey and lower even than Microsoft Excel (see exhibit 2, below) in terms of functionality. This displeasure can't be attributed to the age of the software, because in-house systems are average in that area -- with a median age in the survey of three to five years old. Nor does the blame seem to lie with the complexity of the overall BPM circumstances in which in-house systems tend to be implemented.

Many respondents to the survey are running more than one piece of budgeting/planning/reporting software, and the survey asked about their satisfaction with their overall solution -- i.e., they weren't asked to rate each piece of software separately. But respondents running software that was developed in-house don't have a particularly complex software environment. Those with in-house applications average only 1.9 separate systems, which is in the lowest third among all the software packages in the survey. Clearly, BPM applications developed in-house are not meeting users' needs.

Likewise, spreadsheets are not excelling for corporate budgeting and reporting in the era of heavy compliance scrutiny. Solutions Satisfaction Survey respondents who use only Excel for budgeting, planning, consolidation, and reporting give that solution low marks in every category except cost, where it's right in the middle (see exhibit 3 below). It would seem that users should be happy with Excel's cost, if nothing else, so this result suggests that dissatisfaction with spreadsheets is pervasive and deep-rooted.

BPM software may not be achieving all of users' goals, but it's better than the alternatives. And for some stand-alone performance management applications, satisfaction is high. The software packages that scored highest in the BPM survey were rated by only a few users, so the results may not be statistically significant. Still, the customers who did respond are clearly satisfied -- verging on extremely satisfied -- with these products.

Interestingly, enterprise resource planning (ERP) vendors, as a whole, rated lower in every category than did best-of-breed, stand-alone software vendors. The difference between ERP and best-of-breed seems especially pronounced on the question of BPM products' ease of use and integration with other software systems. This finding calls into question the simplicity of running an ERP vendor's performance management package as a component of the ERP suite, so it calls into question ERP vendors' key argument for why their ERP customers should buy BPM from them as well.

Setting the Stage for Improvement

Further analysis of the survey data reveals several areas in which improvements -- in both vendors' sales and support processes and companies' BPM buying processes -- may substantially enhance long-term user satisfaction with performance management software. One is customer service. Only 16 percent of respondents to the Solutions Satisfaction Survey are extremely satisfied with their BPM vendors' responsiveness. Twenty-one percent are either dissatisfied or somewhat dissatisfied. This is problematic for the future of BPM software, because customer service directly influences people's opinion of the product.

Sixty-seven percent of survey respondents who are extremely satisfied with their BPM vendor's service are also extremely satisfied with the product's ease of use, while none of the respondents who are extremely satisfied or even somewhat satisfied with customer service is dissatisfied with the product's ease of use. At the same time, no users who are dissatisfied with their vendor's service are extremely satisfied on ease of use, and 83 percent of those who are dissatisfied with customer service are either dissatisfied or somewhat dissatisfied with the product's ease of use.

Likewise, 89 percent of people who are extremely satisfied with their vendor's customer service are either extremely satisfied or somewhat satisfied with the cost of implementing and maintaining their BPM product. Ninety-four percent of people who are dissatisfied with their BPM vendor's service are either dissatisfied or somewhat dissatisfied with the product's cost.

Even more telling, among respondents who are extremely satisfied with their business performance management software provider's customer service, 82 percent believe the product will meet their future budgeting, planning, and reporting needs. But among those who are dissatisfied with the vendor's customer service, only 11 percent believe the software will meet their future needs (see exhibit 4, below). Vendors that have been narrowly focused on product differentiation need to put more resources into customer service to ensure long-term success.

They should also make a stronger case for why organizations need BPM and specify ways in which they can benefit from it. The BPM Pulse Survey indicates that the most common reason companies today are implementing performance management software is to replace the aging systems they already have in place. This reason was cited by 49 percent of respondents to that survey, whereas only 36 percent said they are using the product's projected returns as a justification for their investment. Buying software simply for the purpose of updating poor-performing legacy systems is a recipe for dissatisfaction.

The BPM Pulse Survey also suggests that the vendor selection process takes more than three months for 69 percent of purchasers and more than six months for 17 percent. A sharper focus on ROI could simultaneously enhance the long-term satisfaction of buyers and lay a solid foundation for a more efficient, less overwhelming vendor selection process.

Another factor that correlates closely with BPM satisfaction is the consolidation of budgeting, planning, and reporting into a single system. According to the BPM Pulse Survey, 56 percent of BPM buyers purchase budgeting functionality, 47 percent buy analytics, 38 percent acquire a dashboard, and 20 percent buy financial consolidation tools. But only 26 percent buy a full suite that includes all of these capabilities (see exhibit 5, below).

The Solutions Satisfaction Survey raises a red flag on the practice of piecing together an assortment of different BPM products. The fewer performance management-related systems an organization runs, the happier its users are. More than half of respondents are running only one budgeting/planning/reporting application. Among these users, 52 percent are either extremely satisfied or somewhat satisfied with their software's ease of use, compared with 32 percent of those who run four or more systems and only 20 percent of those with three systems. And 58 percent of single-system respondents are extremely or somewhat satisfied with their system's functionality, compared with 48 percent of those with two applications, 31 percent of those with three, and 41 percent of those with four or more.

Although many buyers are shying away from comprehensive suites, the research offers some signs that companies are looking to consolidate their performance management capabilities into fewer total systems. No respondents to the Solutions Satisfaction Survey who have implemented BPM software within the past year run four or more performance management systems, and 75 percent of them run only one (see exhibit 6, below). Not only will this trend presumably make for more satisfied BPM users, but it should save vendors from some of the system-integration problems that damage future sales. Fifty-nine percent of users running one BPM system expect their current software provider to be able to meet their future needs, while only 36 percent of users running four or more BPM applications have the same confidence.

Time Will Tell

The next couple of years will be critical in the development of the market for business performance management software and services. Analysts project that growth will continue, and the BPM Pulse Survey bears out this prediction. The proportion of respondents who have no plans to implement performance management software decreased from 30 percent in BPM Partners' late-2003 survey to 26 percent in late 2004. At the same time, the proportion who have a project in progress, completed, or planned for the short term rose from 52 percent in 2003 to 64 percent.

To keep BPM initiatives on track and to increase the satisfaction of end users, vendors need to present more realistic projections of the time and resources each implementation will require. They also need to spend more time on boosting their customer service. If vendors plow ahead, business as usual, in a market where so many implementations are both exceeding budgets and failing to meet goals, the 22 percent of BPM Pulse Survey respondents who are in the planning stages of an initiative may rethink their investment. Substantial performance management purchases may be postponed until BPM's costs and benefits become more predictable. The alternatives aren't good, but performance management software vendors cannot take market growth for granted.

BPM buyers, for their part, need to focus on returns and on the benefits they can achieve by consolidating various facets of performance management into one software system. The promises of always up-to-date and precise data for all decision-makers become more difficult to achieve with every level of complexity added to the overall solution. Buying performance management from an ERP vendor isn't necessarily the answer, as those products seem to be more difficult, on average, to deploy and use than stand-alone software. But connecting eight systems (as one Solutions Satisfaction Survey respondent has done) is not a path to easy implementation and maintenance or to spectacular system performance.

The performance management software marketplace is a rough-and-tumble world. But over the next few years, as BPM moves into smaller companies that have tighter budgets and less interest in being leading edge for the sake of being leading edge, vendors will have to provide systems that do more than upgrade decades-old legacy applications. That approach is akin to slapping a Band-Aid on a gaping wound. Performance management applications have a lot to offer, but to stay healthy as they mature, they need to begin delivering more value on clearer expectations.

Meg Waters is editor in chief of BPM magazine and of the monthly e-mail newsletter BPM Express.