Organizations have been using Business Intelligence (BI), Business Process Management (BPM), and Business Rule Engines (BREs) successfully as standalone technologies for several years. But today a number of factors - including an increased demand for IT to partner with the rest of the business and a need for effectiveness and agility in addition to productivity and efficiency - are driving demand to use these technologies together. A simplified description of a business lifecycle shows why this is so:
- A business has requirements, which drive business policies and decisions
- Policies create processes and the need for rules
- Processes and rules create and consume data
- Data is analyzed and transformed into usable information
- Business people use this information to refine policies and make better decisions
BI, BPM, and BRE technologies each automate a part of this lifecycle, and each provides value when used alone. Used together, they can create tremendous synergy and even greater value for the organization.
A brief discussion of the three individual technologies
Before discussing how BI, BRE, and BPM can best fit together it makes sense to explore, at a high level, the purpose of each technology independent of the others.
Business Intelligence (BI) - Decision Support for management
As transactional information systems have grown in usage over the past decades, the amount of raw data that is available for analysis has grown exponentially. The goal of BI technologies is to turn all that raw data into usable information to support better business decision making. To that end, BI tools collect, integrate, and analyze data, and present it using charts, reports, and so on.
BI systems often provide information in three primary ways:
- On-line Analytical Processing (OLAP) - Reports generated on aggregated data
- Key Performance Indicators (KPIs) - The current state of the business based on individual statistics
- Predictive Analytics - The probability of a future event occurring. For example, a credit score predicts making future payments on a loan.
The information produced from BI systems is primarily used by business managers. Decisions that used to be made mostly by gut instinct can be increasingly fact-based affairs as quantitative information becomes more readily available and easier to interpret and understand.
Business Process Management (BPM) - Operational support for the masses
Viewing work as a basic process that can be studied and improved as an idea dates to Frederick Taylor at the turn of the last century, if not before. What is somewhat more recent is the idea of automating those processes using software. For the purposes of this article, Business Process Management (BPM) will be covered as three primary functions:
- Business Process Modeling - Mapping and documenting a business process
- Workflow - Automated or semi-automated execution of a business process
- Business Activity Monitoring (BAM) - Using data to analyze and optimize a production workflow; primarily used to find bottlenecks or problems in a process so they can be remedied
The three major BPM functions fit nicely into a BPM lifecycle, shown in Figure A, which very closely relates to the old Systems Development Lifecycle (SDLC):
BPM systems are used to help workers get their work done as quickly and efficiently as possible.
Business Rules Engine (BRE) - Business logic execution for everyone
The concept of Business Rule Engines seems to have started over 30 years ago with the concept of expert systems. They have come a long way since then and in their current state are more frequently called Business Rule Management Systems (BRMSs) and tend to encompass four main areas of functionality:
- Authoring - Declaring the rules in a variety of different forms, including syntax, business or natural language expressions, and decision tables
- Storage & Management - Store rules centrally and provide the ability to secure them, retain a history of changes to them, and deploy them to different environments
- Integration - Provide the ability to have applications work with the rules both at design time and runtime
- Execution - Processing data and rules in a business rule engine
Business rule engines are intended to be embedded in other applications, whether custom applications built in-house or commercial applications meant to be sold with the engine capabilities included. Unlike BI and BPM systems that provide functionality whether or not they are integrated with other systems, business rule engines must be integrated into an application to provide significant value over and above rule declaration. Since rules can be embedded into a variety of systems they can be used to assist both management and work-related functions.
Business Rule Engines can be used by knowledge workers (subject matter experts such as analysts, actuaries, and risk managers) and by developers and other IT professionals to define and maintain decision logic in the form of rules and calculations. Because the decision logic is easier to find and update, rule-based applications are highly customizable and provide the agility to change to meet changing business requirements.
Organizational agility - How BI, BPM & BRE can work together
The three technologies, while each serving specific needs, when properly implemented can enhance one another, as shown in Figure B.
The interaction of BI, BPM, and BRE technologies
From this diagram it's clear that there's a cycle driven by the use of BI and BPM. Data from business transactions and the monitoring of workflows (BAM) is consolidated into databases and data warehouses. Management uses information backed by that data to determine how the business can be improved. Business processes are then created or changed accordingly using a BPM modeling tool. Finally, the new processes are executed using the BPM engine.
Admittedly the cycle described above is nothing new; managers have always used the best information available to support ideas, make decisions, and alter the way their business works. The difference is that technology is becoming better at turning raw data into usable information, and helping managers transform ideas and decisions into action more quickly than before.
Outside influences through reporting and compliance
Compliance is a major factor driving the demand for these technologies. As penalties for non-compliance become stricter, companies are increasingly faced with the need not just to be compliant with regulations, but to demonstrate and prove their compliance. Compliance has become a part of the lifecycle because it is a leading reason that outside parties can impact organizational processes. Organizations are virtually always expected to report something to the outside world. The information reported is frequently used to determine if an audit is required to ensure laws and regulations are being followed.
Process models are helpful for achieving compliance; even more important are actual data proving transactions were performed according to regulations and rules. Take, for example, recent inquiries by various government bodies into lending activity. Organizations are being asked to prove that loans were made in good faith and with a reasonable level of analysis. Financial institutions that can provide that proof quickly have a much better chance of weathering the storms in the current mortgage industry.
Rules are everywhere
Rules play a part throughout the systems represented. By adding business rules to the mix, management teams can get better information from their BI systems and push change through the processes more quickly. Further, execution logic that is clearly documented in the form of rules allows for clear proof of compliance.
There are four primary areas in which rules can be used in conjunction with BI and BPM Systems.
- Dynamic Decisioning in Workflows - Increase readability and maintainability of organizational workflows
- Compliance Support - Be sure that systems are compliant at all times. Further, be able to prove compliance and speed the process of conducting audits
- Decision Support - Provide tools and information that help management make better decisions
- Data Transformation - BI systems need to aggregate data from numerous sources; a BRE can help
Dynamic decisioning in workflows
The logic and input behind operational business decisions change frequently and is understood best by subject matter experts (analysts, risk managers, etc.) or SMEs. Dynamic decisioning gives SMEs control over the rules, logic, and calculations at the heart of essential operations. Dynamic decisioning frees IT staff from the burden of frequent updates to business logic, yet keeps them in control. In the context of BPM it is the outcome of properly embedding rule-based decision points within a workflow. This is possible when the BRE allows the deployment of rules as web services, and/or provides a rich SDK/API for tight integration points.
The benefits of embedding rules within workflow include:
- Workflow logic exists in the workflow tool, while complex decision logic exists in rule form; this streamlines the workflow and makes it easier to understand and maintain (See Figure C.)
- Provide the ability to change assignment rules for routing work to the most qualified performer
- Determine whether to automatically or manually handle different types of transactions
Moving complex decision logic out of the workflow tool and into a BRE streamlines the workflow.
As discussed earlier, organizations must no longer just be compliant they are increasingly being asked to prove it. Further, the rules and laws to which they must comply are in a constant state of change. Organizations that can most quickly and efficiently ensure they are compliant as well as prove compliance are in the best position to survive crises and thrive while their competitors struggle.
There are three main areas of interest where are BRE used in conjunction with BI and BPM can support organizational compliance:
- Tools for rapid development of rule-based applications, giving SMEs the ability to construct an application that supports a business process. An example is scorecards used to conduct audits. An auditor with the ability to create his or her own rules-driven scorecard can help close an investigation more quickly. Another example is a questionnaire used to capture information from people outside the organization. During a job interview there are a variety of regulations that dictate what an interviewer is not allowed to ask. Having the ability to quickly alter an interview questionnaire based on changing regulations can help prevent EEOC violations.
- The ability to reconstruct how transactions are performed, capturing a variety of data points about what fields were changed and which rules were executed.
- The ability to secure process and rule logic, with a rule catalog that allows or denies access based on permissions, and versions rules so that a history of revisions can be tracked.
There are many management problems for which a BRE could be leveraged with BI and BPM tools to provide valuable information to decision makers. Three of the primary issues a BRE can help answer are:
- Should we automate all or part of a specific process? Example: All claims are currently processed manually. Managers want to know if automating them would result in better compliance with policies and procedures. By taking transactions that were handled manually and running them through a set of rules, then comparing the results side by side with a BI system, management can then determine whether to automate all, none, or only certain types of transactions.
- What is the impact of making a business policy change? Example: Disputing a credit card transaction is labor intensive. What would happen if management decided to raise the auto-approval threshold to $15 from $10? Using a BRE, rules can be changed and run against historical data to create data for a what-if analysis in a BI system. An additional benefit to this approach is that these decisions now have demonstrable quantitative backing.
- Can this process be more efficient? Example: It currently takes two days to issue an invoice. What are the bottlenecks in the process? Many BREs have the ability to generate detailed execution data, which can be used for performance metrics in a BI system. This data, once consolidated properly in a BI tool, can help managers identify process bottlenecks.
A tremendous amount of data is generated by rule execution, business activity monitoring and transactional data from the course of doing business. When management can't get the information they need from their BI system it's often because that data is either not aggregated correctly or not currently loaded into the BI system. Some BREs offer the option of using multiple data structures from completely different sources. Rules are used to read inbound data from one source (for example from an ERP or other transactional system) and determine if and how it should be formatted for storage in another source. Semi-technical business analysts can use rules to adjust their BI data feeds directly, providing a means of properly loading data into databases and/or the BI system.
In summary, Business Intelligence (BI), Business Process Management (BPM), and Business Rule Engines (BREs) provide value when used as stand-alone technologies. When used together, they can provide even greater value to organizations seeking to improve their agility, streamline their processes, and make more informed business decisions.
Organizations that don't use any of these technologies can start by evaluating one or two of the technologies, based on where they see the most need for improvement.
Jim Wray is a Pre-Sales Engineer for InRule Technology. Jim has a 10+ year background handling everything from integrating systems to account management to process improvement. He is responsible for working with potential customers to understand their needs and help them determine how InRule can meet those needs. Prior to InRule Technology, Jim was an Account Manager with CS STARS (a software division of MMC) and a Senior Consultant with Deloitte. He holds degrees in both MIS and BPM from the Kelley School of Business at Indiana University.