Knowledge Portfolio

Table of Contents

Introduction ………………. 2
Business Process Integration Framework ………………. 2
Data Integration ………………. 3
Some Issues ………………. 3
Enterprise Information Integration (EII) ………………. 3
EAI vs. Information Integration ………………. 4
Data Integration vs. Information Integration …………… 4
Information Integration Solutions ………………. 4
Challenges ………………. 5
Markets Merging ………………. 5
Metadata Management ………………. 5
Best Practices for Information Integration ……………… 6
Conclusion ………………. 7
Sources and Links ………………. 7

Information Integration and BPM

Introduction
A decade ago, Michael Hammer pushed the idea that process reengineering was the next business revolution. He promised companies that if they overhauled their business processes the businesses would become more efficient. Most businesses that bought into the argument laid a lot of people off, but the expected efficiency eluded them. In part, this was because it was hard to get everybody who was left to agree to change their work processes and in part because even if they did, companies had no mechanism to enforce the changes. But today, the concept is back in the guise of BPM software, which provides a way to monitor and/or enforce efficient business practices. BPM software does this by extracting data from a company's business applications and doing one of two things with it: tracking how the information is used to perform a task so that you can map an existing business process, or escorting the data through a set of tasks to ensure that a business process is being followed.

BPM is defined as follows:
Supporting business processes using methods, techniques, and software to design, enact, control, and analyze operational processes involving humans, organizations, applications, documents and other sources of information.

Business Process Integration Framework
There are four distinct forms of integration. An example of framework of integration in an organization is as under.

Business Process Management
Supporting Processes
Applications
Data



















Fig. 1 Information Integration framework

1. Business-process management coordinates processes across application and possibly enterprise boundaries, such as those involved in a supply-chain relationship. Web services and their derivatives are becoming important here.
2. Supporting processes- Integration of all supporting processes to reduce redundancy.
3. Application integration, in which applications that do similar or complementary things communicate with each other, is typically focused on data transformation and message queuing, increasingly in the XML (Extensible Markup Language) domain.
4. Information integration, wherein complementary data are either physically (through warehousing tools) or logically brought together, makes it possible for applications to be written to and make use of all the relevant data in the enterprise, even if the data are not directly under their control. A typical example of this would be a new customer relationship application that combines the relational call log with the speech-to-text translated call itself.
Although the four models of integration are complementary, in this portfolio, I will focus on data integration in an organization.

Data Integration
Business today increasingly demands a unified view of information. Massive amounts of data are already stored throughout the business, but in a fragmented and disintegrated manner. In order to be efficient and responsive, business users need the ability to use this data transparently, wherever and however it resides, without concern for its timeliness, consistency or security. They require real, useful information. They demand that the IT infrastructure take care of all the details of how the data can be integrated into the information they need.

Some Issues
This demand for integration of information is not easily satisfied today. Some of the required functionality does not exist. Other enabling technology is still emerging. Available functionality is spread over many products in different technology categories.Businesses typically use the traditional integration approaches of extract, transform and load (ETL) and replication. These approaches, here called data placement or consolidation, integrate information by physically consolidating it locally in advance of using it. An emerging approach, known as distributed access or enterprise information integration (EII) enables users to obtain direct access to data in its original locations.

Enterprise Information Integration (EII)

EII is simultaneously an old and new idea. Data integration was the earliest form of integration. Data warehouses and data marts were developed to create an aggregated view of corporate information residing in disparate systems and to give business users better access to corporate information.
The workhorse of data integration has been ETL tools. They were created to extract the information, transform it into a consolidated view, and then load it into a data warehouse in a batch mode. The data volumes involved were generally large, the load cycles long, and information in the data warehouse typically a day to a week old. For synchronizing data across operational systems, operational data stores were created, which enabled the real-time update of information.
But the problem with each of these solutions was the need to physically move large volumes of data from source systems to multiple consolidated data stores including the data warehouse, distributed data marts, operational data stores, and analytical multi-dimensional databases. While these consolidated data sources continue to be important to organizations, latencies and inconsistencies are pretty much a given with such an architecture.

EAI vs. Information Integration
The batch ETL solutions of the past were not capable of meeting the real-time integration needs of the new breed of online systems. Information that is days old is not acceptable for real-time solutions. While the ETL tools continue to serve a valuable function in organizations, they became the step child of integration.
The newer Enterprise Application Integration (EAI) solutions came along and solved the data latency problem by synchronizing changes across systems in real time. However, EAI less adequately addressed the need to aggregate and consolidate data and information across the enterprise. EAI can effectively move data among systems in real time, but does not define an aggregated view of the data objects or business entities.
For example, a customer service representative on the phone needs to be able to answer a customer’s question in real time, without having to figure out which system is involved. This requires the ability to make a query across distributed data sources as if they were a single database. EAI does not address this problem at all. Enterprise Information Integration (EII) does.

Data Integration vs. Information Integration
Along with the movement to real time, the need to integrate different kinds of information has also become more important. For example, a Web site or portal could aggregate data from multiple databases and synchronize updates to all of them, as well as present other types of unstructured data such as graphics, audio and video. While data integration generally focuses on structured data, managed by databases, the term information integration includes both structured data and unstructured electronic media. Enterprise Content Management (ECM) provides these capabilities, and may also provide some application integration and workflow capabilities, redundantly providing these integration services also provided by other technologies in the infrastructure. Some of the emerging EII tools will handle both structured and unstructured information. Further, most also provide metadata management solutions in more open repositories. In other words, EII is fast becoming the part of the infrastructure that manages the information across the enterprise.

Information Integration Solutions
Information integration can be used for the following kinds of applications:
Creating a single view of a customer or other business entity
Enterprise data inventory and management
Real-time reporting and analysis, and creating management dashboards
Updating a data warehouse
Creating a virtual data warehouse
Updating common information across information sources
Creating portal applications containing both structured and unstructured data from disparate systems
Integrating unstructured data, including documents, audio, video and other electronic media, into applications.
Providing an infrastructure for enterprise information management, including all forms of digital media
Information Integration simplifies the creation of all these applications by enabling the information so that it can be accessed and managed as if it came from a single data source.

Challenges
The long-term goal of information integration research is to build systems that are able to provide seamless access to a multitude of independently developed, heterogeneous data sources. These systems should have the following capabilities:
integrate sources at scale (hundreds of thousands of sources
support automated discovery of new data sources
be easy to configure, manage and maintain
protect data privacy
incorporate structured, semi-structured, text, multimedia, and data streams, and possibly inconsistent data
provide flexible querying and exploration of the sources and the data
adapt in the presence of unreliable sources
support secure data access.

Markets Merging
Enterprise information integration is an emerging market sector. EII provides the data aggregation capabilities of the old ETL tools, combined with access to real-time information that EAI provides. The market includes the extract, transform, and load (ETL) tools popular for batch data synchronization, as well as the emerging EII and ECM solutions. Many of the ECM solutions have also added workflow and business process management, so those markets are also overlapping. Moreover, some EAI vendors are adding EII capabilities, because their customers are demanding it. Yet another category of solutions is integrated platforms that will do it all. The bottom line is that most large organizations at some point will need it all. The questions is how to make it all work together, and leverage the analysis, design and implementation from one integration project to the next. A good place to start is metadata management.

Metadata Management
In addition to providing real time access to aggregated information, EII provides an infrastructure for integrated enterprise data management. While the graphical EAI data mapping tools are easy to use and speed the integration process, the information they capture is valuable corporate information required for enterprise data quality management. The analysis required to capture the metadata to drive data transformations, is the same information required for enterprise information management.
In an ideal world, the metadata repository would manage the data that is used by the transformation engine. In the real world, semantic metadata is in multiple places, and not centrally managed. Metadata management is an issue for long-term quality of distributed information. An Aberdeen research report states that database administration costs now dominate the TCO (total cost of ownership) of applications below the 500-user level, and they continue to increase in importance for all sizes of applications (“Enterprise Information Integration: The New Way to Leverage E-information”, July 2003, Aberdeen Group). This is a pressing need that EAI tools were not designed to address, but one that the enterprise needs to address to ensure long-term quality of corporation information. After all, it is of little value to provide real-time access to inaccurate information. Merely creating a canonical data format will not be sufficient to create long-term value.

Best Practices for Information Integration
As with most technologies, success depends more on how you use it than which product you buy. Managing enterprise metadata as a valuable corporate asset will go a long way toward creating long-term value, agility, and reuse from integration efforts. The ultimate value of real-time information access will depend largely on the accuracy of the information itself. There are a number of ways an organization can improve accuracy, increase reuse, and maximize the investment made in discovering and defining aggregated data definitions:
Conduct design reviews: A metadata model represents an aggregated definition of data from different systems in a canonical format. The only way to ensure the common definition is correct is to have the model verified by all the stakeholders those who have knowledge of each of the systems, and those who need to utilize and integrate the data.
Create an enterprise metadata repository: To ensure long-term value, an enterprise metadata repository, based on standards, provides a platform for storing, accessing and managing metadata, and access to information across the organization. It is the Rosetta stone to disparate enterprise data. The repository can grow over time, on a project-by-project basis. However it needs to be actively managed to ensure integrity and data quality, and maximize reuse.
Manage metadata at an enterprise level. It is not sufficient to simply create canonical data formats. The work of researching, defining and verifying the intent and meaning of data in systems, which forms the foundation for integration, needs to be managed and leveraged. It represents a considerable investment and a valuable and reusable resource for the organization. While different projects may work with different data, a centralized group, such as an integration competency center, can track and manage how the metadata is used across projects.
Move toward semantically rich metadata. The more meaning the metadata contains, the less work the programmers need to do. Semantically rich metadata enables electronic transactions to be implemented across systems without needing to add application or database code to ensure the integrity of the data. It is the key to enabling e-commerce faster and cheaper than ever before. The standards bodies, including the OMG and the W3C, are currently working on semantic metadata standards. The Semantic Web is an example of such an effort.

Conclusion
Metadata management is another old idea that is new again. In the very early days of distributed systems, organizations started their efforts by defining an enterprise data dictionary. Unfortunately, many of these efforts failed because they were too large in scope, and there was no underlying integration infrastructure to automatically convert data from one format to another. The data dictionaries gathered dust on the shelves until the efforts were dispersed.
Fast-forward to the present, when companies are creating metadata throughout the organization, but failing to manage it on an enterprise level. These efforts may succeed in the short run, but fail to meet future needs. Creating an enterprise information architecture and strategy, and managing aggregated enterprise data at an enterprise level, will ultimately provide those companies willing to make the investment with the largest long-term ROI and greatest agility for meeting new business requirements. While there is a large number of integration technologies and solutions in the market, the emerging EII tools provide an excellent foundation for data consolidation and aggregation, and providing accurate and timely enterprise information to whoever needs it, when they need it, and where they need it.

Sources:

http://www.informationweek.com/story/IWK19990920S0008
http://www.qilinsoft.com.cn/library/White_Papers/Next_Generation_Business_Process_Integration.pdf.
http://tmitwww.tm.tue.nl/staff/wvdaalst/Publications/p222.pdf

Useful Links:
http://www.javaworld.com/javaworld/jw-08-2002/jw-0809-eai.html - This link provides some detailed information on data integration.

http://csdl.computer.org/comp/proceedings/hicss/2002/1435/09/14350290.pdf -This link defines a generic Integration Framework and describes the key steps in extending this framework to a logical and then physical architecture.

Business Process Modeling (Sanjay and Abhra)

Reflection on Business Process Modeling with special reference to the Article “Business Process Modeling and simulation for Manufacturing Management” by Barber, Dewhurst, Burns and Rogers

Traditional view vs Systems view of an organization: The challenge

The general notion that we gathered through the various readings on Business Processes was that to identify, manage and improve the processes within an organization, the first step is to adopt a systems view of the organization. To do so, managers need to break away from the inertia of thinking in terms of functional units or departments like marketing, sales, finance, etc. This isolated, “silo” way of thinking has to be phased out in favor of a 10,000 foot view of the organization.

After the holistic view of the organization is arrived at, the organization needs to be analyzed and modeled at three levels:

1. The organization and its environment
2. Value chains, processes, and sub-processes
3. Activities and performance

The core competency of the organization has to be identified in terms of a single major value chain, specifically, a core business process. A process classification framework maybe used to break-up the core process into a number of sub-processes. An advantage of using a standardized framework (like that developed by APQC): Use a generic process-centric template to map onto the existing cross-functional processes thereby rising above the functional silos.

A very helpful approach would be to record every business produce and their movement through individual processes. We draw the analogy of a Bill of Material that can throw light on the series of processes that generates its contents. In this regard, the article by Nigam would be relevant. Considering each tangible business outcome (or instance of an outcome) as an artifact and recording the flow of such artifacts between the beginning and end of processes, the processes themselves can be clearly identified, and their scopes and boundaries delineated.

The real challenge is to model the various sub-processes that make up this holistic business process and integrate them into a coherent whole.

The Article by Barber:

The article talks about the typical business process reengineering methods adopted by firms and their limitations. A Business Process reengineering process is typically formed of the following steps:
1. Process identification
2. Process Mapping/Modeling
3. Process measurement
4. Process improvement through projects.

Within our present scope of discussion of process modeling, particularly relevant are the traditional “as is” and “what –if” process diagrams/models that the article refers to. While “as is” diagrams are static representations of the process flow, what-if diagrams take into account the temporal aspect of the processes. However, static modeling tools are unable to deal with the complexity imposed by the time-dependent considerations of a dynamic process. Their dynamic counterparts are, mainly due to hardware and software limitations, not sufficiently scalable to capture large business processes.

Another approach to capture the dynamicity of the processes is Business Process Simulation. The practice is to extract individual sub-processes and run simulation tests to identify bottle-necks, flaws, etc. Further, process improvement techniques are applied on these small processes to meet the target process efficiency.

However, there are serious limitations to these traditional approaches. As discussed in class as well as in the article, some of them are:

1. Studying and improving a single process is often done without the impact analysis w.r.t. other sub-processes within a process-unit and/or the whole core business process.
2. It is common to find that a single process improvement takes place at the expense of the holistic process in terms of efficiency or costs.
3. Even though the results of simulation on small, individual processes (manufacturing industry, for example) maybe satisfactory, the collective whole becomes too unwieldy and complex.

Given the constraints of the existing software and hardware platforms, the general recommendation is to interface the larger static models with smaller dynamic models of processes. The basic idea is to conceive of the core business process as a multi-threaded environment where each end-to-end thread represents a sub-process. A target thread is chosen on the basis of its present (under)performance and potential for improvement. Simulations are done on it and improvements suggested through sessions of brainstorming activities. At the end, this is incorporated within the holistic process and the results benchmarked against the previous static model. The process is repeated till the desired level of cost-benefit ratio is achieved.

Advantages of this method:

We particularly approve of this incremental benefit model. Like any other investment, a Business Process Reengineering initiative (which requires modeling and simulation) must be evaluated project-per-project basis and not as a whole. Marginal benefit calculation is particularly important because the organization has to weigh the costs of reengineering a sub-process that might have possible cost and performance implications on the whole business.

External Vs Internal customers

In our last class we discussed about Value Streams and Support Processes. We also focused on the question- Who is more important for a company internal or external customer?
In my view, we have to keep a balance between the two. Both external and internal customers are important for an organization. If we want keep our customer happy we have to give them good customer service and that can only be achieved by if we have good, efficient, and happy employees. Internal customer service is the service we provide fellow employees and other departments within our own organizations, as well as our suppliers and anyone else with whom we work to get our jobs done.
Providing exceptional customer service lies at the heart of the mission of many organizations. It is the central theme of books, articles, motivational seminars and business courses. Its value is undisputed in business circles. What many companies fail to focus on, however, is the primary path to exceptional customer service: internal customer service.

Superior internal customer service improves morale, productivity, employee retention, external customer service and, ultimately, profitability. As Arthur M. Blank, co-founder of Home Depot and owner of the recently acquired Atlanta Falcons football team said in his keynote at the Metro Atlanta Chamber of Commerce's Small Business Person of the Year Award luncheon, caring for your "associates" is fundamental to caring for your customers and shareholders. (http://www.entrepreneur.com/article/0,4621,299619,00.html)

Business Process Integration

Business Process Integration

In our last class we talked about business strategy and value chains. We focused on Porter’s theory of competitive advantage and process integration. Porter argues that “if a company focuses on fit, it will make very hard for competitors to quickly match efficiencies that company achieves.”(Paul Harmon, Business Process Change: A Manager's Guide to Improving, Redesigning, and Automating Processes, 2003). He further says that it is process integration or fit that provides the basis for long-term competitive advantage.

I agree with Porters ideas. Although he expressed these ideas in his second book “Competitive Advantage: Creating and sustaining Superior Performance” which was published in 1985, they are still valid for most of the companies. The rapid pace of e-business is driving companies to launch business process integration (BPI) initiatives, allowing for the real-time convergence of business processes not only within an enterprise, but across organizational boundaries as well. Today, companies are focusing on improving all parts of the supply chain rather than improving just their internal processes. Many companies are using business process integration (BPI) softwares that enable them to model and then automate the execution of businesses processes across multiple enterprise systems and the Internet.

With BPI and highly visual modeling capabilities, business managers can design integrated processes, and then invoke applications to perform the business processes in real-time. This result in increased collaboration to deliver new services, streamlined operations, reduced operating costs and improved responsiveness across the extended enterprise. With BPI, you can model and automate the interaction among business process components and other business processes.

Business process integration helps businesses:
· Ensure continuity of business operations in the event of a disruption
· Simplify and streamline core or strategic business applications and processes
· Comply with and respond to industry and government regulations
· Improve operational efficiency
· Comply with value-net leader mandates
· Automate core business processes

Cisco developed an IT strategy that was very effective for their company (Cisco Systems, Inc.: Implementing ERP, HBR). In 1998, the firm decided to integrate all of its processes by implementing ERP. The new system allowed nearly all of its business functions to utilize single application packages. Additionally, the new IT platform allowed high customer, partner, and supplier interaction. It also revamped their vendor’s access to production, testing and inventory. The implementation of ERP at Cisco has made them a successful firm in the networking market.

http://www.bijonline.com/PDF/BPIPatternsJarman.pdf
The above link provides some useful information about BPI. It identifies several patterns to manage and support business process automation and integration across a distributed organization.

Key words: BPI, ERP