FMEA AND PMBOK APPLIED TO PROJECT RISK MANAGEMENT FMEA AND PMBOK APPLIED TO PROJECT RISK MANAGEMENT

This paper presents a risk management tool based on two well-known sets of concepts: FMEA (Failure Mode and Effect Analysis) and PMBOK (Project Management Body of Knowledge). After presenting an adherence analysis between the suggested model and PMBOK, we apply the proposed instrument in a real case study: an ERP implementation at the largest Brazilian mail service and logistics organization. The main results show that the proposed model was largely successful because it identified and classified risks. Furthermore, the model helped to document the strategies and action plans needed to respond to these risks.


INTRODUCTION
The acquisition and implementation of a new technology normally require a huge effort from the organizations because they deal with factors such as complexity, innovation, scarce resources (both human and financial), and tight schedules, among others.These factors are important challenges and, in order to face them, it is usual for companies to implement new technologies as special projects.
The statistics concerning project success rates show that successful projects are not the rule.The Standish Group stated that out of the 30,000 Information Technology projects executed in the United States, nearly 63% run late and 49% cost more than the expected.The amounts involved in project failure reached roughly US$ 38 billion.On the other hand, success rates, in terms of schedule and budget, represent only 28% of the entire number of projects studied (HARTLEY, 2004).Thus, the development and implementation of successful project management methodologies, risk management mechanisms in particular, are cornerstones of successful new technology projects.
In the project management scenario, the Project Management Institute (PMI), a global community of 150,000 acting members in project management distributed across more than 150 countries, plays an important role.PMI works towards developing and disseminating best practices, carrying out research, offering training, testing and certification.Project management best practices are consolidated in a publication entitled Project Management Body of Knowledge (PMBOK).The third edition of PMBOK was published in 2004.Its contents cover the nine knowledge areas and processes of project management: integration, scope, time, costs, quality, human resources, communications, purchase and risk -the focus of this paper.
Concern regarding risk management is not new.In fact, some manufacturing experts have been using tools such as FMEA (Failure Mode and Effect Analysis) to identify possible failures in products and processes (SAKNAR and PRABHU, 2001;PALADY, 1997;US MILITARY STANDARD 1629A, 1980).
Combining two sets of techniques, FMEA and PMBOK risk management, we attempt to answer the following question: Could the risk management model of PMBOK be utilized with FMEA to create a new model for project risk management?
In order to tackle this subject, we structure our paper as follows.First, we briefly justify the relevance of the paper and clarify the concepts of FMEA and PMBOK.Then, we propose a model derived from these two techniques.Next, we present an adherence analysis between the suggested model and PMBOK's Model of Risk Management.After identifying the existing gaps, we apply the model to a real case study in the implementation of an Enterprise Resource Planning (ERP) piece of software for billing at Brazil's largest mail service company.The last section discusses the results and concludes this paper.

FMEA AND PMBOK: AN OVERVIEW
The process of acquiring and implementing new technologies usually requires considerable effort on the part of the organizations, as it involves factors such as complexity, scarce resources (both financial and human), and normally tight schedules.In order to face such difficulties, the process of new technology deployment is generally addressed through projects (DOOLEY, LUPTON & SULLIVAN, 2005).However, many of the projects fail at an astonishing rate (MATTA and ASHKENAS, 2003), i.e. they run behind schedule or incur unexpected costs.In order to avoid such scenarios, it is essential to establish risk management strategies (OLSSON, 2007).
The risk management methodology of the Project Management Institute (PMI) presented in the Project Management Body of Knowledge -PMBOK -is perhaps one of the most used technical developments for controlling risks.(PMI, 2004).Despite its dissemination throughout the world, we consider that there is some room for improvement towards a more structured device to manage risks.We believe that the well-known concepts of the Failure Mode and Effect Analysis (FMEA) might be complementary to PMBOK, giving rise to a new integrated framework for project risk management.Carbonne and Tippet (2004) point out that the FMEA method is a natural addition to the project risk management process due to its ease of use, familiar format, and a comprehensive structure.
The foundations of these techniques are briefly described.

2.1) FAILURE MODE AND EFFECT ANALYSIS (FMEA)
Failure Mode and Effect Analysis (FMEA) was first developed as a formal design methodology in the 1960s by the aerospace industry with their obvious reliability and safety requirements (SANKAR and PRABHU, 2001).Later its use spread to other industries, such as the automotive and oil & gas (PALADY, 1997).
FMEA aims to identify and prioritize possible imperfections in products and processes (PUENTE et al., 2001).More precisely, FMEA can be defined as "the set of procedures by which each potential failure mode in a system is analyzed to determine the results or effects thereof on the system and to classify each potential failure mode according to its severity" (US MILITARY STANDARD 1629A, 1980. p. 4).
In the past few years, we have seen several deployments derived from FMEA analysis in order to quantify and analyze project risk issues.In this sense, Carbonne and Tippet (2004) have coined the expression RFMEA (Project Risk FMEA).The US Department of Defense incorporates into the existing FMEA framework a critical analysis obtaining the FMECA method (US MILITARY STANDARD 1629A, 1980).More recently, Bertolini et al (2006) presented a new methodological approach named 'Dysfunction Mode and Effects Critical Analysis' (DMECA) to determine and analyze possible dysfunctions in complex management processes.
In this paper, we focus on the classic FMEA and the following discrete steps are used to perform a FMEA (SANKAR and PRABHU, 2001 Recommendation for corrective action by identifying corrective design or other actions required to eliminate the failure or control the risk Document the analysis and summarize the problems which could not be corrected by design and identify proper controls to reduce risks. The standard FMEA process evaluates failure modes for severity, occurrence and detection.The multiplication of these values leads to what is known as the RPN -Risk Priority Number (CARBONNE and TIPPET, 2004).

2.2) PMBOK AND PROJECT RISK MANAGEMENT
PMI considers the project as a temporary endeavor, unique, in line with the organization's strategy and conceived to create a product that has never been carried through before (PMI, 2004).As the objective of a project is to generate an unknown product, it usually involves risk because the steps to achieve the proposed targets are not known by the people in charge of project development.The concept of project risk is related to all events or conditions that can produce positive or negative effects in at least one project objective.Risks can be classified as internal, when the project team can influence or control them, and external when the project team are unable to control and influence them (PMI, 2004).
Considering the nine areas of knowledge mentioned in the Project Management Body of Knowledge (PMBOK), risk is the fourth most cited area in the relevant literature that documents research in project management, with a 10% quote rating (KLOPPENBORG et al, 2002).
There are several tools available for risk management in projects, most of which concern risk identification.They present a list of risks that have already occurred in other projects or critical success factors, as mentioned by Jiang et al (2000Jiang et al ( , 2002)), Jiang & Klien (2001), Datta & Mukherjee (2001), Royer (2000) and Gambôa (2004).
Once the risk survey has been completed, it is necessary to find out how to analyze and address them.Some researchers suggest making an evaluation of the risks based on criteria impact and magnitude of the impact (NAKASHIMA and CARVALHO, 2004), others consider impact and possibility of occurrence (PYRA and TRASK, 2002;FERREIRA, 2003;PMI, 2000), observing that the impact can be shown as Expected Monetary Value -EMV- (ROYER, 2000;PMI, 2000).Royer (2002) elaborated his risk management tool creating links between utility theory and cognitive psychology.Ferreira (2003) suggests the application of the effectiveness control concept to project risk management.Focusing on Information Technology (IT) projects, Stewart (2008) offers a framework to select IT projects based on a range of monetary and non-monetary benefits and risks.
PMBOK addresses Project Risk Management processes in the 11th Chapter (PMI, 2004) Regardless of the type of risk management process, the application of risk management has a positive effect on finding and taking action to avoid events that could lead to negative consequences for the project and the organization (OLSSON, 2007).In the following section, we present a simple model that attempts to integrate the "classic" FMEA approach into PMBOK's risk management process (PMI, 2004).

THE MODEL
In this section, we first present the general outline of our risk management model.Then, we compare our model with the PMBOK guidelines (see them in the lines below, PMI, 2004, Ch. 11).Such a comparison allows us to estimate the fit between the two models in section 4, when we apply it to a real case study related to the implementation of Enterprise Resource Planning (ERP) software.
According to the classification of case studies presented by Roesch (1999), this work uses a hybrid research strategy.In order to capture the characteristics of the case and draw some conclusions, this paper combines exploratory and interpretative analysis.It is also uses the construction method theory (BANDEIRA DE MELLO and CUNHA, 2006) so that by applying certain concepts developed by other researchers in other fields, it is possible to expand the previously existing theory.
Our main objective is to provide a simple, useable tool for managers and other people involved in project risk analysis.In this sense, our model captures the main topics of the well-known FMEA approach.We incorporate FMEA concepts (usually used in product and process development in manufacturing) into a broader perspective oriented to the Project Risk Management environment.As we do not intend to "reinvent the wheel", we maintain the expressions and jargon of FMEA in our model.
In general, the model is made up of the following steps: 1) Identify project's macro objectives 2) Recognize the risk groups and their corresponding effects 3) List the underlying risks of each risk group 4) Assign values for severity to each group or types of risks 5) Assign values for the probability of occurrence of each risk 6) Assign values for the ability of detection of each risk 7) Calculate the risk factor 8) List the risks in descending order according to the risk factor 9) Define a common strategy for each group or each risk 10) Take possible actions for the selected risk based on the adopted strategy This model applies to the different phases of the project cycle, from its initial planning to specific sub-projects inside the main project.The framework above has some products: a Risk Diagram, an Evaluation Matrix and an Action List.
The Risk Diagram can be elaborated immediately after step 3, as illustrated in Figure 1.We can use some Total Quality Control (TQC) techniques, such as the Ishikawa Diagram, and Brainstorming to prepare the Risk Diagram.From the visual point of view, the Risk Diagram offers a hierarchical representation, with the macro objective at the top followed by the risk effects and risk causes.In order to link with the concepts of FMEA, the risk to be avoided in step 1 is related to the Failure Mode, i.e. the manner in which a failure is observed (PALADY, 1997).Similar to FMEA, the effects listed in step 2 represent the consequences an individual or a set of risks has on the operation of a specific item.In order to identify these factors, the team must recognize the process in the following manner: "if this risk happens, then that effect will occur" (CARBONNE and TIPPETT, 2004).The underlying risks listed in the step 3 can be interpreted as failure causes, which are the reasons for failure (US MILITARY STANDARD 1629A, 1980).
Step Description The Risk Evaluation Matrix explores the risks for each group by assigning values to: severity, probability of occurrence and detection of each risk (see Figure 2).The multiplication of these three values leads to the risk index, which is similar to FMEA's RPN (CARBONNE and TIPPETT, 2004).To avoid bias, we recommend the involvement of more than one person in the process of assigning values.
In our model, we assume the principles of the RPN (Risk Priority Number) technique, which ranks the severity of a failure effect, the probability of the failuremode occurrence and the probability of the failure being detected on a numeric scale from 1 to 10 (SANKAR and PRABHU, 2001).
In our model, we also consider that the severity dimension captures the consequence of a certain failure mode and that the occurrence is related to the likelihood that a cause will create the failure associated with those effects (PALADY, 1997).In the same vein, we assume that the detection dimension concerns the ability of foreseeing the risk event (either failure mode or failure cause) in time to plan for a contingency and act upon the risk (CARBONNE and TIPPETT, 2004).
Step Description After obtaining the risk index (RPN), we can rank them from the highest value to the lowest in order to help establish strategies to address the several types of risks.In practice, for each risk we can choose different strategies using the guidelines from PMI: avoidance, transference, and mitigation for negative risks.On the other hand, three other responses to deal with potentially positive impacts are suggested: exploit, share and enhance.The acceptance option can be adopted in both cases, either negative or positive (PMI, 2004).
According to the scope of the chosen strategy, we can list the possible actions to be adopted.At the end of step 10, an action list can be created, as illustrated in Figure 3.
Step Description 8 List the risks in descending order according to the risk factor 9 Define a common strategy for each group or each risk 10 Draw up actions based on the adopted strategy

Actions List
Step Description

4) RESULTS AND DISCUSSION
As previously mentioned, we adopt two steps for assessing the consistency of our model.First, we execute a sort of gap analysis between the suggested model and PMBOK's chapter on risk management (PMI, 2004, Chapter 11).Then we apply our risk management tool to a real case study.

4.1) Gap Analysis between the suggested Model and PMBOK's Model of Risk Management
Two main characteristics differentiate the risk management model suggested in this work from that recommended by PMI: the criteria of severity and detection.
First, according to PMI, severity must be expressed in a monetary value (US$) based on the calculated value of the tasks involved and the probability of occurrence of the risk related to these tasks.The impact of an activity which has a low calculated value can cause great disturbance in a project -customs clearance of a spare part or imported equipment or a license to be obtained from a regulatory agency are typical examples.Furthermore, differing perceptions regarding monetary values can alter the outcomes of severity analysis since different professionals with distinct visions might be involved in the process.
Second, the detection criterion is absent in PMI's methodology.Although the time dimension is described in PMBOK as a crucial factor for both risk identification and the adoption of action in response to risks, we need to take into account that an exact response at the wrong time can lead to unsatisfactory results.As we have seen, the detection dimension is widely used in project management.In addition, the consequences of a problem later identified are certainly greater than the consequences of the same problem identified at an earlier phase of the project.Thus, the ease of detection of an error is essential in risk management and has been incorporated into our model.
In our analysis we compared the existing fit between the proposed model and PMBOK.The results of this analysis are available in Figures 4 to 9. The numbers in parenthesis refer to the corresponding section of the PMBOK chapter dedicated to risk management (11.1; 11.2, etc).
The Risk Management Plan suggested in PMBOK in chapter 11.1 includes methodology, roles and responsibilities, and budget (PMI, 2004).Because PMBOK's approach is more generic and has a broader scope, our model's adherence is low.

Risk Management Plan (11.1) Low
The suggested model has a narrower scope than PMBOK's 11.1 process Qualitative Risk Analysis in PMBOK involves the evaluation of the impact and the probabilities of the identified risks.It produces an overall risk ranking for the project, a list of prioritized risks, a list of risks for additional analysis and management, and trends in qualitative risks analysis results (PMI, 2004, Ch. 11.3).
Our analysis shows a high adherence to the overall risk ranking for the project as our model calculates a risk factor that evaluates global risk.Likewise, we observe a high adherence to the List of Prioritized Risks and to the List of Risks for Additional Analysis and Management since the same risk factor can be classified in a descending order.This can shed some light on prioritized risks and on those that might need additional management.Lastly, when we consider the dimension "Trends in Qualitative Risks Analysis Results" we observe a medium adherence.However, this result has validity only if we repeat the analysis several times.By doing so, we are able to identify trends and tendencies.PMBOK's Quantitative Risk Analysis involves impacts and the calculation of estimated probabilities for each risk identified.The outcome of this phase involves a prioritized list of quantified risks, probabilistic analysis of the project and probability of achieving the objectives of cost and time and trends in quantitative risk analysis of results (PMI, 2004, Ch. 11.4).

PMBOK's
Because our model allow managers to evaluate the impact of each risk, it is possible to observe high adherence to the prioritized list of quantified risks.On the other hand, there is low adherence to the probabilistic analysis of the project and probability of achieving the objectives of cost and time as the suggested model does not encompass the required simulations and it does not evaluate the variables' time and cost directly.The gap analysis results for the dimension "trends in quantitative risk analysis of results" showed medium adherence and this only can be analyzed if the process were repeated.PMBOK's Risk Response Planning involves taking action to maximize the opportunities and to minimize the threats, which might jeopardize the objectives and goals of the project.Besides the risk response plan itself, Risk Response Planning includes the assessment of residual risks, secondary risks.It also must to take into account contractual agreements, the inputs for other processes and inputs for a revised project planning (PMI, 2004, Ch. 11.5).

PMBOK's
In general, our examination shows high adherence to the risk response plan because the suggested model allows managers to establish a plan containing proper responses to critical risks.As regards residual and secondary risks, the gap analysis results have medium adherence: the elaboration of the action list (see Figure 3) allows the evaluation of the new scenario for the project.For contractual agreements subdimension, our analysis also demonstrated medium adherence.In fact, our examination indicates a link between the model and PMI´s transference strategy.Put differently, contractual agreements are related to the adopted risk strategy.The inputs to other processes are a specific generic item of PMBOK that binds the analyzed process with others without going into specific details.The gap analysis results for the inputs for a revised plan of the project have medium adherence because the application of the suggested model provides some information for an eventual revision of the project plan.For example, a particular project, which has been evaluated as having too many severe high probability risks, could make project execution unfeasible.On the other hand, a project might present few risks and therefore is subject to having its schedule shortened or its scope increased.PMBOK's Risk Monitoring and Control are a set of processes that run throughout the project because risks change during the project life cycle.It involves risk tracking; monitoring of residual and new risks that could present themselves after an action; and the execution of the action plan and evaluation of its deployment.Its product is composed of workaround plans, corrective actions, project change requests, update of the risk response plan, risk database and updates to the risk identification checklist (PMI, 2000;2004 Ch. 11.6).

PMBOK's
From the gap analysis, we can observe high adherence to workaround plans because the action list contains the activities that must be carried out and those that can be considered alternative actions or workarounds.In the same vein, our investigation illustrates high adherence tocorrective actions, as these actions are part of the action list (Figure 3).The dimension project change requests has medium adherence with the suggested model.This is because the action list supplies elements that allow the project manager to analyze demand for change.We observe the same behavior for "update of the risk response plan" as this activity can only be carried out if the suggested model is applied more than once during the project.On the other hand, the analysis for "risk database" and "updates to risk identification checklist" showed low adherence with our model.Despite the fact that the suggested model supplies some inputs to risk database, it should not be considered a specific repository for future projects, rather a tool for risk management of current projects.In summary, the above analysis has covered the 24 processes of project risk management presented in PMBOK.(PMI, 2000;2004).Figure 10 shows that most of the processes (71%) present high or medium adherence, which indicates that our model (FMEA) satisfies most of PMBOK's project risk management demands.This confirms that FMEA can be considered a powerful tool for use in project risk management.

Case Study: ERP implementation in postal and logistics services
In order to evaluate the model's feasibility, we apply the proposed tool to a real case study.The organization chosen is Company A, a state-owned company, which is the Brazilian market leader in postal services.Company A also operates in logistics and communication services.According to its annual report, Company A's revenues were roughly US$ 5.7 billion in 2007 and it employs around 110,000 people.
In the year 2000, Company A started the implementation of ERP software.The company's size and its corresponding complexity contributed to the delay of the project, which took almost seven years to be complete (PAVANI, 2007).We tested our model in a specific sub-project: the billing process of service "X", not identified here for reasons of confidentiality.Service X concerns the implementation of a customer invoicing service and is responsible for collecting about 25% of its revenue.
Service X has some particular characteristics, which meant that its billing process required a high degree of customization in the original programs of the ERP system.Project managers decided that the billing process of service X would have to be the first to be implemented in the ERP system, while the other services would continue to be invoiced using the former legacy systems.The complexity and importance of Service X for the company's income meant that there was a need for careful evaluation of the risks involved.
A project team was put together to manage the risks.One project manager, two consultants and two key-users made up the team members.They carried out the activities listed in the 10 step model described in section 3 and generated the three products of the suggested model: Risk Diagram, Evaluation Matrix and Action List.Section 4.2.1 shows a summary of the products created in the above-mentioned subproject at Company A.

Risk Diagram
The execution of the steps 1 to 3 (see Section 3) required several meetings of the risk management team members.The first step attempted to identify the objective of minimizing risks: to implement service X's billing process successfully, as shown in Figure 11.The second step, level two or FMEA's failure effect, lists the groups of possible problems that can jeopardize the objective identified in level 1.The third step, level three or FMEA's cause of imperfection, lists the risks of each group described in level 2. The risks described in level 3 are the ones which made up the list of risks, process 11.2 in PMBOK.(Level 1: Objective)

4.2.2) Evaluation Matrix
With the list of risks drawn up as described in section 4.2.1, we can now address the FMEA criteria: severity, occurrence and detection (Figure 12).For the sake of simplicity, we demonstrate only a small part of the possible failure causes that are grouped under team problems.In the column Failure Cause, the risks are listed in descending order according to the RPN value.The column Risk Strategy shows which of the 4 strategies are recommended for PMBOK (to prevent, to transfer, to mitigate and to accept) to guide the action to be taken to deal with the risks.For the untrained risk and lack of users the chosen strategy by the project team was mitigation and for the lack of consultants risk the chosen strategy was acceptance of the risk.The column Action shows the recommended actions to respond to each of the risks in the column Failure Cause.

CONCLUSIONS
In many projects, it is rarely possible to foresee action that will be needed in the future (POLLACK, 2007).Given these circumstances, we have to manage events that might exacerbate any harmful effects.It is necessary, therefore, to establish mechanisms for risk management.In this paper, we have presented a simple model that can be adopted by project managers.By combining the well-known concepts of FMEA with the requisites of PMBOK, the proposed tool analyzed here may be helpful in reducing risks in a project.Using the same reasoning as Bertolini et al. (2006), the approach of the model proposed here allows the user to analyze a generic process of a company in a straightforward but detailed and structured way.The model's simplicity not only facilitates its use and dissemination, but it can also help companies to save both financial and human resources.In other words, simple ideas can be used to solve complex problems (PYRA and TRASK, 2002).
Our analysis of PMBOK's requirements for project risk management and the available tools in our model based on FMEA have demonstrated that the proposed model fulfills 71% of PMBOK's requirements to a high or a medium degree.This indicates that FMEA can be considered as a tool to be tested in project risk management in other areas other than product development in manufacturing, such as Information Technology projects.
The application of the suggested model to a real case indicates that it presents a feasible alternative for project risk management.Furthermore, the model enables outcomes that provide concrete results: a Risk Diagram, an Evaluation Matrix and Action List.
The main characteristics that differentiate our model from PMI methodology for risk management lies in the way severity is addressed and detection criterion.In our model, we assign values on a numerical scale rather than in terms of monetary values.Naturally, financial issues can continue to drive the analysis, however, using a scale makes it easier to carry out a risk analysis as it enables the construction of a numerical risk index (or according to FMEA concept the Risk Priority Number -RPN).The inclusion of the detection dimension adds value to PMI's risk management analysis as it provides a structured way of predicting failures and avoiding them before they occur.
It must be emphasized that the proposed model was tested without the assistance of a specific information system for project management.If an information system had been used, the risk management process would have been boosted.The automatization provided by IT tools could also evaluate the model's robustness, stressing either its strengths or weaknesses.We intend to do this in the future.
; US MILITARY STANDARD 1629A, 1980): Define the system to be analyzed and construct block diagrams which illustrate the operation Identify all potential failure modes (the manner by which a failure is observed) Estimate the severity of the failure mode List the potential causes of the imperfection Estimate the frequency of occurrence of failure Describe failure detection method Estimate the Risk Priority Number (RPN) Figure 1: Risk Diagram Figure 2: Risk Evaluation Matrix Figure 3: Action List

Figure 10 :
Figure 10: Gap analysis between PMBOK's project risk management requirements and FMEA's characteristics

Figure
Figure 11: Risk Diagram

Figure 12 :
Figure 12: Schematic representation of Evaluation Matrix . The content given in PMBOK is useful for Project Managers to manage and communicate project risks.The six processes included in PMBOK's Project Risk Management chapter are: