Managing project risks and uncertainties
Forest Ecosystems volume 2, Article number: 2 (2015)
This article considers threats to a project slipping on budget, schedule and fit-for-purpose. Threat is used here as the collective for risks (quantifiable bad things that can happen) and uncertainties (poorly or not quantifiable bad possible events). Based on experience with projects in developing countries this review considers that (a) project slippage is due to uncertainties rather than risks, (b) while eventuation of some bad things is beyond control, managed execution and oversight are still the primary means to keeping within budget, on time and fit-for-purpose, (c) improving project delivery is less about bigger and more complex and more about coordinated focus, effectiveness and developing thought-out heuristics, and (d) projects take longer and cost more partly because threat identification is inaccurate, the scope of identified threats is too narrow, and the threat assessment product is not integrated into overall project decision-making and execution. Almost by definition, what is poorly known is likely to cause problems. Yet it is not just the unquantifiability and intangibility of uncertainties causing project slippage, but that they are insufficiently taken into account in project planning and execution that cause budget and time overruns. Improving project performance requires purpose-driven and managed deployment of scarce seasoned professionals. This can be aided with independent oversight by deeply experienced panelists who contribute technical insights and can potentially show that diligence is seen to be done.
This article considers the risks and uncertainties to completing projects within budget, on time and fit-for-purpose. What are the risks and uncertainties? When, where and how do they commonly impact on projects? How might controls be applied to reduce slippage?
Over the past decade ‘planning fallacy’ (underestimating the time and cost to complete a task) has been argued as the soul of project slippage (Flyvbjerg et al. 2003; Ansar et al. 2013; Flyvbjerg 2014) and Kahneman (2012). Though attributed to Kahneman and Tversky (1979), a related idea of planning shortfalls is stated by Parkinson’s Law that ‘a task will take all the time you give it’, and a bit more (Parkinson 1958). ‘Fit-for-purpose’ (the project meets the needs for which it was undertaken to satisfy) is less explicitly addressed than time and cost, but implicitly shortcomings in the quality of the product translate into more time and cost (or less benefit). The Flyvbjerg-Kahneman remedy is to use reference class forecasting (the performance of previous similar projects as the basis for predicting), and realistic and transparent scheduling and estimation of costs and benefits. This article addresses the shortcomings of project planning and execution, focussing on risk and uncertainty controls particularly on the environmental and social aspects of infrastructure, natural resource and socio-economic projects in developing countries.
The framework for this article is the constraints analysis in Figure 1.
The project: risk and uncertainty
Various contemporary perceptions of the nature of risk are described by Blennow et al. (2013). In this article risk is a bad thing that might happen and for which the probability and the consequence can be quantified. Bad things that potentially can happen in a project and that are poorly to not quantifiable are uncertainties. Threat is here the collective term for risk and uncertainty. Here good things are not threats, but that they might not come about constitutes a risk or uncertainty of obstruction to good things eventuating. This is consistent with what management tries to do – remove obstacles or ease constraints as explained in The Goal (Goldratt 1986). While risk is often used loosely on projects, the stricter usage here does not invalidate conventions (e.g. likelihood x consequence = severity), as considered below.
Are the threats that most materially affect projects risks or uncertainties? Consider four South African infrastructure construction projects (Table 1). For the three water supply projects the degree of achievement is generously stated by the principal. The environmental and social impact assessment (ESIA) for the Olifants Water Resource Development Project (ORWRDP) started in 2004. Obtaining environmental authorization took more than two years. This delayed dam and water delivery construction. On the Mooi-Mgeni Transfer Scheme (MMTS), delays in environmental authorization, started in 1999, set back achievement more than stated in Table 1, constructing the transfer pipeline started only in late 2013, and the land dwellers displaced by inundation of Spring Grove Dam have not yet moved into their replacement housing. The Mokolo-Crocodile Water Augmentation Project (MCWAP) has been delayed further by a big flood in March 2014, and later a dam failure. The national road project was designed, and its servitude purchased, more than 40 years ago. The road was never built. The ESIA has been in preparation for 4 years, it is now released for public comment, it has yet to be submitted to the authorities, it might still be contested in court, and construction could be subject to industrial action. In the meanwhile costs are mounting – no return on already incurred costs, and the accidents and lives lost on the present dangerous road.
Students of project management might interpret Table 1 by the ‘break-fix’ model (Flyvbjerg 2014). The project stalls (the break) for want of anticipating threats, so a new version of the project is generated (the fix) with revised budget and timelines. The previous cost and completion date are erased from memory. Iterated ‘break-fix’ can make any project look good.
Time-consuming ESIA preparation and authorization, stakeholder intervention, court injunction, industrial action, rare events and defective construction are the types of threat that are chief project show-stoppers. They are poorly quantifiable. They are not risks, but uncertainties.
Project proponents typically employ professionals to manage and execute. Why then would oversight be required?
Projects can slip for many reasons. Flyvbjerg et al. (2003) consider project politics, planning and execution, and attribute slippage to ambition, egos, vested interests, changes in currency exchange rates, altering or developing project scope and design, underestimated cost, overestimated benefits, optimism that everything goes according to plan (EGAP), and disregard of safety requirements and the full environmental and social externalities. van Marrewijk et al. (2008) present a more benign view with project participants in an alliance collaborating to realise a shared vision. The cultural setting of the project might indeed determine degrees of autocracy, accountability, groupthink, project loyalty, etc., and in many projects many participants perform professionally to complete the project satisfactorily. In hydropower projects the amount of slippage did not correlate with the degree of autocracy/democracy, but slippage occurred possibly for different reasons under the differing cultures (Ansar et al. 2013). Both autocratic and democratic project leaders have vested interests. They want to know whether the project is running to plan, what shortcomings exist, how achievement might be improved, and neither will want to advertise cost and time overruns. Even if the most able are appointed as leaders, oversight institutions and mechanisms are warranted if shortcomings are to be avoided. Shortcomings range from limitations in expertise, experience and capacity, through vested interests, to corruption and dishonesty. In practice these are not simply told apart. No one is omniscient. All people are dishonest. Unless projects are transparent and publicly reported upon then society’s default assumption – understandable and justified by the behavioural economists’ research (Ariely 2012) – is that there is something to hide.
The World Bank prescribes that construction projects which it funds be reviewed periodically by panels of independent engineering, environmental and social experts. Financial institutions adopting the Equator Principles require peer review for ESIA, construction and operations phases of funded projects. Kaplan and Mikes (2012) and the Flyvbjerg-Kahneman school advocate the ‘outside view’. To some project proponents these panels are unwarranted and unwanted expense and interference. However, a perceived utility of the panels is illustrated by them continuing after World Bank loans close, and by them being adopted on projects without World Bank funding. The panellists act as sounding boards and devils’ advocates, examining and criticizing designs, explaining why things might fail, and suggesting improvements, with savings running to US$ millions. Also, the mere fact of surveillance moderates excesses of ambition and optimism, and is recommended by Ariely (2012) to curtail dishonesty that everyone is prone to if not watched.
The project plan
Key components of the project plan are objectives and scope, what will be delivered and how, resources including budget, and schedule. Objectives and scope are dealt with under stakeholder engagement below, and the other components here. The boundary conditions of our generic project are related. Time is money, and so is fit-for-purpose. If project completion is delayed it likely costs more, because the contractor incurs more cost, and because the benefits, revenue streams and loan payback are postponed. If the product is not fit-for-purpose, further time and expense are incurred to effect remedy which, if not possible, decreases benefit.
From Table 1, project delays, and hence increased costs, are common. In four megaprojects construction costs were 125% to 180% of the initial estimate (Flyvbjerg et al. 2003). While some projects come in close to budget the Flyvbjerg figures understate the overrun in the typical public and public private partnership projects (PPPs), as illustrated for hydropower projects (Ansar et al. 2014) and Table 2. Ansar et al. (2013) found cost overruns increase with project size and duration. This is not evident from Table 2, but the Flyvbjerg-Kahneman school would say the projects in Table 2 are of different types.
In some business projects there is an impression of thorough cost-benefit analysis (CBA) or equivalent. It is not the same on PPPs. Why is that? In some public projects the benefit might be very large, and the project is managed not so much to hone the cost-benefit ratio but to expedite the project while containing the cost. The benefit of a 555 km long fuel pipeline connecting economic hubs and costing US$2.4 billion might be infinitely better than a daily 65 km long convoy of 30 000 litre tankers on an already busy freeway (Table 2, second item). Yet in many cases it can be vital to determine the benefits accurately in order to know what cost can be afforded yet the project remain viable. Suppose a pumped storage scheme. The prices of buying power (to pump water when there is surplus power on the grid) and selling power (when stored water is released to generate hydropower), and the amount of power to be bought and sold, must be known to set an upper limit to the construction, mitigation and operation costs. It is not enough for the ESIA to conclude qualitatively that ‘there are no fatal flaws’ – all impacts can in theory be mitigated, but at what cost? How many households must be relocated? What area of arable land will be lost? How many livelihoods will be destroyed? What will mitigation – probably spanning decades – cost? The cost might be unaffordable, in which case the facile ‘no fatal flaws’ is mistaken.
Done thoroughly at feasibility, with ‘reference class forecasting’ (Ansar et al. 2013), CBA provides a necessary though incomplete guide on project go-ahead. Conventionally it is numerical. Inexactness of number inputs into the CBA can be entered as statistical distributions and Monte Carlo techniques used to produce expected values with confidence limits. The most likely development (MLD) with variation about it should be adopted rather than ‘planning fallacy’ (Kahneman 2012) or optimistic EGAP (Flyvbjerg et al. 2003) that is unlikely ever to arise. Contemporary sensitivity analysis software helps detect where good data are needed, and which are the most critical cost and benefit aspects of the model. If the confidence limits from the Monte Carlo modelling are wide it means that even the numbers are warning that the project is risky. If the confidence limits are narrow there are still the uncertainties that the numbers have not captured, in particular about the future. Circumstances will change but precisely how is unknowable. To illustrate, return to our example of pumped storage schemes. These are huge investments with long operating lives over which payback is required. Yet even now a pilot pumped-heat electricity storage is being developed, anticipated to store energy at 75% of the cost of conventional pumped storage. Might today’s pumped storage schemes become obsolete before capital outlay is repaid? Ansar et al. (2014) urge agile energy alternatives with shorter time horizons than conventional hydropower projects. To limit planning fallacy not only is experience of previous similar ventures relevant, but the proponents should not assume their new project is manned by smarter operators and thereby immune to past perils (Kahneman 2012).
Having done the CBA using reference class forecasting, and though there will remain greater or lesser uncertainties, the project proponents and managers must work within budget and schedule. Providing cost and time reserves or contingencies will, by Parkinson’s Law, guarantee project overrun. I recall one build-operate-transfer project in which the concessionaire had to find his own funding. Is it coincidence or selective memory that there was no project slippage? Follow-up CBAs, conducted through the construction and operation phases, can help keep the project viable (e.g. whether to cut costs, where, by how much, what tariff increases to seek, if and when to abort the project).
In the previous section project objective and scope were mentioned as key project components. What is the purpose of the project? Project aims can be too vague, too numerous, misdirected or unachievable. A project can solve the wrong problem. What stakeholders want is not enough. What the project proponents think the stakeholders need, can be misperception. Projects must meet real needs, yet often they do not, constituting waste of donor, investor or taxpayer funds. Involuntary stakeholders must be identified and properly informed, which may extend to teaching them how to assess the impacts of a project on their lives, properties and livelihoods. A few examples follow.
Polak and Warwick (2013) report a project to help poor families in Haiti making charcoal from sugarcane waste, providing fuel, jobs and income. A method to make charcoal was indeed developed. What did not materialize though was an efficient technique to produce a competitive and saleable fuel to benefit buyers and enrich producers. What was needed and not obtained was insight into the circumstances. When are sugarcane waste and labour available? At what cost can the charcoaling equipment be bought to produce a saleable product at what price and quality to pay back the investment and make a profit?
In another example, road users in the Gauteng Freeway Improvement Project (GFIP) were not engaged on the scope, nature of improvement, method of funding, and options for payback. All this was decided by the authorities, because they know best (Table 2, fifth item). Now 80% of road users refuse to register and pay electronic road tolls. In a year the road agency has racked up US$200 million in unpaid toll fees. The users have no ownership over GFIP. There is no social contract.
Polak and Warwick (2013) urge project proponents to clarify the central problem, talk to the people with the problem, and listen to and understand them, to craft the essential intent (McKeown 2014), including scope and boundary conditions. Infrastructure projects are engineer-driven and top-down. Stakeholder engagement in southern Africa is typically authoritarian and only informs and invites comment. No wonder problems arise, as with GFIP. The World Bank’s experience with land reform shows that achieving success goes beyond technical land management and depends on the institutional environment and factors that determine investment and profitability (Binswanger 2007). Insight into the lives of the stakeholders requires surveys to seek unbiased perceptions, one-on-one dialogue and forum discussion, and stakeholder buy-in is helped by facilitating meetings where stakeholders decide their priorities, and by involving them in implementation (Decker et al. 2012). For a project to be sustainable, it must be profitable and ideally all stakeholders must benefit (Polak and Warwick 2013; Simanis and Duke 2014). Porter and Kramer (2006) reshape corporate social responsibility from philanthropy to corporate social integration through designing business activities in which society shares value. One of their examples is Nestlé developing a dairy business at Moga, India. Nestlé built refrigerated dairies and sent trucks to collect milk, at the same time dispensing veterinary and agronomic services, paying more for milk, increasing local farmers from 180 to 75 000, prompting competitive dairies and milk factories and spawning an industry cluster.
If stakeholders are not engaged early in dialogue, and not involved in pilot trials and the like, the project becomes ‘yours’ not ‘ours’, potential benefits are liable to be overlooked, and uncertainties abound that the project will not address real needs, fall short in delivering sustainable benefits, and might be obstructed by civil disobedience, industrial action or court injunction. Participative stakeholder engagement is a necessary measure that, done well, can swing obstructers and objectors into neutral observers, supporters and even ‘make-it-happen’ allies.
Conventionally risk (and uncertainty) management involves two components: (a) assessment comprising threat identification, classification, prioritizing, and devising controls, and (b) applying the controls.
ESIA is the most visible form of threat management for projects. A decade ago Flyvbjerg et al. (2003) observed that ESIA was learning only slowly, and shortcomings concerned (a) lack of accuracy in impact prediction, (b) the narrow scope of impacts and their time horizons, and (c) inadequate organization, scheduling and institutional integration of the ESIA process in overall project decision-making. What has happened in the interim, as reflected by project performance in a developing region?
In South Africa ESIAs are done to obtain government authorization, not to control threats (Mentis 2010). Often the ESIA passes over serious threats. For example for big infrastructure projects the loss of households, cultivated fields and jobs of the poor may be overlooked. Houses are big assets to most people, and poor rural people depend on cultivated fields for their livelihood. Houses can be replaced but in southern Africa arable land is scarce, and restoring the lost livelihood of a poor land-dependent household is difficult. Unskilled labour is not mobile and hard to retrain and retool. Without identifying these issues they are not quantified and included in the CBA, and not considered in control measure affordability and project feasibility. With overlooked threats, the environmental and social management plan (ESMP) is defective, the proposed controls are superficial and might not comply with the law that, for example, in Lesotho requires ‘full description’, and possibly insufficient to enable a tendering contractor to prepare a threat management budget. Often there is poorly defined responsibility for the fuzzily and incorrectly identified threats so that during the project impacts are not avoided and not mitigated, and the project bequeaths detrimental externalities.
In developing regions there is a skills shortage – a scarcity, at all levels, of professionals experienced in planning, constructing and operating projects. Principals too often appoint impact assessors without experience on that reference class of project – a virtual guarantee for overlooked threats, poorly controlled detrimental externalities, and overruns. Blennow et al. (2013) make the point that forest owners seeing and believing the effects of climate change improve prospects for adopting adaptive measures. Awareness, perceptions, belief, commitment and implementation might be aided by forums (e.g. lessons learnt) that rarely take place even within single organizations undertaking several big projects.
In Table 3 examples of methods to identify threats are presented. These are indeed just examples because there is no single right way to assess threats. Rather, good assessment is a mindset and the expert assessor adapts her method to the specific circumstances of a project. Table 3 therefore is intended to illustrate, not prescribe.
Examples of contemporary methods can be categorized into hindsight, informal, checklist and matrix, and input–output, constraints, scenario and what-if analyses. A chief reason why many ESIAs are weak is that threat identification is informal. It involves cursory thinking (System 1 or automated thinking of Kahneman (2012)). For example, the local rural poor people are asked and they say that their cultivated fields are unproductive, not because they do not depend on the fields, but because they want employment on the upcoming construction project. The assessor accepts this at face value. He ignores hindsight, and does not use checklists and matrices – what he or others learnt from previous similar projects. He does not entertain the possibility that this new project might have novel aspects and effects. He does not deconstruct the project into components. He does not scrutinize each component for gains and losses of energy, materials, rights and opportunities. He never tries to assemble his threat identification into a coherent picture of cause and effect, as in a constraints analysis. He never gets into deep and effortful thinking (System 2 of Kahneman (2012)).
The identified threats form the project threat register – a list of what might go wrong. Threat management ultimately needs to recognize which threats are material and warrant attention, and which trivial. Conventionally the threats are categorized according to the classical organization functional units such as engineering, environment, finance, health and safety. Done this way project integration can face problems of ‘apples with pears’ comparisons. Rather classify threats by criteria that relate to the manner of control. Group threats into preventable, strategy and external types (Kaplan and Mikes 2012).
Preventable threats are internal. They arise within the functioning of the organization. Examples are biodiversity loss, dust, fuel and oil spills, noise, water contamination and waste. They are controllable, and the aim with them is to avoid or eliminate impacts cost-effectively. Checklists and matrix methods, and input–output analysis (IOA) are effective means to identify preventable threats. A rules-based system is developed (e.g. keep site clear of litter and waste at all times). Most ESIAs, environmental authorizations and ESMPs focus on the preventable. In some projects, such as MMTS (Table 1, item 2) in which good rapport was struck between consulting engineer and construction contractor, control of preventable threats was done to a high standard. This fits with the alliance model of van Marrewijk et al. (2008). However, in other projects where the preventable threats were poorly controlled, relations between principal, consulting engineer and contractor were more adversarial and confrontational – of course cause and effect are unclear.
Strategy threats relate to how work is done – commitments, methods, standards, values, etc. The aim of managing strategy threats is to reduce likelihood of occurrence and mitigate consequence if the threat materializes. Unlike with the preventable, strategy threat is not entirely under management control. An example is a strategy to adopt a just-in-time (JIT) inventory. For instance in a construction project the aggregate, cement and sand are not stockpiled months in advance, but procured and delivered as needed. The benefits are reduced cost, small lay-down area and therefore reduced project footprint. The downside is possible disruption in the supply chain – the portion of the threat outside managerial control if an external supplier is used. In constructing a roller-compacted-concrete dam wall, disruption might lead not only to project delays and budget overruns, but interruptions in concrete laying could impair the product. Prevention – not adopting JIT – would seem prudent for roller-compacted-concrete dam walls if outside suppliers are used.
Another strategy example concerns project-affected people. A road agency might disregard loss of jobs that arise because a realigned road causes, say, closure of a filling station, on a rationale that employment destroyed here is recreated there. However, unskilled labour is not mobile, and cannot move readily if ‘here’ and ‘there’ are far apart. Hence the agency strategy does not accord with the economist’s precept of at least one person better off and none worse off, nor with the World Bank’s standard of maintaining or improving livelihoods. The road agency might be sued for lost livelihoods. In contrast the strategy in Lesotho Highlands Water Project (LHWP) is fair and prompt compensation for losses suffered, and no unfortunate precedents. But even with the best plans, the unanticipated can happen – the head of the household loses his identity document or dies, or ownership of the lost asset is disputed, or the losses are unfairly calculated or underestimated so the claimants go to court and the cost balloons, or the loss is unfortunately overrated and this sets a precedent for further claims against the project.
Uncontrollable events fall in the class of external threats. Examples are flood, drought, earthquake, disease epidemic, communications blackout, power failure, price escalation, currency fluctuation, industrial action and political turmoil. Threat management here is to reduce the consequence of eventuation, to develop organizational resilience to Black Swans (unpredictable events of big consequence such as the 2008 global financial meltdown, and the recent seismic events and tsunamis in Indonesia and Japan (Taleb 2008)). The control measures centre about developing resilience and emergency response and preparedness, and design of organizational and operational structure to retain function in the face of adversity and catastrophe. The logic is to design the organization so it is robust and safe-in-failure (Mentis 2014; Linkov et al. 2014).
The approach to strategy and external threats is to stage internal workshops, use embedded expertise in scenario analysis, ‘stress test’ the plans not only in the business-as-usual context but also for when things are unusual, and supplement with independent review. Strategy and external threats are poorly quantifiable, so they are uncertainties rather than risks. On most projects the ESIAs, environmental authorizations, ESMPs and day-to-day management poorly cover strategy and external threats. Managing the project affected people on LHWP is an exception. Hundreds of households were relocated, water and sanitation programs instituted, living standards maintained, and advice given on investing compensation payments, not without shortcomings and failings, but with policy formulation, planning, budgetary provision, diligent implementation and learning from experience. On other projects, building a fish barrier is done better than relocating displaced land dwellers, culvert design for a new road takes precedence over possible destroyed livelihoods that the road could cause, and a behind-schedule project does not have time to take flood protection precautions and so slips further when flood occurs.
Possibly the only limit to what can go wrong with a project is imagination. If the project threat register has been prepared diligently, the list of potential problems will be long. On Sasol’s Mozambique-Secunda pipeline 708 threats were identified in an operations assessment. The top 10% were prioritized. Even that was ambitious. Finally only 2% of 708 were addressed. There are many reasons to prioritize. Simplest is that no one has the resources of time, budget and manpower to do everything, and the more resources are distributed across many controls the less effective can be control on each threat. This is argued about life generally by McKeown (2014). Notwithstanding this, environmental authorities want comprehensive ESIAs and ESMPs, and if some omitted item, even trivial, is picked up then environmental authorization is withheld. Yet a thought experiment on the economics of threat control shows that prioritization is imperative. Suppose the following. First, resources of budget, staff and time are finite. Second, control measures have variable effectiveness. There is always a bigger or smaller residual impact – remaining effect after the avoidance and mitigation are done. Figure 2 shows the principle of this. As deployment of resources increases so the residual impact reduces, in the form of a decay curve. This is the best case and assumes that resources are deployed first where they have biggest impact reduction. Of course the cost rises as control measures are increased, as reflected by the line rising from left to right in Figure 2. The total cost (add the decay curve and the straight line) gives a saddle-shaped curve which implies that beyond some moderate level of control costs outweigh benefits. Hence, even if funding were infinite, controlling for all threats is not a proposition.
Examples of methods to prioritize threats are given in Table 4. As with threat identification, there is no single right method, and the expert adapts her technique to the specifics of her project. The ‘impact assessor’s rating’ in Table 4 is in government guidelines, and widely and often uncritically applied. Its rating of consequence is complicated. It implicitly assumes that threats are absolute whereas they are relative to context. For example, in the rich world the loss of a house is easily managed by paying cash compensation, but in the poor world the lost house must be replaced by the project in a complex and costly exercise. On a South African mine transported personnel are required by mine health and safety law to be buckled up, whereas labour on a farm or timber plantation is transported on the back of an open lorry. The ‘impact assessor’s rating’ does not contextualize threat severity, and often it invokes circular logic: a threat is significant because it needs mitigation, and mitigation is necessary so the threat is significant.
Simple qualitative rating might be weak also because it assumes threats are absolute, and because it might be too simple and insensitive. At the other extreme, the probability-cost technique can be expensive to undertake for screening many threats, especially when few threats are ever going to be controlled. Probability-cost also assumes absolutism or otherwise that the expected value yielded by the product of probability and cost is the supreme measure of severity (Brown et al. 2001; Evans 2012). Dollar threat levels might group together events that differ in legal obligation (e.g. buckling up, as above), society’s norms or expectations (e.g. in Gauteng, South Africa, you do not pay your electronic road toll but in other places you do), or commitment in corporate policy (e.g. road agency disregards loss of livelihoods, but LHWP commits to compensate promptly and fairly). By and large, society objects to a project’s detrimental externalities irrespective of the actual dollar amount, so dollars might be the wrong criterion in threat screening.
Ordinal rating has several merits. Not all relevant variables can be expressed in ratio or interval data, but it is always possible to convert ratio and interval data to ordinal. Ordinal rating can provide a common currency enabling comparison between different types of threat, and different levels of knowledge on individual threats. An example of an ordinal rating is given in Table 5. It easily accommodates qualitative issues such as whether control of threat is legally required, expected by the stakeholders, and prescribed by corporate policy. The scoring can be calibrated to account for context. For example, once all threats in the project threat register have been scored, they can be summarized in a matrix where cells in the matrix are the number of threats corresponding to the relevant rated likelihood and rated consequence (Figure 3). The locations of a few well known threats in the matrix are then used as a basis to draw a threshold between material threats that warrant control by such criteria as legal requirement, societal expectation or corporate policy, and the other immaterial threats.
Having discriminated between the material and non-material, the question arises: Can the project afford to control the threats rated as material? If not then the project should not proceed. Now the control costs, not the unavoided impact costs, need to be determined. It should be the primary function of ESIAs, CBAs and indeed the whole of threat management to put the project decision-makers in an informed position so the affordability of controlling threats can be determined, as explained above about pumped storage, displaced households and lost arable land. If the necessary costing of environmental and social controls is not estimated under the aegis of the ESIA, who is going to do it? Few ESIAs get near this, and the claim of Flyvbjerg et al. (2003) that impact assessment does not integrate into overall project decision-making applies.
The above prioritization procedure does not work for Black Swans. Stock market crashes – with likelihood calculated from normal distributions fitted to daily stock market fluctuations – are so improbable as to be implausible (unlikely since the Big Bang), yet they happen (Taleb 2008). Interestingly there are some significant non-random variables that affect our lives and projects. Examples in addition to stock market price variations are weather and stream flow. A feature of Black Swans is not a single cause but contingency of events and circumstances. Airliner crashes often do not have one cause that usually the vigilant crew would easily remedy. Rather there is a combination of mechanical failure, lapse in surveillance and preparedness, bad weather conditions, and so on. Similarly with dams. Exceptional heavy rain is accompanied by, say, an uncommonly strong wind that blows water spilling over the dam into the galleries, pumps have not been maintained so the galleries flood, and the low level release valves cannot be operated. Now add reservoir induced seismicity (the dam was never so full) and the implausible event of dam failure suddenly looms large. The poor weather conditions might also disrupt communications so downstream people cannot be forewarned of impending disaster.
Work through the project threat register and flag threats with low likelihood but possible high consequence. These are potential Black Swans that though not above the threshold in Figure 3 warrant control.
There are several ways of controlling threats: avoid, mitigate, offset, transfer, insure, accept and prepare (Table 6). The troublesome ones are preparedness – discussed above in relation to external threats – and offset and transfer.
Offsetting involves compensating for one set of lost resources by investing in another, preferably of equivalent type, extent and value. It might work, as in case of a grant to an organization undertaking environmental protection or remediation (e.g. protected area management, reclamation of contaminated or eroded land, restoration of wetland, afforestation to sequester carbon). Offsetting greenhouse gas emissions by growing trees is widely publicized. At Eskom’s Ingula Pumped Storage Scheme in South Africa (Table 2, item 4) a condition of environmental authorization was to offset wetland and other resource loss by acquiring, rehabilitating and managing surrounding land, including a large wetland, as protected area. The prospects for effective offset are good because one large area of land is involved and the manager (Eskom) is on site. In a contrasting case of the Mooi-Mgeni Transfer Scheme (Table 1, item 2), offset of wetlands lost by inundation by the Spring Grove Dam is required by the environmental authorities. A dozen or so wetland areas on nearby private land are being viewed. Physical interventions – such as closing drains, erosion control structures and withdrawal of farming – are contemplated. This kind of offset is poorly feasible and has high opportunity cost. Landowners will have to be compensated. The many isolated pieces of land will require management and policing, otherwise land use and condition are liable to return to the pre-offset situation.
The notion of transferring threats might seem to the uninitiated like cheaply disposing of responsibilities. In many recent projects in South Africa the project documents have specified that the project proponent or principal is ultimately responsible for conforming to the conditions of project authorization. As watertight as this might seem, it is a loophole for contractors to evade expending resources on controlling their impacts. The low effectiveness of threat control (<50%) on many projects will continue until the quality of ESIAs improves as a basis for risk transfer. Tenderers for contracts must be informed of the scope of threat control that the principal will transfer explicitly in the contract. If the tenderer is not informed he does not estimate (generous estimates make bids uncompetitive), then when the contract is awarded impacts arise that were not budgeted for. A belated compensation event or variation order is then sought, or the impact is not addressed at all. Either way the project expense increases via more control or detrimental externality costs. With the threat transfer in place, the principal must have the contractor and subcontractors indemnify the principal against contractor and subcontractor negligence and misconduct (Brown et al. 2001). The fact that most projects today have several to many role players means that threat transfer is central to overall project threat control.
Related to transferring threats are organizational design and responsibilities. The typical organogram for southern African projects is shown in Figure 4. The design is fine but delegation of responsibility not. Often the government authorization requires the independent overseer to ensure the project complies with conditions of authorization and laws of the land. Technically the overseer can do no such thing because he has no executive authority and if he did he would lose his independence. Practically this is another loophole for the contractor: compliance is not his responsibility. Project resources are then wasted arguing about responsibilities while consequences of unavoided impacts aggravate. An analogous situation arises with the contractor and his threat control officer. Project documents specify that the threat control officer is responsible for threat control. This is based on the add-on design for threat control where there is an environmental protection department whose responsibility it is to clean up, fix and rehabilitate. This does not work because the rest of the organization then does not give a hoot for the mess left, and it can be unfixable. Rather, like health and safety, threat control must be designed into work procedures. This is the build-in model where the contractor manager is responsible for threat control, not his threat control officer who observes and advises his manager who in turn instructs his supervisors. One client requires each work group to do a mini threat assessment at the start of every working day. The activities, main threats and controls are summarily written down, and everyone signs. If something goes wrong the incident investigation has a head-start. Was the threat identified? Was the control appropriate? Was it applied? Should procedures be revised? Is disciplinary action warranted?
In the mindset of the authorities, project proponents and threat assessors, ESIAs are once-off project approval exercises, rather than tools to protect people, the environment and the project over its lifetime (Mentis 2010). Even if CBAs, ESIAs, ESMPs and related documents were perfect at the time of compilation, the circumstances change: the project moves on, knowledge and experience grow, prices and currencies alter, laws and norms get stricter, the competitive environment shifts, and technology advances. There are several measures that can be adopted to adjust and improve project execution and stay abreast of change.
First, there needs to be measurement against the project objective and deliverables. Because resources are finite and some things are hard to measure, not everything can be measured. A small set of key performance indicators is needed. To contain time and cost here it is imperative to have powerful simple metrics. Yet in project after project armies of scientists are deployed to ‘measure everything’. For example, fish are among the parameters in determining the condition of regulated rivers. Common monitoring practice involves deploying the whole fish sampling arsenal on every occasion – boats, electro-shockers, dip nets, fyko nets, seine nets, and hook-and-line. Is it possible to develop a reduced fish monitoring method with skeletal equipment that can be carried in a backpack so even the remote streams in mountainous Lesotho might be sampled to produce data with confidence limits?
Gigerenzer (2007, 2014) urges use of heuristics – rules of thumb – rather than complex and costly models. He questions ‘big data’, contending that more data can worsen rather than improve decision-making. Gigerenzer’s heuristics have potential to save time and cost. To be sure, use of pure gut feelings (System 1 of automated thinking and unthinking action (Kahneman 2012)) is unsuitable for projects funded by investors or taxpayers. Though Gigerenzer finds business executives frequently make decisions by gut feel, and some of these might be brilliant, no statistics are quoted on the success rate of gut feel decisions versus the more carefully rationalized decisions. Are gut feel decisions better than flip of the coin? In cases such as fire-fighting in houses or forests, fast intuitive decisions might be necessary for survival, but in other situations can they offer a cost-effective substitute to slow and complex methods? Is it possible to develop, using Kahneman’s System 2 of effortful mental activity, the ‘fast and frugal’ methods sought by Gigerenzer?
Kahneman and Klein (2009) consider that evaluating the likely quality of an intuitive judgment (heuristic) requires an assessment of the predictability of the environment in which the judgment is made and of the individual’s opportunity to learn the regularities of that environment. They distinguish between high-validity systems, where heuristics might develop, and zero-validity situations, such as financial markets and politics, where predictability is low and establishment of expertise constrained.
There is limited incentive to be ‘fast and frugal’. Big engineering and construction companies bid for big projects. Contractors get paid a percentage of any subcontracts. There is every reason to go big rather than small. Vested interests (Flyvbjerg et al. 2003) are alive and well.
Iterated plan-do-review-revise is a form of error elimination and a prime way of learning and improving the project. This should take place at least annually, and whenever there is major change in the project, e.g. shift from construction to operation phase. This type of updating rarely happens. The ideal is for the review to include the effectiveness of threat controls, monitoring data, audit and any emergency, accident, incident and near-miss reports, and for the revision to include fresh threat assessment. In many projects the impacts of the project on people and the environment parallel environmental and people effects on the project. For example, if sewer pipelines and water treatment works are not maintained then there are dangers of disease and eutrophication of the environment. At the same time, leaking sewers lower water quality and require subsequent extra purification treatment of the recycled water. If service delivery is poor, for example because of inadequate maintenance and then mechanical breakdown, people can vent their frustration by vandalizing project infrastructure. Understandably there is a reluctance to revisit contemporary ESIAs and ESMPs because of the labour in their preparation, and because their perceived utility is low. However, that should not detract from the benefits of iterated threat management applied throughout the project from prefeasibility to decommissioning.
In the short time frame, project managers use threats and issues log (Barker and Cole 2007). Issues are threats which have eventuated. The log is a concise list of the current threats and issues, and names and briefly describes each threat or issue, and identifies action, responsibility and time frame. The log is reviewed and revised weekly.
As part of, or in addition to, iterated plan-do-review-revise, attention must be given to emergencies, accidents, incidents, near misses and areas of weak performance against the objectives and deliverables. Possibly the most visible example is the airline industry. The fact of hundreds of tons of metal, plastic, liquid and people hurtling thousands of km through thin air at nearly the speed of sound, mostly without accident or even incident, is truly astonishing. The airline industry boasts the most economical and safest form of transport invented. How did it do it? By relentless pursuit of efficiency and safety. Safe procedures are followed on every flight. Every emergency, accident, incident and even near miss are investigated. No effort is spared to deliver the better safer flight. This is how we got from Kittyhawk to Boeing. In another context, former Anglo American CEO, Cynthia Carroll, could not accept ‘mining is inherently dangerous’ as the reason for more than 40 deaths a year on Anglo mines (Carroll 2012). In four years she reduced the fatality rate by 60%. In comparison, big infrastructure (dams, pipelines, power stations, railways, roads, transmission lines) is widely regarded as inherently messy, with big environmental and social impacts. But can the practices of the airline industry and Cynthia Carroll be applied? Can project participants review what they are doing – focus on the failings, shortcomings and weaknesses, and fix them? Can projects be transformed from Kittyhawks into Boeings? Reference class forecasting recommended by the Flyvbjerg-Kahneman school helps but is not sufficient to improve procedures at the work face.
Project performance might be improved further by attending to a few generics: purpose, organization and communication.
Much project activity is not focused on the objective and how the deliverables are achieved (see project plan above), and thereby squanders time and money. Many project plans and reports I review every year are prefaced with pages of background, and burdened generally with marginally relevant material. Often it is hard to find the purpose of the document, and the purpose of the task being considered. How much project resource is consumed compiling these documents? How much is consumed by users searching for the document purpose and trying to use the document? Are the documents indeed read and used? If purpose does not drive and circumscribe thought, word and action then it is easy to stray in Kahneman’s System 1 mode. Focus is System 2 thinking. Everything on a project should be purpose driven – what must we deliver, how, by when and with what budget? The adage applies: ‘Begin [proceed and finish] with the end in mind’ (Covey 1992).
Document organization, or rather the lack of it, is frequently another cost and time sink for projects. The typical project document has, for example, policy, principle, procedure and detail all on the same page. This can be challenging to understand and use, it makes review and revision difficult, and puts error elimination, and improvement from experience, out of easy reach. Policy concerns commitment. Its principles are the laws, customs, expectations and norms on which it is based. Procedure is about how the commitments are met. Detail includes the data. Each aspect has different properties and fulfils different functions. Reviewing and revising or updating any of the individual aspects separately is quite feasible, but mixed up on each of most pages of a document, the distinction between the aspects blurs, and the task of review and revision is so daunting that no one attempts it. Again, if this is how things are written, it probably reflects how project participants think and act – carried along in automatic System 1 mode, and not resorting to deep System 2 thinking that, though demanding to do, saves time and money in the end. An aid here is the precedent of the one-page project manager (Campbell 2007). The principle is to capture project status by the project manager displaying the key issues of objectives, tasks, timeline, cost and owners on one page. The project manager appoints owners of individual tasks each of which would have his own one page to capture the key issues of his responsibility. The task owner might have several assistants or subcontractors, each given ownership of subtasks for each of which there is another one page summary. Project information is thereby hierarchically arranged, and it is possible to zoom in or out depending on the level of information of interest, or the aspect of concern.
The one-page project manager has the dual benefit of helping the project manager and his staff to see the wood for the trees, and of communicating project status and performance to the principal and stakeholders. It is perhaps optimistic to dream that CBAs, ESIAs, ESMPs, audits and the like get condensed into one-page summaries. Currently these documents are poor at communicating. Often they do not have an executive summary, and the purpose of document and purpose of task being addressed are obfuscated by tiresome detail. If the reader is a senior manager she simply does not have time to distil the important issues. In consequence, she proceeds in ignorance of the insight that her staff have expensively gathered. Composing an executive summary has the benefit of clarifying in the mind of the report author what the key issues are to his manager and the project. He should be the best person to do this. He is also challenged to ensure the right message is communicated. That I am asked to review draft reports without executive summaries is indicative of the inefficiency of project management in the developing world.
Issues arising from this review are that (a) project slippage is due to uncertainties rather than risks, (b) while eventuation of some bad things is beyond managerial control, efficient management and execution are the prime means of staying ‘within budget, on time and fit-for-purpose’, (c) improving project delivery is less about ‘bigger and more complex’ and more about coordinated effectiveness, focus and developing thought-out heuristics, and (d) projects take longer and cost more partly because threat identification is inaccurate, the identified threats are too narrow, and the threat assessment product is not integrated into overall project decision-making and execution.
Table 1 suffices to dispel the view that project performance is mostly about risk management and that uncertainties are an aside. The main reasons for slippage are non-numerical:
the CBA is not done
the CBA is necessary but insufficient because of the uncertainties that lie outside it
reference class forecasting is not used
there is limited transparency
the stakeholders are not engaged and involved from project inception
threat identification in the ESIA is defective
threats are not classified according to type of control and not evaluated relative to the pervading socio-cultural context in a common currency enabling like with like comparisons
responsibility for threat control or transfer of threats is not explicit in contract documents
incentive is for project participants to go big rather than be fast and frugal
error elimination – the learning process that made Toyota and other organizations world leaders – is not used
project activity is not purpose driven
project documents are disorganized
communication is weak.
These observations conflict with project management currently but mistakenly fixated on risk. Guides devote pages to numerical economic evaluation with only mention that uncertainties exist, then mostly in relation to inexactness of the number inputs to the models (e.g. Sanral 2013). In the words of one project evaluation economist ‘we go only by the facts’ (i.e. the quantifiable costs and benefits). Publicity given to Black Swans (Taleb 2008) and resilience management (Linkov et al. 2014) might help to redress overreliance on certainty.
It is appropriate to question the management paradigm that if the project slips then management is deficient (Sage et al. 2014). The true effect of management can be difficult to distinguish from good or bad fortune (Taleb 2007). Nevertheless this does not dispose of the need to manage. As a consultant one walks in and out the front door of many an organization, and my experience is summed up by the sign at the entrance gate of the now defunct Duvha Colliery near Witbank, South Africa: ‘Through these gates passes the world’s finest team of coal miners’. Team members believed in themselves. Their performance was legendary. Why was that? They were ordinary people. The single most important cause was mine leadership. As one employee said to me years later ‘We were effective because of the Mine Manager. He did walkabouts. If he found anything broken or short it was fixed, by tomorrow.’ To be sure, strategy and external threats may eventuate irrespective of management, and the political and social environment matters on every project. These are givens. If management is viewed as a fixed suite of interventions then these givens could make or break a project. However, the contemporary manager is expected to be adaptable to circumstances (Fernández-Aráoz 2014) in which case it is up to management to succeed or fail with whatever the givens.
There is ongoing belief that good science and technology involve big data, of the ratio or interval sort, used in complex models to guide management infallibly, and anything less is stamp collecting. One is reminded of Laplace’s demon – given the means to collect and interpret the data this super-scientist could describe the present in such detail as to be able to reconstruct the past exactly, and predict the future perfectly. If this dream ever materializes it is so far in the future as to be irrelevant to current project management. As things are now, impact assessment and project authorization are overly complicated and getting more so. The cost of an ESIA has soared 20-fold in the past 15 years, with doubtful improvement in project benefits, and in conflict with the experience curve (Henderson’s Law) stating that cost falls by a constant percentage with every doubling of product volume. A present challenge is to deliver better value for money which bigger data and greater complexity are not yielding, but which a drive for heuristics might.
The main defects of threat assessment flagged by Flyvbjerg et al. (2003) a decade ago persist. Threat identification is so inaccurate that big threats can be missed altogether. The scope and time horizons of perceived threats may indeed be narrow. Control is often preoccupied with preventable threats and neglectful of strategy and external threats that are chief reasons why EGAP fails and projects slip. Inaccurate narrow ESIAs and ESMPs can hardly contribute to overall project decision-making and execution. The consequently belatedly or weakly controlled, or otherwise unavoided and unmitigated, environmental and social impacts are at financial cost to the project, and probably to future projects because they give threat control a bad name so no one wants to invest resources there.
In conclusion, the main obstacles to projects being within budget, on schedule and fit-for-purpose are not the quantifiable issues generally called risks, but uncertainties. Almost by definition it is what is poorly known that is problematic. Yet it is not just the unquantifiability and intangibility of uncertainties that cause project slippage, but that they are barely recognized and taken into account in project planning and execution. Improving project performance requires purpose-driven and managed deployment of scarce seasoned professionals. This can be aided with independent oversight of deeply experienced panellists who contribute technical insights and can potentially show that diligence is seen to be done.
Ansar A, Flyvbjerg B, Budzier A, Lunn D (2014) Should we build more large dams? The actual costs of hydropower megaproject development. Energy Policy (2014), http://dx.doi.org/10.1016/j.enpol.2013.10.069
Ariely D (2012) The (Honest) Truth About Dishonesty. Harper, London, p 314
Barker S, Cole R (2007) Brilliant Project Management. Pearson, Harlow, p 161
Binswanger HP (2007) Empowering rural people for their own development. Agr Econ 37:13–27
Blennow K, Persson J, Wallin A, Vareman N, Persson E (2013) Understanding risk in forest ecosystem services: implications for effective risk management, communication and planning. Forestry 2013: 0, 1–10, doi:10.1093/forestry/cpt032
Brown AR, Lane MR, Martin JH (2001) Triple Bottom Line Risk Management. John Wiley & Sons, New York, p 314
Campbell CC (2007) The One-page Project Manager. John Wiley & Sons, Hoboken, p 140
Carroll C (2012) Getting serious about safety. Harv Bus Rev June 2012: 43-46
Covey SR (1992) The Seven Habits of Highly Effective People. Simon and Schuster, London, p 358
Decker DJ, Riley SJ, Siemer WF (2012) Human Dimensions of Wildlife Management, 2nd edition. Johns Hopkins University Press, Baltimore, p 286
Evans D (2012) Risk Intelligence: How to Live with Uncertainty. Atlantic Books, London, p 276
Fernández-Aráoz C (2014) 21st century talent spotting. Har Bus Rev June 2014:60–73
Flyvbjerg B (2014) What you should know about megaprojects and why: an overview. Project Manage J 45(2):6–19
Flyvbjerg B, Bruzelius N, Rothengatter W (2003) Megaprojects and Risk. Cambridge University Press, Cambridge, p 207
Gigerenzer G (2007) Gut Feelings: Short Cuts to Better Decision Making. Penguin, London, p 280
Gigerenzer G (2014) Risk Savvy: How to Make Good Decisions. Penguin, New York, p 322
Goldratt EM (1986) The Goal: A process of Ongoing Improvement. North River Press, New York, p 273
Kahneman D (2012) Thinking, Fast and Slow. Penguin, London, p 499
Kahneman D, Klein G (2009) Conditions for intuitive expertise. Am Psychol 64(6):515–526
Kahneman D, Tversky A (1979) Intuitive prediction: biases and corrective procedures. TIMS Studies Manage Sci 12:313–327
Kaplan RS, Mikes A (2012) Managing risks: a new framework. Harvard Business Rev June 2012:48–60
Linkov I, Fox-Lent C, Keisler J, Sala SD, Sieweke J (2014) Perspective: risk and resilience lessons from Venice. Environ Syst Decisions 34:378–382. doi:10.1077/s10669-014-9511-8
McKeown G (2014) Essentialism: The Disciplined Pursuit of Less. Crown Publishers, New York, p 260
Mentis M (2010) Environmental Risk Management in South Africa. Mentis, Hillcrest, p 172
Mentis M (2014) Science writing in the real world. Forest Ecosystems 1:2
Parkinson CN (1958) Parkinson’s Law: The Pursuit of Progress. John Murray, London, p 128
Polak P, Warwick M (2013) The Business Solution to Poverty. Berrett-Koehler Publishers, San Francisco, p 245
Porter ME, Kramer MR (2006) Strategy and society: The link between competitive advantage and corporate social responsibility. Harvard Business Rev 2006:78–92
Sage D, Dainty A, Brookes N (2014) A critical argument in favor of theoretical pluralism: Project failure and the many and varied limitations of project management. Int J Project Manage 32(4):544–555
Sanral (2013) Drainage Manual. South African National Roads Agency Ltd, Pretoria, p 464 (www.sanral.co.za)
Simanis E, Duke D (2014) Profits at the bottom of the pyramid. Harv Bus Rev 2014:94–105
Taleb NN (2007) Fooled by Randomness: The Hidden Role of Chance in Life and the Markets. Penguin, London, p 316
Taleb NN (2008) The Black Swan: The Impact of the Highly Improbable. Penguin, London, p 366
van Marrewijk A, Clegg SR, Pitsis TS, Veenswijk M (2008) Managing public-private megaprojects: Paradoxes, complexity, and project design. Int J Project Manage 26:591–600
The author declares that he has no competing interests.
About this article
Cite this article
Mentis, M. Managing project risks and uncertainties. For. Ecosyst. 2, 2 (2015). https://doi.org/10.1186/s40663-014-0026-z