Open Access

A decision framework for identifying models to estimate forest ecosystem services gains from restoration

  • Zachary L. Christin1,
  • Kenneth J. Bagstad2Email author and
  • Michael A. Verdone3
Forest Ecosystems20163:3

DOI: 10.1186/s40663-016-0062-y

Received: 26 October 2015

Accepted: 8 February 2016

Published: 10 February 2016

Abstract

Restoring degraded forests and agricultural lands has become a global conservation priority. A growing number of tools can quantify ecosystem service tradeoffs associated with forest restoration. This evolving “tools landscape” presents a dilemma: more tools are available, but selecting appropriate tools has become more challenging. We present a Restoration Ecosystem Service Tool Selector (RESTS) framework that describes key characteristics of 13 ecosystem service assessment tools. Analysts enter information about their decision context, services to be analyzed, and desired outputs. Tools are filtered and presented based on five evaluative criteria: scalability, cost, time requirements, handling of uncertainty, and applicability to benefit-cost analysis. RESTS uses a spreadsheet interface but a web-based interface is planned. Given the rapid evolution of ecosystem services science, RESTS provides an adaptable framework to guide forest restoration decision makers toward tools that can help quantify ecosystem services in support of restoration.

Keywords

Decision support Ecosystem services Forest restoration Modeling Valuation Comparative tools assessment

Introduction

Over 2 billion hectares of degraded and deforested land across the world have lost their ability to provide benefits to people and other species; restoring the productivity of this land has now become a global priority (International Union for the Conservation of Nature [IUCN] 2015a). Several United Nations (UN) conventions have adopted goals focused on restoring degraded land (UN 2014), and a prominent commitment to restoration was made at the New York Climate Summit in 2015. The Bonn Challenge, a global initiative to begin restoring 150 million hectares of degraded forest and agricultural land by 2020 (IUCN 2015b), was created to advance these efforts. To date 11 countries have signed on to the Bonn Challenge by committing to restore more than 59 million hectares of degraded land. Further, global programs like REDD+ and natural capital accounting increasingly need to monitor forest ecosystem services and their change over time (Caplow et al. 2011; Obst et al. in press).

Many of restoration’s benefits come in the form of ecosystem goods and services, so incorporating their assessment into restoration decision making is important. To date, however, their inclusion has been limited because of the perceived shortcomings in ecosystem service assessment and modeling tools (Benayas et al. 2009). Yet, as the movement to restore the world’s degraded land continues to grow so has the number and sophistication of ecosystem service assessment tools (Bagstad et al. 2013).

The number of such tools creates both an opportunity and a challenge for restoration decision-makers. On one hand, the ability to quantify and value the ecosystem service impacts of alternative restoration scenarios has never been greater and the information produced from these tools can be used to help decision makers compare alternative restoration strategies to meet an increasingly diverse set of restoration goals. On the other hand, there are now so many tools with such a wide range of requirements and capabilities that it can be difficult for analysts and decision makers to know which tool best suits their decision-making needs.

Forest landscape restoration and ecosystem services

Forest Landscape Restoration (FLR) is an umbrella term that refers to at least four techniques of restoring ecosystem services produced by forests (Rietbergen-McCracken et al. 2007). Natural regeneration can be used when degraded land is located near existing forest edges, creating a high chance of seed and species dispersal and colonization from the forest frontier into the degraded landscape (Chazdon 2008). Agroforestry, the inclusion of woody perennial species within farming systems, has been used in various agricultural and ecological settings to increase nutrient flows from forest to farm and improve the supply of timber available from agricultural landscapes. Improved farm fallows increase the length of fallows and plant woody species such as Sesbania, which improve soil fertility, increase crop yields, and enhance biodiversity on agricultural lands (Haggblade et al. 2004). Watershed restoration is commonly used to reduce fuel loads and restore historical fire regimes to watersheds that provide ecosystem services for downstream communities (Elliot et al. 2010).

Reversing landscape degradation offers society many tangible benefits in the form of ecosystem services. Restoration has been used to increase water supplies, terrestrial carbon stocks, and aesthetic and cultural values (Marin-Spiotta et al. 2007; Chazdon 2008; Dodds et al. 2008). Reforested lands can produce timber and non-timber forest products (e.g., mushrooms, berries, and game animals) and offer new livelihood opportunities for forest-dependent communities. For instance, forest thinning and restoration in California’s Sierra Nevada increased streamflow water yields by up to 6 % over a 10-year period (Podolak et al. 2015). In Costa Rica, restored forests have supported ecotourism, offering more opportunities for forest recreation and wildlife watching (IUCN 2015b). Between 1985 and 2004, over 300,000 ha of Acacia and Miombo forests were restored in Shinyanga, Tanzania, following a near collapse of the ecosystem as part of tsetse fly eradication and cash crop-based agricultural expansion (Barrow 2014).

The types of ecosystem services countries hope to restore through large-scale restoration can be seen in commitments to the Bonn Challenge (Table 1). Countries need to restore land to produce commodities, like food, fuel, or fiber that can improve local livelihoods and reduce poverty or be sold to finance restoration activities; however, they also need to restore landscapes to produce public goods like watershed protection, disaster risk reduction, and biodiversity conservation. These competing demands on landscapes point toward inevitable tradeoffs facing decision makers in designing restoration strategies across multiple spatial scales and forest types. Countries like Rwanda, Uganda, the United States, and Pakistan have started collaborative processes, using methodologies such as the Restoration Opportunity Assessment Methodology (ROAM), developed by the International Union for the Conservation of Nature and the World Resources Institute (IUCN and WRI 2014) to understand the tradeoffs of different restoration strategies, but many countries still have not.
Table 1

Characteristics of restoration commitments made to the Bonn Challengea

Country/Region

Restoration area commitment (ha)

Forest types

Desired ecosystem service benefits

Desired social/ financial benefits

Restoration activities

Gross Domestic Product (GDP)/capita (2014)

Brazil's Atlantic Forest Restoration Pact

1,000,000

Mangrove; Tropical/subtropical moist broadleaf forest

Carbon, biodiversity, timber, water, pollination, pest control

Improve human well-being and economic conditions

Total planting, enrichment, natural regeneration, agroforestry

$11,612

Colombia

1,000,000

Tropical/subtropical moist broadleaf forest

Watershed functions, erosion control, biodiversity, control invasive species

Poverty alleviation

Natural and assisted reforestation, avoided deforestation, agroforestry

$7720

Democratic Republic of the Congo

8,000,000

Tropical/subtropical moist broadleaf forest; Tropical/subtropical coniferous forest; Savannah and dry grasslands

Carbon, biodiversity, food, fuelwood

Improve human well-being and economic conditions

Natural regeneration, avoided deforestation, agroforestry

$475

Guatemala

1,200,000

Tropical/subtropical moist broadleaf forest; Tropical/subtropical coniferous forest

Carbon, tourism, biodiversity,

Poverty reduction, alternative livelihoods,

Agroforestry, others

$3703

Initiative 20 × 20 (Peru, Mexico, Ecuador, Chile)

20,000,000

Mangrove; Tropical/subtropical moist broadleaf forest; Tropical/subtropical coniferous forest

Carbon, biodiversity, water, food

Improve human well-being and economic conditions

Natural and assisted reforestation, avoided deforestation, agroforestry

$6291 - $14,520

Pakistan - Khyber Pakhtunkhwa Province

380,000

Temperate coniferous forest; Boreal/taiga forest

Carbon, water, fuelwood

Improve human well-being and economic conditions, especially for youth

Enrichment planting, afforestation, protective forests,

$1333

Uganda

2,500,000

Tropical/subtropical moist broadleaf forest; Savannah and dry grasslands

Carbon, food, fuel, fiber, water, biodiversity, erosion control

Poverty reduction, livelihoods, economic development

Agroforestry, protective forests, natural regeneration, silviculture

$677

United States

15,000,000

Temperate broadleaf and mixed forests; Temperate coniferous forest

Carbon, water, biodiversity, disaster risk reduction

Rural development, fire risk reduction

Fuel reduction, watershed restoration

$54,630

aSources: World Bank (2015) for GDP data; IUCN (2015c) for all other data

Biophysical, social, and financial factors make it unlikely that landscapes can be restored to simultaneously meet all the demands people place on them without concerted efforts to evaluate the ecosystem service tradeoffs of different restoration strategies and activities (Goldstein et al. 2012). Understanding such tradeoffs in forest restoration is also important because most ecosystem services are public goods. Since their producers cannot capture the benefits and recoup the costs of their production, this removes their incentive to produce many ecosystem services. Quantifying and valuing ecosystem services is a first step in designing policies that can send landowners a price signal, creating a means of capturing the value of ecosystem services when landscapes might otherwise be converted to other uses or used in ways that undermine the production of these services.

Clear and credible information is needed to determine where restoration is most needed, guide the selection of alternative scenarios, develop policies to incentivize restoration on private land, identify restoration options on public lands, and overcome institutional and policy bottlenecks. However, as both the number of ecosystem service assessment tools and the diversity of needed restoration decisions continue to grow it is difficult for decision makers and analysts to know which tools can best meet their informational needs.

The ecosystem service assessment tool landscape

The importance of ecosystem services modeling is widely recognized in the scientific and policy push to understand ecosystem services and use information about them in decision making (Burkhard 2012). While economic valuation methods for ecosystem services and biophysical models of natural processes have existed for decades, the rise of dedicated ecosystem services modeling tools is a more recent development. This notably followed the release of the Millennium Ecosystem Assessment in 2005 (MA 2005) and, shortly after, the launch of systematic and sustained ecosystem services modeling approaches like Integrated Valuation of Ecosystem Services and Tradeoffs (InVEST, Sharp et al. 2014) and Artificial Intelligence for Ecosystem Services (ARIES, Villa et al. 2014).

A 2013 review provided a snapshot description of the ecosystem services tool “landscape,” but periodic re-reviews of ecosystem services tools will be necessary, particularly while the field remains in a state of rapid evolution (Bagstad et al. 2013). That review also focused on the use of ecosystem service tools in decision making for a U.S. government agency, the Bureau of Land Management, charged with managing a substantial portion of the U.S. landscape (99 million ha of land and 283 million ha of subsurface mineral estate). Since this review, improved functionality and documentation with new versioning has occurred for several ecosystem services tools; others have been renamed or embraced new funding models (i.e., moving from free/open source to fee for use/proprietary). Some tools appear to no longer be supported or in use, while other new tools have emerged.

In support of forest restoration activities by IUCN and others, here we present a comparative analysis of ecosystem service assessment tools that can quantify ecosystem service tradeoffs. In the following sections, we present an overview of ecosystem service models relevant to restoration decision making and characterize the tools based on five key criteria. These criteria include their ability to quantify the benefits of restoration activities and scenarios in a timely and cost-effective manner across different geographic locations and scales, ecosystem service types, decision contexts, and under varying levels of uncertainty. We use that information to build a spreadsheet-based decision framework and a future online selector that we call the Restoration Ecosystem Service Tool Selector (RESTS). This framework will allow analysts to identify ecosystem service assessment tools that are best able to provide relevant information to guide the restoration decision-making process. We describe two hypothetical FLR decision contexts to illustrate how RESTS would guide an analyst toward particular ecosystem service assessment tools under different situations, and conclude by considering best practices and next steps for using ecosystem service assessments in restoration decision making.

RESTS decision framework

We designed the initial RESTS framework in a spreadsheet; IUCN plans to build it into a website that will allow an analyst to interactively navigate between different decision contexts and tools of potential use in assessing forest ecosystem services. This framework allows restoration analysts to input basic information into RESTS about decision-making needs for a proposed forest restoration project. Rather than prescribing one specific tool, the framework is designed to remove inappropriate tools, giving the user flexibility to further evaluate and choose appropriate tool(s) for their project. Based on a list of filter criteria, tools inappropriate to the user’s decision context are removed, leaving an output that describes suggested tools that may be appropriate to their needs (Fig. 1). Each tool has a list of attributes for evaluative criteria that align with these evaluation questions, plus further information including references and a Uniform Resource Locator (URL) that enables the user to further research appropriate tools. The framework is flexible, so new tools can be easily added to the spreadsheet and planned website, and existing tools can be updated when their characteristics and capacities change as the science of ecosystem services advances.
Fig. 1

Restoration Ecosystem Service Tool Selector (RESTS) decision framework. Acronyms: Artificial Intelligence for Ecosystem Services (ARIES), Ecosystem Valuation Toolkit (EVT), Integrated Valuation of Ecosystem Services and Tradeoffs (InVEST), Multiscale Integrated Models of Ecosystem Services (MIMES)

The RESTS framework starts from an initial list of ecosystem service assessment tools (Phase 1). The restoration analyst next provides inputs based on a series of questions that describe their decision context and information needs (Phase 2). Answers to these questions filter the initial list of tools. Finally, the filtered tool list is evaluated according to a series of five criteria for each tool, including its scalability, cost, time, handling of uncertainty, and applicability to benefit-cost analysis (Phase 3). At the end of the workflow, the resulting tools list will include a set of attributes with their corresponding “ratings.” These rated attributes will assist the analyst in choosing the appropriate tool for their project. The ratings were created on a scale of 1 to 3, with 1 representing a tool’s ability to address the criterion most strongly, and 3 representing a tool’s inability to address the criterion as strongly.

Phase 1: Initial list of assessment tools

Numerous “ecosystem-based management tools” exist (EBM Tools 2015), particularly related to forest hydrologic and ecological processes, mature fields of study based on many decades past research. However, most of these tools are ecological, hydrologic, or other biophysical process models that lack an explicit focus on ecosystem services. To limit the scope of this review to ecosystem service assessment tools, we used five criteria to select tools for inclusion in RESTS. First, we sought to include tools that enable restoration analysts to incorporate monetary or nonmonetary valuation (i.e., nonmonetary prioritization or analysis of cultural ecosystem services), or provide outputs that are easily monetizable. This typically requires that a tool connects ecosystem services to beneficiaries in some way, rather than simply quantifying ecological processes. For instance, a tool could provide direct monetary values or outputs in biophysical units that are amenable to valuation. Second, we considered the current level of development of the tool. Tools should be sufficiently developed to run reliably, use established models, produce replicable results, and have their methods, assumptions, strengths, and limitations well documented as part of a user manual and peer-reviewed journal articles, which may include validation exercises. Tools that are well developed and documented have greater transparency and credibility, which can improve trust with decision makers and the public. We thus excluded tools that lack full documentation and user support. Third, we excluded tools that can only be applied across limited geographic extents. IUCN’s global forest restoration efforts cover Africa, Latin America, Europe, Australia, Asia, and North America; if an assessment tool was not able to be applied to two or more of these regions we excluded it from RESTS. Because of this requirement, we also excluded non-English-language tools, though in our literature search we did not identify any non-English language tools intended for broader than regional use. Fourth, we included valuation databases with functionality for users to construct valuation maps and summaries (Troy and Wilson 2006; Earth Economics 2015), but excluded valuation databases that simply provide users with a searchable list of nonmarket valuation studies (van der Ploeg and de Groot 2010; EVRI 2015; MESP 2015). Finally, we excluded ad hoc approaches for biophysical modeling (Martinez-Harms and Balvanera 2012) and public participatory geographic information system (GIS) approaches for cultural ecosystem service mapping that were not connected to a specific assessment tool (Brown and Fagerholm 2015).

An initial screening for RESTS evaluated 24 tools that built on a past review of ecosystem service tools (Bagstad et al. 2013) and additional tools that have been recently developed. From this list, we eliminated tools that lacked an explicit connection to ecosystem service beneficiaries (Arnold and Fohrer 2005; Stolton and Dudley 2009; Ecotrust 2011; Patel et al. 2011; Eslinger et al. 2012; Vogl et al. 2015), that were at too early a stage of development to independently apply (RFF 2014; Willis et al. 2014), and/or that covered a limited geographic extent (Loomis et al. 2008; iTree 2014; RFF 2014; FEST 2015). The remaining 13 tools have potential to meet analysts’ needs for ecosystem service quantification associated with FLR (Table 2).
Table 2

List of assessment tools and descriptions

Abbreviation

Tool name

Developer

Tool description & reference

ARIES

Artificial Intelligence for Ecosystem Services

Basque Centre for Climate Change (BC3)

Framework to integrate multiple modeling paradigms in spatial modeling and mapping of ecosystem services. Supports artificial intelligence-based data and model selection through semantic modeling to quantify ecosystem service flows from ecosystems to beneficiaries (Villa et al. 2014, http://aries.integratedmodelling.org/).

Co$ting Nature

Co$ting Nature

King’s College London and AmbioTEK

Mapping and modeling tool for multiple ecosystem services using global datasets. Quantifies ecosystem services as opportunity costs (i.e., avoided cost of producing those services from a non-natural capital substitute) (Mulligan 2015, http://www.policysupport.org/costingnature).

EcoMetrix

EcoMetrix

EcoMetrix Solutions Group and Parametrix

Field-based tool designed for use at relatively fine spatial scales. Primary use is to illustrate the effects of human activities (i.e., development or restoration scenarios) on ecosystem services (Ecometrix Solutions Group 2013, http://www.ecometrixsolutions.com/ecometrix.html).

EnSym

Environmental Systems Modelling Platform

State of Victoria, Australia

Environmental systems modeling platform for researchers to apply process-based models. Designed to provide information on how and where to invest to maximize environmental outcomes (Ha et al. 2010, https://ensym.dse.vic.gov.au/cms/).

Envision

Envision

Oregon State University

GIS-based tool for scenario-based planning and environmental assessment. Enables “multi-agent modeling” to represent human decisions on landscape simulations (Guzy et al. 2008, http://envision.bioe.orst.edu/).

ESR for IA

Ecosystem Services Review for Impact Assessment

World Resources Institute

Method to address project impacts and dependencies on ecosystem services within the environmental and social impact assessment process. It identifies measures to mitigate project impacts on benefits provided by ecosystems and to manage operational dependency on ecosystems (Landsberg et al. 2011, http://www.wri.org/publication/ecosystem-services-review-impact-assessment).

EVT

Ecosystem Valuation Toolkit

Earth Economics

Provides monetary values for natural assets under multiple modules. Includes a Researcher’s Library, searchable database of ecosystem service values, and SERVES, a web-based tool for calculating ecosystem service values (Earth Economics2015, http://esvaluation.org/).

InVEST

Integrated Valuation of Ecosystem Services and Tradeoffs

Natural Capital Project

Spatial mapping and modeling of multiple ecosystem services. Includes a diverse set of provisioning, regulating, and cultural services from marine and terrestrial environments. The models primarily provide results in biophysical terms to which valuation can be applied (Sharp et al. 2014, http://www.naturalcapitalproject.org/).

LUCI

Land Utilisation and Capability Indicator

Victoria University of Wellington

Explores the capability of a landscape to provide a variety of ecosystem services. It compares the services provided by the current use of the landscape and its potential capability. The model uses this information to identify areas where change or maintenance of current conditions may be most beneficial (Jackson et al. 2013, http://www.lucitools.org/).

MIMES

Multiscale Integrated Models of Ecosystem Services

Afordable Futures

Modeling platform designed to quantify causal linkages between ecosystems and the economy. MIMES allows an individual to map decisions/policies, and the output illustrates how those choices affect the economy and ecosystems (Boumans et al. 2015, http://www.afordablefutures.com/orientation-to-what-we-do/services/mimes).

NAIS

Natural Assets Information System

Spatial Informatics Group

Integrated valuation database and reporting engine. The database is integrated with proprietary spatial modeling tools to characterize ecosystems and flow of services on the landscape (Troy and Wilson 2006, http://www.sig-gis.com/services/ecosystem-services/).

SolVES

Social Values for Ecosystem Services

U.S. Geological Survey (USGS)

Spatial mapping and modeling tool primarily for quantifying cultural ecosystem services using public participatory GIS (Sherrouse et al. 2011, http://solves.cr.usgs.gov/).

TESSA

Toolkit for Ecosystem Service Site-based Assessment

BirdLife International

A process using flow charts to describe how ecosystem services benefit society under current conditions and alternative scenarios (Peh et al. 2013, http://tessa.tools/).

Each tool above was developed to quantify and/or monetize ecosystem services, often over time and across landscapes. Maps, tabular summaries, and tradeoff analyses provide managers and decision makers with information to better consider nature’s benefits in resource planning decisions. The final tool list includes qualitative screening tools like the Ecosystem Services Review for Impact Assessment (ESR for IA) and Toolkit for Ecosystem Service Site-based Assessment (TESSA) that can be used for coarse scales of analysis. Tools like Co$ting Nature are useful for quantitative rapid assessments of restorations’ impact on ecosystem services, but lack the modeling complexity of other quantitative tools. Spatially explicit modeling tools like ARIES, Environmental Systems Modelling Platform (Ensym), InVEST, Land Utilisation and Capability Indicator (LUCI), and Multiscale Integrated Models of Ecosystem Services (MIMES) can quantify biophysical metrics for ecosystem services using varying modeling paradigms, complexity, and underlying assumptions. Valuation databases such as Ecosystem Valuation Toolkit (EVT) and Natural Assets Information System (NAIS) can be used to link biophysical with monetary values. Finally, specialized tools may be appropriate for a specific use—for example, EcoMetrix for site-scale quantitative analysis or Social Values for Ecosystem Services (SolVES) for cultural ecosystem service mapping. Depending on the decision context, using more than one tool throughout a more extensive process leading from broad-scale scoping and project prioritization to more specific site selection and monetary valuation may be appropriate (Fig. 2).
Fig. 2

Potential steps in ecosystem services assessment process for forest landscape restoration. Acronyms: Artificial Intelligence for Ecosystem Services (ARIES), Environmental Systems Modelling Platform (ENSYM), Ecosystem Services Review for Impact Assessment (ESR for IA), Ecosystem Valuation Toolkit (EVT), Integrated Valuation of Ecosystem Services and Tradeoffs (InVEST), Land Utilisation and Capability Indicator (LUCI), Multiscale Integrated Models of Ecosystem Services (MIMES), Natural Assets Information System (NAIS), Social Values for Ecosystem Services (SolVES), Toolkit for Ecosystem Service Site-based Assessment (TESSA)

Phase 2: Restoration analyst filter criteria inputs

Three questions are shown in the user input questions box in Fig. 1, and will appear as dropdown windows on the planned online RESTS user interface. For each of the tools listed in Table 2, we evaluated its suitability for different decision contexts, ecosystem services capable of being analyzed, and outputs produced by the tool (Additional file 1). We obtained information for each tool based on a past review (Bagstad et al. 2013), supplemented by a new review of each tool’s updated website and publications, and where necessary, conversations with each tool’s developers.

Question 1: What is the decision context of your project?

Ecosystem service assessments can range from broad scoping studies to fine-scale studies that require much greater precision. For instance, FLR might start with a national scale screening study that ranked several alternative restoration scenarios but did not require monetary valuation or high spatiotemporal resolution. Such a study could rely on expert opinion or coarser global data and models. Alternatively, when precisely designed on-the-ground restoration scenarios are being compared or when considering payments for ecosystem services (PES) program design, greater precision data and models might be needed to support accurate modeling and valuation. This “fit for purpose” is extremely important, because while novice users may quickly gravitate toward the most detailed and complex models, these may not always be necessary and their application to simpler decision contexts may waste time and resources (Schröter et al. 2015). RESTS thus asks users which of five categories their decision context fits: (1) a scoping/screening level study (i.e., where expert opinion, rankings of ecosystem service provision, or global data and models are adequate), (2) detailed tradeoff analysis (i.e., where detailed local data and models with stronger linkages to monetary valuation are needed), or (3-5) whether the project has specific needs that might drive the selection of a particular tool, in addition to the first two context categories (e.g., for site-scale assessment, monetary valuation, or cultural ecosystem service mapping).

Question 2: Which ecosystem services will be analyzed?

The restoration analyst next chooses among ecosystem service types classified using six provisioning, nine regulating, and four cultural services proposed by the Millennium Ecosystem Assessment (MA 2005). The MA has known limitations and newer classifications have been proposed (e.g., Common International Classification of Ecosystem Services (CICES, Ecosystem Service Partnership 2014) and Final Ecosystem Goods and Services (FEGS, Landers and Nahlik 2013)). However, these systems are more complex and not as widely known as the MA, so we chose to use the best known classification system for most analysts. If one of the tools from Table 2 cannot be applied to one or more of the user-selected ecosystem services, then that tool will not appear in the final list at the end of the workflow.

Question 3: Which outputs are desired?

Ecosystem services can be quantified in many ways—such as biophysical units, monetary values, or relative rankings in expert-based estimations or cultural ecosystem service assessments. For this question, the user selects whether desired outputs are monetary, biophysical, recreational, or cultural metrics, and whether spatial and/or temporal outputs are needed. If one of the tools from Table 2 cannot produce one or more of the user-selected desired tool outputs, then the tool will not appear in the final list at the end of the workflow.

Responses to each of the three questions are used as criteria to filter the Phase 1 tool list. This subset list of tools is the basis under which Phase 3—the evaluation of each tool based on five criteria categories—begins.

Phase 3: Tool evaluation criteria

Five criteria are used to “evaluate” each tool identified during RESTS’ Phase 2 for their applicability to the ROAM-guided project’s decision-making context. The tools are evaluated using five criteria that can assist analysts in selecting the most appropriate tools for each project. To minimize subjectivity in evaluating the tools, we ranked each on a 3-point scale from typically most (1) to least (3) suitable for each criterion. The criteria build on a past review (Bagstad et al. 2013) and have been further refined in conjunction with IUCN. They include (1) scalability, (2) cost requirements, (3) time requirements, (4) reporting uncertainty, and (5) applicability to benefit cost analysis (BCA). A matrix showing each tool’s evaluation against the five criteria is provided in Additional file 2. The intent of the criteria is to show an analyst how each tool performs relative to other tools. This relative comparison will allow the analyst to choose from among a pool of suitable tools to select the tool(s) that best fit(s) their needs.

Evaluation criterion 1: Scalability

Ecosystem service tools can support assessments at multiple spatial scales. One study derived a scale distribution, which we adopted, based on a review of 47 ecosystem service studies: village/farm scale: <60 km2; municipal scale: 60-8709 km2; provincial scale: 8709-83,000 km2; national scale: 83,000-1,220,000 km2; continental scale: >1,220,000 km2 (Malinga et al. 2015). They found that a majority of ecosystem service studies were conducted across intermediate extents (i.e., municipal and provincial level). We evaluated each tool based on its ability to provide accurate results at each scale identified above. Modeling tools with flexible input data resolution and analysis scale were ranked as 1. Tools limited to site-scale assessment were ranked as 3; remaining tools were ranked as 2.

Evaluation criterion 2: Cost requirements

The cost of using a tool takes several forms. First, a tool itself may be proprietary, requiring paid access or a subscription to use. There may be costs to acquire data or to support consultants to run the tool or tailor it to local contexts. Additionally, while some tools may be free to use, added training may be required to sufficiently implement the tool. Finally, tools that use very intensive data and models may require greater computing and data storage resources. Tools that can be freely obtained and applied independently without the need for proprietary software were ranked as 1. Tools that always require the use of proprietary software, subscriptions, or consulting services were ranked as 3.

Evaluation criterion 3: Time requirements

As the time required to apply a tool decreases, it becomes increasingly practical for widespread use in time-sensitive decision-making processes. We evaluated each tool based on a rough estimate of its time requirements to provide results for a “standard” restoration analyst (i.e., a Masters-level specialist trained in ecosystem service analysis, including GIS and/or economic valuation). Additionally, evaluations reflect time requirements to develop customized input data, to run, test, and calibrate models, and to apply scenario analysis. Rapid assessment tools were ranked as 1; time-intensive tools were ranked as 3.

Evaluation criterion 4: Uncertainty

Reporting uncertainty and providing mechanisms to understand and to later reduce large error margins will strengthen restoration projects. Reporting a single value can inspire false confidence in the certainty of results, so uncertainty estimates are a valuable addition to the set of tool outputs. Tools with built-in methods for assessing and displaying uncertainty were ranked as 1. Tools capable of generating uncertainty estimates with user-supplied variation in inputs were ranked as 2; remaining tools were ranked as 3.

Evaluation criterion 5: Applicability to benefit-cost analysis

A primary interest of ROAM is to provide ecosystem service outputs amenable to BCA. Some tools provide direct monetary outputs. Others rely on external valuation data to derive monetary outputs that can be associated with, for example, biophysical units or relative rankings, while some have no ability to provide monetary values. A higher evaluation is associated with a tool’s ability to provide monetary outputs without the need to collect valuation data external to the tool’s application. Tools that directly provide monetary values were ranked as 1. Tools that provide biophysical outputs that can be easily paired with valuation data were ranked as 2; remaining tools were ranked as 3.

Applying RESTS to hypothetical forest planning scenarios

Here we apply RESTS to two hypothetical FLR scenarios, to illustrate how it can guide an analyst to different ecosystem service assessment tools for different assessment contexts. In the first scenario, we apply RESTS to a screening decision-making context for a national environmental authority that wants to prioritize country-level restoration investment. In the second scenario, we apply RESTS to a precision decision-making context, where a water utility is creating a PES program to incentivize private landowners to restore forests to improve water yield and quality. This requires the utility to quantify and value the services provided by individual landowners participating in the PES market. We have not yet performed “road tests” of the RESTS tool with FLR users outside IUCN, but this is a next step in the development of the web tool.

Scenario 1

In 2014, the country of Uganda committed to begin restoring the economic and ecological productivity of 2.5 million hectares of degraded land by 2020 under the Bonn Challenge. This effort is part of a broader development strategy designed to recover important ecosystem goods and services, such as water purification, food and timber production, and erosion control that degraded lands no longer provide (IUCN 2015d). The restoration analyst provides the following answers to the Phase 2 filter criteria inputs.

Question 1: What is the decision context of your project?

The country must spatially prioritize restoration investments at a national level to maximize the benefits from limited restoration budgets.

Question 2: Which ecosystem services will be analyzed?

The country wants to identify modeling tools that can be applied at the national level to assess benefits for agricultural and timber yields, to reduce erosion, and increase water purification.

Question 3: Which outputs are desired?

At this early stage of the restoration planning process, decision makers are most interested in the biophysical impacts of restoration activities; later studies will focus on economic valuation of these benefits.

Once the screening criteria from Phase 2 are applied to the list of ecosystem service modeling tools in Table 2, the following table of appropriate tools was identified (Table 3). The spreadsheet and planned RESTS interface also provide a more detailed description of what the numeric rankings mean and web links to direct the analyst toward each tool’s website. Based on this list, the analyst can conduct further research about each tool to identify the best tool(s) for use in their project.
Table 3

Appropriate tools for hypothetical national scale screening context in Uganda

Tool

Scalability

Cost requirements

Time requirements

Uncertainty

Applicability to BCA

ARIES

1

1

3

1

2

Co$ting Nature

2

1

1

3

2

EnSym

1

1

3

2

2

Envision

1

2

3

2

2

InVEST

1

1

2

2

2

LUCI

1

3

3

3

2

MIMES

1

3

3

2

2

TESSA

3

1

1

3

3

Acronyms: ARIES Artificial Intelligence for Ecosystem Services, ENSYM Environmental Systems Modelling Platform, InVEST Integrated Valuation of Ecosystem Services and Tradeoffs, LUCI Land Utilisation and Capability Indicator, MIMES Multiscale Integrated Models of Ecosystem Services, TESSA Toolkit for Ecosystem Service Site-based Assessment

Scenario 2

In the United States, 68 operational watershed PES programs in 28 states enroll over 260,151 ha of land to improve water quantity and/or quality (Bennett et al. 2013). Using PES schemes to efficiently provide watershed services is predicated on the ability of water utilities to value and pay for a given level of service (Pagiola and Platais 2007). Ecological production functions coded within ecosystem service models can quantify these benefits (Benayas et al. 2009). This information can be used with cost and budget information to design an efficient PES program. A restoration analyst provides the following answers to the Phase 2 filter criteria inputs.

Question 1: What is the decision context of your project?

The utility needs to identify a set of modeling tools that can assess the expected water yield and water quality benefits of different site-level restoration activities.

Question 2: Which ecosystem services will be analyzed?

The utility is interested in tools that can model surface water yield and water quality.

Question 3: Which outputs are desired?

The utility is interested in tools that can also produce economic outputs that can be used to help design PES price structures.

Once the screening criteria from Phase 2 were applied to the list of ecosystem service modeling tools in Table 2, the following table of appropriate tools was identified (Table 4).
Table 4

Appropriate tools for hypothetical precision decision-making context in PES scheme for watershed services

Tool

Scalability

Cost requirements

Time requirements

Uncertainty

Applicability to BCA

ARIES

1

1

3

1

2

EnSym

1

1

3

2

2

Envision

1

2

3

2

2

EVT

2

3

1

3

1

InVEST

1

1

2

2

2

LUCI

1

3

3

3

2

MIMES

1

3

3

2

2

NAIS

2

3

1

3

1

Acronyms: ARIES Artificial Intelligence for Ecosystem Services, ENSYM Environmental Systems Modelling Platform, EVT Ecosystem Valuation Toolkit, InVEST Integrated Valuation of Ecosystem Services and Tradeoffs, LUCI Land Utilisation and Capability Indicator, MIMES Multiscale Integrated Models of Ecosystem Services, NAIS Natural Assets Information System

Conclusions

As shown in the hypothetical scenarios above, ecosystem service assessment tools and decision-making contexts differ in their complexity, meaning that it is seldom appropriate to recommend a single tool for every job. Although novice modelers may be tempted to use more complex models, these are not always appropriate or necessary in all decision contexts, nor do they always provide added information that justifies their often greater level of complexity, time, and money to apply (Tallis and Polasky 2011). Simpler, screening-level models could be paired with more technical models as a decision context increases in complexity (Fig. 2). As shown in our example above, analysts might start with a coarser national-scale analysis to prioritize restoration at the regional scale, then move to a precise, quantitative tradeoff analysis of restoration alternatives at a finer spatial scale.

Results obtained from each of the 13 tools we analyzed can be credible when they are used in a scientifically credible fashion, as outlined by the developers of each tool. General best practices to improve credibility include model validation, clear documentation of the assumptions and data sources used, and participatory modeling that involves experts and other stakeholders throughout the project. Ideally, such participatory approaches will, at a minimum, involve stakeholders in scoping, ecosystem service, data, and model selection during a project’s kickoff; a review of initial results at a project’s midpoint; and review of results and discussion prior to their dissemination at the conclusion of a project.

Despite rapid advances made in the science of ecosystem services and the sophistication of tools, many tools remain time and resource intensive to run (Bagstad et al. 2013). Key scientific and data gaps remain; however, these challenges exist alongside strong policy needs to standardize ecosystem service information for decision making (Polasky et al. 2015). Further, the learning curve to run some tools can be high. This may present more difficulty in some contexts than others. For example, in a middle-income country with good non-governmental organization and university research support, capacity to apply ecosystem service assessment tools may be relatively strong. In a developing country without these resources, the same learning curve may present a substantial barrier to applying more technically complex ecosystem service assessment tools.

The goal of this project was to characterize the types of restoration decisions that decision makers may face based on the experiences of countries who have committed to restore degraded land to forests. The ecosystem service assessment tools included in the RESTS framework provide the ability to quantify the benefits of restoration activities and scenarios with differing levels of detail and costs across different geographic locations and scales, ecosystem service types, decision contexts, and under varying levels of uncertainty. This allows analysts and other decision makers to identify the best ecosystem service assessment tools that can inform the restoration decision-making process. Further, the flexible nature of RESTS, which is developed as spreadsheets and a planned web interface, will allow new ecosystem service tools to be added and the characteristics of existing tools to be updated as their capabilities change over time. This provides a flexible framework for tracking ecosystem service tools’ capacity to support restoration decision making as the science of ecosystem services continues to advance.

Declarations

Acknowledgments

We thank Jared Soares and Paula Wood of Earth Economics for research assistance and figure design, respectively. Chetan Kumar of IUCN provided project oversight. Funding for this work was provided by UK aid.

Disclaimer

Any use of trade, product, or firm names is for descriptive purposes only and does not imply endorsement by the U.S. Government.

Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Authors’ Affiliations

(1)
Earth Economics
(2)
U.S. Geological Survey, Geosciences & Environmental Change Science Center
(3)
International Union for the Conservation of Nature

References

  1. Arnold JG, Fohrer N (2005) SWAT2000: current capabilities and research opportunities in applied watershed modeling. Hydrol Process 19(3):563–572View ArticleGoogle Scholar
  2. Bagstad KJ, Semmens DJ, Waage S, Winthrop R (2013) A comparative assessment of tools for ecosystem services quantification and valuation. Ecosyst Serv 5:27–39View ArticleGoogle Scholar
  3. Barrow E (2014) 300,000 hectares restored in Shinyanga, Tanzania — but what did it really take to achieve this restoration? SAPIENS 7(2)
  4. Benayas JMR, Newton AC, Diaz A, Bullock JM (2009) Enhancement of biodiversity and ecosystem services by ecological restoration: a meta-analysis. Science 325:1121–1124View ArticleGoogle Scholar
  5. Bennett G, Carroll N, Hamilton K (2013) Charting New Waters: State of Watershed Payments 2012. Forest Trends, Washington, DC, Available at http://www.ecosystemmarketplace.com/reports/sowp2012, accessed on 15 April 2015Google Scholar
  6. Boumans R, Roman J, Altman I, Kaufman L (2015) The Multiscale Integrated Model of Ecosystem Services (MIMES): simulating the interactions of coupled human and natural systems. Ecosyst Serv 12:30–41View ArticleGoogle Scholar
  7. Brown G, Fagerholm N (2015) Empirical PPGIS/PGIS mapping of ecosystem services: a review and evaluation. Ecosyst Serv 13:119–133View ArticleGoogle Scholar
  8. Burkhard B (2012) Solutions for sustaining natural capital and ecosystem services. Ecol Indic 21:1–6View ArticleGoogle Scholar
  9. Caplow S, Jagger P, Lawlor K, Sills E (2011) Evaluating land use and livelihood impacts of early forest carbon projects: lessons for learning about REDD+. Environ Sci Policy 14:152–167View ArticleGoogle Scholar
  10. Chazdon RL (2008) Beyond deforestation: restoring forests and ecosystem services on degraded lands. Science 320:1458–1460View ArticlePubMedGoogle Scholar
  11. Dodds WK, Wilson KC, Rehmeier RL, Knight GL, Wiggam S, Falke JA, Dalgleish HJ, Bertrand KN (2008) Comparing ecosystem goods and services provided by restored and native lands. Bioscience 58(9):837–845View ArticleGoogle Scholar
  12. Earth Economics (2015) Ecosystem Valuation Toolkit. Available at http://esvaluation.org/, accessed 15 April 2015
  13. Ecometrix Solutions Group (2013) EcoMetrix Method Development Overview. Available at http://www.ecometrixsolutions.com/assets/ecometrix_method_oct2013.pdf, accessed 15 April 2015
  14. Ecosystem-Based Management (EBM) Tools Database (2015) EBM Tools Database. Available at http://ebmtoolsdatabase.org/, accessed 23 October 2015.
  15. Ecosystem Service Partnership (ESP) (2014) Public consultation on CICES V4 classification of ecosystem services. Available at http://www.fsd.nl/esp/80003/9/0/50, accessed 15 April 2015
  16. Ecotrust (2011) Madrona decision support framework documentation. Available at http://ecotrust.github.io/madrona/docs/, accessed 15 April 2015
  17. Elliot WJ, Miller SI, Audin L (2010) Cumulative watershed effects of fuel management in the western United States. U.S. Department of Agriculture, Forest Service, Rocky Mountain Research Station, Fort CollinsGoogle Scholar
  18. Environmental Valuation Reference Inventory (EVRI) (2015) EVRI. Available at https://www.evri.ca/, Accessed 19 August 2015
  19. Eslinger DL, Carter HJ, Pendleton M, Burkhalter S, Allen M (2012) OpenNSPECT: The Open-source Nonpoint Source Pollution and Erosion Comparison Tool. NOAA Office for Coastal Management. Charleston, SCGoogle Scholar
  20. Forest Ecosystem Services Toolkit (FEST) (2015) Tools. Available at http://www.forestecoservices.net/tools.php, accessed 15 April 2015
  21. Goldstein JH, Caldarone G, Duarte TK, Ennaanay D, Hannahs N, Mendoza G, Polasky S, Wolny S, Daily GC (2012) Integrating ecosystem-service tradeoffs into land-use decisions. Proc Natl Acad Sci U S A 109(19):7565–7570PubMed CentralView ArticlePubMedGoogle Scholar
  22. Guzy MR, Smith CI, Bolte JP, Hulse DW, Gregory SV (2008) Policy research using agent based modeling to assess future impacts of urban expansion into farmlands and forests. Ecol Soc 13(1):37Google Scholar
  23. Ha J, Eigenraam M, Forbes G, Lewis W, Chua J (2010) The Environmental Systems Modelling Platform (EnSym) to Assess Effects of Land Use Change on Groundwater Recharge. Proceedings of the 2010 International Environmental Modelling and Software Society
  24. Haggblade S, Tembo G, Donovan C (2004) Household level financial incentives to adoption of conservation agricultural technologies in Africa. Food Security Research Project, Food Security Research Project, Lusaka, ZambiaGoogle Scholar
  25. International Union for the Conservation of Nature (IUCN) (2015a) Forest Landscape Restoration. Available at https://www.iucn.org/about/work/programmes/forest/fp_our_work/fp_our_work_thematic/fp_our_work_flr/, accessed 22 August 2015
  26. International Union for the Conservation of Nature (IUCN) (2015b) Bonn Challenge: What Are the Benefits of Restoration? Available at http://www.bonnchallenge.org/content/restoration-benefits, accessed 15 April 2015
  27. International Union for the Conservation of Nature (IUCN) (2015c) Bonn Challenge: Commitments. Available at http://www.bonnchallenge.org/commitments, accessed 15 April 2015
  28. International Union for the Conservation of Nature (IUCN) (2015d) Bonn Challenge: Commitments: Uganda Available at http://www.bonnchallenge.org/content/Uganda, accessed 15 April 2015
  29. International Union for the Conservation of Nature (IUCN), World Resources Institute (WRI) (2014) Assessing forest landscape restoration opportunities: a handbook. Identifying, analysing and mapping restoration opportunities at a national or sub-national level. IUCN, GlandGoogle Scholar
  30. i-Tree (2014) i-Tree User's Manual v5. Technical Report. Available at http://www.itreetools.org, accessed 15 April 2015
  31. Jackson B, Pagella T, Sinclair F, Orellana B, Henshaw A, McIntyre N, Reynolds B, Wheater H, Eycott A (2013) Polyscape: a GIS mapping toolbox providing efficient and spatially explicit landscape-scale valuation of multiple ecosystem services. Landsc Urban Plan 112:74–88View ArticleGoogle Scholar
  32. Landers DH, Nahlik AM (2013) Final ecosystem goods and services classification system (FEGS-CS). EPA/600/R-13/ORD-004914. U.S. Environmental Protection Agency, Office of Research and Development, Washington, DCGoogle Scholar
  33. Landsberg F, Ozment S, Stickler M, Henninger N, Treweek J, Wenn O, Mock G (2011) Ecosystem Services Review for Impact Assessment: Introduction and Guide Scoping., World Resources Institute, Available at http://www.wri.org/sites/default/files/ecosystem_services_review_for_impact_assessment_introduction_and_guide_to_scoping.pdf, accessed 15 April 2015Google Scholar
  34. Loomis J, Kroeger T, Richardson L, Casey F (2008) A benefit transfer toolkit for fish, wildlife, wetlands, and open space. West Econ Forum 7:33–43Google Scholar
  35. Malinga A, Gordon L, Jewitt G, Lindborg R (2015) Mapping ecosystem services across scales and -continents – a review. Ecosyst Serv 13:57–63View ArticleGoogle Scholar
  36. Marine Ecosystem Service Partnership (MESP) (2015) Ecosystem Service Library. Available at http://mesp2.env.duke.edu/explore, accessed 15 April 2015
  37. Marin-Spiotta E, Ostertag R, Silver W (2007) Long-term patterns in tropical reforestation: plant community composition and aboveground biomass accumulation. Ecol Appl 17(3):828–839View ArticlePubMedGoogle Scholar
  38. Millennium Ecosystem Assessment (2005) Ecosystems and Human Well-being: Synthesis. Island Press, Washington, DCGoogle Scholar
  39. Mulligan M (2015) Trading off agriculture with nature’s other benefits, spatially. In: Zolin CA, Rodrigues RAR (eds) Impact of climate change on water resources in agriculture. CRC Press, Boca Raton, pp 184–204View ArticleGoogle Scholar
  40. Obst C, Hein L, Edens B (in press) National accounting and the valuation of ecosystem assets and their services. Forthcoming in: Environ Resour Econ
  41. Patel H, Gopal S, Kaufman L, Carleton M, Holden C, Pasquarell V, Ribera M, Shank B (2011) MIDAS a spatial decision support system for monitoring marine management areas. Int Reg Sci Rev 34(2):191–214View ArticleGoogle Scholar
  42. Peh KS, Balmford AP, Bradbury RB, Brown C, Butchart SHM, Hughes FMR, Stattersfield AJ, Thomas DHL, Walpole M, Bayliss J, Gowing D, Jones JPG, Lewis SL, Mulligan M, Pandeya B, Stratford C, Thompson JR, Turner K, Vira B, Willcock S, Birch JC (2013) TESSA: a toolkit for rapid assessment of ecosystem services at sites of biodiversity conservation importance. Ecosyst Serv 5:51–57View ArticleGoogle Scholar
  43. Podolak K, Edelson D, Kruse S, Aylward B, Zimring M, Wobbrock N (2015) Estimating the water supply benefits from forest restoration in the Northern Sierra Nevada. The Nature Conservancy, San Francisco, CAGoogle Scholar
  44. Polasky S, Tallis H, Reyers B (2015) Setting the bar: standards for ecosystem services. Proc Natl Acad Sci U S A 112(24):7356–7361PubMed CentralView ArticlePubMedGoogle Scholar
  45. Resource for the Future (RFF) (2014) Forest Conservation Targeting Tool. Centers for the Management of Ecological Wealth. Available at http://www.rff.org/centers/management_of_ecological_wealth/Pages/Forest-Conservation-Targeting-Tool.aspx, accessed 22 April 2015
  46. Rietbergen-McCracken J, Maginnis S, Sarre A (2007) The Forest Landscape Restoration Handbook. Earthscan, LondonGoogle Scholar
  47. Schröter M, Remme RP, Sumarga E, Barton DN, Hein L (2015) Lessons learned for spatial modeling of ecosystem services in support of ecosystem accounting. Ecosyst Serv 13:64–69View ArticleGoogle Scholar
  48. Sharp R, Tallis HT, Ricketts T, Guerry AD, Wood SA, Chaplin-Kramer R, Nelson E, Ennaanay D, Wolny S, Olwero N, Vigerstol K, Pennington D, Mendoza G, Aukema J, Foster J, Forrest J, Cameron D, Arkema K, Lonsdorf E, Kennedy C, Verutes G, Kim CK, Guannel G, Papenfus M, Toft J, Marsik M, Bernhardt J, Griffin R, Glowinski K, Chaumont N, Perelman A, Lacayo M, Mandle L, Hamel P, Vogl AL (2014) InVEST User’s Guide. The Natural Capital Project, Stanford, CAGoogle Scholar
  49. Sherrouse B, Clement JM, Semmens DJ (2011) A GIS application for assessing, mapping, and quantifying the social values of ecosystem services. Appl Geogr 31:748–760View ArticleGoogle Scholar
  50. Stolton S, Dudley N (2009) The Protected Areas Benefits Assessment Tool: A Methodology., Food and Agriculture Organization of the United Nations. World Wide Fund for Nature, Available at http://wwf.panda.org/?174401/PABATru, accessed 15 April 2015Google Scholar
  51. Tallis H, Polasky S (2011) How much information do managers need? The sensitivity of ecosystem service decisions to model complexity. In: Kareiva P, Tallis H, Ricketts TH, Daily GC, Polasky S (eds) Natural capital: Theory and practice of mapping ecosystem services, ed. Oxford University Press, Oxford, pp 264–277View ArticleGoogle Scholar
  52. Troy A, Wilson M (2006) Mapping ecosystem services: practical challenges and opportunities in linking GIS and value transfer. Ecol Econ 4:435–449View ArticleGoogle Scholar
  53. United Nations (UN) (2014) Forest action statements and action plans. Available at http://www.rff.org/centers/management_of_ecological_wealth/Pages/Forest-Conservation-Targeting-Tool.aspx, accessed 15 April 2015
  54. Van der Ploeg S, de Groot RS (2010) The TEEB Valuation Database – a searchable database of 1310 estimates of monetary values of ecosystem services. Foundation for Sustainable Development, Wageningen, NetherlandsGoogle Scholar
  55. Villa F, Bagstad KJ, Voigt B, Johnson GW, Portela R, Honzak M, Batker D (2014) A methodology for adaptable and robust ecosystem services assessment. PLoS ONE 9(3):1–18View ArticleGoogle Scholar
  56. Vogl A, Tallis H, Douglass J, Sharp R, Wolny S, Veiga F, Benitez S, Leon J, Game E, Petry P, Guimeraes J, Lozano JS (2015) Resource Investment Optimization System: Introduction & theoretical documentation. Natural Capital Project, Stanford, CAGoogle Scholar
  57. Willis K, Vandvik V, Nogué S (2014) Development of a new automated tool for ecosystem service evaluation (EcoSET)., Oxford Institute on Biodiversity, Available at http://www.vista.no/project/vis.html?tid=53069, accessed 15 April 2015Google Scholar
  58. World Bank (2015) World Bank Open Data. Available at http://data.worldbank.org/indicator/NY.GDP.PCAP.CD, Accessed 15 April 2015

Copyright

© Christin et al. 2016