Using equilibrium policy gradients for spatiotemporal planning in forest ecosystem management

  • Crowley M
  • 9


    Mendeley users who have this article in their library.
  • 2


    Citations of this article.


Spatiotemporal planning involves making choices at multiple locations in space over some planning horizon to maximize utility and satisfy various constraints. In Forest Ecosystem Management, the problem is to choose actions for thousands of locations each year including harvesting, treating trees for fire or pests, or doing nothing. The utility models could place value on sale of lumber, ecosystem sustainability or employment levels and incorporate legal and logistical constraints on actions such as avoiding large contiguous areas of clearcutting. Simulators developed by forestry researchers provide detailed dynamics but are generally inaccesible black boxes. We model spatiotemporal planning as a factored Markov decision process and present a policy gradient planning algorithm to optimize a stochastic spatial policy using simulated dynamics. It is common in environmental and resource planning to have actions at different locations be spatially interelated; this makes representation and planning challenging. We define a global spatial policy in terms of interacting local policies defining distributions over actions at each location conditioned on actions at nearby locations. Markov chain Monte Carlo simulation is used to sample landscape policies and estimate their gradients. Evaluation is carried out on a forestry planning problem with 1,880 locations using a variety of value models and constraints. Index

Author-supplied keywords

  • Computational sustainability
  • Ecosystem management
  • Forestry planning
  • Machine learning
  • Markov decision processes
  • Optimization
  • Policy gradient planning
  • Reinforcement learning

Get free article suggestions today

Mendeley saves you time finding and organizing research

Sign up here
Already have an account ?Sign in

Find this document


  • Mark Crowley

Cite this document

Choose a citation style from the tabs below

Save time finding and organizing research with Mendeley

Sign up for free