Preparing for the BES Mid-Term Review: What Does NSF Want?

Here are the current expectations as stated by the National Science Science Foundation that will be used in evaluating BES.  The Mid-Term Review will take place in October, and this Charge to the Review Team is provided to help the BES community prepare for the review:

NSF Review Criteria

The visiting review team is asked to do the following things.
The central intellectual aim of the LTER program is to understand long-term ecological patterns and processes at multiple spatial scales. Each site is organized around a unique scientific theme as articulated in a reviewed proposal, but all sites must address the same five core areas: 1) patterns and controls of primary production, 2) spatial and temporal distributions of populations selected to represent trophic interactions or food web dynamics, 3) patterns and controls of organic matter accumulation and decomposition in surface layers and sediments, 4) patterns of inorganic inputs and movements of nutrients through soils, groundwater and surface waters, and 5) patterns and frequency of disturbances.
The 25 active LTER sites were established at different times, ranging from 8 to 32 years ago. The questions and research directions proposed in a site’s most recent proposal were therefore placed within the context of long-term research and motivated by data sets of varying length. This proposal, its conceptual basis, and the research proposed form the context for the mid-term site review. During the review, the site PI may choose to discuss long-term studies so that current research can be placed in a temporal perspective.
Mid-term site reviews are an essential part of the ongoing evaluation cycle of the LTER program and of each LTER site. Sites are funded for six years at a time. A mid-term site review is conducted in the third year of a six-year award to evaluate the strengths and weaknesses of a site’s performance over the past three years and its plans for completing its research goals over the coming three years. The review advises NSF as to whether or not the site is
  •  fulfilling its proposed research goals in a timely manner
  •  collecting and managing core data sets
  • carrying out research at the leading edge of ecology that will advance the field
  •  managed effectively
  •  maintaining a unique research focus
  •  making data and metadata readily accessible
  • integrating research with education, training and outreach.

The evaluation should be based on the research proposed in the site’s most recent renewal award, the extent to which the PI has addressed concerns raised by review panel in 2009, and progress in accomplishing the site’s proposed goals.  No funding decisions are attached to the mid-term review.
Your report is an evaluation written to NSF. We ask that you document the strengths and weaknesses of the site’s activities and provide your value judgments about the extent to which it has achieved its stated goals.

Suggested sections of the report are:

I. Executive Summary: major findings viz. strengths and weaknesses. Is the site on track?  Are there any major problems or areas for concern?  Is the science cutting-edge, focused on exciting questions that require long-term data or that can only be posed after a site has been studied for a long time?  Recommendations should specify problems and weaknesses and identify solutions if possible.
II. Body of report – room to elaborate, justification for strengths and weaknesses. Suggested categories for evaluation include:
  •  core data collection
  • compelling, site-specific long-term ecological and related research
  • cross-site or broader scale synthetic research
  •  outreach, education, and training
  •  information management
  •  project  management, including institutional relations, personnel mgt, decision making, diversity, leadership and transitions

III. Please keep in mind that it is not the role of the site review team to suggest major changes in focus or direction that would deviate from the funded proposal.  Be careful when suggesting alternate research questions or areas for future studies.  No site can do everything with the limited funds NSF provides, so you need to be fair in your expectations. The report may include some suggestions for changes, but these should be limited and well justified.
IV. Other considerations may include PI transition plans, major new directions, investments in major experiments, and what needs to be changed or sacrificed to accommodate these.
LTER site visits are valuable opportunities for site researchers to discuss ideas with the review team. It is important for you to interact with all LTER participants – PIs, collaborators, postdoctoral researchers, students, techs, staff, etc. Please ask a lot of questions.
NSF will send the report to the PI along with a cover letter that highlights aspects of the report and your evaluations from an agency perspective.  PIs may then respond to the site review report, and your report along with the site’s response will become part of the review history for that site. Please keep in mind that many LTER sites post their mid-term review reports, so you should not consider the report to be anonymous.

Budgets: During each funding cycle, sites receive a fixed, flat budget for each year. As a result, there are multiple and sometimes conflicting demands on the funds received between long-term data collection and new research directions. Most sites obtain significant research funds from other NSF programs, funding agencies or sources, with an average of about 3 non-LTER dollars for every NSF/LTER dollar. It can be difficult to sort out what NSF funds from related projects, but it is important for you to evaluate the ability of a site to retain a clear research focus that addresses important questions. The potential for a site to be come diffuse in its research is obvious when many of the resources obtained are not tied to core LTER goals.