STScI Logo

Hubble Space Telescope
Cycle 22 Peer Review Guidelines

Panel Orientation Presentations


Cycle 22 Peer Review Agenda and Shuttle Schedule

Cycle 22 Welcome Letter

Introduction

The Hubble Space Telescope science peer review process has changed considerably in recent years so even if you have served previously on an HST Panel or TAC, please review this website carefully.

HST proposals are of three main types: General Observer (GO), Snapshot (SNAP), and Archival Research (AR). Briefly, GO proposals request observing time in an integral number of spacecraft orbits; GO proposals are classified as Small (1-34 orbits), Medium (35-74 orbits), or Large (75 or more orbits which are summed over Cycles 22-24). SNAP programs use small amounts of otherwise idle time in the HST schedule; they are allocated a number of targets of which in practice only a fraction, ~30 - 40%, will actually be observed. AR programs are to carry out investigations using the HST Archive and are awarded a dollar amount (proposers from U.S. institutions only). We are continuing in Cycle 22 three important proposal categories which were started for the first time in Cycle 11: Treasury proposals are a special class of GO proposals that provide enhanced data products for the community; AR Legacy proposals are a comparable type of AR proposal; and Theory proposals, which are now supported as part of the HST AR program.

Small and Medium GO, SNAP, AR and Theory proposals will be reviewed by fourteen broad Review Panels: Planets (2), Stars (3), Stellar Populations (2), Galaxies (3), AGN and IGM (2) and Cosmology (2). The Telescope Allocation Committee (TAC) consists of the Chairs of the Panels plus a TAC Chair (who is not associated with any of the Panels) and three members at-large. The principal role of the TAC is to review and recommend allocation of the Large GO, Treasury and AR Legacy proposals. The TAC will also produce a final ranking for Pure Parallel and Medium programs, and will adjudicate any cross-panel scientific issues, as needed. The TAC and Panel recommendations on the Cycle 22 science program are advisory to the STScI Director, who is responsible for the final allocation of HST observing time and funding.

The main tasks of the Panels and TAC are threefold: (1) to rank the proposals in order of scientific merit; (2) to recommend the resources that should be allocated to each proposed program; and (3) to provide written comments about the recommendations.

Panel members, excluding panel chairs, will be assigned to grade 2/3s of the proposals before their panel for the preliminary grades/triage process. They will then need to read any additional proposals that make it thru the Triage process for the panel meeting as well as discuss and submit final grades during the review.

TAC Members will be assigned to grade 2/3s of the proposals before their panel for the preliminary grades/triage process.

The deadline for submission of preliminary grades is Wednesday May 28, 2014. Ranked proposal lists will be distributed by Friday, May 30, 2014 and reviewers should read the proposals above the triage line for which they have not submitted preliminary grades. All unconflicted reviewers are required to submit final grades for proposals discussed at the Panel/TAC meeting. At the meeting, for each proposal above the triage line or revived from triage (triaged proposals can be raised for discussion by any unconflicted TAC member), the primary and secondary reviewers will lead discussion.

It is extremely important that panelists submit their preliminary grades to STScI by the May 28th deadline. The primary and secondary reviewers assigned to each proposal will lead the discussion; preliminary comments are not required. If you wish to enter any preliminary comments prior to the review, we have implemented a "Private" Field in the Reviews section for each reviewer that is only visible to yourself. This will help lessen any bias with the preliminary grading process. Panel members should also review, but not grade, the TAC (Large, AR Legacy and Treasury) proposals assigned to their panel prior to the actual panel meetings.

Declare any Conflicts to your PSS as soon as possible.

What's New for Cycle 22

Unconscious bias: We have been continuing to look at ways of making minor adjustments to help reviewers focus on the science being proposed rather than who is leading the proposal. We have decided to make a few changes in the proposal format for Cycle 22. We will no longer list the name of the PI on the front page of the proposal – note that the PI is listed on the second page of the HST proposal with all the co-investigators, so that information is still available, just not highlighted to the same extent. In addition, we will replace forenames with initials in listing all investigators, and the pdf files will no longer include the PI surname as part of their filename.

Panelist Workload: Each proposal will receive preliminary grades from 6 panelists; one of those panelists will be designated the primary reviewer and a second will be the secondary reviewer. The reviewers for each proposal will be assigned through the usual process of science keyword matching so we optimize use of the expertise on the panel. The preliminary grading process will be uniform across all panels irrespective of the total number of proposals submitted to an individual panel.

In the case of the Large, Treasury and Legacy proposals submitted to the TAC, each proposal will receive 12 preliminary grades.

With 9 panelists, each panelist will be submitting preliminary grades for 2/3rds of the proposals submitted to a given panel. Those grades will be submitted via the TAC webtool. As in past cycles, panelists will be able to see and download all proposals assigned to their panel. To ensure we minimize confusion, panelists will only be able to enter preliminary grades for their assigned proposals prior to the preliminary grade deadline.

Preliminary grades (including those for TAC proposals) will be due at least 10 days before the meeting (the deadline, May 28th). We will use those preliminary grades to compute triage lists for each panel and the TAC and will circulate the results by May 30th.

Unless there is a conflict, panelists grade all proposals that are discussed at the panel meeting. Panelists should therefore review proposals above the triage line which they have not yet read in order to participate in the discussion of those proposals and provide final grades.

External Reviews for TAC Proposals: To help ensure that each proposal has the required conflict-free expert review, we will be obtaining 3 written reviews from community experts for each proposal evaluated by the TAC. Those experts have been chosen to avoid conflicts from both expert panelists and non-panel members. As with NSF proposals, will be asked to comment on the strengths and weaknesses of each proposal, not whether they believe a given proposal should be awarded time. We will collect those reviews and distribute them to the TAC members prior to their submitting preliminary grades before the TAC meeting.

Confidentiality

All HST Phase I proposals and documents related to the proposal review are strictly confidential.

  • They should not be distributed or used in any manner not directly related to the review process.
  • They may not be discussed with anyone other than those STScI staff participating in the review activities.
  • There should be no discussions with your fellow committee members other than those during the actual committee meeting.
  • The proposals and all related documents are to be brought to the Panel or TAC meetings for use there and collection afterward. Please purge any review files from your computer after the review.
  • Your involvement in the Cycle 22 review process itself should also be kept confidential until after the review has been completed, at which time membership lists are published in the STScI Newsletter.

Schedule

The Space Telescope Science Institute (STScI) will conduct the HST Cycle 22 Proposal Review June 08 - 13, 2014. The Review meetings will take place at the Space Telescope Science Institute. Fourteen Review Panels will meet for 2 1/2 days each, and the overall TAC will meet for 2 days, immediately after the panel meetings.

  • Preliminary Grades:
    • Wednesday May 28, 12pm EDT
  • Review Orientation:
    • Sunday June 08, 7:30pm EDT
  • Panels:
    • Monday June 09 - Wednesday June 11 (will end no later than 2pm)
  • TAC:
    • Wednesday June 11 (2-3pm) - Friday June 13 (will end around 5pm)

Conflicts of Interest

Our goal is informed, unbiased discussion of each proposal:

  • Voting committee members should have neither direct nor indirect interest vested in the outcome of the review.
  • The subset of the review committee discussing the proposal should have sufficient knowledge to assess the science.

It is critically important that conflicts of interest, or the appearance thereof, are avoided during the selection process. Here we identify two types of conflicts, Major and Minor. A panelist is considered to have a conflict of interest on a proposal under any of the following:

Major

  • Personal involvement (PI or Co-I)

    Direct gain from proposal success

  • Recent former advisor/student of PI or Co-I

    Indirect gain from proposal success

  • Involvement in a closely competing proposal regardless of whether that proposal is also before your panel. Proposals are judged to be 'closely competing' if their scientific goals, or their observations of the same targets, are sufficiently similar that an impartial panel, knowing one had been approved, would be unlikely to approve the other. Such conflicts should be direct and specific; a Panelist or Chair should not be excluded from considering an entire sub-category of proposals).

    Direct gain from proposal failure

  • Close personal ties (family, etc.) with PI or Co-I

    Indirect gain from proposal success

Minor

  • Institutional conflict, same institution as PI or Co-I

    Indirect gain from proposal success or failure

  • Close collaborator of PI or Co-I on the proposal
  • Any other reason for discomfort
  • Who qualifies as a close collaborator?
    • An active collaborator on a current research program (including Cycle 22 HST Proposals).
    • Active co-author on 3 or more papers in last 3 years
      i.e. more than a participant in a large project (e.g. SDSS).
  • Active collaborator on several recent programs
    At least 3 projects completed in last 3 years.

Key Question:

Would my personal research benefit (or would there be an appearance of benefit) if this proposal is accepted or rejected?

If the answer is yes, then there's a conflict.

Major Conflicts leave the room during the proposal discussion, while Minor conflicts declare their conflict, then they can participate in the discussion if they so choose, but do not vote on that proposal.

Panel Chairs should be very sensitive to these issues and always err on the side of caution. They should appoint a deputy chair to guide the discussions in the case where they have a conflict and must leave the room. If there is any possibility of a conflict or even the perception of a conflict, then it should be treated as such; questions concerning conflicts of interest should be directed to the Science Policies Group.

Chairs are responsible for avoiding and tracking conflict situations during the meetings, with assistance from the Panel Support staff. Each panelist must sign and turn in the Conflicts of Interest Disclosure Form prior to the end of the meeting.

Reviewer Activities

Step-by-Step Details

  1. Read all of the Documentation. Now thru Early May
  2. Check for Conflicts of Interest. Before May 7th
  3. Declare any Conflicts to your PSS as soon as possible. Before May 7th
  4. Review the proposals based upon Criteria for Selection as spelled out below in the Call for Proposals. Now thru May 28th
  5. Submit your Preliminary Grades by Wednesday 5/28.
  6. Review the Triage list. May 30 - June 7
  7. Read any proposals that are above the Triage line that you didn't already read and grade. May 30 - June 7
  8. Travel to the review. June 7 & June 8
  9. Review Meeting steps:
    1. Orientation Sunday June 8, 7:30pm
    2. Discuss Triage Proposals Monday Morning June 9
    3. Discuss and Grade all non-Triaged Proposals June 9 - June 10
    4. Rank Proposals June 10 - June 11
    5. Finalize Notification Comments June 11
    6. End of Panel Meetings June 11
    7. TAC Panel Meeting begins afternoon of June 11
    8. End of TAC Meeting late afternoon of June 13

Web-Reviewer Tool Helpful Hints

The Web-Reviewer tool can be found at: https://tac.stsci.edu/tacreview
Please contact Brett S. Blacker for access to the system if you don't recall your userid and/or password.

  • Simple Documentation or Full Documentation **This documentation will be updated in early May **.
  • Please download your panel(s) package from the Proposal Data Area.
  • The tool will be activated for preliminary grading in Early May and we will notify everyone that it's available. We will enable the reviews section of the tool several weeks prior to the review.

The System has been tested on FireFox 24.0 and above, IE 9.0 and above, Chrome 25.0 and above and Safari 6.0 and above.

Technical Information

This review is organized by the Science Policy Group (SPG) of the Science Mission Office (SMO) of the Space Telescope Science Institute (STScI). As a member of the TAC/Panels for Cycle 22, you are a critical part of the review process. We value your comments and suggestions, and welcome feedback throughout the process.


This page is maintained by Brett S. Blacker

Last Updated: August 29, 2014