- Panel Orientation Presentations
- Important Documentation
- Selection Criterion
- What's New for Cycle 23
- Important Considerations
- Conflicts of Interest
- Reviewer Activities
- Web-Reviewer Tool
- Technical Information
Panel Orientation Presentations POSTED 06/07/15 9:24PM
- Cycle 23 Introduction to Cycle 23 Review Panels by Kathy Flanagan
- Cycle 23 HST Status Presentation to Cycle 23 Review Panels by John MacKenty
- Cycle 23 Review Process Presentation to Cycle 23 Review Panels by Claus Leitherer
The Hubble Space Telescope science peer review process has changed considerably in recent years so even if you have served previously on an HST Panel or TAC, please review this website carefully.
HST proposals are of three main types: General Observer (GO), Snapshot (SNAP), and Archival Research (AR). Briefly, GO proposals request observing time in an integral number of spacecraft orbits; GO proposals are classified as Small (1-34 orbits), Medium (35-74 orbits), or Large (75 or more orbits which are summed over Cycles 23-25). SNAP programs use small amounts of otherwise idle time in the HST schedule; they are allocated a number of targets of which in practice only a fraction, ~30 - 40%, will actually be observed. AR programs are to carry out investigations using the HST Archive and are awarded a dollar amount (proposers from U.S. institutions only). We are continuing in Cycle 23 three important proposal categories which were started for the first time in Cycle 11: Treasury proposals are a special class of GO proposals that provide enhanced data products for the community; AR Legacy proposals are a comparable type of AR proposal; and Theory proposals, which are now supported as part of the HST AR program.
Small and Medium GO, SNAP, AR and Theory proposals will be reviewed by fourteen broad Review Panels: Planets (2), Stars (3), Stellar Populations (2), Galaxies (3), AGN and IGM (2) and Cosmology (2). The Telescope Allocation Committee (TAC) consists of the Chairs of the Panels plus a TAC Chair (who is not associated with any of the Panels) and three members at-large. The principal role of the TAC is to review and recommend allocation of the Large GO, Treasury and AR Legacy proposals. The TAC will also produce a final ranking for Pure Parallel, and will adjudicate any cross-panel scientific issues, as needed. The TAC and Panel recommendations on the Cycle 23 science program are advisory to the STScI Director, who is responsible for the final allocation of HST observing time and funding.
The main tasks of the Panels and TAC are threefold: (1) to rank the proposals in order of scientific merit; (2) to recommend the resources that should be allocated to each proposed program; and (3) to provide written comments about the recommendations.
Panel members, excluding panel chairs, will be assigned to grade 2/3s of the proposals before their panel for the preliminary grades/triage process. They will then need to read any additional proposals that make it thru the Triage process for the panel meeting as well as discuss and submit final grades during the review.
TAC Members will be assigned to grade 2/3s of the proposals before their panel for the preliminary grades/triage process.
The deadline for submission of preliminary grades is Monday June 1, 12:00PM EDT, 2015. Ranked proposal lists will be distributed by Wednesday, June 03, 2015 and reviewers should read the proposals above the triage line for which they have not submitted preliminary grades. All unconflicted reviewers are required to submit final grades for proposals discussed at the Panel/TAC meeting. At the meeting, for each proposal above the triage line or revived from triage (triaged proposals can be raised for discussion by any unconflicted TAC member), the primary and secondary reviewers will lead discussion.
It is extremely important that panelists submit their preliminary grades to STScI by the June 1st deadline. The primary and secondary reviewers assigned to each proposal will lead the discussion; preliminary comments are not required. If you wish to enter any preliminary comments prior to the review, we have implemented a "Private" Field in the Reviews section for each reviewer that is only visible to yourself. This will help lessen any bias with the preliminary grading process. Panel members should also review, but not grade, the TAC (Large, AR Legacy and Treasury) proposals assigned to their panel prior to the actual panel meetings.
Declare any Conflicts to your PSS as soon as possible.
Selection CriterionReviewers should review all proposals against the following criteria:
- The scientific merit of the program and its potential contribution to the advancement of scientific knowledge
- The program’s importance to astronomy in general
- The extent to which the proposal demonstrates sufficient understanding to assure a thorough analysis of the data
- A demonstration that the unique capabilities of HST are required to achieve the science goals of the program
- A demonstration of timely publication of the results of any previous HST programs
- Evidence for a coordinated effort to maximize the scientific return.
Reviewers should ensure that the comments that proposers receive address some or all of these primary criteria. Additional criteria for specific types of proposals are listed here.
What's New for Cycle 23
Medium Proposals: Medium GO Proposals are those requesting between 35 and 74 orbits.
Continuing this cycle, they are being treated as a separate category of
proposal with its own pool of orbits. We expect that about 15
such proposals will be recommended for approval by the TAC.
Each panel will rank the Medium GO Proposals alongside the Small Proposals based on scientific merit. Given the limited resources, and high proposal pressure especially in the medium category, we expect that on average less than one medium proposal per panel will be approved. We will group the proposals by mirror panels and use the fraction of the whole to assign a medium proposal allocation to each set of mirror panels. Fractional Time will be rounded. The medium proposals above the cut-off line are identified for further discussion. There should be nominally no more than three medium proposals above the cut-off line. The candidate proposals will be circulated to the mirror panels on Tuesday evening.
On Wednesday morning each mirror panel will discuss and rank the candidate medium proposals. Conflicts of interest will be treated as usual procedures. The ranked lists from each set of mirror panels will be combined and averaged to forma a list of medium proposals recommended for approval. The panels can adjust their own rankings if they want to support any medium proposals that did not make the merged panel cut. The TAC will get a report of the recommended medium proposals before it considers the Large and Treasury programs.
Unconscious bias: We have been continuing to look at ways of making minor adjustments to help reviewers focus on the science being proposed rather than who is leading the proposal. We have decided to make a few changes in the proposal format for Cycle 23. We will no longer list the name of the PI on the front page of the proposal – note that the PI is listed on the second page of the HST proposal with all the co-investigators, so that information is still available, just not highlighted to the same extent. In addition, we will replace forenames with initials in listing all investigators, and the pdf files will no longer include the PI surname as part of their filename.
Panelist Workload: Each proposal will receive preliminary grades from 6 panelists; one of those panelists will be designated the primary reviewer and a second will be the secondary reviewer. The reviewers for each proposal will be assigned through the usual process of science keyword matching so we optimize use of the expertise on the panel. The preliminary grading process will be uniform across all panels irrespective of the total number of proposals submitted to an individual panel.
In the case of the Large, Treasury and Legacy proposals submitted to the TAC, each proposal will receive 10 preliminary grades.
With 9 panelists, each panelist will be submitting preliminary grades for 2/3rds of the proposals submitted to a given panel. Those grades will be submitted via the TAC webtool. As in past cycles, panelists will be able to see and download all proposals assigned to their panel. To ensure we minimize confusion, panelists will only be able to enter preliminary grades for their assigned proposals prior to the preliminary grade deadline.
Preliminary grades (including those for TAC proposals) will be due at least 10 days before the meeting (the deadline, June 1st). We will use those preliminary grades to compute triage lists for each panel and the TAC and will circulate the results by June 3rd.
Unless there is a conflict, panelists grade all proposals that are discussed at the panel meeting. Panelists should therefore review proposals above the triage line which they have not yet read in order to participate in the discussion of those proposals and provide final grades.
External Reviews for TAC Proposals: To help ensure that each proposal has the required conflict-free expert review, we will be obtaining 3 written reviews from community experts for each proposal evaluated by the TAC. Those experts have been chosen to avoid conflicts from both expert panelists and non-panel members. As with NSF proposals, the experts will be asked to comment on the strengths and weaknesses of each proposal, not whether they believe a given proposal should be awarded time. We will collect those reviews and distribute them to the TAC members prior to their submitting preliminary grades before the TAC meeting.
All HST Phase I proposals and documents related to the proposal review are strictly confidential.
- They should not be distributed or used in any manner not directly related to the review process.
- They may not be discussed with anyone other than those STScI staff participating in the review activities.
- There should be no discussions with your fellow committee members other than those during the actual committee meeting.
- The proposals and all related documents are to be brought to the Panel or TAC meetings for use there and collection afterward. Please purge any review files from your computer after the review.
- Your involvement in the Cycle 23 review process itself should also be kept confidential until after the review has been completed, at which time membership lists are published in the STScI Newsletter.
The Space Telescope Science Institute (STScI) will conduct the HST Cycle 23 Proposal Review June 07 - 12, 2015. The Review meetings will take place at the Space Telescope Science Institute. Fourteen Review Panels will meet for 2 1/2 days each, and the overall TAC will meet for 2 1/2 days, immediately after the panel meetings.
- Preliminary Grades:
- Monday June 1st, 12pm EDT
- Review Orientation:
- Sunday June 07, 7:30pm EDT
- Monday June 08 - Wednesday June 10 (will end no later than 2pm)
- Wednesday June 10 (2-3pm) - Friday June 12 (will end around 5pm)
Conflicts of Interest
Our goal is informed, unbiased discussion of each proposal:
- Voting committee members should have neither direct nor indirect interest vested in the outcome of the review.
- The subset of the review committee discussing the proposal should have sufficient knowledge to assess the science.
It is critically important that conflicts of interest, or the appearance thereof, are avoided during the selection process. Here we identify two types of conflicts, Major and Minor. A panelist is considered to have a conflict of interest on a proposal under any of the following:
Personal involvement (PI or Co-I)
Direct gain from proposal success
Recent former advisor/student of PI or Co-I
Indirect gain from proposal success
Involvement in a closely competing proposal regardless of whether that proposal is also before your panel. Proposals are judged to be 'closely competing' if their scientific goals, or their observations of the same targets, are sufficiently similar that an impartial panel, knowing one had been approved, would be unlikely to approve the other. Such conflicts should be direct and specific; a Panelist or Chair should not be excluded from considering an entire sub-category of proposals).
Direct gain from proposal failure
Close personal ties (family, etc.) with PI or Co-I
Indirect gain from proposal success
Institutional conflict, same institution as PI or Co-I
Indirect gain from proposal success or failure
- Close collaborator of PI or Co-I on the proposal
- Any other reason for discomfort
- Who qualifies as a close collaborator?
- An active collaborator on a current research program (including Cycle 23 HST Proposals).
- Active co-author on 3 or more papers in last 3 years
i.e. more than a participant in a large project (e.g. SDSS).
- Active collaborator on several recent programs
At least 3 projects completed in last 3 years.
Would my personal research benefit (or would there be an appearance of benefit) if this proposal is accepted or rejected?
If the answer is yes, then there's a conflict.
Major Conflicts leave the room during the proposal discussion, while Minor conflicts declare their conflict, then they can participate in the discussion if they so choose, but do not vote on that proposal.
Panel Chairs should be very sensitive to these issues and always err on the side of caution. They should appoint a deputy chair to guide the discussions in the case where they have a conflict and must leave the room. If there is any possibility of a conflict or even the perception of a conflict, then it should be treated as such; questions concerning conflicts of interest should be directed to the Science Policies Group.
Chairs are responsible for avoiding and tracking conflict situations during the meetings, with assistance from the Panel Support staff. Each panelist must sign and turn in the Conflicts of Interest Disclosure Form prior to the end of the meeting.
- How do I receive my Proposals?
- What are my Review Assignments?
- How do I make Travel/Hotel Arrangements?
- Why do we need to submit Preliminary Grades and why do we do Triage?
- Do I need to submit Preliminary Comments?
- As a Panel Chair what are my additional duties?
- What do we do as a Panel with the UV Initiative Proposals?
- What do we do as a Panel with the Large Proposals?
- How are the TAC Proposals assigned?
- What happens during the Panel Meetings?
- What happens during the TAC Meeting?
Step-by-Step Details See Step-by-Step Web-Reviewer Guide
- Read all of the Documentation. Now thru Early May
- Check for Conflicts of Interest. Before May 7th
- Declare any Conflicts to your PSS as soon as possible. Before May 8th
- Review the proposals based upon Selection Criterion. Now thru June 1st
- Submit your Preliminary Grades by Monday 6/1.
- Review the Triage list. June 3 - June 6
- Read any proposals that are above the Triage line that you didn't already read and grade. June 3 - June 6
- Travel to the review. June 6 & June 7
- Review Meeting steps:
- Orientation Sunday June 7, 7:30pm
- Discuss Triage Proposals Monday Morning June 8
- Discuss and Grade all non-Triaged Proposals June 8 - June 9
- Rank Proposals June 9 - June 10
- Finalize Notification Comments June 10
- End of Panel Meetings June 10
- TAC Panel Meeting begins afternoon of June 10
- End of TAC Meeting late afternoon of June 12
Web-Reviewer Tool Helpful Hints
- What's in a Proposal?
- What are some minor policy issues to keep in mind?
- What Special Proposal Categories do we need to worry about?
- Medium Proposals
- UV Initiative Proposals
- Long Term Proposals
- Treasury Proposals
- Legacy Archival Research Proposals
- Theory Proposals
- Calibration AR Proposals
- Calibration GO Proposals
- Calibration Snap Proposals
- Joint HST-Chandra Proposals
- Joint HST-NOAO Proposals
- Joint HST-NRAO Proposals
- Joint HST-SPITZER Proposals
- Joint HST-XMM Proposals
- Target of Opportunity Proposals
- Where is the HST Cycle 23 Call for Proposals or HST Primer or the HST Instrument Handbooks?
- Where is that welcome letter from Claus?
- Who is supporting our Panel from STScI?
- What about NICMOS and WFC3 vs AO?
- What do all of these pesky acronyms and abbreviations mean?
- Peer Review Contact List for any Questions
This review is organized by the Science Policy Group (SPG) of the Science Mission Office (SMO) of the Space Telescope Science Institute (STScI). As a member of the TAC/Panels for Cycle 23, you are a critical part of the review process. We value your comments and suggestions, and welcome feedback throughout the process.
This page is maintained by Brett S. Blacker
Last Updated: August 01, 2015