The HST Proposal Selection Process

The following article by Meg Urry [cmu@stsci.edu], the head of the STScI Science Program Selection Office, appeared in the April 1999 STScI Newsletter.

Evolution of the HST Proposal Selection Process:
Changes for Cycle 9

INTRODUCTION

Yet after eight Cycles, the system has begun to show signs of strain. The number of submitted proposals has more than doubled, and the fundamental structure of panels and TAC has simply been scaled up commensurately, with little structural change. (See the January 1999 Newsletter for a description of the two-step panel/TAC review process for Cycle 8.)

As a result, the TAC has doubled in size, to more than 20 people in Cycles 7 and 8. Its primary responsibility has always been to establish priorities among scientific disciplines. With so many TAC members, the kind of in-depth, focused discussion needed to set these priorities has become increasingly difficult.

With so many proposals, the TAC spends most of its time debating smallish proposals in the "gray area" -- not the top ranked science -- and has less time to consider allocations to large proposals of 100 or more orbits, where TAC expertise is most needed.

With as many as 1300 proposals coming in, maintaining 80 to 90 proposals per panel has meant increasingly narrow science focus in each panel, a sort of "Balkanization" of the proposed science. This inadvertantly encourages undue attention to minute details, at the expense of "the big picture." The quality of the science discussions declines -- it must be very boring to discuss several dozen nearly identical proposals! And scientific areas are given an "entitlement" of HST time simply by designation of the panel topics.

The natural sociology of the proposal selection process also works against larger programs, which are increasingly seen as a vital part of the HST program.

These trends threaten our ability to maintain the excellence of HST science as we move into the second decade of operations.

CHANGES IN CYCLE 9

Major changes include:

• Fewer panels. With more orbits to allocate, each panel will have more flexibility, especially to approve the somewhat larger proposals.
• Even fewer scientific categories. Panels will be dedicated to very broad science topics, perhaps as few as five -- one for solar system, two for Galactic science, two for extragalactic science. *Hence it will be essential for proposers to describe the impact of the proposed science on astronomy in general ("the big picture").* Science balance among subtopics, previously determined by the TAC, will be achieved within the selection panels. This approach, long practiced in the high-energy astrophysics community, also simplifies the avoidance of conflicts of interest.
• TAC focus on large programs. The main role of the TAC will be to approve the best large programs (100 or more orbits), for which they will have up to 1,000 orbits to allocate. This means the TAC could approve 2 to 3 programs averaging 300 orbits each. With scientific balance achieved within the broad selection panels, the TAC is not expected to address the small proposals at all. *Large proposals will therefore have an excellent chance of success, and are strongly encouraged.
• Incentives for medium proposals. Starting in Cycle 7, there have been "orbit subsidies" to encourage panels to approve medium proposals (more than 30 orbits). This meant that a fraction of the orbit request was not charged against the panel quota. This system worked reasonably well, leading in Cycle 8 to an average acceptance rate that was independent of proposal size. Most of the community was not aware, however, that "medium" proposals were given this advantage to compensate for the natural reluctance to allocate so much to one program.
  For Cycle 9, we plan to implement progressive subsidies, meaning the subsidized fraction increases with size. We will adjust the algorithm from one Cycle to the next to try to keep the oversubscription rate approximately the same independent of the size of the proposal (under the assumption that quality is independent of size). *We strongly encourage proposers to ask for the observing time they need to achieve their scientific goals, without strategizing in response to perceived notions of advantages or disadvantages with respect to size.*
• Allocation of coordinated Chandra/AXAF time.Proposals of a fundamentally multi-wavelength nature, requiring both HST and the Chandra X-ray Observatory (AXAF), can be submitted to either the present HST Cycle 9 or the next AXAF review. By agreement with the AXAF Science Center (ASC), STScI will be able to award up to 400 kiloseconds of AXAF observing time, and similarly the ASC will be able to award up to 100 orbits of HST time, to highly-rated proposals meeting the multiwavelength criterion.

REVISE AND CONSENT

1. Proposers must stress why their science is critically important and why it requires HST. The panelists reading the proposals will have a broader expertise -- and there will be fewer specialists in the particular topic -- so more introductory material may be necessary. At the same time, we can use as reviewers more expert HST users who in the past were excluded because they submitted too many proposals to meet our conflict-of-interest guidelines.
2. Larger proposals are strongly encouraged and are expected to be at least as successful as small proposals. The oversubscription rate for HST hovers around 5:1, so it is never easy to get time but *the odds of success are the same for proposals over 100 orbits and those under 10 orbits.* So if you have a project that requires a large investment of HST observing time, do not hesitate to propose it.
3. Science requiring both HST and Chandra/AXAF can be proposed to only one Observatory, eliminating the "double jeopardy" that unfairly disadvantages multiwavelength science. Proposals should be submitted to the observatory that represents the prime science, i.e, where the predominant panel expertise (IR/opt/UV vs. X-ray) is most relevant.
4. Write clearly. Reviewers have always had a difficult job reading ~100 or so proposals carefully. In Cycle 9, that number may be doubled. So take the time to write clearly and coherently, explaining what you want to do and why. This is good proposal strategy whatever the review process may be.

Some of the changes for Cycle 9 were suggested by experienced members of past HST proposal reviews and other interested astronomers. Similar input from the entire HST community is very welcome.

You may also be interested to read the article by Mike Shara, the former head of the STScI Science Program Selection Office, which appeared in the January 1999 STScI Newsletter.

The HST Proposal Selection Process

Next month I'll step down as Head of the STScI Science Programs Selection Office (SPSO), after overseeing proposal Cycles 5, 6 and 8. The Panels and TACs I've had the privilege of working with have allocated about 10,000 HST orbits, equivalent to roughly a billion dollars (when you amortize the HST mission cost over 20 years). Their hard work, and that of the SPSO staff, is directed at selecting the very best proposals received each year.

The oversubscription rate is typically four- or five to one, with 1000+ proposals each year, so it's inevitable that some very good proposals are declined each cycle. I'd like to share with you some details of how the process works, what the Panels and TAC look for in supporting the truly outstanding proposals, and some ideas that you might find useful when you write your next HST proposal.

How the Proposal Selection Process Works

Observations and Suggestions

1. The best proposals can be understood in their entirety by a first-year graduate student. In other words, a clear and succinct statement of the broad science background is essential. Convincing the Panel that the proposed observations will significantly advance our understanding of a class of objects or physical process is no less crucial. Narrowly-focused proposals or those proposing only modest increments of knowledge rarely succeed.
2. Tell the panel why HST is essential to the success of your proposal. If it can be done from the ground, even with difficulty, your proposal will not receive time.
3. Signal-to-noise calculations must be thoroughly documented. Vague descriptions force STScI technical personnel to do Exposure Time Calculations in support of panels, and they may have to guess at proposers' intentions. Significant doubt about the technical feasibility of a proposal is usually fatal to that proposal; the onus is on the proposer!
4. Panels look very hard at what you've done with your past allocations of HST time. Requesting more time when you haven't published observations from two or more cycles ago regularly leads to rebukes and proposal downgrading by panels. Timely publication in refereed journals is regarded by panels as a strong plus.
5. Proposers who request exactly the number of orbits that they really need, regardless of the "Small," "Medium," or "Large" orbit boundaries, tend to do best. Panels and TAC have an uncanny sense of orbit inflation and punish it. They also recognize a truly great, exciting proposal right away and support it with little regard for orbit cost.
6. Edit your proposals carefully and have a colleague read them for clarity. Missing references, mislabelled figures, garbled sentences and proposals exceeding the page limit rapidly sour panelists trying to absorb up to 100 competing proposals.

Final Thoughts

Finally, consider submitting your proposal a few days ahead of the deadline. Please. The internet didn't melt down on September 11, as your proposals came flooding in, despite the fact that Ken Starr's report was competing for bandwidth. But did you really need the adrenaline rush of watching your proposal vanish into the ether with less than 100% probability of receipt before the deadline?

[.........]

Owner: Brett S. Blacker

Technical Questions: HST Help Desk

Last updated: June 11, 1999 by Roeland van der Marel.