About this Article

Science thrives best in an equitable and open environment. The Hubble Space Telescope owes its unparalleled scientific productivity in no small part to its constitution as an international observatory, welcoming proposals from anyone in the worldwide astronomical community. As the prime steward for Hubble science, the Institute is committed to ensuring that the community competes for observing time on a level playing field. Fairness and equity are an integral part of the peer-review process used for proposal evaluations, so this is a commitment that we take very seriously.

It is disquieting to see a systematic offset in the success rates of proposals led by male and female principal investigators (PIs). While small for any given observing cycle, this offset cannot be explained as statistical noise in the review process, as it is persistent and always in the same sense: women do not fare as well as expected based on the percentage of proposals submitted. [1] This leads us to believe that subtle, but important, unconscious biases may be a factor in the proposal reviews. Such biases are well documented in many disciplines and are known to affect the peer review process (see bibliography in reference [2]). Hubble is not the only observatory showing this effect—similar trends have been seen in the proposal selection processes for other observatories as well. This raises the possibility that there may be common biases affecting proposal assessments, possibly extending beyond gender to race, geographic location, home institution, seniority, and other factors that have not yet been fully evaluated.

Following discussions with the Space Telescope Users Committee (STUC), we implemented progressive changes in the proposal format over the past several observing cycles to obscure the identity of the PI while preserving the investigator list in an alphabetized order. We also brought in an outside expert, Dr. Stefanie Johnson (University of Colorado, Leeds School of Business), as a consultant and observer at the Cycle 25 Time Allocation Committee (TAC) meeting in 2017. Dr. Johnson has extensive professional experience in developing strategies to mitigate unconscious bias. Her analysis of the Hubble results shows that the proposal format changes we have implemented led to more balanced results in the preliminary grades, which were submitted prior to the face-to-face discussions of the proposals. However, a gender bias was still evident in the final set of proposals recommended to the Director for implementation after the in-house panel review. Dr. Johnson noted that some proposal discussions continued to focus on the proposers rather than scientific objectives. Her recommendation was to consider fully anonymizing the identities of the proposers, which would have the added benefit of minimizing conflicts of interest and reducing unknown biases, as well as those identified. While this approach is common in other fields, it represents a significant departure from the norm for much of the astronomical community.

With that in mind, I asked the STScI Science Mission Office (SMO) and the HST Mission Office (HSTMO) to constitute a working group [see Table 1] to identify an appropriate process for rendering proposals anonymous and developing instructions and guidelines for writing and reviewing proposals. As part of their deliberations, the working group solicited community feedback, receiving 60 responses: many were highly supportive, others less so. Separately, I also received letters, some in support and others suggesting alternatives to fully anonymizing the proposals. Some alternative suggestions favored a random draw of proposals meeting minimum qualifications over peer review; some suggested making the entire process open so that both reviewers and proposers are known to each other; and others advocated actively reordering proposal rankings near the cutoff to achieve a specified result. In my opinion, these alternative approaches neither preserve the value of the peer-review process nor produce a level playing field in a manner consistent with the thoughtfulness and effort invested in these proposals by the community.

The most pressing area of concern raised by the community centers on vetting the proposing teams—specifically, ensuring that proposers have the expertise to write not only compelling proposals, but also to conduct the science proposed. In earlier days, this was a primary evaluation criterion for Hubble proposals, but in recent years has been de‑emphasized. There is also some unease about the effort required to write proposals that will need to conform to new guidelines for proposal anonymity.

The recommendations of the working group address these concerns directly. In short, the proposals will be reviewed strictly on science merit, without revealing the proposers' names. Proposals will need to be written in a manner that eliminates self-identification. Specific recommendations for how that should be done, along with additional information, are available at the websites below. [2, 3] After the science evaluations and ranking are completed, the names of the investigators and a self-provided description of qualifications will be reviewed for proposals ranked above the science merit cutoff. These latter expertise assessments will not alter the proposal rankings, but any identified deficiencies will be brought to the attention of the Director for consideration in the final approval process.

Based on the working group's recommendations and the endorsement of the Space Telescope Institute Council (STUC), and the Goddard Space Flight Center HST Project Office, the Institute will implement an anonymous review process in Cycle 26. I’d like to thank everyone involved in reaching this decision. Our overarching goals are to place the focus of the reviews squarely on the science and to give every proposal team an equal platform to make their case. I recognize that this change may be unfamiliar to some, yet steadfastly believe that this is the fairest and most defendable approach to ensuring impartial reviews of all proposals. The Institute will continue to monitor and publish the outcomes of the peer reviews, and measure progress against the well-established history of the observatory. Time will tell, but I am confident we will look back and view this change as a positive watershed moment for the mission.

References

[1] Reid, I. N. 2014, PASP, 126, 923

[2] https://outerspace.stsci.edu/display/APRWG [Working group recommendations]

[3] https://hst-docs.stsci.edu [HST Cycle 26 Call for Proposals]