Space Telescope Users Committee

Attended: Dave Axon, Marc Davis, James Dunlop, Debra Elmegreen, Martin Elvis, Suzanne Hawley, Holland Ford, Chris Impey, John Kormendy, Karen Meech, George Miley (Chair), Peter Nugent, Dave Sanders, Karl Stapelfeldt, John Stocke, Lisa Storrie-Lombardi

1. GENERAL - STUC PORTFOLIOS

The chairman welcomed 5 new members to the committee, Dave Axon, Martin Elvis, Karen Meech, Peter Nugent and Lisa Storrie-Lombardi. The following revised list of STUC portfolios for the various issues concerning STUC was agreed:

(i) Instrumental Issues:

(e.g. observing modes, calibration, performance, capabilities and upgrades),

ACS/ WFPC2: Elmegreen, Ford, Impey, Stapelfeldt

COS/ STIS: Axon, Hawley, Meech,Stocke

NICMOS/ WFC3: Dunlop, Kormendy, Sanders, Storrie-Lombardi

(ii) Operational Issues:

Proposal Handling and Scheduling: Axon, Impey, Storrie-Lombardi

Software Analysis Tools: Elvis, Impey, Sanders

Targets of Opportunity: Nugent

Solar System Issues: Meech, Stapelfeldt

Archive: Elvis, Davis, Dunlop

(iii) Miscellaneous issues

TAC: Dunlop, Hawley, Stocke

GO Funding: Elmegreen, Nugent, Stocke

HST Lifetime: Davis, Ford, Kormendy

Users who have input of general interest about any of these issues should contact the relevant STUC portfolio-holder or communicate directly with George Miley. Information about the portfolios should be displayed in the STUC section of the STScI web site.

2. THE SM3B SERVICING MISSION AND STATE OF THE PROJECT

The meeting was held about 1 month after the SM3b servicing mission. There were several presentations describing the mission and the present state of the telescope and instruments. STUC congratulates all concerned on the complete success of this demanding mission. The users appreciate the dedication, expertise and hard work of the hundreds of engineers, managers, scientists and other workers who contributed to the endeavour and the courage of the astronauts who carried out the delicate job of fitting the solar panels and new instruments to the telescope.

We are thrilled that all the major aims of the EVAs were accomplished. The HST has been restored to full health and its scientific capabilities have been increased substantially.

The STUC is excited by the huge improvement in sensitivity and resolution provided by the ACS. First light was obtained on 22 March and orbital verification appears to be proceeding according to plan. We were privileged to see some of the beautiful images obtained during the calibration program and congratulate the ACS IDT and the STScI ACS team on the progress to date. The ACS performance seems to be excellent and within specs. We have no doubt that the powerful combination of the HST and ACS will have a substantial impact on fundamental science.

During the past year we have emphasized the scientific importance of restoring NICMOS to health, particularly as an infrared complement to the ACS. We were therefore gratified that NASA was able to install the cryocooler, despite the many other demands of the mission. At the time of our meeting, cooling of NICMOS was still in progress, and first tests of the retrofitted NICMOS were scheduled for a few days later. Further, the new solar arrays have been shown to deliver a 27% increase of power. We compliment NASA on their success.

The SM3b mission was accomplished with a minimum loss of HST time. Routine observations were being performed within a few days of the completion of the servicing. This is a tribute to all those at STScI who are involved in scheduling the telescope and ensuring that as many orbits as possible are used for scientific discovery.

3. THE PROPOSAL SELECTION PROCESS

As agreed during our last meeting, considerable time at this meeting was devoted to a discussion of the results of the Cycle 11 TAC and the planned procedures for Cycle 12. After the unprecedented over-subscription of proposals for Cycle 11, the institution of new classes of programs and experimentation with new procedures this topic was regarded as one that merited close scrutiny by STUC. Several topics related to the TAC were considered.

3.1. Fairness. On the basis of (i) the procedures presented to us and (ii) the experience of the 3 STUC members who participated in the Cycle 11 TAC process, all 16 members of the STUC concluded that the TAC was just and fair. Since members of the STScI staff were excluded from participating in the TAC, their success in the winning Cycle 11 time cannot be attributed to any bias in the process. We note that the formal statistics were weighted heavily by the success of one or two large proposals involving PIs from the STScI. The STUC compliments the STScI in conducting well-founded and balanced TAC procedures during the past 11 years.

3.2. Balance between large, intermediate and small programs. During Cycle 11, 42% of the telescope time was allocated to large and treasury programs, considerably above the 30% target. The initiation of the Treasury program, as well as the community's anticipation of the increased power of ACS, certainly contributed to this increase.

For Cycle 12, the STUC recommends that the decisions on large and treasury proposals be made after the panel reviews of the smaller proposals, which will give the TAC perspective on the full suite of proposals before committing a large fraction of the telescope time. Until we learn more from the ongoing studies of the HST metrics, we continue to recommend that 1/3 of the telescope time be the target allocation for large and treasury proposals.

Setting the optimum balance between different sizes and categories of programs continues to be a challenge for HST and all observatories. The goal should be to steer this balance so as to optimize the fundamental scientific impact of the facility, but the measurement of this impact, as discussed in the HST metrics presentation, remains imprecise. We are pleased that a comparison of the impact of past programs of different types is underway and expect that the conclusions of this study will be an important input for specifying the optimum balance in the future.

The high visibility and potential impact of large and treasury programs must be weighed against the demonstrated success of small programs, which have yielded 85% of HST's scientific publications to date.

3.3. Feedback to Proposers. STUC welcomes the decision of the Institute to reinstate feedback to users on Cycle 11 proposals and to provide enhanced feedback for Cycle 12. We note that feedback to proposers is a feature of every major NASA and NSF peer-reviewed process.

While comments cannot convey the complete reason for a proposal’s ranking, it is clear that a large number of users would appreciate more comprehensive feedback than was eventually provided for Cycle 11. The STUC believes that it is perfectly acceptable for the feedback to follow some time after the announcement of the TAC results in order to ensure the quality and accuracy of the comments.

It is our impression that the ease with which such feedback can be provided would be assisted by introducing a more rigorous system of proposal assessment by panel members prior to the TAC meeting, categorizing the various classes of standard comments (e.g. “Other proposal for similar science was regarded as preferable”) and making more use of electronic tools in disseminating and checking the comments. Encouraging a higher degree of discipline in the completion of TAC

comments prior to the meeting and dissemination to panel members would also stimulate a more intensive and deeper consideration of proposals by panel members. The advantage of such a system would be to enforce a professional and consistent approach and provide a document prior to the TAC/ panel meetings, from which accurate feedback can be easily culled. It would also facilitate efficient operation of the panel meetings. Specifically, we suggest considering the provision of electronic versions of the comments and assessments to all panel members at least a week before the TAC meeting. Consideration may need to be given to providing incentives to TAC members to ensure that deadlines are met.

3.4. Archival and theory programs. STUC is satisfied that the current threshold above which archival and theory proposals are approved (oversubscription of ~ 3 to 1) is appropriate. However, we urge that any future metrics include a careful assessment of the scientific impact of archival research, since the funding of archive research leverages a steadily growing HST resource.

The committee recommends that there be a section in the proposal forms, for both archival and theoretical proposals, where the PI's and Co-I's are compelled to list their current and pending sources of research support, together with the title of the associated research projects. This would help avoid duplication of research funding. In addition the proposers should be asked to describe how the proposed research is unique/distinct from previous endeavors by themselves or others.

3.5. Shortening the Proposal Process. The STUC considered the proposal by the STScI to shorten the time line between submission and delivery on the telescope, thereby shifting the proposal deadline from September to January. We regarded this initiative as timely and highly desirable. Shortening the process has no obvious penalty and a number of logistic benefits for the user community. Further, the simplification of user overheads in the Phase I and Phase II preparation, inherently implied by this revised time-line, would be welcome.

3.6. Guidelines to TAC. STUC praises the plan to provide guidelines to Cycle 12 TAC members regarding the limitations of ground-based adaptive optics and the uniqueness of the science that can be carried out by NICMOS. Such guidelines would have been helpful to the Cycle 11 TAC in their evaluation of high spatial resolution of faint objects and high contrast observations of circumstellar regions.

3.7. STIC Committee. There is no perfect TAC procedure and specifying an optimum one is one of the most difficult problems in administering highly oversubscribed public scientific facilities. We note that the Space Telescope Institute Council is presently instituting a committee to consider such questions. STUC will be happy to provide input to this committee or any other help deemed to be useful.

4. SOFTWARE

4.1. The Astronomers Proposal Tool. The APT has now becoming an important user tool and the APT project is a vital element in shortening the time between proposal submission and scheduling. During Cycle 12, the APT will replace RPS2 as the standard Phase 2 preparation tool.

The STUC was provided with demonstrations of the new functionality of the APT and was pleased with general progress in its development. We were particularly impressed with the apparent increase in speed of the new tool compared with RPS2.

A successful final development phase for APT has become crucial, since APT has moved from being an optional resource to being the only tool for proposal preparation and submission. We advise the STScI to obtain as much feedback from users as possible during the next six months, to make sure that APT contains the features most required by the user community. Relevant STUC portfolio holders will be happy to assist with this. We look forward to the APT becoming an easy-to-use stable environment that will support the Institute for several years. The new options should ease both the Phase 1 and Phase 2 processes for users.

The STUC has some worry that the present development, while concentrating on satisfying the majority of users, is neglecting small but important segments of the community. The ability to handle moving targets is essential for planetary astronomy and this needs to be addressed in APT as a high priority item. Support for planetary astronomy is being implemented in a similar tool that is being developed for SIRTF and it might therefore be of benefit for the APT team to discuss the matter with the appropriate staff at the SIRTF Science Center.

We also consider cross-platform support for the APT to be a high priority. Linux is now used by a substantial fraction of the astronomical community (perhaps the majority) and we therefore consider it essential that APT should run on Linux. Macintosh support should also be provided as soon as resources allow. In previous reports we have advocated spending more attention on providing such cross-platform support for all user support software developed at STScI. We note that number of users of Solaris-Unix (base-system for much of the HST software development) has strongly diminished within the community.

4.2. On-line Documentation. The STUC was pleased to review the new system for online documentation, with its simpler means for updating. We note that the volume of HST documentation is now so large that it is difficult for inexpert users to locate information on how to perform procedures for basic tasks needed in data reduction or proposal preparation.

We therefore urge that the Institute create a set of “threads” or “cookbook recipes” that contain instructions for carrying out simple frequently required tasks for each instrument, including both data analysis and proposal preparation. We suggest that the STScI consider coordinating such an effort with the Chandra and SIRTF Science Centers, with a view to providing a common look and feel.

4.3. SHARE. We received written update about progress in the Study of Hubble Archiving & Reprocessing Enhancements. We endorse the development of the capabilities planned for the next year, particularly the priority given to the basic data products that affect all users, and for which expertise is concentrated at the Institute. These include work to improve astrometric accuracy, create “engineering-grade” catalogs for this purpose, and to develop mosaicing. We suggest an even tighter focus on these enhancements during this first phase.

Given the ECF success with improving the quality of FOS data, the STUC had some concern with the proposal that would remove the ability to return to the original data, for those instruments that currently are processed via GEIS files. We therefore urge that an assessment be made of the effort needed to make these tools usable with FITS input and output formats.

4.4. The STECF Archive Calibration Project. STUC were impressed by the work of the ECF to improve the calibration quality of archival data on the basis of physical modeling of the instruments. The resultant work has led both to a deeper understanding of the instruments and to data of improved quality. We regard the routine integration of this approach into instrument development at an earlier stage to be desirable. Consideration should be given to developing a resource plan involving the Institute and the ECF that would allow the extension of physical modeling -based calibration studies to the ACS, NICMOS, WFC3, COS and the GHRS.

5. SCIENTIFIC EFFECTIVENESS OF HST

The STUC was delighted to see substantial progress on the project to develop “Science Metrics” to measure the effectiveness of the HST. There are at least three distinct goals for such a study; (i) to document the productivity and impact of the HST for distribution to management and funding agencies and to the general public, (ii) to evaluate the impact of HST in a form that is useful to the astronomical community and (iii) to study the impact of HST programs as a function of proposal parameters, i.e. as a tool for optimizing the TAC process.

Interest remains high in comparing the impact of the HST with that of other facilities. We reiterate that paper counts are a useful means to measure productivity for some purposes (e.g. i), but that citation counts provide the best measure of scientific impact for other purposes (e.g. ii, iii).

Data gathering, which is clearly the most work-intensive part of the project, is now well under way. STScI is now poised to start the most important phase of the study, namely using the data to ask specific questions. We particularly encourage the Institute to pursue this aspect of the work. In doing so, the following points deserve special emphasis:

5.1. Methodology.

1. Citation statistics should be based not only on the papers that publish the primary HST data sets, but also on the subsequent papers that perform the scientific analysis of these data. Frequently, the most important scientific analyses are not contained in the first data paper, whose timing is motivated frequently by a perceived responsibility to make the data available quickly. To omit the follow-up analysis papers could do a significant injustice to the estimated impact of a particular project or of the HST in general.

2. The citation analyses should if possible be based on the full suite of citing papers used in the printed Science Citation Index, or in the Web of Science, and not on the more restricted set of citing papers represented in the online ISI database. It is also important, for any statistic under discussion, to state clearly which of the above databases was used.

3. We are exceedingly interested in the highest impact papers and projects that are already being discussed. But it would also be useful to evaluate the median- and the lowest-impact programs. It is possible that some programs that were awarded time did not succeed in their aim, or that they succeeded but had minimal impact.

5.2. Applications.

4. The results to date show the enormous impact made by the HST on fundamental science. Continuing the comparison of HST with other telescopes is of great interest.

5. A comparison should be made of the absolute impact (no normalization) and the relative impact (normalized by number or orbits) of small, medium and large proposals. This would be an important addition to the relative analyses of these categories in evaluating the TAC process.

However, the results of such analyses should be treated with caution. For example, citation statistics could demonstrate convincingly that past large programs (including Key Projects) have had very high impact, and the results might be used to argue that substantial resources (say, > 1/3 of the telescope time) should continue to be spent on large programs. This reasoning would be flawed if scientific impact is strongly correlated with the maturity of the telescope or instrument. Although the first few large programs (such as deep surveys) were obvious, well justified and destined to have high impact, the 10^th or 20^th such large program might well not have as much impact as the first few. The substantial HST resources that are required to carry out large programs argues for special care in their selection and in the interpretation of metrics that evaluate them.

6. It would be interesting to investigate the correlation between program impact and the grade that its proposal received during the TAC process. We can think of no more direct way to test the effectiveness of the TAC process. When a program produces a large number of papers, it is probably most relevant to correlate citations with proposal grade for the highest-impact one or two papers

The STUC looks forward to reviewing further progress in this study during our next meeting. We are grateful to all the Institute staff involved for their careful hard work on this always-tricky subject.

6. THE FUTURE OF HUBBLE

In our last report we pointed out that a reevaluation of the strategy for maintaining the HST over the next decade would be appropriate. We suggested that serious consideration should be given to the cost effectiveness of taking proactive measures to ensure that the performance of the HST is kept up to modern technological standards, at least until such time as the NGST is likely to be scientifically productive.

Recently two new developments have reinforced the need to examine the cost effectiveness of extending the lifetime of Hubble. First, the present NGST launch date has now slipped until mid-2010. Secondly, an independent study predicts that the chance of HST failure is a strong function of the time following the previous servicing mission. The likelihood of complete failure increases sharply after 3 years. If the last HST servicing mission is in 2004, the probability that the telescope will still be functioning in mid-2010 is only about 25%. Given the uncertainty in predicting launch dates, we regard it as very likely that there will be a substantial gap between the cessation of HST operation and the start of NGST science operations.

We recognize and appreciate that NASA is concerned with providing a balanced scientific program within the limited available resources. However, we believe that there are now several compelling reasons for reevaluating the HST long-term strategy. We therefore urge all those concerned to reconsider this question.

7. NEXT MEETING

The dates of the next STUC meeting will be 21 and 22 October 2002. Possible items for consideration at this meeting include (i) review of the GO funding allocation procedures, (ii) calibration issues for the ACS and other instruments, (iii) the status of the APT, (iv) WFC3 and COS , (v) the science metrics project and (vi) a status report on the SHARE project.