HST Data Handbook for COS

1.2 Obtaining Data via the MAST Web

---

HDA datasets can be searched for, previewed, and retrieved via the World Wide Web. The starting point for Web-based searches of the HDA is the Multimission Archive at STScI (MAST) Web site at:

  http://archive.stsci.edu1

1.2.1 MAST Overview

The MAST home page is shown in Figure 1.1. A powerful feature of MAST is that all of its mission archives, including the HDA, can be searched simultaneously. This is done with the Quick Target Search option shown on the MAST home page. This search will return all datasets for all missions available for a given object or coordinates, according to the search constraints specified by the user (based on the wavelength region of interest), and will provide hypertext links to these datasets. If only HST datasets are desired, they can be accessed separately by clicking on the MAST home page from the "Missions" pull-down menu. Searches of the HDA by object class can also be made with the VizieR Catalog Search tool at:

  http://archive.stsci.edu/vizier.php

The HST section of MAST offers tutorials about the HDA, as well as news and a Frequently Asked Questions page. It also provides links to HST "Prepared" datasets such as the Ultra Deep Field and the Hubble Deep Field images. Clicking on the "Main Search Form" option of the "Search and Retrieval" menu in the HST section brings up the page shown in Figure 1.2. Here the user may query on several search parameters, such as Object Name, Instrument, and Proposal ID. Once these are entered, clicking the "Search" button returns a page listing the datasets found. An example search results page is shown in Figure 1.3. More information about an individual dataset, including a preview (for most datasets), are also available by clicking on the dataset name (Figure 1.4).

Datasets can be selectively marked for retrieval in the search results page. After marking datasets for retrieval, press the button labeled "Submit marked data for retrieval from STDADS". This brings you to the Retrieval Options page, shown in Figure 1.5. Here you may select which files (calibrated, uncalibrated, etc.) you would like retrieved, and where you would like them delivered.

If you are retrieving proprietary data, you will need an archive account. Please refer to Section 1.1.1 for information on how to request one.

Non-proprietary data may be retrieved with or without an archive account. To retrieve non-proprietary data without an archive account, type "anonymous" in the "Archive Username" field, and your email address in the password field. (The email address is needed so that we can notify you when the retrieval request is finished.)

Options for data delivery include direct ftp and sftp, staging, and CD or DVD. If ftp/sftp delivery is specified, you will be need to provide the name of the computer and directory to which the files are to be delivered, as well as your username and password on that computer. (Retrieval requests are encrypted, so there is no danger of your login information being stolen.)

Shortly after submitting the request, you will receive an e-mail message acknowledging its receipt. Another message will be sent after all the requested files have been transferred. The status of the request, including how many files have been transferred and any errors that have occurred, can be checked on a Web page at the address given in the acknowledgment message.

Datasets retrieved to the staging disk using an Archive username and password may be accessed through ftp from archive.stsci.edu using this username and password. (Data that you stage this way will only be visible to you. Therefore, proprietary data as well as non-proprietary data may be safely staged.) Datasets that were retrieved as "anonymous" can be accessed using either an account username and password or through regular anonymous ftp.

Figure 1.1: MAST home page.


 
Figure 1.2: HST Archive web search form.


 
Figure 1.3: HST Archive search results page..


 
Figure 1.4: HST Archive dataset preview page.


 
Figure 1.5: HST retrieval options page.


 

1.2.2 Hubble Legacy Archive

The Hubble Legacy Archive (HLA) is designed to enhance science from the Hubble Space Telescope by augmenting the data collected by the HDA and by providing advanced browsing capabilities. The primary enhancements are:

1. The data is online and available for immediate access.
2. A footprint service makes it easier to browse and download images.
3. More extensive "composite images" (e.g., stacked, color, mosaics) are being developed.
4. The absolute astrometry has been improved from 1 - 2 arcsec to ~0.3 arcsec.
5. Source lists are available for many fields.

The HLA is a joint project of the STScI, the European Coordinating Facility (ST-ECF), and the Canadian Astronomy Data Centre (CADC).

The HLA is in its Data Release 2 (DR2) phase since September 2008. This encompasses enhanced HLA products for most of the ACS and WFPC2 non-proprietary data, and access to MAST products for NICMOS, STIS, FOS, and GHRS. DR2 provides both DAOphot and SExtractor lists for ~70% of ACS images. In addition, user-interface improvements have been added with DR2, including the capability for faster downloads of multiple files, an enhanced plotting tool, GSC2, 2MASS, SDSS, and FIRST catalog overlay, user-defined search lists, and much faster footprints.

Future releases will include "ACS-like-products" (i.e., cosmic-ray rejected, MultiDrizzle-combined) for other existing data such as NICMOS and STIS images, as well as for the new instruments WFC3 and COS. A variety of spectroscopic products are also being considered (e.g., coadded multi-visit spectra, multi-wavelength spliced spectra), and a more advanced user interface (e.g., quick look spectral extractions, line identification lists). Some of the more general goals of the HLA are to make HST data compatible with the Virtual Observatory (VO), to move toward a sky atlas user-view rather than a collection of datasets, and to develop an "all-HST-sky" source list. Figures 1.6, 1.7, and 1.8 show examples of the HLA Web pages.

The HLA can be accessed at: http://hla.stsci.edu.

Figure 1.6: HLA example page with ACS WFC broad band and coadded (color) images.


 
Figure 1.7: STIS Eta Car spectral image and segments of a wavelength calibrated spectrum.


 
Figure 1.8: FOV of all HST instruments. Observation FOVs are overlaid on 30 Doradus.


 

1.2.3 High-Level Science Products

MAST also contains a number of High-Level Science Products (HLSP), which are accessible at http://archive.stsci.edu/hlsp/index.html. High-Level Science Products are fully processed (reduced, coadded, cosmic-ray cleaned, etc.) images and spectra that are ready for scientific analysis. HLSP also include files such as object catalogs, spectral atlases, and README files describing a given set of data. The data originate from the Treasury, Archival Legacy and Large Programs (TALL) from Cycle 11 onward, but contain contributions from smaller HST programs and other MAST missions.

A screen shot of the Web page for the ACS product Galaxy Evolution from Morphology and SEDs (GEMS) is shown in Figure 1.9.

Users who are interested in contributing to the HLSP, are referred to the Guidelines for Contributing High-Level Science Products to MAST (http://archive.stsci.edu/hlsp/hlsp_guidelines.html, please make sure to get the latest version). Furthermore, they are asked to contact the archive scientist involved as soon as they start working on the data.

Figure 1.9: Example High-Level Science Product: GEMS.


 
¹ European archive users should generally use the ST-ECF Archive at http://archive.eso.org. Canadian users should request public archival data through the CADC Web site at http://cadcwww.dao.nrc.ca. Proprietary data are only available through STScI.