Space Telescope Science Institute
Intro to HST Data Handbooks 8.0 May 2011
help@stsci.edu
Table of Contents Previous Next Index Print


Introduction to the HST Data Handbooks > Chapter 1: Obtaining HST Data > 1.2 Obtaining Data via the MAST Web

1.2
HDA datasets can be searched for, previewed, and retrieved via the World Wide Web. The starting point for Web-based searches of the HDA is the Multimission Archive at STScI (MAST) Web site:
http://archive.stsci.edu1
or the Hubble Legacy Archive (described in Section 1.2.2) at:
http://hla.stsci.edu
1.2.1
The MAST home page is shown in Figure 1.1. A powerful feature of MAST is that all of its mission archives, including the HDA, can be searched simultaneously. This is done with the Quick Target Search option shown on the MAST home page. This search will return all datasets for all missions available for a given object or coordinates, according to the search constraints specified by the user (based on the wavelength region of interest), and will provide hypertext links to these datasets. If only HST datasets are desired, they can be accessed separately by clicking on the MAST home page from the “Missions” pull-down menu. Searches of the HDA by object class can also be made with the VizieR Catalog Search tool at:
http://archive.stsci.edu/vizier.php
The HST section of MAST offers tutorials about the HDA, as well as news and a Frequently Asked Questions page. It also provides links to HST “Prepared” datasets such as the Ultra Deep Field and the Hubble Deep Field images. Clicking on the “Main Search Form” option of the “Search and Retrieval” menu in the HST section brings up the page shown in Figure 1.2. Here the user may query on several search parameters, such as Object Name, Instrument, and Proposal ID. Once these are entered, clicking the “Search” button returns a page listing the datasets found. An example search results page is shown in Figure 1.3. More information about an individual dataset, including a preview (for most datasets), are also available by clicking on the dataset name (Figure 1.4).
Datasets can be selectively marked for retrieval in the search results page. After marking datasets for retrieval, press the button labeled “Submit marked data for retrieval from STDADS”. This brings you to the Retrieval Options page, shown in Figure 1.5. Here you may select which files (calibrated, uncalibrated, etc.) you would like retrieved, and where you would like them delivered.
If you are retrieving proprietary data, you will need an archive account. Please refer to Section 1.1.1 for information on how to request one.
Non-proprietary data may be retrieved with or without an archive account. To retrieve non-proprietary data without an archive account, type “anonymous” in the “Archive Username” field, and your email address in the password field. (The email address is needed so that we can notify you when the retrieval request is finished.)
Options for data delivery include direct ftp and sftp, staging, and CD or DVD. If ftp/sftp delivery is specified, you will need to provide the name of the computer and directory to which the files are to be delivered, as well as your username and password on that computer. (Retrieval requests are encrypted, so there is no danger of your login information being stolen.)
Shortly after submitting the request, you will receive an e-mail message acknowledging its receipt. Another message will be sent after all the requested files have been transferred. The status of the request, including how many files have been transferred and any errors that have occurred, can be checked on a Web page at the address given in the acknowledgment message.
Datasets retrieved to the staging disk using an Archive username and password may be accessed through ftp from archive.stsci.edu using this username and password. (Data that you stage this way will only be visible to you. Therefore, proprietary data as well as non-proprietary data may be safely staged.) Datasets that were retrieved as “anonymous” can be accessed using either an account username and password or through regular anonymous ftp.
Figure 1.1: MAST Home Page
Figure 1.2: HST Archive Web Search Form
Figure 1.3: HST Archive Search Results Page
Figure 1.4: HST Archive Dataset Preview Page
Figure 1.5: HST Retrieval Options Page
1.2.2
The Hubble Legacy Archive (HLA) is designed to facilitate access to Hubble Space Telescope data in a form suitable for direct use by a broad variety of science projects. The HLA does this by providing enhanced data products and advanced search and browsing capabilities. The HLA is a joint project of the STScI, the European Coordinating Facility (ST-ECF-Note: ST-ECF ceased operations Jan. 1, 2011; their archives are now served out of ESO/ST-ECF), and the Canadian Astronomy Data Centre (CADC).
At the core of the HLA is a browser-based search and display interface through which users can identify data of interest based on instrument, position on the sky, exposure time, level of processing, availability of catalogs, and other properties of the data. Users can visualize the outline (“footprint”) of all data within a desired region of the sky superimposed on a Digital Sky Survey image, display small previews or cutout views of each dataset, or view the properties of the data offerings in tabular form. Additional selections based on exposure time, date of observation, filter, or other properties are supported. The interface also allows immediate visualization within the browser of the science data at their full resolution, and the data are online and available for immediate download. A shopping cart interface facilitates the selection and retrieval of multiple data sets.
HST datasets are listed in the HLA and are available for visualization within the footprint viewer within approximately one to three weeks after the observations. The advanced HLA data products are added approximately once per year. (A system that will allow much more frequent updates is under development.)
Proprietary data are also listed in the HLA interface, although the datasets cannot be downloaded by the public. Data previews will not be available, but using the footprints viewer the HLA interface may still be useful in order to visualize proprietary observations in the context of other observations of the same sky region.
The offerings of the HLA are rapidly expanding. At the time of this writing (January 2011), the HLA provides enhanced data products for most ACS, NICMOS, and WFPC2 non-proprietary data, including visit-level combined images, and access and immediate display for MAST products for STIS, FOS, and GHRS. Mosaics combining multi-band ACS data from multiple visits are available for nearly 70 pointings, and provide deeper, wider images than can be obtained from individual visits. Extracted spectra are available for most NICMOS and ACS grism data. Source lists for most of the ACS and WFPC2 images have been generated using both DAOPhot and Source Extractor; object positions can be overplotted on the image display, together with sources from external lists such as SDSS, GSC2, 2MASS, GALEX, and FIRST. In addition, over a dozen sets of user-provided high-level science products (HLSP) can be accessed through the same search and display interface; this includes the results of many of the HST Treasury and public programs, such as HDF, UDF, COSMOS, GOODS, GEMS, ANGST, and several others. These are a subset of the HLSP described in Section 1.2.3.
Extensive descriptions of the HLA products and their properties are available through the HLA Help Center. The Release Notes summarize the evolution of HLA products over time. Figures 1.2.3, 1.7, and 1.8 show examples of the HLA Web pages. The HLA main page can be accessed at:
http://hla.stsci.edu.
Figure 1.6: HLA Example Page With ACS WFC Broad Band and Coadded (Color) Images
Figure 1.7: STIS Eta Car Spectral Image and Segments of a Wavelength Calibrated Spectrum
Figure 1.8: HLA Footprint view showing HST Instrument. FOVs Overlaid on 30 Doradus.
The recent “Data Release 5” includes new and expanded data products and significant interface enhancements. Products planned for release in DR5 or soon thereafter include combined WFC3 images, prototype COS and STIS products, and a greatly expanded number of multi-visit ACS mosaics. A new footprint display allows panning and zooming and is more fully integrated with the rest of the interface. Other improvements under consideration for future releases include expanding the generation of source lists and mosaics to additional instruments, more spectroscopic products (e.g., coadded multi-visit spectra, multi-wavelength spliced spectra), and a more advanced user interface for spectra (e.g., quick look spectral extractions, line identification lists). Some of the more general goals of the HLA are to make more HST data available through the Virtual Observatory (VO), to move toward a sky atlas user-view rather than a collection of datasets, and to develop an “all-HST-sky” source list.
The HLA is designed to be complementary to the MAST interface for access to HST data. Users needing access to known data sets, or those who plan to run independent data reduction and image combination, will often find it more convenient to use the MAST interface to access and retrieve the data of interest. Those who need access to proprietary data may also prefer to use the MAST interface. On the other hand, users who wish to search for data covering a region of interest and are interested in a graphical display of their relative position, or users who may be interested in quick-look capabilities, science-ready combined data, and/or source listings, will generally find the HLA offerings to be more convenient and more directly accessible.
1.2.3
MAST also contains a number of user-contributed, High-Level Science Products (HLSP), which are accessible at http://archive.stsci.edu/hlsp/index.html. High-Level Science Products are generally fully processed (reduced, coadded, cosmic-ray cleaned, etc.) images and spectra that are ready for scientific analysis. HLSP also include files such as object catalogs, spectral atlases, and README files describing a given set of data. Most of the data originate from the Treasury, Archival Legacy and Large Programs (TALL) from Cycle 11 onward, but some contributions from smaller HST programs and other MAST missions are also included.
A screen shot of the Web page for the ACS product Galaxy Evolution from Morphology and SEDs (GEMS) is shown in Figure 1.9.
Users who are interested in contributing to the HLSP, are referred to the Guidelines for Contributing High-Level Science Products to MAST (http://archive.stsci.edu/hlsp/hlsp_guidelines.html, please make sure to get the latest version). Furthermore, they are asked to contact the archive scientist involved as soon as they start working on the data.
Figure 1.9: Example High-Level Science Product: GEMS

1
European archive users should generally use the ESO/ST-ECF Archive at http://archive.eso.org/cms. Canadian users should request public archival data through the CADC Web site at http://cadcwww.dao.nrc.ca. Proprietary data are only available through STScI.


Table of Contents Previous Next Index Print