About this Article

Introduction

There's no way around it, the community-workhorse Image Reduction and Analysis Facility (IRAF) is getting on in years. It has served astronomy for three productive and fruitful decades and is appreciated by many. But as with many things in the software realm, the landscape has changed significantly since the inception of IRAF. Most modern astronomy analysis tools are built in languages like Python, IDL, and C/C++. As the tide has turned towards these newer languages, IRAF has become more and more difficult to build and maintain on current 64-bit architectures. A large portion of the IRAF tasks cannot be compiled as a 64-bit executable, and must be built as a 32-bit program. For these reasons the Space Telescope Science Institute (STScI) has been working towards IRAF independence for all our instrumentation and calibration work. This effort has included the development of transition resources, re-writes of older IRAF scripts, and some additions to Astropy (the current community-supported Python Astronomy package) when needed. If you are interested in transitioning from IRAF, this article is for you.

De-IRAFing the institute was made possible by both cross-divisional communications and effort within the institute, and making extensive use of GitHub. To ensure a smooth transition, the Data Analysis Tools Branch worked closely with the Instruments Division, the HST Mission Office and the Data Science Mission Office to gather the needs and requirements of STScI staff, as well as feedback and testing through the development of new tools.  GitHub was an effective forum to track the work being done, as well as feedback and reviews from internal users. In particular, GitHub became indispensible for the project when communication was needed with Astropy on existing community tools, and for having Instruments Division staff review new transition content.

The STAK Notebooks

If you browse through Astropy or the IDL Astronomy User's Library you'll notice that some of the most used IRAF functions have been replicated in Python and IDL. But sometimes it can be a bit daunting to put together a set of unfamiliar commands when switching to a new language. We recognize this, and to address it have developed a translation resource for both the STScI and the external community. This is the essence of the STAK Notebooks: a collection of Jupyter Notebooks available to download or as an online reference. 

Jupyter notebooks are the ideal tool for this type of resource. A Jupyter notebook is a web application that can contain live code, visualizations, and narrative text. If the individual notebook is opened locally, the user can run individual sections of code, edit, and save their changes. The notebooks also translate nicely to HTML so that they can be posted as online documentation. For a broader introduction to Jupyter notebooks see their quick-start guide.

The STAK notebooks are organized by IRAF module, with each section showing a Python example of a particular IRAF task. We attempted to cover some of the most commonly used IRAF tasks, including images.imutil, tables.ttools, and images.imfilter. Figure 1 shows an example of the online documentation for the IRAF task tables.fitsio.catfits. You’ll see that there are several boxes containing code for each task. The imports will always be in the top, followed by the Python code used to provide the task functionality. If this notebook were to be downloaded locally, the code segments could be run interactively. There are 19 sub-packages in total currently covered in the STAK notebooks. We have also included some introductory materials for new Python users. In addition to the STAK notebooks, we are also migrating a handful of STIS post-pipeline user tools to the Python stistools library.

We highly encourage community users to contribute to these transition notebooks! This is an open-source project and we will happily accept pull requests. If you have found a good Python replacement for an IRAF task you commonly use, it would be of great value to the community to include it in this resource. 

Future Developments

With the transition from IRAF to a more mainstream programming language (Python) it is important to realize that there has also been a major change in workflow. The traditional IRAF workflow consists of individual tasks that take a FITS file as input, run a data processing algorithm on the input file, and return an updated output file for the user. However, in Python data manipulation is more a flow of operations expressed in regular code than individual tasks that operate on files. You can read a bit more about this in the Introduction section of the STAK notebooks, along with an introduction to FITS I/O. 

While the STAK Notebook effort was primarily to remove IRAF dependencies in the internal STScI workflow, within the next few years, STScI will slowly phase out active support of our IRAF/PyRAF-based software as well. We will continue the distribution of the Astroconda Python 2.7 environment which includes PyRAF, but this will be frozen and there will be no updates for dependencies, bug fixes, and user support.

The end of IRAF support in the community is a challenge best overcome by teamwork. Contributions from the community can be made in various ways. As stated above, additions to the STAK notebooks would be very welcome, but there are other ways to help. Astropy has been the hub of astronomy-tool development in Python for quite a few years now, but there is always more work to be done. The perspective of newer Astropy users is especially valuable, and specific feedback can be communicated through issues on the GitHub repository.  Even more helpful are contributions (via GitHub pull requests) from the community, even if it's a small change to make the documentation more clear. There is currently active development on tools for spectroscopy, photometry, and world-coordinate systems underway within the Astropy community, and now is a good time to test early releases and request features or suggest changes.

Acknowledgements

We would like to thank all the staff members of the institute who participated in this effort and provided feedback on the STAK notebooks. This project could not have been a success without the teamwork shown by the institute staff. We would also like to thank Cristina Oliveira for acting as our INS liaison during this project.