Using Astroquery to download HEASARC observation data#
Learning Goals#
This notebook will teach you:
How to retrieve HEASARC ‘master’ catalogs, which summarize all observations taken by a particular telescope.
How to filter an observation summary table to find relevant observations for a single source, based on how close the observation was to the source.
How to download the data files associated with those relevant observations.
Introduction#
This bite-sized tutorial will show you how to find and retrieve observation data files from HEASARC using Astroquery.
HEASARC hosts a large number of catalogs; the vast majority relate to the properties and/or locations of astrophysical sources and have been produced by scientists during their research. A small subset of the catalogs served by HEASARC are ‘master’ (or ‘observation summary’) catalogs, which act as the primary index of every observation taken by a particular telescope.
We will filter an ‘observation summary’ catalog to find ‘relevant’ observations and then show you how to acquire those data files.
Runtime#
As of 17th July 2026, this notebook takes ~5-minutes to run to completion on Fornax using the ‘small’ server with 8GB RAM/ 2 cores.
Please note that this runtime is heavily dependent on archive servers, and the speed of your internet connection, which means runtime may vary.
Imports#
import glob
import os
from astropy.coordinates import SkyCoord
from astropy.units import Quantity
from astroquery.heasarc import Heasarc
1. Listing HEASARC’s master catalogs#
We assume that you already have a basic understanding of HEASARC’s Astroquery interface and how you search for HEASARC catalogs – if not, please see the ‘Find specific HEASARC catalogs using Python’ bite-sized tutorial.
To list the ‘observation summary’ catalogs (see the introduction) hosted by HEASARC, we run:
all_obs_cat = Heasarc.list_catalogs(master=True)
To examine the contents of the returned table (see the
‘Exploring the contents of HEASARC catalogs using Python’
tutorial, for an explanation of why we use pprint_all()) we can run:
# The 'pprint' stands for 'pretty print'
all_obs_cat.pprint_all()
name description
---------- -------------------------------------------------------------
ascamaster ASCA Master Catalog
burcbmastr BurstCube Master Observation Catalog
chanmaster Chandra Observations
cmbmaster LAMBDA Cosmic Microwave Background Experiments Master Catalog
erassmastr eROSITA All-Sky Survey Master Catalog
erosmaster eROSITA Observations Master Catalog
euvemaster EUVE Archive and Observation Log
exomaster EXOSAT Master Observation List
halomaster HaloSat Master Catalog
hitomaster Hitomi Master Catalog
ixmaster IXPE Master Catalog
maximaster MAXI Master Catalog
nicermastr NICER Master Catalog
numaster NuSTAR Master Catalog
rassmaster ROSAT All-Sky Survey Archival Data
rosmaster ROSAT Archival Data
spitzmastr Spitzer Space Telescope Observation Log
suzamaster Suzaku Master Catalog
swiftmastr Swift Master Catalog
uitmaster Ultraviolet Imaging Telescope Master Catalog
verimaster VERITAS Source Catalog
xlcalmastr XL-Calibur Master Catalog
xmmmaster XMM-Newton Master Log & Public Archive
xrismmastr XRISM Master Catalog
xtemaster XTE Master Catalog
2. Choosing an observation summary catalog#
We can pick out the name of the observation summary catalog we want by examining the name and
description columns in the table above, or we can use a keyword search to find a more specific
match. See the ‘Find specific HEASARC catalogs using Python’
tutorial for an explanation of keyword searches):
filt_obs_cat = Heasarc.list_catalogs(keywords="suzaku", master=True)
filt_obs_cat
| name | description |
|---|---|
| str10 | str21 |
| suzamaster | Suzaku Master Catalog |
Then extract the name of the catalog:
# The filtered table has one row (accessed using [0])
obs_cat_name = filt_obs_cat[0]["name"]
obs_cat_name
np.str_('suzamaster')
3. Filtering observations by distance from a source#
One of the most common ways to select observations relevant to your science goal is to require the nominal central coordinate of the pointing (or ‘sky tile’, for all-sky surveys) to be within some radius of the source of interest.
The specific matching radius you choose will depend on:
The mission you need observations from - every mission has a different field of view (FoV).
Which instrument you are most interested in (different instruments on the same mission will oftentimes have different FoVs).
Whether the instrument’s FoV is circular, square, or rectangular (Chandra’s ACIS-S, for instance, is often used in a rectangular configuration that is much longer than it is wide).
Your source and science goal – a low-redshift galaxy cluster, for instance, might motivate a larger matching radius as the whole source may not fit within the instrument’s FoV.
If you only want observations where your source is near the center of the field, where many high-energy telescopes are the most sensitive and have the smallest point spread function (PSF).
Each of HEASARC’s observation summary catalogs has a default search radius, which can be found using:
default_search_rad = Heasarc.get_default_radius(obs_cat_name)
default_search_rad
We can also define our own search radius – in this instance let’s assume we only want to select observations that have our source in the very center of the FoV:
custom_search_rad = Quantity(3, "arcmin")
custom_search_rad
Speaking of sources, for this demonstration we’re going to search for Suzaku observations of PDS 456, a nearby radio-quiet quasar:
# Blank space can be included in names passed to a name resolver
source_name = "PDS 456"
A string source name variable can be passed directly into the search function we’re about to use, which will then use a name resolver to fetch the coordinate. Alternatively, we can trigger a name resolver ourselves and pass the coordinate in to the search function:
source_coord = SkyCoord.from_name(source_name)
source_coord
<SkyCoord (ICRS): (ra, dec) in deg
(262.08245562, -14.26551526)>
You should always treat the output coordinates of a name resolver with a little caution. They are likely to be very reliable for point-like sources, but for extended sources in particular there is often not a single ‘correct’ position definition, and you will want to check exactly what coordinate has been returned.
In fact, you might want to define your own coordinate directly:
# The same coordinate we got from the name lookup, but demonstrates how to
# manually define a coordinate object.
manual_source_coord = SkyCoord(262.0825, -14.2655, unit="deg")
manual_source_coord
<SkyCoord (ICRS): (ra, dec) in deg
(262.0825, -14.2655)>
Finally, we can run the query that will filter our table of observations:
source_obs_res = Heasarc.query_region(
position=source_name, catalog=obs_cat_name, radius=custom_search_rad
)
source_obs_res
| name | ra | dec | time | obsid | exposure | processing_date | public_date | __row |
|---|---|---|---|---|---|---|---|---|
| deg | deg | d | s | d | d | |||
| object | float64 | float64 | float64 | object | float64 | float64 | float64 | object |
| PDS 456 | 262.08050 | -14.26170 | 56354.821597222224 | 707035020 | 164810.70000 | 57610.834282407406 | 56018.0 | 1283 |
| PDS 456 | 262.08060 | -14.26170 | 56359.50015046296 | 707035030 | 108339.70000 | 57610.79914351852 | 56018.0 | 1284 |
| PDS 456 | 262.08020 | -14.26100 | 56344.89074074074 | 707035010 | 182299.00000 | 57610.75497685185 | 56018.0 | 1285 |
| PDS 456 | 262.08070 | -14.26040 | 54155.74865740741 | 701056010 | 190600.20000 | 57537.58505787037 | 54757.0 | 1286 |
| PDS 456 | 262.08090 | -14.25370 | 55636.62546296296 | 705041010 | 125527.20000 | 57601.286261574074 | 56050.0 | 1287 |
Note
We could have passed position=source_coord or position=manual_source_coord to the
above query, as we defined those coordinates in the discussion above.
4. Downloading observation data files#
Now that we’ve identified some observations that are relevant to our source of interest (see the end of Section 3), we can move on to downloading their data files.
The first step is to pass the return from our Heasarc.query_region(...) call (an Astropy Table
assigned to source_obs_res) and pass it to the locate_data(...) method of Heasarc. This function
will construct a table of ‘datalinks’, which describe where the relevant observation data are
actually stored, and provide us an easy way of accessing them:
source_obs_datalinks = Heasarc.locate_data(source_obs_res)
source_obs_datalinks
| ID | access_url | sciserver | aws | content_length | error_message |
|---|---|---|---|---|---|
| byte | |||||
| object | object | str33 | str46 | int64 | object |
| ivo://nasa.heasarc/suzamaster?1283 | https://heasarc.gsfc.nasa.gov/FTP/suzaku/data/obs/7//707035020/ | /FTP/suzaku/data/obs/7/707035020/ | s3://nasa-heasarc/suzaku/data/obs/7/707035020/ | 5196242980 | |
| ivo://nasa.heasarc/suzamaster?1284 | https://heasarc.gsfc.nasa.gov/FTP/suzaku/data/obs/7//707035030/ | /FTP/suzaku/data/obs/7/707035030/ | s3://nasa-heasarc/suzaku/data/obs/7/707035030/ | 2688162469 | |
| ivo://nasa.heasarc/suzamaster?1285 | https://heasarc.gsfc.nasa.gov/FTP/suzaku/data/obs/7//707035010/ | /FTP/suzaku/data/obs/7/707035010/ | s3://nasa-heasarc/suzaku/data/obs/7/707035010/ | 5097270881 | |
| ivo://nasa.heasarc/suzamaster?1286 | https://heasarc.gsfc.nasa.gov/FTP/suzaku/data/obs/7//701056010/ | /FTP/suzaku/data/obs/7/701056010/ | s3://nasa-heasarc/suzaku/data/obs/7/701056010/ | 4423025493 | |
| ivo://nasa.heasarc/suzamaster?1287 | https://heasarc.gsfc.nasa.gov/FTP/suzaku/data/obs/7//705041010/ | /FTP/suzaku/data/obs/7/705041010/ | s3://nasa-heasarc/suzaku/data/obs/7/705041010/ | 3790196712 |
The table has several columns, including:
ID – A unique International Virtual Observatory (IVO) ID for the data.
access_url – A URL to one of the locations the data are stored, HEASARC’s FTP server.
sciserver – Path to the data if working on SciServer (see the HEASARC@SciServer user guide).
aws – A URI that points to where the data are stored in the HEASARC Amazon Web Services (AWS) S3 bucket (see the registry of open data on AWS).
This means that when we come to download the data, we have a choice of where to download it from. Unless you are working on SciServer, we generally recommend pulling data from our S3 bucket.
Now we set up a new directory using os.makedirs(...) (the exist_ok=True argument ensures that no
error is raised if that directory already exists), and start the download.
We pass our datalink table (source_obs_datalinks), tell the function to download from the
HEASARC S3 bucket (host='aws'), and make sure the downloaded files are placed in the
directory specified by download_dir (they would be placed in your current directory if
you didn’t pass anything to the location= argument):
# Define download path, and create the directories
download_dir = f"heasarc_data/{obs_cat_name}"
os.makedirs(download_dir, exist_ok=True)
# Triggers the download
Heasarc.download_data(links=source_obs_datalinks, host="aws", location=download_dir)
INFO: Downloading data AWS S3 ... [astroquery.heasarc.core]
INFO: Enabling anonymous cloud data access ... [astroquery.heasarc.core]
INFO: downloading s3://nasa-heasarc/suzaku/data/obs/7/707035020/ [astroquery.heasarc.core]
INFO: downloading s3://nasa-heasarc/suzaku/data/obs/7/707035030/ [astroquery.heasarc.core]
INFO: downloading s3://nasa-heasarc/suzaku/data/obs/7/707035010/ [astroquery.heasarc.core]
INFO: downloading s3://nasa-heasarc/suzaku/data/obs/7/701056010/ [astroquery.heasarc.core]
INFO: downloading s3://nasa-heasarc/suzaku/data/obs/7/705041010/ [astroquery.heasarc.core]
INFO: Enabling anonymous cloud data access ... [astroquery.heasarc.core]
INFO: downloading s3://nasa-heasarc/suzaku/data/obs/7/707035020/ [astroquery.heasarc.core]
INFO: downloading s3://nasa-heasarc/suzaku/data/obs/7/707035030/ [astroquery.heasarc.core]
INFO: downloading s3://nasa-heasarc/suzaku/data/obs/7/707035010/ [astroquery.heasarc.core]
INFO: downloading s3://nasa-heasarc/suzaku/data/obs/7/701056010/ [astroquery.heasarc.core]
INFO: downloading s3://nasa-heasarc/suzaku/data/obs/7/705041010/ [astroquery.heasarc.core]
Caution
If the specified data files already exist in your download_dir, then this process will
overwrite them.
Finally, we can take a look at the contents of the download directory:
os.listdir(download_dir)
['707035020', '707035030', '707035010', '701056010', '705041010']
As well as the contents of one of the observation directories:
glob.glob(os.path.join(download_dir, "707035020") + "**/*")
['heasarc_data/suzamaster/707035020/auxil',
'heasarc_data/suzamaster/707035020/hxd',
'heasarc_data/suzamaster/707035020/log',
'heasarc_data/suzamaster/707035020/xis']
Then a specific instrument directory:
glob.glob(os.path.join(download_dir, "707035020", "xis") + "**/*")
['heasarc_data/suzamaster/707035020/xis/event_cl',
'heasarc_data/suzamaster/707035020/xis/event_uf',
'heasarc_data/suzamaster/707035020/xis/hk',
'heasarc_data/suzamaster/707035020/xis/products']
About this notebook#
Author: David Turner, HEASARC Staff Scientist
Updated On: 2026-07-16
Additional Resources#
Support: HEASARC Helpdesk
Latest Astroquery Documentation
We provide several bite-sized tutorials on accessing HEASARC catalogs using Python and Astroquery:
To learn how to use Python and Astroquery to search for a particular HEASARC catalog, please see the ‘Find specific HEASARC catalogs using Python’ tutorial.
To learn how to use Python and Astroquery to retrieve and explore the contents of HEASARC catalogs, please see the ‘Exploring the contents of HEASARC catalogs using Python’ tutorial.
To learn how to use Python and Astroquery to cross-match a local catalog (either locally on-disk or loaded into local memory) to a catalog hosted by HEASARC, please see the ‘Cross-matching local and HEASARC catalogs using Python’ tutorial.
Acknowledgements#
References#
Ginsburg, Sipőcz, Brasseur et al. (2019) – astroquery: An Astronomical Web-querying Package in Python