ESA EUCLID Archive (astroquery.esa.euclid)

Euclid is an ESA mission to map the geometry of the dark Universe. The mission investigates the distance-redshift relationship and the evolution of cosmic structures. The space telescope creates a great map of the large-scale structure of the Universe across space and time by observing billions of galaxies out to 10 billion light-years, across more than a third of the sky. It achieves this by measuring shapes and redshifts of galaxies and clusters of galaxies out to redshifts ~2, or equivalently to a look-back time of 10 billion years. It therefore explores how the Universe has expanded and how structure has formed over cosmic history, revealing more about the role of gravity and the nature of dark energy and dark matter.

This package allows access to the European Space Agency Euclid Archive (https://eas.esac.esa.int/).

The Euclid Survey is done in a ‘step-and-stare’ mode, where the telescope points to a position on the sky and then imaging and spectroscopic measurements are performed on an area of ~0.48 deg2 around this position. The telescope consists of two cameras, the visible instrument (VIS) and the Near Infrared Spectrometer and Photometer (NISP) instrument that observe simultaneously using a light splitting dichroic.

For the survey standard operating mode, the telescope undertakes a 4-point dither pattern. At each position VIS and NISP each take a 560s exposure, consisting of a direct visible image and a red grism exposure. This is followed by further NISP exposures in the Y, J, and H band filters (87 seconds each). The telescope is then dithered, and the sequence is repeated starting with a different grism position angle. There are actually two operational grisms oriented 180 degrees from each other. Each grism will be used twice in this sequence, but with slight angular offsets (+/- 4 degrees), effectively creating the four different grism angles (Scaramella et al. 2022, A&A 662, A112).

This standard four-dithers operating mode sequence is called a single observation and all the individual exposures associated with each observation are organized by Observation ID in the archive. The SGS also processes all of its imaging into merged mosaics, which can contain multiple different observations. All products associated with these mosaics are organized by Tile ID in the archive.

astroquery.esa.euclid provides the astroquery interface to the metadata and datasets provided by the European Space Agency EUCLID Archive using a TAP+ REST service. TAP+ is an extension of Table Access Protocol (TAP: http://www.ivoa.net/documents/TAP/) specified by the International Virtual Observatory Alliance (IVOA: http://www.ivoa.net).

The TAP query language is Astronomical Data Query Language (ADQL: https://www.ivoa.net/documents/ADQL/20231215/index.html ), which is similar to Structured Query Language (SQL), widely used to query databases.

TAP provides two operation modes: Synchronous and Asynchronous:

  • Synchronous: the response to the request will be generated as soon as the request received by the server. (Do not use this method for queries that generate a large amount of results.)

  • Asynchronous: the server starts a job that will execute the request. The first response to the request is the required information (a link) to obtain the job status. Once the job is finished, the results can be retrieved.

ESA EUCLID TAP+ server provides two access modes: public and authenticated:

  • Public: this is the standard TAP access. A user can execute ADQL queries and upload tables to be used in a query ‘on-the-fly’ (these tables will be removed once the query is executed). The results are available to any other user, and they will remain in the server for a limited space of time.

  • Authenticated: some functionalities are restricted to authenticated users only. The results are saved in a private user space and they will remain in the server forever (they can be removed by the user).

    • ADQL queries and results are saved in a user private area.

    • Persistence of uploaded tables: a user can upload a table in a private space. These tables can be used in queries as well as in cross-matches operations.

If you use public Euclid data in your paper, please take note of our guide on how to acknowledge and cite Euclid data.

This python module provides an Astroquery API access.

Examples

It is highly recommended checking the status of Euclid TAP before executing this module. To do this:

>>> from astroquery.esa.euclid import Euclid
>>> Euclid.get_status_messages()

This method will retrieve the same warning messages shown in EUCLID Science Archive with information about service degradation.

0. Euclid science archive systems

The Euclid Science Archive has several environments serving different purposes for the Euclid Consortium members.

1. The OTF (“on-the-fly”) environment of the Euclid science archive, first started at the start of science operation exposed data as processed by the SGS (Science Ground Segment) soon after acquisition to provide an access as soon as possible. In this environment the data will not be reprocessed and the processing is therefore heterogeneous.

2. The REG (for non-regression testing) environment of the Euclid science archive, where a large area in the sky is processed with the same version for all data products. The first campaign was run in September 2024, for area of about 500 square degrees (~1000 observations), the next campaign shall be run in March-April 2025.

3. The IDR (Internal Data Release) environment of the Euclid science archive holds the data that will then become public. The first release Q1 opened on the 6th of November 2024, with a first pass on the three Euclid deep fields (EDFN, EDFS and EDFF) as well as observations on the Lynds Dark Nebula LDN1641.

4. The PDR (Public Data Release) environment of the Euclid science archive holds the public data. Euclid Q1 data was publicly released on March 19, 2025. The main component of the Q1 data contains Level 2 data of a single visit (at the depth of the Euclid Wide Survey) over the Euclid Deep Fields (EDFs): 20 deg2 of the EDF North, 10 deg2 of EDF Fornax, and 23 deg2 of the EDF South. The deep fields will be visited multiple times during the mission.

The description of these data products can be found on the Data Product Definition Document (DPDD)

By default, the object Euclid

>>> from astroquery.esa.euclid import Euclid

makes use of the PDR environment. In order to make use of a different one, it is necessary to instantiate the class EuclidClass

>>> from astroquery.esa.euclid import EuclidClass
>>> euclid = EuclidClass(environment='IDR')

The parameter environment is limited to IDR, OTF, PDR or REG.

1. Non authenticated access

1.1. Getting public tables metadata

Table and column metadata are specified by IVOA TAP recommendation (to access to the actual data, an ADQL query must be executed).

To load only table names metadata (TAP+ capability):

>>> from astroquery.esa.euclid import Euclid
>>> tables = Euclid.load_tables(only_names=True, include_shared_tables=True)
INFO: Retrieving tables... [astroquery.utils.tap.core]
INFO: Parsing tables... [astroquery.utils.tap.core]
INFO: Done. [astroquery.utils.tap.core]
>>> print("Found", len(tables), "tables")
Found 34 tables
>>> print(*(table.name for table in tables), sep="\n")
ivoa.obscore
public.dual
sedm.raw_detector
sedm.raw_frame
sedm.raw_quadrant
sedm.aux_calibrated
sedm.aux_mosaic
sedm.aux_stacked
sedm.basic_download_data
sedm.calibrated_detectors
sedm.calibrated_frame
sedm.column_values
sedm.combined_spectra
...
tap_config.coord_sys
tap_config.properties
...

To load all table metadata (TAP compatible):

>>> from astroquery.esa.euclid import Euclid
>>> tables = Euclid.load_tables()
INFO: Retrieving tables... [astroquery.utils.tap.core]
INFO: Parsing tables... [astroquery.utils.tap.core]
INFO: Done. [astroquery.utils.tap.core]
>>> print(tables[0])
TAP Table name: ivoa.obscore
Description: None
Size (bytes): 0
Num. columns: 34

To load only a table (TAP+ capability) and inspect its columns:

>>> from astroquery.esa.euclid import Euclid
>>> raw_detector_table = Euclid.load_table('sedm.raw_detector')
>>> print(raw_detector_table)
TAP Table name: sedm.raw_detector
Description: None
Size (bytes): 0
Num. columns: 12
>>> print(*(column.name for column in raw_detector_table.columns), sep="\n")
crpix1
crpix2
crval1
crval2
detector_id
detector_oid
l1_raw_frame_oid
q1_oid
q2_oid
q3_oid
q4_oid
to_be_published

To get the list of products associated with a given Euclid observation_id or tile_index (for mosaic):

>>> from astroquery.esa.euclid import Euclid
>>> product_list_results = Euclid.get_product_list(tile_index="102018211", product_type="DpdMerBksMosaic")
>>> print("Found", len(product_list_results), "results")
Found 12 results
>>> print(product_list_results)
                                    file_name                                      mosaic_product_oid tile_index instrument_name filter_name category second_type     ra       dec   technique
                                      str255                                             int64          int64         str255        str255    str255     str255    float64   float64   str255
---------------------------------------------------------------------------------- ------------------ ---------- --------------- ----------- -------- ----------- ---------- ------- ---------
EUC_MER_BGSUB-MOSAIC-DES-I_TILE102018211-31E2C9_20241018T143048.358037Z_00.00.fits               1399  102018211           DECAM     DECAM_i  SCIENCE         SKY 57.9990741   -51.5     IMAGE
  EUC_MER_BGSUB-MOSAIC-VIS_TILE102018211-ACBD03_20241018T142710.276838Z_00.00.fits               1395  102018211             VIS         VIS  SCIENCE         SKY 57.9990741   -51.5     IMAGE
EUC_MER_BGSUB-MOSAIC-DES-G_TILE102018211-D9D163_20241018T143010.768685Z_00.00.fits               1394  102018211           DECAM     DECAM_g  SCIENCE         SKY 57.9990741   -51.5     IMAGE
EUC_MER_BGSUB-MOSAIC-NIR-H_TILE102018211-42F1AD_20241018T142558.469987Z_00.00.fits               1396  102018211            NISP       NIR_H  SCIENCE         SKY 57.9990741   -51.5     IMAGE
EUC_MER_BGSUB-MOSAIC-NIR-J_TILE102018211-E044A1_20241018T142600.459089Z_00.00.fits               1393  102018211            NISP       NIR_J  SCIENCE         SKY 57.9990741   -51.5     IMAGE
EUC_MER_BGSUB-MOSAIC-NIR-Y_TILE102018211-E5CAE1_20241018T142558.172837Z_00.00.fits               1397  102018211            NISP       NIR_Y  SCIENCE         SKY 57.9990741   -51.5     IMAGE
EUC_MER_BGSUB-MOSAIC-DES-R_TILE102018211-1078B7_20241018T142927.232351Z_00.00.fits               1401  102018211           DECAM     DECAM_r  SCIENCE         SKY 57.9990741   -51.5     IMAGE
EUC_MER_BGSUB-MOSAIC-DES-Z_TILE102018211-83C32F_20241018T143526.104818Z_00.00.fits               1398  102018211           DECAM     DECAM_z  SCIENCE         SKY 57.9990741   -51.5     IMAGE

The method returns a list of products as an astropy.table.

1.3. Query object

This query searches for all the objects contained in an arbitrary rectangular projection of the sky.

WARNING: This method implements the ADQL BOX function that is deprecated in the latest version of the standard (ADQL 2.1, see: https://ivoa.net/documents/ADQL/20231107/PR-ADQL-2.1-20231107.html#tth_sEc4.2.9).

The following example searches for all the sources contained in an squared region of side = 0.1 degrees around a specific point in ra/dec coordinates. The results are sorted by distance (dist) in ascending order.

The method returns the job results as astropy.table

>>> # Search for objects around a given position with the default catalog catalogue.mer_catalogue
>>> from astroquery.esa.euclid import Euclid
>>> from astropy.coordinates import SkyCoord
>>> import astropy.units as u
>>> coord = SkyCoord(ra=60.3372780005097, dec=-49.93184727724773, unit=(u.degree, u.degree), frame='icrs')
>>> table = Euclid.query_object(coordinate=coord, width=u.Quantity(0.1, u.deg), height= u.Quantity(0.1, u.deg))
>>> print("Found a total of", len(table), "query results")
Found a total of 2000 query results
>>> print(table)
         dist         avg_trans_wave_g_ext_decam avg_trans_wave_g_ext_hsc avg_trans_wave_g_ext_jpcam avg_trans_wave_g_ext_lsst avg_trans_wave_h avg_trans_wave_i_ext_decam ... sersic_fract_z_ext_panstarrs_disk_sersic sersic_fract_z_ext_panstarrs_disk_sersic_err she_flag spurious_flag     spurious_prob      variable_flag vis_det
--------------------- -------------------------- ------------------------ -------------------------- ------------------------- ---------------- -------------------------- ... ---------------------------------------- -------------------------------------------- -------- ------------- ---------------------- ------------- -------
3.566798805594703e-06            4826.7998046875                       --                         --                        --               --             7826.669921875 ...                                       --                                           --       --             0    0.15743961930274963            --       1
0.0004459918667892947            4826.7998046875                       --                         --                        --               --             7826.669921875 ...                                       --                                           --       --             0   0.004427384119480848            --       1
0.0011813971416470212            4826.7998046875                       --                         --                        --               --             7826.669921875 ...                                       --                                           --       --             0     0.1833316683769226            --       1
0.0015542789169486976            4826.7998046875                       --                         --                        --               --             7826.669921875 ...                                       --                                           --       --             0    0.12239421904087067            --       0
0.0015885047273778879            4826.7998046875                       --                         --                        --               --             7826.669921875 ...                                       --                                           --       --             0 0.00021384040883276612            --       1
                  ...                        ...                      ...                        ...                       ...              ...                        ... ...                                      ...                                          ...      ...           ...                    ...           ...     ...
  0.03958455791235079            4826.7998046875                       --                         --                        --               --             7826.669921875 ...                                       --                                           --       --             0     0.1343534141778946            --       0
  0.03958823626200475            4826.7998046875                       --                         --                        --               --             7826.669921875 ...                                       --                                           --       --             0 0.00015592691488564014            --       1
  0.03959898295410331            4826.7998046875                       --                         --                        --               --             7826.669921875 ...                                       --                                           --       --             0     0.0365927591919899            --       1
 0.039605684988334174            4826.7998046875                       --                         --                        --               --             7826.669921875 ...                                       --                                           --       --             0     0.0831669270992279            --       0
  0.03960602180308949            4826.7998046875                       --                         --                        --               --             7826.669921875 ...                                       --                                           --       --             1     0.6376287937164307            --       1
 0.039606556762811496            4826.7998046875                       --                         --                        --               --             7826.669921875 ...                                       --                                           --       --             0   0.012528697960078716            --       1
  0.03962541836711639            4826.7998046875                       --                         --                        --               --             7826.669921875 ...                                       --                                           --       --             0   0.003399776993319392            --       1
Length = 2000 rows

Synchronous queries like this one return a limited number of rows -> 2000

The previous query can be executed as an asynchronous version:

>>> from astroquery.esa.euclid import Euclid
>>> from astropy.coordinates import SkyCoord
>>> import astropy.units as u
>>> coord = SkyCoord(ra=60.3372780005097, dec=-49.93184727724773, unit=(u.degree, u.degree), frame='icrs')
>>> width=u.Quantity(0.1, u.deg)
>>> height= u.Quantity(0.1, u.deg)
>>> table_async = Euclid.query_object(coordinate=coord, width=width, height=height, async_job=True)
INFO: Query finished. [astroquery.utils.tap.core]
>>> print("Found a total of", len(table_async), "query results")
Found a total of 2895 query results
>>> print(table_async)
         dist         avg_trans_wave_g_ext_decam avg_trans_wave_g_ext_hsc avg_trans_wave_g_ext_jpcam avg_trans_wave_g_ext_lsst avg_trans_wave_h avg_trans_wave_i_ext_decam ... sersic_fract_z_ext_panstarrs_disk_sersic sersic_fract_z_ext_panstarrs_disk_sersic_err she_flag spurious_flag     spurious_prob      variable_flag vis_det
--------------------- -------------------------- ------------------------ -------------------------- ------------------------- ---------------- -------------------------- ... ---------------------------------------- -------------------------------------------- -------- ------------- ---------------------- ------------- -------
3.566798805594703e-06            4826.7998046875                       --                         --                        --               --             7826.669921875 ...                                       --                                           --       --             0    0.15743961930274963            --       1
0.0004459918667892947            4826.7998046875                       --                         --                        --               --             7826.669921875 ...                                       --                                           --       --             0   0.004427384119480848            --       1
0.0011813971416470212            4826.7998046875                       --                         --                        --               --             7826.669921875 ...                                       --                                           --       --             0     0.1833316683769226            --       1
0.0015542789169486976            4826.7998046875                       --                         --                        --               --             7826.669921875 ...                                       --                                           --       --             0    0.12239421904087067            --       0
0.0015885047273778879            4826.7998046875                       --                         --                        --               --             7826.669921875 ...                                       --                                           --       --             0 0.00021384040883276612            --       1
                  ...                        ...                      ...                        ...                       ...              ...                        ... ...                                      ...                                          ...      ...           ...                    ...           ...     ...
  0.03958455791235079            4826.7998046875                       --                         --                        --               --             7826.669921875 ...                                       --                                           --       --             0     0.1343534141778946            --       0
  0.03958823626200475            4826.7998046875                       --                         --                        --               --             7826.669921875 ...                                       --                                           --       --             0 0.00015592691488564014            --       1
  0.03959898295410331            4826.7998046875                       --                         --                        --               --             7826.669921875 ...                                       --                                           --       --             0     0.0365927591919899            --       1
 0.039605684988334174            4826.7998046875                       --                         --                        --               --             7826.669921875 ...                                       --                                           --       --             0     0.0831669270992279            --       0
  0.03960602180308949            4826.7998046875                       --                         --                        --               --             7826.669921875 ...                                       --                                           --       --             1     0.6376287937164307            --       1
 0.039606556762811496            4826.7998046875                       --                         --                        --               --             7826.669921875 ...                                       --                                           --       --             0   0.012528697960078716            --       1
  0.03962541836711639            4826.7998046875                       --                         --                        --               --             7826.669921875 ...                                       --                                           --       --             0   0.003399776993319392            --       1
Length = 2000 rows

1.4. Synchronous query

The results of a synchronous query are stored at the user side (i.e., they are not saved in the server). These queries can only be used when the amount of data to be retrieved (number of rows) is small, otherwise, a timeout error can be raised. The output of the synchronous queries is limited to 2000 rows. If you need more than that, you must use asynchronous queries. The results of asynchronous queries can be saved in memory (default) or in a file but are also stored on the server/archive so you can access the results by logging in on the archive website too.

Query without saving results in a file:

>>> from astroquery.esa.euclid import Euclid
>>> #query content: getting some galaxies from the mer catalogue
>>> job = Euclid.launch_job("SELECT right_ascension, declination, segmentation_area, fluxerr_vis_1fwhm_aper, ellipticity, kron_radius FROM catalogue.mer_catalogue  WHERE ellipticity > 0 ORDER BY ellipticity ASC")
>>> source_results_table = job.get_results()
>>> print("Found", len(source_results_table), " query results")
Found 2000  query results
>>> print("The results table includes the following", len(source_results_table.colnames), "columns: ", source_results_table.colnames)
The results table includes the following 6 columns:  ['right_ascension', 'declination', 'segmentation_area', 'fluxerr_vis_1fwhm_aper', 'ellipticity', 'kron_radius']
>>> print(source_results_table[:15])
 right_ascension       declination     segmentation_area fluxerr_vis_1fwhm_aper      ellipticity          kron_radius
------------------ ------------------- ----------------- ---------------------- ---------------------- ------------------
  60.3372780005097  -49.93184727724773                45   0.024313488975167274 1.1569118214538321e-05 10.145233154296875
 59.92581284609097 -48.117835930359156               165   0.035201895982027054 4.3500345782376826e-05 10.814051628112793
 62.91963955425831  -45.60370330289406             43783                    nan  5.609192521660589e-05  884.3989868164062
 54.38946012012026 -28.843720993232775               213    0.03697587549686432  6.897230923641473e-05  11.36937141418457
 267.3583266451287  63.994872950197674                31    0.01883346401154995  8.067921589827165e-05 10.564066886901855
52.507667893100944 -29.445192357858655                44   0.020872678607702255 0.00010451683920109645 10.120616912841797
 61.43102565450044  -48.13296805386111                53   0.023762457072734833 0.00010995510092470795 10.149212837219238
274.03172675714404   65.86352270374569               134    0.03499231114983559 0.00011172338417964056 10.804966926574707
 271.4749139468259   68.66799925831447               725    0.07235158979892731 0.00013952785229776055 15.325024604797363
 64.08716465009101  -47.39574129846509                21   0.025711175054311752  0.0001506721746409312  8.137764930725098
 62.46388138426946 -48.771499634650795               293    0.04792384058237076  0.0001519227953394875  12.14012336730957
 59.12696116576496  -50.70917518433447                35   0.021673062816262245 0.00015204095689114183  13.88205337524414
 64.00922297351093 -47.458008931760105               221     0.0443091057240963 0.00015916737902443856 11.836610794067383
 63.12619441593158  -46.21602917569897                36   0.023102710023522377 0.00016669274191372097  9.809906959533691
   270.19726289254   68.13460591203628               157    0.03000682406127453 0.00018387728778179735 10.963176727294922

The method returns a Job object.

Query saving results in a file (you may use ‘output_format’ to specify the results’ data format. Available formats are: ‘votable’, ‘votable_plain’, ‘fits’, ‘csv’ and ‘json’, default is ‘votable’):

>>> from astroquery.esa.euclid import Euclid
>>> job = Euclid.launch_job("SELECT right_ascension, declination, segmentation_area, fluxerr_vis_1fwhm_aper, ellipticity, kron_radius FROM catalogue.mer_catalogue  WHERE ellipticity > 0 ORDER BY ellipticity ASC", dump_to_file=True, output_format='votable')
>>> print(job.outputFile)
1668863838419O-result.vot.gz
>>> r = job.get_results()
>>> print(r)
<Table length=2000>
 right_ascension       declination     segmentation_area fluxerr_vis_1fwhm_aper      ellipticity          kron_radius
     float64             float64             int32              float64                float64              float64
------------------ ------------------- ----------------- ---------------------- ---------------------- ------------------
  60.3372780005097  -49.93184727724773                45   0.024313488975167274 1.1569118214538321e-05 10.145233154296875
 59.92581284609097 -48.117835930359156               165   0.035201895982027054 4.3500345782376826e-05 10.814051628112793
 62.91963955425831  -45.60370330289406             43783                     --  5.609192521660589e-05  884.3989868164062
 54.38946012012026 -28.843720993232775               213    0.03697587549686432  6.897230923641473e-05  11.36937141418457
 267.3583266451287  63.994872950197674                31    0.01883346401154995  8.067921589827165e-05 10.564066886901855
               ...                 ...               ...                    ...                    ...                ...
 274.4463705921747   64.52235051000498               139     0.0359811969101429  0.0021674882154911757 11.007905006408691
275.12508717512287   65.27789133746404                89   0.017870161682367325  0.0021674996241927147  9.524261474609375
62.343396630172755  -49.57660112316788               164   0.023227984085679054   0.002168711507692933  21.94918441772461
 60.42143918751039 -47.698201261387545               124    0.03315594792366028  0.0021696146577596664 10.711515426635742
  59.5694307527783  -47.00910465608437               162   0.030248118564486504  0.0021698300261050463 11.478931427001953

You can inspect the status of the job by typing:

>>> print(job)
<Table length=2000>
         name           dtype  n_bad
---------------------- ------- -----
       right_ascension float64     0
           declination float64     0
     segmentation_area   int32     0
fluxerr_vis_1fwhm_aper float64    67
           ellipticity float64     0
           kron_radius float64     0
Jobid: None
Phase: COMPLETED
Owner: None
Output file: 1

Note: to obtain the current location, type:

>>> import os
>>> print(os.getcwd())

1.5. Asynchronous query

Asynchronous queries save all results on the server side, which means they take up the user’s file quota in the archive. These queries can be accessed at any time on the archive website. For anonymous users, results are kept for three days. For authenticated users, the asynchronous results are kept at the server side forever (until the user decides to remove them). Make sure to delete the results you don’t need anymore every once in a while to make sure you don’t reach the archive user quota. When that happens, all future jobs/queries will start failing. You can delete the jobs on the archive website (and soon through python too).

The results of the execution of the queries can be stored locally in memory (by default) or in a file. The following command stores the result in memory:

>>> from astroquery.esa.euclid import Euclid
>>> # query content: getting the mosaic file name corresponding to the first source in the last query
>>> output_folder="my_temp_folder"
>>> query = "SELECT file_name, file_path, datalabs_path, mosaic_product_oid, tile_index, instrument_name, filter_name, ra, dec FROM sedm.mosaic_product WHERE (instrument_name='VIS') AND (((mosaic_product.fov IS NOT NULL AND INTERSECTS(CIRCLE('ICRS', 60.3372780005097, -49.93184727724773,"+ str(0.5/60) + "), mosaic_product.fov)=1))) ORDER BY mosaic_product.tile_index ASC"
>>> job_async = Euclid.launch_job_async(query, dump_to_file=True, output_file=output_folder + "async_result.csv", output_format="csv", verbose=False)
>>> print("Started async job with id:", job_async.jobid)
Started async job with id: 1738523825791TIDR
>>> #to check on the status of the async job
>>> print(job_async)
Jobid: 1738523825791TIDR
Phase: COMPLETED
Owner: None
Output file: example_outputs/async_result.csv
Results: None
>>> mosaic_file_results_table = job_async.get_results()
>>> print("Found", len(mosaic_file_results_table), " query results")
Found 1  query results
>>> print(mosaic_file_results_table)
                                   file_name                                                          file_path                                   datalabs_path              mosaic_product_oid tile_index instrument_name filter_name     ra      dec
-------------------------------------------------------------------------------- --------------------------------------------------- --------------------------------------- ------------------ ---------- --------------- ----------- ---------- -----
EUC_MER_BGSUB-MOSAIC-VIS_TILE102019591-BDF6EF_20241018T164804.324145Z_00.00.fits /euclid/repository_idr/iqr1/Q1_R1/MER/102019591/VIS /data/euclid_q1/Q1_R1/MER/102019591/VIS               1081  102019591             VIS         VIS 60.0509927 -50.0

To get all the asynchronous jobs:

>>> from astroquery.esa.euclid import Euclid
>>> joblist = Euclid.list_async_jobs()
>>> for j in joblist:
...     print(j, "\n")
Jobid: 1730889961292TIDR
Phase: COMPLETED
Owner: None
Output file: None
Results: None

Jobid: 1730889844173TIDR
Phase: COMPLETED
Owner: None
Output file: None
Results: None

To remove asynchronous jobs:

>>> from astroquery.esa.euclid import Euclid
>>> job = Euclid.remove_jobs(["job_id_1", "job_id_2", ...])

1.6. Synchronous query on an ‘on-the-fly’ uploaded table

‘On-the-fly’ queries allow you to submit a votable and perform a query using that table all in one command; unlike tables uploaded with the methods described in section 2.5. Uploading table to user space, these tables will be deleted after the query is complete.

You have to provide the local path to the file you want to upload. In the following example, the file ‘my_table.xml’ is located to the relative location where your python program is running. See note below.

>>> from astroquery.esa.euclid import Euclid
>>> upload_resource = 'my_table.xml'
>>> j = Euclid.launch_job(query="select * from tap_upload.table_test",
... upload_resource=upload_resource, upload_table_name="table_test", verbose=True)
>>> r = j.get_results()
>>> r.pprint()
source_id alpha delta
--------- ----- -----
        a   1.0   2.0
        b   3.0   4.0
        c   5.0   6.0

Note: to obtain the current location, type:

>>> import os
>>> print(os.getcwd())
/Current/directory/path

2. Authenticated access

Authenticated users are able to access to TAP+ capabilities (shared tables, persistent jobs, etc.) In order to authenticate a user, login method must be called. After a successful authentication, the user will be authenticated until the logout method is called.

All previous methods (query_object, cone_search, load_table, load_tables, launch_job) explained for non authenticated users are applicable for authenticated ones.

The main differences are:

  • Asynchronous results are kept at the server side forever (until the user decides to remove one of them).

  • Users can access to share tables.

2.1. Login/Logout

There are several ways to log in to the Euclid archive.

Login through graphic interface

Note: The Python Tkinter module is required to use the login_gui method.

>>> from astroquery.esa.euclid import Euclid
>>> Euclid.login_gui()

Login through command line

>>> from astroquery.esa.euclid import Euclid
>>> Euclid.login()
>>> User: user
>>> Password: pwd (not visible)

or

>>> from astroquery.esa.euclid import Euclid
>>> Euclid.login(user='userName', password='userPassword')

It is possible to use a file where the credentials are stored:

The file must contain user and password in two different lines.

>>> from astroquery.esa.euclid import Euclid
>>> Euclid.login(credentials_file='my_credentials_file')

To perform a logout:

>>> Euclid.logout()

2.2. Getting public tables metadata

To get the list of products associated with a given EUCLID observation_id or tile_index (for mosaic):

>>> from astroquery.esa.euclid import Euclid
>>> product_list_results = Euclid.get_product_list(tile_index="102018211", product_type="DpdMerBksMosaic")
>>> print("Found", len(product_list_results), "results")
Found 12 results
>>> print(product_list_results)
                                    file_name                                      mosaic_product_oid tile_index instrument_name filter_name category second_type     ra       dec   technique
                                      str255                                             int64          int64         str255        str255    str255     str255    float64   float64   str255
---------------------------------------------------------------------------------- ------------------ ---------- --------------- ----------- -------- ----------- ---------- ------- ---------
EUC_MER_BGSUB-MOSAIC-DES-I_TILE102018211-31E2C9_20241018T143048.358037Z_00.00.fits               1399  102018211           DECAM     DECAM_i  SCIENCE         SKY 57.9990741   -51.5     IMAGE
  EUC_MER_BGSUB-MOSAIC-VIS_TILE102018211-ACBD03_20241018T142710.276838Z_00.00.fits               1395  102018211             VIS         VIS  SCIENCE         SKY 57.9990741   -51.5     IMAGE
EUC_MER_BGSUB-MOSAIC-DES-G_TILE102018211-D9D163_20241018T143010.768685Z_00.00.fits               1394  102018211           DECAM     DECAM_g  SCIENCE         SKY 57.9990741   -51.5     IMAGE
EUC_MER_BGSUB-MOSAIC-NIR-H_TILE102018211-42F1AD_20241018T142558.469987Z_00.00.fits               1396  102018211            NISP       NIR_H  SCIENCE         SKY 57.9990741   -51.5     IMAGE
EUC_MER_BGSUB-MOSAIC-NIR-J_TILE102018211-E044A1_20241018T142600.459089Z_00.00.fits               1393  102018211            NISP       NIR_J  SCIENCE         SKY 57.9990741   -51.5     IMAGE
EUC_MER_BGSUB-MOSAIC-NIR-Y_TILE102018211-E5CAE1_20241018T142558.172837Z_00.00.fits               1397  102018211            NISP       NIR_Y  SCIENCE         SKY 57.9990741   -51.5     IMAGE
EUC_MER_BGSUB-MOSAIC-DES-R_TILE102018211-1078B7_20241018T142927.232351Z_00.00.fits               1401  102018211           DECAM     DECAM_r  SCIENCE         SKY 57.9990741   -51.5     IMAGE
EUC_MER_BGSUB-MOSAIC-DES-Z_TILE102018211-83C32F_20241018T143526.104818Z_00.00.fits               1398  102018211           DECAM     DECAM_z  SCIENCE         SKY 57.9990741   -51.5     IMAGE

The method returns a list of products as an astropy.table.

It is possible to download a product given its file name or product id:

>>> #makeing a folder for the output files
>>> import os
>>> from astroquery.esa.euclid import Euclid
>>> output_folder= 'example_outputs/'
>>> if not os.path.exists(output_folder):
       os.makedirs(output_folder)
>>>
>>> example_file_name = "EUC_MER_BGSUB-MOSAIC-NIR-H_TILE102158889-ED035A_20241024T212936.705156Z_00.00.fits"
>>> print("Getting file:", example_file_name)
Getting file: EUC_MER_BGSUB-MOSAIC-NIR-H_TILE102158889-ED035A_20241024T212936.705156Z_00.00.fits
>>> path = Euclid.get_product(file_name=example_file_name, output_file=output_folder + example_file_name,verbose=True)
Retrieving data.
Data request: TAPCLIENT=ASTROQUERY&RELEASE=sedm&FILE_NAME=EUC_MER_BGSUB-MOSAIC-NIR-H_TILE102158889-ED035A_20241024T212936.705156Z_00.00.fits&RETRIEVAL_TYPE=FILE
------>https
host = easidr.esac.esa.int:443
context = /sas-dd/data
Content-type = application/x-www-form-urlencoded
200
Reading...
Done.
>>> #display the downloaded product (since this is a calibrated frame the different detectors are stored as different extensions - we are displaying only one extension)
>>> from astropy.io import fits
>>> import matplotlib.pyplot as plt
>>> from astropy.visualization import astropy_mpl_style, ImageNormalize, PercentileInterval, AsinhStretch, LogStretch
>>> hdul = fits.open(path[0])
>>> print(fits.info(path[0]))
WARNING: File may have been truncated: actual file length (103579232) is smaller than the expected size (1474565760) [astropy.io.fits.file]
Filename: example_notebook_outputs/EUC_MER_BGSUB-MOSAIC-DES-I_TILE102018211-31E2C9_20241018T143048.358037Z_00.00.fits
No.    Name      Ver    Type      Cards   Dimensions   Format
  0  PRIMARY       1 PrimaryHDU      48   (19200, 19200)   float32
None
>>> image_data = hdul[0].data
>>>
>>> plt.figure()
<Figure size 800x600 with 0 Axes>
<Figure size 800x600 with 0 Axes>
>>> plt.imshow(image_data, cmap='gray', origin='lower', norm=ImageNormalize(image_data, interval=PercentileInterval(99.9), stretch=AsinhStretch()))
>>> colorbar = plt.colorbar()
EUC_MER_BGSUB-MOSAIC-NIR-H_TILE102158889-ED035A_20241024T212936.705156Z_00.00.fits

The method downloads the fits file(s) and returns the local path where the product(s) is saved.

To download the products for a given EUCLID observation_id (observations) or tile_index (mosaics):

>>> #downloading all products for observation id: 102018211
>>> from astroquery.esa.euclid import Euclid
>>> mos_id = 1399
>>> path = Euclid.get_observation_products(id=mos_id, product_type='mosaic', filter="VIS", output_file=f"{output_folder}/products_{mos_id}.fits", verbose=True)

For big files the download may require a long time.

2.4. Listing shared tables

In the Euclid archive user tables can be shared among user groups.

To obtain a list of the tables shared to a user type the following:

>>> from astroquery.esa.euclid import Euclid
>>> tables = Euclid.load_tables(only_names=True, include_shared_tables=True)
>>> for table in tables:
...   print(table.get_qualified_name())

2.5. Uploading table to user space

It is now possible to store a table in the private user space. The table to be uploaded can be in a VOTable located at a given URL, a table stored in a local file in the user machine, a pre-computed Astropy table file or a job executed in the Euclid archive.

Each user has a database schema described as: ‘user_<user_login_name>’. For instance, if a login name is ‘joe’, the database schema is ‘user_joe’. Your uploaded table can be referenced as ‘user_joe.table_name’

2.5.1. Uploading table from URL

An already generated VOTable, accessible through a URL, can be uploaded to Euclid archive.

The following example launches a query to Vizier TAP (‘url’ parameter). The result is a VOTable that can be uploaded to the user’s private area.

Your schema name will be automatically added to the provided table name:

>>> from astroquery.esa.euclid import Euclid
>>> Euclid.login()
>>> # Provide a URL pointing to valid VOTable resource
>>> url = ("https://tapvizier.cds.unistra.fr/TAPVizieR/tap/sync/?"
...        "REQUEST=doQuery&lang=ADQL&FORMAT=votable&"
...        "QUERY=select+*+from+TAP_SCHEMA.columns+where+table_name='II/336/apass9'")
>>> job = Euclid.upload_table(upload_resource=url, table_name="table_test_from_url",
... table_description="Some description")
Job '1539932326689O' created to upload table 'table_test_from_url'.

Now, you can query your table as follows (a full qualified table name must be provided, i.e.: user_<your_login_name>.<table_name>. Note that if the <table_name> contains capital letters, it must be surrounded by quotation marks, i.e.: user_<your_login_name>.”<table_name>”):

>>> from astroquery.esa.euclid import Euclid
>>> full_qualified_table_name = 'user_<your_login_name>.table_test_from_url'
>>> query = 'select * from ' + full_qualified_table_name
>>> job = Euclid.launch_job(query=query)
>>> results = job.get_results()

2.5.2. Uploading table from file

A file containing a table can be uploaded to the user private area. Only a file associated to any of the formats described in https://docs.astropy.org/en/stable/io/unified.html#built-in-table-readers-writers, and automatically identified by its suffix or content can be used. Note that for a multi-extension fits file with multiple tables, the first table found will be used. For any other format, the file can be transformed into an astropy Table (https://docs.astropy.org/en/stable/io/unified.html#getting-started-with-table-i-o) and passed to the method.

The parameter ‘format’ must be provided when the input file is not a votable file.

Your schema name will be automatically added to the provided table name.

>>> from astroquery.esa.euclid import Euclid
>>> Euclid.login()
>>> job = Euclid.upload_table(upload_resource="1535553556177O-result.vot", table_name="table_test_from_file", format="votable")
Sending file: 1535553556177O-result.vot
Uploaded table 'table_test_from_file'.

Now, you can query your table as follows (a full qualified table name must be provided, i.e.: user_<your_login_name>.<table_name>. Note that if the <table_name> contains capital letters, it must be surrounded by quotation marks, i.e.: user_<your_login_name>.”<table_name>”):

>>> from astroquery.esa.euclid import Euclid
>>> full_qualified_table_name = 'user_<your_login_name>.table_test_from_file'
>>> query = 'select * from ' + full_qualified_table_name
>>> job = Euclid.launch_job(query=query)
>>> results = job.get_results()

2.5.3. Uploading table from an astropy Table

A votable can be uploaded to the server in order to be used in a query. Your schema name will be automatically added to the provided table name.

>>> from astroquery.esa.euclid import Euclid
>>> from astropy.table import Table
>>> a=[1,2,3]
>>> b=['a','b','c']
>>> table = Table([a,b], names=['col1','col2'], meta={'meta':'first table'})
>>> # Upload
>>> Euclid.login()
>>> Euclid.upload_table(upload_resource=table, table_name='table_test_from_astropy')

Now, you can query your table as follows (a full qualified table name must be provided, i.e.: user_<your_login_name>.<table_name>. Note that if the <table_name> contains capital letters, it must be surrounded by quotation marks, i.e.: user_<your_login_name>.”<table_name>”):

>>> from astroquery.esa.euclid import Euclid
>>> full_qualified_table_name = 'user_<your_login_name>.table_test_from_astropy'
>>> query = 'select * from ' + full_qualified_table_name
>>> job = Euclid.launch_job(query=query)
>>> results = job.get_results()

2.5.4. Uploading table from job

The results generated by an asynchronous job (from a query executed in the Euclid archive) can be ingested in a table in the user’s private area.

The following example generates a job in the Euclid archive and then, the results are ingested in a table named: user_<your_login_name>.’t’<job_id>:

>>> from astroquery.esa.euclid import Euclid
>>> Euclid.login()
>>> job_1 = Euclid.launch_job_async("select top 10 * from Eucliddr3.Euclid_source")
>>> Euclid.upload_table_from_job(job=job_1)
Created table 't1539932994481O' from job: '1539932994481O'.

Now, you can query your table as follows (a full qualified table name must be provided, i.e.: user_<your_login_name>.”t<job_id>”. Note that the previous table name must be surrounded by quotation marks since it contains capital letters.):

>>> from astroquery.esa.euclid import Euclid
>>> full_qualified_table_name = 'user_<your_login_name>."t1710251325268O"'
>>> query = 'select * from ' + full_qualified_table_name
>>> job = Euclid.launch_job(query=query)
>>> results = job.get_results()

2.6. Deleting table

A table from the user’s private area can be deleted as follows:

>>> from astroquery.esa.euclid import Euclid
>>> Euclid.login_gui()
>>> job = Euclid.delete_user_table(table_name="table_test_from_file")
Table 'table_test_from_file' deleted.

2.7. Updating table metadata

It can be useful for the user to modify the metadata of a given table. For example, a user might want to change the description (UCD) of a column, or the flags that give extra information about a certain column. This is possible using:

>>> from astroquery.esa.euclid import Euclid
>>> Euclid.login_gui()
>>> Euclid.update_user_table(table_name, list_of_changes)

where the list of changes is a list of 3 items:

[“column name to be changed”, “metadata parameter to be changed”, “new value”]

The metadata parameter to be changed can be ‘utype’, ‘ucd’, ‘flags’ or ‘indexed’:

  • values for ‘utype’ and ‘ucd’ are free text. See VOTable specification (sections UType and UCD), UCD specification and UTypes usage.

  • value for ‘flags’ can be ‘Ra’, ‘Dec’, ‘Mag’, ‘Flux’ and ‘PK’.

  • value for ‘indexed’ is a boolean indicating whether the column is indexed or not.

It is possible to apply multiple changes at once. This is done by putting each of the changes in a list. See example below.

In this case, we have a table (user_joe.table), with several columns: ‘recno’, ‘nobs’, ‘raj2000’ and ‘dej2000’.

We want to set:

  • ‘ucd’ of ‘recno’ column to ‘ucd sample’

  • ‘utype’ of ‘nobs’ column to ‘utype sample’

  • ‘flags’ of ‘raj2000’ column to ‘Ra’

  • ‘flags’ of ‘dej2000’ column to ‘Dec’

We can type the following:

>>> from astroquery.esa.euclid import Euclid
>>> Euclid.login_gui()
>>> Euclid.update_user_table(table_name="user_joe.table",
...                        list_of_changes=[["recno", "ucd", "ucd sample"],
...                                         ["nobs","utype","utype sample"],
...                                         ["raj2000","flags","Ra"],
...                                         ["dej2000","flags","Dec"]])
Retrieving table 'user_joe.table'
Parsing table 'user_joe.table'...
Done.
Table 'user_joe.table' updated.

2.8. Tables sharing

It is possible to share tables with other users. You have to create a group, populate that group with users, and share your table to that group. Then, any user belonging to that group will be able to access your shared table in a query.

2.8.1. Creating a group

>>> from astroquery.esa.euclid import Euclid
>>> Euclid.login()
>>> Euclid.share_group_create(group_name="my_group", description="description")

2.8.2. Removing a group

>>> from astroquery.esa.euclid import Euclid
>>> Euclid.login()
>>> Euclid.share_group_delete(group_name="my_group")

2.8.3. Listing groups

>>> from astroquery.esa.euclid import Euclid
>>> Euclid.login()
>>> groups = Euclid.load_groups()
>>> for group in groups:
...     print(group.title)

2.8.4. Adding users to a group

>>> from astroquery.esa.euclid import Euclid
>>> Euclid.login()
>>> Euclid.share_group_add_user(group_name="my_group",user_id="<user_login_name")

2.8.5. Removing users from a group

>>> from astroquery.esa.euclid import Euclid
>>> Euclid.login()
>>> Euclid.share_group_delete_user(group_name="my_group",user_id="<user_login_name>")

2.8.6. Sharing a table to a group

>>> from astroquery.esa.euclid import Euclid
>>> Euclid.login()
>>> Euclid.share_table(group_name="my_group",
...                  table_name="user_<user_login_name>.my_table",
...                  description="description")

2.8.7. Stop sharing a table

>>> from astroquery.esa.euclid import Euclid
>>> Euclid.login()
>>> Euclid.share_table_stop(table_name="user_<user_login_name>.my_table", group_name="my_group")

Reference/API

astroquery.esa.euclid Package

European Space Astronomy Centre (ESAC) European Space Agency (ESA)

Classes

EuclidClass(*[, environment, ...])

Constructor for EuclidClass.

Conf()

Configuration parameters for astroquery.esa.euclid.