.. doctest-skip-all .. _astroquery.cosmosim: **************************************** CosmoSim Queries (`astroquery.cosmosim`) **************************************** This module allows the user to query and download from one of three cosmological simulation projects: the MultiDark project, the BolshoiP project, and the CLUES project. For accessing these databases a CosmoSim object must first be instantiated with valid credentials (no public username/password are implemented). Below are a couple of examples of usage. Requirements ============ The following packages are required for the use of this module: * requests * keyring * getpass * bs4 Getting started =============== .. code-block:: python >>> from astroquery.cosmosim import CosmoSim >>> CS = CosmoSim() Next, enter your credentials; caching is enabled, so after the initial successful login no further password is required if desired. >>> CS.login(username="uname") uname, enter your CosmoSim password: Authenticating uname on www.cosmosim.org... Authentication successful! >>> # If running from a script (rather than an interactive python session): >>> # CS.login(username="uname",password="password") To store the password associated with your username in the keychain: >>> CS.login(username="uname",store_password=True) WARNING: No password was found in the keychain for the provided username. [astroquery.cosmosim.core] uname, enter your CosmoSim password: Authenticating uname on www.cosmosim.org... Authentication successful! Logging out is as simple as: >>> CS.logout(deletepw=True) Removed password for uname in the keychain. The deletepw option will undo the storage of any password in the keychain. Checking whether you are successfully logged in (or who is currently logged in): >>> CS.check_login_status() Status: You are logged in as uname. Below is an example of running an SQL query (BDMV mass function of the MDR1 cosmological simulation at a redshift of z=0): >>> sql_query = "SELECT 0.25*(0.5+FLOOR(LOG10(mass)/0.25)) AS log_mass, COUNT(*) AS num FROM MDR1.FOF WHERE snapnum=85 GROUP BY FLOOR(LOG10(mass)/0.25) ORDER BY log_mass" >>> CS.run_sql_query(query_string=sql_query) Job created: 359748449665484 #jobid; note: is unique to each and every query Managing CosmoSim Queries ========================= The cosmosim module provides functionality for checking the completion status of queries, in addition to deleting them from the server. Below are a few examples of functions available to the user for these purposes. .. code-block:: python >>> CS.check_all_jobs() JobID Phase --------------- --------- 359748449665484 COMPLETED >>> CS.delete_job(jobid='359748449665484') Deleted job: 359748449665484 >>> CS.check_all_jobs() JobID Phase --------------- --------- The above function 'check_all_jobs' also supports the usage of a job's phase status in order to filter through all available CosmoSim jobs. .. code-block:: python >>> CS.check_all_jobs() JobID Phase --------------- --------- 359748449665484 COMPLETED 359748449682647 ABORTED 359748449628375 ERROR >>> CS.check_all_jobs(phase=['Completed','Aborted']) JobID Phase --------------- --------- 359748449665484 COMPLETED 359748449682647 ABORTED Additionally, 'check_all_jobs' (and 'delete_all_jobs') accepts both phase and/or tablename (via a regular expression) as criteria for deletion of all available CosmoSim jobs. But be careful: Leaving both arguments blank will delete ALL jobs! .. code-block:: python >>> CS.check_all_jobs() JobID Phase --------------- --------- 359748449665484 COMPLETED 359748449682647 ABORTED 359748449628375 ERROR >>> CS.table_dict() {'359748449665484': '2014-09-07T05:01:40:0458'} {'359748449682647': 'table2'} {'359748449628375': 'table3'} >>> CS.delete_all_jobs(phase=['Aborted','error'],regex='[a-z]*[0-9]*') Deleted job: 359748449682647 (Table: table2) Deleted job: 359748449628375 (Table: table3) Note: Arguments for phase are case insensitive. Now, check to see if the jobs have been deleted: >>> CS.check_all_jobs() JobID Phase --------------- --------- 359748449665484 COMPLETED Getting rid of this last job can be done by deleting all jobs with phase COMPLETED, or it can be done simply by providing the 'delete_job' function with its unique jobid. Lastly, this could be accomplished by matching its tablename to the following regular expression: '[0-9]*-[0-9]*-[0-9]*[A-Z]*[0-9]*:[0-9]*:[0-9]*:[0-9]*'. All jobs created without specifying the tablename argument in 'run_sql_query' are automatically assigned one based upon the creation date and time of the job, and is therefore the default tablename format. Deleting all jobs, regardless of tablename, and job phase: .. code-block:: python >>> CS.check_all_jobs() JobID Phase --------------- --------- 359748449665484 ABORTED 359748586913123 COMPLETED >>> CS.delete_all_jobs() Deleted job: 359748449665484 Deleted job: 359748586913123 >>> CS.check_all_jobs() JobID Phase --------------- --------- In addition to the phase and regex arguments for 'check_all_jobs', selected jobs can be sorted using two properties: >>> CS.check_all_jobs(phase=['completed'],regex='[a-z]*[0-9]*',sortby='tablename') JobID Phase Tablename Starttime --------------- --------- --------- ------------------------- 361298054830707 COMPLETED table1 2014-09-21T19:28:48+02:00 361298050841687 COMPLETED table2 2014-09-21T19:20:23+02:00 >>> CS.check_all_jobs(phase=['completed'],regex='[a-z]*[0-9]*',sortby='starttime') JobID Phase Tablename Starttime --------------- --------- --------- ------------------------- 361298050841687 COMPLETED table2 2014-09-21T19:20:23+02:00 361298054830707 COMPLETED table1 2014-09-21T19:28:48+02:00 Exploring Database Schema ========================= A database exploration tool is available to help the user navigate the structure of any simulation database in the CosmoSim database. Note: '@' precedes entries which are dictionaries .. code-block:: python >>> CS.explore_db() Must first specify a database. Projects Project Items Information ------------------------ ------------- -------------------------------------------------------------------------------------- @ Bolshoi @ tables id: 2 description: The Bolshoi Database. ------------------------ ------------- -------------------------------------------------------------------------------------- @ BolshoiP @ tables id: 119 description: Bolshoi Planck simulation ------------------------ ------------- -------------------------------------------------------------------------------------- @ Clues3_LGDM @ tables id: 134 description: CLUES simulation, B64, 186592, WMAP3, Local Group resimulation, 4096, Dark Matter only ------------------------ ------------- -------------------------------------------------------------------------------------- @ Clues3_LGGas @ tables id: 124 description: CLUES simulation, B64, 186592, WMAP3, Local Group resimulation, 4096, Gas+SFR ------------------------ ------------- -------------------------------------------------------------------------------------- @ MDPL @ tables id: 114 description: The MDR1-Planck simulation. ------------------------ ------------- -------------------------------------------------------------------------------------- @ MDR1 @ tables id: 7 description: The MultiDark Run 1 Simulation. ------------------------ ------------- -------------------------------------------------------------------------------------- @ cosmosim_user_username @ tables id: userdb description: Your personal database ------------------------ ------------- -------------------------------------------------------------------------------------- .. code-block:: python >>> CS.explore_db(db='MDPL') Projects Project Items Tables ----------- ------------- ------------- --> @ MDPL: --> @ tables: @ FOF id @ FOF5 description @ FOF4 @ FOF3 @ FOF2 @ FOF1 @ BDMW @ Redshifts @ LinkLength @ AvailHalos @ Particles88 .. code-block:: python >>> CS.explore_db(db='MDPL',table='FOF') Projects Project Items Tables Table Items Table Info Columns ----------- ------------- ------------- ------------ ---------- -------- --> @ MDPL: --> @ tables: --> @ FOF: id: 934 y id @ FOF5 @ columns x description @ FOF4 description: z @ FOF3 ix @ FOF2 iz @ FOF1 vx @ BDMW vy @ Redshifts vz @ LinkLength iy @ AvailHalos np @ Particles88 disp size spin mass axis1 axis2 axis3 fofId phkey delta level angMom disp_v axis1_z axis1_x axis1_y axis3_x axis3_y axis3_z axis2_y axis2_x NInFile axis2_z snapnum angMom_x angMom_y angMom_z .. code-block:: python >>> CS.explore_db(db='MDPL',table='FOF',col='fofId') Projects Project Items Tables Table Items Columns ----------- ------------- ------------- -------------- ------------ --> @ MDPL: --> @ tables: --> @ FOF: --> @ columns: --> @ fofId: id @ FOF5 id @ disp description @ FOF4 description @ axis1_z @ FOF3 @ axis1_x @ FOF2 @ axis1_y @ FOF1 @ ix @ BDMW @ iz @ Redshifts @ axis3_x @ LinkLength @ axis3_y @ AvailHalos @ axis3_z @ Particles88 @ vx @ vy @ vz @ axis2_y @ axis2_x @ size @ axis1 @ axis2 @ axis3 @ iy @ angMom @ NInFile @ np @ axis2_z @ disp_v @ phkey @ delta @ snapnum @ spin @ level @ angMom_x @ angMom_y @ angMom_z @ mass @ y @ x @ z Downloading data ================ Query results can be downloaded and used in real-time from the command line, or alternatively they can be stored on your local machine. .. code-block:: python >>> CS.check_all_jobs() JobID Phase --------------- --------- 359750704009965 COMPLETED >>> data = CS.download(jobid='359750704009965',format='csv') >>> print(data) (['row_id', 'log_mass', 'num'], [[1, 10.88, 3683], [2, 11.12, 452606], [3, 11.38, 3024674], [4, 11.62, 3828931], [5, 11.88, 2638644], [6, 12.12, 1572685], [7, 12.38, 926764], [8, 12.62, 544650], [9, 12.88, 312360], [10, 13.12, 174164], [11, 13.38, 95263], [12, 13.62, 50473], [13, 13.88, 25157], [14, 14.12, 11623], [15, 14.38, 4769], [16, 14.62, 1672], [17, 14.88, 458], [18, 15.12, 68], [19, 15.38, 4]]) Unless the filename attribute is specified, data is not saved out to file. >>> data = CS.download(jobid='359750704009965',filename='/Users/uname/Desktop/test.csv',format='csv') |==========================================================================================================================| 1.5k/1.5k (100.00%) 0s Other formats include votable, votableb1, and votableb2 (the latter two are binary files, for easier handling of large data sets; these formats can not be used in an interactive python session). Data can be stored and/or written out as a `~astropy.io.votable`. .. code-block:: python >>> data = CS.download(jobid='359750704009965',format='votable') >>> data >>> data = CS.download(jobid='359750704009965',filename='/Users/uname/Desktop/test.xml',format='votable') >>> |==========================================================================================================================| 4.9k/4.9k (100.00%) 0s Reference/API ============= .. automodapi:: astroquery.cosmosim :no-inheritance-diagram: