!pip install aiohttp matplotlib pandas pyarrow "xarray[io]"
import pandas as pd
META_URL = "https://s3.echo.stfc.ac.uk/mast/fairmast-meta"
Quickstart#
This notebook shows some very simple examples of getting started.
Metadata for every shot, source and signal is stored as Parquet files in the public S3 bucket under s3://mast/fairmast-meta/, which can be read over HTTPS at https://s3.echo.stfc.ac.uk/mast/fairmast-meta/:
File |
Contents |
|---|---|
|
One row per shot: shot metadata and CPF summary statistics |
|
One row per source (group of signals) per shot |
|
One row per signal for a single shot |
The l1_ files describe the level 1 data and the l2_ files describe the level 2 data.
List All Shots#
shots_df = pd.read_parquet(f"{META_URL}/l2_shots.parquet")
shots_df
| shot_id | facility | equi_li2_max_current | shot_summary | type_ | equi_max_li2 | generic_min_radius_max_plasma | title | shot_experiment_number | generic_max_poloidal_area | ... | nbi_total_injected_energy_ss | nbi_injected_energy_ss_ruby_time | nbi_power_max_ss | nbi_power_truby_ss | nbi_power_ss_max_current | shot_scenario | scenario | rad_o2ratio | radii_c2ratio | shot_flat_top_duration | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 11766 | MAST | 0.902606 | COMMISSIONED ENGINEERING PLASMA WITH MOST PLAN... | dcat:Dataset | 1.890299 | 0.080 | Shot Dataset | 11766 | 1.824400 | ... | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN |
| 1 | 11767 | MAST | 0.897837 | COMMISSIONED ENGINEERING PLASMA WITH MOST PLAN... | dcat:Dataset | 1.444762 | 0.080 | Shot Dataset | 11767 | 1.846246 | ... | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN |
| 2 | 11768 | MAST | 0.901121 | COMMISSIONED ENGINEERING PLASMA WITH MOST PLAN... | dcat:Dataset | 1.603340 | 0.080 | Shot Dataset | 11768 | 1.842971 | ... | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN |
| 3 | 11769 | MAST | 0.798088 | COMMISSIONED ENGINEERING PLASMA WITH MOST PLAN... | dcat:Dataset | 1.902339 | 0.070 | Shot Dataset | 11769 | 1.899469 | ... | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN |
| 4 | 11771 | MAST | 0.809877 | COMMISSIONED ENGINEERING PLASMA WITH MOST PLAN... | dcat:Dataset | 2.214325 | 0.070 | Shot Dataset | 11771 | 1.903582 | ... | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN | NaN |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 11568 | 30467 | MAST | 0.807955 | 'Data from neutron camera and magnetics obtain... | dcat:Dataset | 6.996288 | 0.110 | Shot Dataset | 30467 | 1.972266 | ... | 0.340425 | 0.000195 | 2.080261 | NaN | 2.180491 | S6 | 3.0 | NaN | NaN | NaN |
| 11569 | 30468 | MAST | 0.799021 | 'Data from neutron camera and magnetics obtain... | dcat:Dataset | 8.578611 | 0.155 | Shot Dataset | 30468 | 1.973768 | ... | 0.288250 | 0.000208 | 2.083544 | NaN | 1.985447 | S8 | 2.0 | NaN | NaN | NaN |
| 11570 | 30469 | MAST | 0.814006 | 'Data from neutron camera and magnetics obtain... | dcat:Dataset | 2.681262 | 0.140 | Shot Dataset | 30469 | 1.943575 | ... | 0.328446 | 0.000189 | 2.163367 | NaN | 2.288206 | S6 | 3.0 | NaN | NaN | NaN |
| 11571 | 30470 | MAST | 0.802867 | 'Data from neutron camera and magnetics obtain... | dcat:Dataset | 12.264058 | 0.135 | Shot Dataset | 30470 | 1.972500 | ... | 0.294526 | 0.000207 | 2.204641 | NaN | 2.107413 | S8 | 2.0 | NaN | NaN | NaN |
| 11572 | 30471 | MAST | 0.963086 | 'Data from neutron camera and magnetics obtain... | dcat:Dataset | 62.251090 | 0.135 | Shot Dataset | 30471 | 1.965805 | ... | 0.171676 | 0.000202 | 2.062899 | NaN | NaN | S8 | 2.0 | NaN | NaN | NaN |
11573 rows × 189 columns
Filter Shots#
The metadata are ordinary pandas DataFrames, so you can filter them and select columns with pandas. For example, to get a few columns for every shot in the M9 campaign:
m9_shots = shots_df[shots_df.campaign == "M9"]
m9_shots[["shot_id", "timestamp", "preshot_description", "divertor_config"]]
| shot_id | timestamp | preshot_description | divertor_config | |
|---|---|---|---|---|
| 9164 | 28752 | 2013-05-20 13:30:00 | \nChange ZREF to +1cm.\n | Conventional |
| 9173 | 28647 | 2013-04-18 12:37:00 | \nRepeat after abort.\n | Conventional |
| 10154 | 28750 | 2013-05-20 12:57:00 | \nChange zref to -1 cm\n | Conventional |
| 10475 | 28631 | 2013-04-17 14:54:00 | \nRepeat, with loop volts tweak and +-25kA P1P... | Conventional |
| 10476 | 28632 | 2013-04-17 15:19:00 | \nRepeat, with -ve P1PS current limit 45kA. Tw... | Conventional |
| ... | ... | ... | ... | ... |
| 11568 | 30467 | 2013-09-27 14:03:00 | \nRepeat with new neutron camera position.\ncH... | Conventional |
| 11569 | 30468 | 2013-09-27 14:21:00 | \nRepeat with new neutron camera position.\ncH... | Conventional |
| 11570 | 30469 | 2013-09-27 14:39:00 | \nRepeat with increased beam power (74 kV)\ncH... | Conventional |
| 11571 | 30470 | 2013-09-27 15:03:00 | \nRepeat last using hydrogen in outboard and c... | Conventional |
| 11572 | 30471 | 2013-09-27 15:20:00 | \nThe last plasma:\nConvert to i/b Helios 1724... | Conventional |
1101 rows × 4 columns
List All Sources#
sources_df = pd.read_parquet(f"{META_URL}/l2_sources.parquet")
sources_df
| shot_id | type_ | description | title | name | endpoint_url | context_ | quality | uuid | url | imas | |
|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 14399 | dcat:Dataset | Source Dataset | gas_injection | https://s3.echo.stfc.ac.uk | {'dct': 'http://purl.org/dc/terms/', 'dqv': 'h... | Not Checked | f39ee009-4405-59ea-9672-443de2c051bd | s3://mast/level2/shots/14399.zarr | gas_injection | |
| 1 | 14399 | dcat:Dataset | Source Dataset | magnetics | https://s3.echo.stfc.ac.uk | {'dct': 'http://purl.org/dc/terms/', 'dqv': 'h... | Not Checked | 6eeee9b5-9803-57c7-9c8d-8d6f249b5983 | s3://mast/level2/shots/14399.zarr | magnetics | |
| 2 | 14399 | dcat:Dataset | Source Dataset | pf_active | https://s3.echo.stfc.ac.uk | {'dct': 'http://purl.org/dc/terms/', 'dqv': 'h... | Not Checked | c8fdcd7b-f882-5ac5-8dee-e5736f9abdec | s3://mast/level2/shots/14399.zarr | pf_active | |
| 3 | 14399 | dcat:Dataset | Source Dataset | pulse_schedule | https://s3.echo.stfc.ac.uk | {'dct': 'http://purl.org/dc/terms/', 'dqv': 'h... | Not Checked | bbd4f35f-bdf7-549d-93d0-7063b029a2e1 | s3://mast/level2/shots/14399.zarr | pulse_schedule | |
| 4 | 14399 | dcat:Dataset | Source Dataset | soft_x_rays | https://s3.echo.stfc.ac.uk | {'dct': 'http://purl.org/dc/terms/', 'dqv': 'h... | Not Checked | 82f89e7e-1eb6-516c-a8fb-39aee47fd2bf | s3://mast/level2/shots/14399.zarr | NaN | |
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 105993 | 14398 | dcat:Dataset | Source Dataset | soft_x_rays | https://s3.echo.stfc.ac.uk | {'dct': 'http://purl.org/dc/terms/', 'dqv': 'h... | Not Checked | 5af5a37d-d8fa-5a3c-a2a9-d32d4ac55538 | s3://mast/level2/shots/14398.zarr | NaN | |
| 105994 | 14398 | dcat:Dataset | Source Dataset | spectrometer_visible | https://s3.echo.stfc.ac.uk | {'dct': 'http://purl.org/dc/terms/', 'dqv': 'h... | Not Checked | 7aa3b4b0-33e6-5bdf-9e1b-a43dd8587152 | s3://mast/level2/shots/14398.zarr | NaN | |
| 105995 | 14398 | dcat:Dataset | Source Dataset | summary | https://s3.echo.stfc.ac.uk | {'dct': 'http://purl.org/dc/terms/', 'dqv': 'h... | Not Checked | a86b4d7d-55dd-528b-aa19-f77ddb8e06f3 | s3://mast/level2/shots/14398.zarr | summary | |
| 105996 | 14398 | dcat:Dataset | Source Dataset | thomson_scattering | https://s3.echo.stfc.ac.uk | {'dct': 'http://purl.org/dc/terms/', 'dqv': 'h... | Not Checked | 8cf090c4-6c4b-59bd-a8cb-8e09a06992f4 | s3://mast/level2/shots/14398.zarr | thomson_scattering | |
| 105997 | 14399 | dcat:Dataset | Source Dataset | equilibrium | https://s3.echo.stfc.ac.uk | {'dct': 'http://purl.org/dc/terms/', 'dqv': 'h... | Not Checked | 1031f2e4-8fbe-5e6e-863c-7b5bfd72c424 | s3://mast/level2/shots/14399.zarr | equilibrium |
105998 rows × 11 columns
List all signals Given a Shot Number#
signals_df = pd.read_parquet(f"{META_URL}/l2_signals/signals_30421.parquet")
signals_df
| shot_id | description | rank | dimensions | url | quality | endpoint_url | type_ | source | title | units | shape | uuid | imas | context_ | name | version | |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | 30421 | 2 | [time, major_radius] | s3://mast/level2/shots/30421.zarr | Not Checked | https://s3.echo.stfc.ac.uk | dcat:Dataset | charge_exchange | Signal Dataset | eV | [73, 160] | 75756c34-c5b1-5e75-a53e-82bd90635d75 | charge_exchange | {'dct': 'http://purl.org/dc/terms/', 'dqv': 'h... | t_i | 0 | |
| 1 | 30421 | 2 | [time, major_radius] | s3://mast/level2/shots/30421.zarr | Not Checked | https://s3.echo.stfc.ac.uk | dcat:Dataset | charge_exchange | Signal Dataset | eV | [73, 160] | dcb14130-bf6a-5cc3-90ff-3985dad388a7 | charge_exchange | {'dct': 'http://purl.org/dc/terms/', 'dqv': 'h... | v_i | 0 | |
| 2 | 30421 | 1 | [time] | s3://mast/level2/shots/30421.zarr | Not Checked | https://s3.echo.stfc.ac.uk | dcat:Dataset | equilibrium | Signal Dataset | [146] | eb848f08-2f4e-54a6-8bd5-98a98823aa2e | equilibrium | {'dct': 'http://purl.org/dc/terms/', 'dqv': 'h... | beta_normal | 0 | ||
| 3 | 30421 | 1 | [time] | s3://mast/level2/shots/30421.zarr | Not Checked | https://s3.echo.stfc.ac.uk | dcat:Dataset | equilibrium | Signal Dataset | [146] | cb95202b-279e-524a-b6af-899729ca9298 | equilibrium | {'dct': 'http://purl.org/dc/terms/', 'dqv': 'h... | beta_pol | 0 | ||
| 4 | 30421 | 1 | [time] | s3://mast/level2/shots/30421.zarr | Not Checked | https://s3.echo.stfc.ac.uk | dcat:Dataset | equilibrium | Signal Dataset | [146] | fd1ec81b-5eb9-5a59-b507-8b2df648e0ad | equilibrium | {'dct': 'http://purl.org/dc/terms/', 'dqv': 'h... | beta_tor | 0 | ||
| ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... | ... |
| 68 | 30421 | 2 | [time, major_radius] | s3://mast/level2/shots/30421.zarr | Not Checked | https://s3.echo.stfc.ac.uk | dcat:Dataset | thomson_scattering | Signal Dataset | eV | [146, 120] | f0b586cd-11ea-5208-927b-4e4f057a6c44 | thomson_scattering | {'dct': 'http://purl.org/dc/terms/', 'dqv': 'h... | n_e | 0 | |
| 69 | 30421 | 1 | [time] | s3://mast/level2/shots/30421.zarr | Not Checked | https://s3.echo.stfc.ac.uk | dcat:Dataset | thomson_scattering | Signal Dataset | eV | [146] | 097cfe72-4d53-5a04-a7dc-97b1898e6e98 | thomson_scattering | {'dct': 'http://purl.org/dc/terms/', 'dqv': 'h... | n_e_core | 0 | |
| 70 | 30421 | 2 | [time, major_radius] | s3://mast/level2/shots/30421.zarr | Not Checked | https://s3.echo.stfc.ac.uk | dcat:Dataset | thomson_scattering | Signal Dataset | eV | [146, 120] | 23c0ad57-6ff8-5222-9a7a-c897b661b9ad | thomson_scattering | {'dct': 'http://purl.org/dc/terms/', 'dqv': 'h... | p_e | 0 | |
| 71 | 30421 | 2 | [time, major_radius] | s3://mast/level2/shots/30421.zarr | Not Checked | https://s3.echo.stfc.ac.uk | dcat:Dataset | thomson_scattering | Signal Dataset | eV | [146, 120] | 6e2f4faf-7eb6-5f9e-a8a3-f0869af4abbf | thomson_scattering | {'dct': 'http://purl.org/dc/terms/', 'dqv': 'h... | t_e | 0 | |
| 72 | 30421 | 1 | [time] | s3://mast/level2/shots/30421.zarr | Not Checked | https://s3.echo.stfc.ac.uk | dcat:Dataset | thomson_scattering | Signal Dataset | eV | [146] | c83667d9-fd83-5808-90b9-bb6750e98b77 | thomson_scattering | {'dct': 'http://purl.org/dc/terms/', 'dqv': 'h... | t_e_core | 0 |
73 rows × 17 columns
Open and plot shot data.#
Here we are visualising some plasma current data.
import xarray as xr
import matplotlib.pyplot as plt
dataset = xr.open_zarr(
"https://s3.echo.stfc.ac.uk/mast/level2/shots/30421.zarr",
group='summary'
)
dataset['ip'].plot()
plt.show()
