EFIT Data

!pip install pandas matplotlib zarr fsspec s3fs intake intake_xarray intake_parquet
Hide code cell output 
Requirement already satisfied: pandas in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (2.0.3)
Requirement already satisfied: matplotlib in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (3.9.0)
Requirement already satisfied: zarr in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (2.15.0)
Requirement already satisfied: fsspec in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (2024.5.0)
Requirement already satisfied: s3fs in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (2024.5.0)
Requirement already satisfied: intake in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (2.0.5)
Requirement already satisfied: intake_xarray in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (0.7.0)
Requirement already satisfied: intake_parquet in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (0.3.0)
Requirement already satisfied: python-dateutil>=2.8.2 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from pandas) (2.9.0.post0)

Requirement already satisfied: pytz>=2020.1 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from pandas) (2024.1)
Requirement already satisfied: tzdata>=2022.1 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from pandas) (2024.1)
Requirement already satisfied: numpy>=1.21.0 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from pandas) (1.25.2)
Requirement already satisfied: contourpy>=1.0.1 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from matplotlib) (1.2.1)
Requirement already satisfied: cycler>=0.10 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from matplotlib) (0.12.1)
Requirement already satisfied: fonttools>=4.22.0 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from matplotlib) (4.53.0)
Requirement already satisfied: kiwisolver>=1.3.1 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from matplotlib) (1.4.5)
Requirement already satisfied: packaging>=20.0 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from matplotlib) (24.0)
Requirement already satisfied: pillow>=8 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from matplotlib) (10.3.0)
Requirement already satisfied: pyparsing>=2.3.1 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from matplotlib) (3.1.2)
Requirement already satisfied: asciitree in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from zarr) (0.3.3)
Requirement already satisfied: fasteners in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from zarr) (0.19)
Requirement already satisfied: numcodecs>=0.10.0 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from zarr) (0.12.1)
Requirement already satisfied: aiobotocore<3.0.0,>=2.5.4 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from s3fs) (2.13.0)
Requirement already satisfied: aiohttp!=4.0.0a0,!=4.0.0a1 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from s3fs) (3.9.5)
Requirement already satisfied: pyyaml in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from intake) (6.0.1)
Requirement already satisfied: appdirs in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from intake) (1.4.4)
Requirement already satisfied: xarray>=02022 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from intake_xarray) (2023.7.0)
Requirement already satisfied: dask>=2.2 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from intake_xarray) (2024.4.1)
Requirement already satisfied: netcdf4 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from intake_xarray) (1.6.4)
Requirement already satisfied: msgpack in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from intake_xarray) (1.0.8)
Requirement already satisfied: requests in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from intake_xarray) (2.31.0)
Requirement already satisfied: fastparquet in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from intake_parquet) (2024.11.0)
Requirement already satisfied: pyarrow in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from intake_parquet) (12.0.1)
Requirement already satisfied: botocore<1.34.107,>=1.34.70 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from aiobotocore<3.0.0,>=2.5.4->s3fs) (1.34.106)
Requirement already satisfied: wrapt<2.0.0,>=1.10.10 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from aiobotocore<3.0.0,>=2.5.4->s3fs) (1.16.0)
Requirement already satisfied: aioitertools<1.0.0,>=0.5.1 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from aiobotocore<3.0.0,>=2.5.4->s3fs) (0.11.0)
Requirement already satisfied: aiosignal>=1.1.2 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->s3fs) (1.3.1)
Requirement already satisfied: attrs>=17.3.0 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->s3fs) (23.2.0)
Requirement already satisfied: frozenlist>=1.1.1 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->s3fs) (1.4.1)
Requirement already satisfied: multidict<7.0,>=4.5 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->s3fs) (6.0.5)
Requirement already satisfied: yarl<2.0,>=1.0 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from aiohttp!=4.0.0a0,!=4.0.0a1->s3fs) (1.9.4)

Requirement already satisfied: click>=8.1 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from dask>=2.2->intake_xarray) (8.1.5)
Requirement already satisfied: cloudpickle>=1.5.0 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from dask>=2.2->intake_xarray) (3.0.0)
Requirement already satisfied: partd>=1.2.0 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from dask>=2.2->intake_xarray) (1.4.2)
Requirement already satisfied: toolz>=0.10.0 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from dask>=2.2->intake_xarray) (0.12.1)
Requirement already satisfied: importlib-metadata>=4.13.0 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from dask>=2.2->intake_xarray) (7.1.0)
Requirement already satisfied: six>=1.5 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from python-dateutil>=2.8.2->pandas) (1.16.0)

Requirement already satisfied: cramjam>=2.3 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from fastparquet->intake_parquet) (2.9.0)
Requirement already satisfied: cftime in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from netcdf4->intake_xarray) (1.6.3)
Requirement already satisfied: certifi in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from netcdf4->intake_xarray) (2024.2.2)
Requirement already satisfied: charset-normalizer<4,>=2 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from requests->intake_xarray) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from requests->intake_xarray) (3.7)
Requirement already satisfied: urllib3<3,>=1.21.1 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from requests->intake_xarray) (2.2.1)
Requirement already satisfied: jmespath<2.0.0,>=0.7.1 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from botocore<1.34.107,>=1.34.70->aiobotocore<3.0.0,>=2.5.4->s3fs) (1.0.1)
Requirement already satisfied: zipp>=0.5 in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from importlib-metadata>=4.13.0->dask>=2.2->intake_xarray) (3.18.1)
Requirement already satisfied: locket in /Users/rt2549/miniconda3/envs/mast/lib/python3.11/site-packages (from partd>=1.2.0->dask>=2.2->intake_xarray) (1.0.0)

import intake
import numpy as np
import matplotlib.pyplot as plt

First let’s take a look at the top level shots. In this table we can find all of the metadata we have about particular shots.

catalog  = intake.open_catalog('https://mastapp.site/intake/catalog.yml')
shots_df = catalog.index.level1.shots().read()
shots_df
url preshot_description postshot_description campaign current_range divertor_config plasma_shape comissioner facility shot_id ... cpf_vol_ipmax cpf_vol_max cpf_vol_truby cpf_wmhd_ipmax cpf_wmhd_max cpf_wmhd_truby cpf_zeff_ipmax cpf_zeff_max cpf_zeff_truby cpf_zmag_efit
0 s3://mast/level1/shots/11695.zarr \n0.1T TF SHOT\n \nOK\n M5 None Conventional None None MAST 11695 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
1 s3://mast/level1/shots/11696.zarr \nSTANDARD 0.3T TF SHOT\n \nOK\n M5 None Conventional None None MAST 11696 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
2 s3://mast/level1/shots/11697.zarr \nRAISE TO 0.5T\n \nOK, ALARMS ARE LOWER\n M5 None Conventional None None MAST 11697 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
3 s3://mast/level1/shots/11698.zarr \nRAISE TO .56T\n \nSTILL ALARMS BUT LOWER AGAIN\n M5 None Conventional None None MAST 11698 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
4 s3://mast/level1/shots/11699.zarr \nRAISE TO .58T\n \nOK\n M5 None Conventional None None MAST 11699 ... NaN NaN NaN NaN NaN NaN NaN NaN NaN NaN
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
15916 s3://mast/level1/shots/30467.zarr \nRepeat with new neutron camera position.\ncH... \nTwo times lower DD neutron rate than referen... M9 700 kA Conventional Connected Double Null None MAST 30467 ... 9.029202 9.046394 0.0 49469.122469 52653.445 0.0 NaN NaN NaN 0.013202
15917 s3://mast/level1/shots/30468.zarr \nRepeat with new neutron camera position.\ncH... \nGood beam.\nGood repeat.\n M9 700 kA Conventional Lower Single Null None MAST 30468 ... 9.102411 9.107017 0.0 48516.962675 49382.133 0.0 NaN NaN NaN 0.012445
15918 s3://mast/level1/shots/30469.zarr \nRepeat with increased beam power (74 kV)\ncH... \nGood shot. Modes present.\n M9 700 kA Conventional Connected Double Null None MAST 30469 ... 8.988730 9.047923 0.0 47466.249616 49115.805 0.0 NaN NaN NaN 0.015299
15919 s3://mast/level1/shots/30470.zarr \nRepeat last using hydrogen in outboard and c... \nNo HF gas.\n M9 700 kA Conventional None None MAST 30470 ... 9.687049 10.055509 0.0 17290.432865 22310.516 0.0 NaN NaN NaN 0.015164
15920 s3://mast/level1/shots/30471.zarr \nThe last plasma:\nConvert to i/b Helios 1724... \nGood shot.\n M9 700 kA Conventional Lower Single Null None MAST 30471 ... 8.817559 9.283702 0.0 38063.582380 40906.090 0.0 NaN NaN NaN 0.014340

15921 rows × 283 columns

EFIT Data#

This notebook contains some examples of loading and plotting equillibrium reconstruction data from EFIT.

First we can have a look at how many EFIT reconstructions are in the database:

catalog  = intake.open_catalog('https://mastapp.site/intake/catalog.yml')
sources_df = catalog.index.level1.sources().read()

sources_df = sources_df.loc[(sources_df.name == 'efm')]
sources_df
description quality uuid shot_id name url
2 Basic EFIT Not Checked fd0a0dc4-1ed8-546f-8c02-455061374fd4 11695 efm s3://mast/level1/shots/11695.zarr/efm
6 Basic EFIT Not Checked c553a338-249f-58d5-9ba8-14f5c507c4e4 11696 efm s3://mast/level1/shots/11696.zarr/efm
10 Basic EFIT Not Checked f2ce95e9-bfe8-595d-a1eb-d860de377ab2 11697 efm s3://mast/level1/shots/11697.zarr/efm
14 Basic EFIT Not Checked f1fbd4b8-d68b-54bd-a9c0-209fddf9cef0 11698 efm s3://mast/level1/shots/11698.zarr/efm
18 Basic EFIT Not Checked 798d44ea-0773-5020-8c3c-083b1bdb3aab 11699 efm s3://mast/level1/shots/11699.zarr/efm
... ... ... ... ... ... ...
307982 Basic EFIT Not Checked 72422662-d3dd-5bb9-abdc-5e5e3427587e 30467 efm s3://mast/level1/shots/30467.zarr/efm
308017 Basic EFIT Not Checked 5f073f88-7f09-53ca-b204-e9c620b0672d 30468 efm s3://mast/level1/shots/30468.zarr/efm
308052 Basic EFIT Not Checked 0bcd11d2-f8f5-5ca3-94e6-235c13b1434d 30469 efm s3://mast/level1/shots/30469.zarr/efm
308087 Basic EFIT Not Checked 91cbcfda-7434-55be-a760-80493d6e8f46 30470 efm s3://mast/level1/shots/30470.zarr/efm
308122 Basic EFIT Not Checked 1b50bfb9-e9f3-5c23-8f5b-5821e441d1b7 30471 efm s3://mast/level1/shots/30471.zarr/efm

13621 rows × 6 columns

Let’s look at the data for a particular shot. Here we are going to use shot 30420 as an example. Below we get the url for the efm data.

shot_id = 30420
url = sources_df.loc[sources_df.shot_id == shot_id].iloc[0].url
url

's3://mast/level1/shots/30420.zarr/efm'

efm source holds the EFIT data as a single dataset.

dataset = catalog.level1.sources(url=url)
dataset = dataset.to_dask()

dataset

<xarray.Dataset>
Dimensions:            (time: 64, psi_norm: 65, n_iterations: 10,
                        fcoil_seg_n: 938, fcoil_n: 101, ffprime_coefs_n: 2,
                        mag_probe_n: 78, psi_loop_n: 46, r: 65, z: 65,
                        profile_r: 129, lcfs_coords: 147, limiter_n: 37,
                        pprime_coefs_n: 2, profile_z: 65)
Coordinates: (12/13)
  * fcoil_n            (fcoil_n) float32 0.0 1.0 2.0 3.0 ... 98.0 99.0 100.0
  * ffprime_coefs_n    (ffprime_coefs_n) float32 0.0 1.0
  * lcfs_coords        (lcfs_coords) float32 0.0 1.0 2.0 ... 144.0 145.0 146.0
  * mag_probe_n        (mag_probe_n) float32 0.0 1.0 2.0 3.0 ... 75.0 76.0 77.0
  * n_iterations       (n_iterations) float32 0.0 1.0 2.0 3.0 ... 7.0 8.0 9.0
  * pprime_coefs_n     (pprime_coefs_n) float32 0.0 1.0
    ...                 ...
  * profile_z          (profile_z) float32 -2.0 -1.938 -1.875 ... 1.938 2.0
  * psi_loop_n         (psi_loop_n) float32 0.0 1.0 2.0 3.0 ... 43.0 44.0 45.0
  * psi_norm           (psi_norm) float32 0.0 0.01562 0.03125 ... 0.9844 1.0
  * r                  (r) float32 0.06 0.09031 0.1206 0.1509 ... 1.939 1.97 2.0
  * time               (time) float32 -0.05 -0.045 -0.04 ... 0.29 0.295 0.3
  * z                  (z) float32 -2.0 -1.938 -1.875 -1.812 ... 1.875 1.938 2.0
Dimensions without coordinates: fcoil_seg_n, limiter_n
Data variables: (12/151)
    all_times          (time) float32 dask.array<chunksize=(64,), meta=np.ndarray>
    areap_c            (time, psi_norm) float32 dask.array<chunksize=(64, 65), meta=np.ndarray>
    betan              (time) float32 dask.array<chunksize=(64,), meta=np.ndarray>
    betap              (time) float32 dask.array<chunksize=(64,), meta=np.ndarray>
    betapd             (time) float32 dask.array<chunksize=(64,), meta=np.ndarray>
    betat              (time) float32 dask.array<chunksize=(64,), meta=np.ndarray>
    ...                 ...
    wpol               (time) float32 dask.array<chunksize=(64,), meta=np.ndarray>
    xpoint1_rc         (time) float32 dask.array<chunksize=(64,), meta=np.ndarray>
    xpoint1_zc         (time) float32 dask.array<chunksize=(64,), meta=np.ndarray>
    xpoint2_rc         (time) float32 dask.array<chunksize=(64,), meta=np.ndarray>
    xpoint2_zc         (time) float32 dask.array<chunksize=(64,), meta=np.ndarray>
    zbdry              (time) float32 dask.array<chunksize=(64,), meta=np.ndarray>
Attributes:
    description:  Basic EFIT
    file_name:    efm0304.20
    format:       IDA3
    mds_name:     None
    name:         efm
    quality:      Not Checked
    shot_id:      30420
    signal_type:  Analysed
    source:       efm
    uda_name:     EFM
    uuid:         1e39c600-8ffb-5f56-900d-2941e352319c
    version:      0
time_index = 50

plasma_current = dataset['plasma_current_rz'].dropna(dim='time')
plasma_current = plasma_current.isel(time=time_index)

polodial_flux_rz = dataset['psirz']
polodial_flux_rz = polodial_flux_rz.dropna(dim='profile_r')
polodial_flux_rz = polodial_flux_rz.isel(time=time_index)

lcfs_R = dataset['lcfs_r'].isel(time=time_index)
lcfs_Z = dataset['lcfs_z'].isel(time=time_index)

# Get the R and Z coordinates of the profiles.
r = dataset['r']
z = dataset['z']
R, Z = np.meshgrid(r, z)

# Get the x-point
xpoint_r = dataset['xpoint2_rc'][time_index]
xpoint_z = dataset['xpoint2_zc'][time_index]

# Get the current centre
mag_axis_r = dataset['current_centrd_r'][time_index]
mag_axis_z = dataset['current_centrd_z'][time_index]


# Get the last closed flux surface (LCFS)
lcfs_r = lcfs_R.values
lcfs_r = lcfs_r[~np.isnan(lcfs_r)]
lcfs_z = lcfs_Z.values
lcfs_z = lcfs_z[~np.isnan(lcfs_z)]

fig, (ax1, ax2) = plt.subplots(1, 2)
ax1.contourf(R, Z, polodial_flux_rz, cmap='magma', levels=50, label='Polodial Flux')
ax1.scatter(xpoint_r, xpoint_z, marker='x', color='green', label='X Point')
ax1.scatter(mag_axis_r, mag_axis_z, marker='o', color='purple', label='Current Centre')
ax1.plot(lcfs_r, lcfs_z, c='blue', linestyle='--', label='LCFS')
ax1.set_title(f'Polodial Flux for Shot {polodial_flux_rz.attrs["shot_id"]}')
ax1.set_ylabel('Z (m)')
ax1.set_xlabel('R (m)')

ax2.contourf(R, Z, plasma_current, cmap='magma', levels=20, label='Plasma Current')
ax2.scatter(xpoint_r, xpoint_z, marker='x', color='green', label='X Point')
ax2.scatter(mag_axis_r, mag_axis_z, marker='o', color='purple', label='Current Centre')
ax2.plot(lcfs_r, lcfs_z, c='blue', linestyle='--', label='LCFS')
ax2.set_title(f'Plasma Current for Shot {plasma_current.attrs["shot_id"]}')
plt.ylabel('Z (m)')
plt.xlabel('R (m)')
plt.legend()
plt.tight_layout()

/var/folders/xr/yr8z575s52b4tbg3fj65qwx00000gp/T/ipykernel_88474/494923854.py:34: UserWarning: The following kwargs were not used by contour: 'label'
  ax1.contourf(R, Z, polodial_flux_rz, cmap='magma', levels=50, label='Polodial Flux')

/var/folders/xr/yr8z575s52b4tbg3fj65qwx00000gp/T/ipykernel_88474/494923854.py:42: UserWarning: The following kwargs were not used by contour: 'label'
  ax2.contourf(R, Z, plasma_current, cmap='magma', levels=20, label='Plasma Current')
_images/04bdedcba43414306f35dd4d9db8306d1817fed6b299c823feda6b74e5e8082d.png