FAIRmat-NFDI · mkuehbach · Aug 31, 2023 · Jun 2, 2023 · Jun 2, 2023 · Jun 10, 2023
diff --git a/README.md b/README.md
@@ -71,6 +71,18 @@ Especially relevant for developers, there exists a basic test framework written
    <
8000
span class='blob-code-inner blob-code-marker ' data-code-marker=" ">python -m pytest -sv tests
 ```
 
+## Run examples
+
+A number of examples exist which document how the tools can be used. For a standalone
+usage convenient jupyter notebooks are available for each tool. To use them jupyter
+and related tools have to be installed in the development environment as follows:
+
+```shell
+python -m pip install jupyter
+python -m pip install jupyterlab
+python -m pip install jupyterlab_h5web
+```
+
 # Questions, suggestions?
 
 To ask further questions, to make suggestions how we can improve these tools, to get advice

diff --git a/dev-requirements.txt b/dev-requirements.txt
@@ -19,6 +19,7 @@ astroid==2.15.5
 attrs==22.1.0
     # via
     #   cattrs
+    #   pytest
     #   requests-cache
 backcall==0.2.0
     # via ipython
@@ -340,6 +341,8 @@ psutil==5.9.2
     #   pyxem
 ptyprocess==0.7.0
     # via pexpect
+py==1.11.0
+    # via pytest
 pycifrw==4.4.5
     # via diffpy-structure
 pycodestyle==2.9.1

diff --git a/examples/apm/Write.NXapm.Example.1.ipynb b/examples/apm/Write.NXapm.Example.1.ipynb
@@ -195,6 +195,20 @@
     "### **Step 3c**: Convert the files in the example into an NXapm-compliant NeXus/HDF5 file."
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "<div class=\"alert alert-block alert-info\">\n",
+    "The deployment_specific YAML file can be used to simplify a specific ELN, e.g. to avoid having to type<br>\n",
+    "repetitive information. A typical use case is an OASIS in a laboratory where there is a fixed number of<br>\n",
+    "instruments for which many details stay the same, these can be carried over via a *.oasis.specific.yaml file.<br>\n",
+    "It is the responsibility of the OASIS maintainer to document and version these specific configurations.<br>\n",
+    "We would be happy to learn how this functionality is useful and learn about your feedback to improve<br>\n",
+    "this feature to make working with NeXus and ELNs more convenient.<br>\n",
+    "</div>"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,
@@ -204,35 +218,38 @@
    "outputs": [],
    "source": [
     "#parser-nexus/tests/data/tools/dataconverter/readers/em_om/\n",
+    "import numpy as np\n",
     "eln_data_file_name = [\"eln_data_apm.yaml\"]\n",
+    "deployment_specific = [\"apm.oasis.specific.yaml\"]\n",
     "input_recon_file_name = [\"Si.apt\",\n",
     "                         \"Si.epos\",\n",
     "                         \"Si.pos\",\n",
-    "                         \"R31_06365-v02.pos\",\n",
-    "                         \"R18_58152-v02.epos\",\n",
-    "                         \"70_50_50.apt\"]\n",
+    "#                         \"R31_06365-v02.pos\",\n",
+    "#                         \"R18_58152-v02.epos\",\n",
+    "#                         \"70_50_50.apt\"]\n",
     "#                         \"R56_01769-v01.pos\"]\n",
     "input_range_file_name = [\"Si.RRNG\",\n",
     "                         \"Si.RNG\",\n",
     "                         \"Si.RNG\",\n",
-    "                         \"R31_06365-v02.rrng\",\n",
-    "                         \"R31_06365-v02.rrng\",\n",
-    "                         \"R31_06365-v02.rrng\"]\n",
+    "#                         \"R31_06365-v02.rrng\",\n",
+    "#                         \"R31_06365-v02.rrng\",\n",
+    "#                         \"R31_06365-v02.rrng\"]\n",
     "#                         \"R56_01769.rng.fig.txt\"]\n",
     "output_file_name = [\"apm.case1.nxs\",\n",
     "                    \"apm.case2.nxs\",\n",
     "                    \"apm.case3.nxs\",\n",
     "                    \"apm.case4.nxs\",\n",
     "                    \"apm.case5.nxs\",\n",
     "                    \"apm.case6.nxs\"]\n",
-    "for case_id in [0]:\n",
+    "for case_id in np.arange(0, 3):\n",
     "    ELN = eln_data_file_name[0]\n",
+    "    OASIS = deployment_specific[0]\n",
     "    INPUT_RECON = input_recon_file_name[case_id]\n",
     "    INPUT_RANGE = input_range_file_name[case_id]\n",
     "    OUTPUT = output_file_name[case_id]\n",
     "\n",
-    "    ! dataconverter --reader apm --nxdl NXapm --input-file $ELN --input-file \\\n",
-    "    $INPUT_RECON --input-file $INPUT_RANGE --output $OUTPUT"
+    "    ! dataconverter --reader apm --nxdl NXapm --input-file $ELN --input-file $OASIS \\\n",
+    "    --input-file $INPUT_RECON --input-file $INPUT_RANGE --output $OUTPUT"
    ]
   },
   {
@@ -258,7 +275,7 @@
    "outputs": [],
    "source": [
     "# H5Web(OUTPUT)\n",
-    "H5Web(\"apm.case1.nxs\")"
+    "H5Web(\"apm.case3.nxs\")"
    ]
   },
   {
@@ -417,7 +434,7 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "<div class=\"alert alert-block alert-danger\">\n",
+    "<div class=\"alert alert-block alert-warning\">\n",
     "Currently, this functionality requires a Python environment with a newer version of the ase library than the one<br>\n",
     "which is used by the installation of pynxtools (which is currently ase==3.19.0). Instead, ase>=3.22.1 should be used.<br>\n",
     "The issue with the specific functionalities used in the *create_reconstructed_positions* function is that when using<br>\n",
@@ -448,12 +465,14 @@
    "cell_type": "markdown",
    "metadata": {},
    "source": [
-    "<div class=\"alert alert-block alert-danger\">\n",
-    "This functionality uses recent features of ase which demands an environment that is currently not supported<br>\n",
+    "<div class=\"alert alert-block alert-warning\">\n",
+    "This functionality uses recent features of ase which demands an environment that is not necessarily supported<br>\n",
     "by NOMAD OASIS. As the here exemplified settings for this example are configured to represent an environment<br>\n",
-    "matching close to NOMAD users who are interested in this developer functionality should do the following:<br>\n",
+    "matching one which is close to NOMAD, users who are interested in this dev functionality should do the following:<br>\n",
     "Run this example in a standalone environment where ase is upgraded to the latest version and then use<br>\n",
     "the generated NeXus files either as is or upload them to NOMAD OASIS.<br>\n",
+    "If the above-mentioned cell detects e.g. that a recent version of ase was installed<br>\n",
+    "(e.g. >3.22.x) then the code in the following cell can be executed without issues.<br>\n",
     "</div>"
    ]
   },
@@ -465,7 +484,7 @@
    },
    "outputs": [],
    "source": [
-    "# ! dataconverter --reader apm --nxdl NXapm --input-file synthesize1 --output apm.case0.nxs"
+    "! dataconverter --reader apm --nxdl NXapm --input-file synthesize1 --output apm.case0.nxs"
    ]
   },
   {
@@ -496,7 +515,7 @@
    "metadata": {},
    "source": [
     "### Contact person for the apm reader and related examples in FAIRmat:\n",
-    "Markus Kühbach, 2023/05<br>\n",
+    "Markus Kühbach, 2023/08/31<br>\n",
     "\n",
     "### Funding\n",
     "<a href=\"https://www.fairmat-nfdi.eu/fairmat\">FAIRmat</a> is a consortium on research data management which is part of the German NFDI.<br>\n",
@@ -527,7 +546,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.8.16"
+   "version": "3.10.12"
   }
 },
  "nbformat": 4,

diff --git a/examples/apm/apm.oasis.specific.yaml b/examples/apm/apm.oasis.specific.yaml
@@ -0,0 +1 @@
+location: Leoben
diff --git a/examples/apm/eln_data_apm.yaml b/examples/apm/eln_data_apm.yaml
@@ -1,82 +1,118 @@
 atom_probe:
   analysis_chamber_pressure:
     unit: torr
-    value: 1.0e-10
+    value: 2.0e-10
   control_software_program: IVAS
-  control_software_program__attr_version: 3.6.4
-  fabrication_capabilities: n/a
-  fabrication_identifier: n/a
+  control_software_program__attr_version: 3.6.8
+  fabrication_identifier: '12'
   fabrication_model: LEAP3000
-  fabrication_vendor: AMETEK/Cameca
+  fabrication_vendor: Cameca
+  field_of_view:
+    unit: nm
+    value: 20.
   flight_path_length:
     unit: m
-    value: 0.9
-  instrument_name: LEAP 3000
-  ion_detector_model: cameca
-  ion_detector_name: none
+    value: 1.2
+  instrument_name: LEAP
+  ion_detector_model: n/a
+  ion_detector_name: n/a
   ion_detector_serial_number: n/a
   ion_detector_type: mcp_dld
-  local_electrode_name: electrode 1
+  local_electrode_name: L1
+  location: Denton
   pulser:
-    laser_source_name: laser
-    laser_source_power:
-      unit: W
-      value: 2.0e-08
-    laser_source_pulse_energy:
-      unit: J
-      value: 1.2e-11
-    laser_source_wavelength:
-      unit: m
-      value: 4.8e-07
-    pulse_fraction: 0.1
+    laser_source:
+    - name: laser1
+      power:
+        unit: nW
+        value: 24.0
+      pulse_energy:
+        unit: pJ
+        value: 24.0
+      wavelength:
+        unit: nm
+        value: 355.0
+    - name: laser2
+      power:
+        unit: nW
+        value: 12.0
+      pulse_energy:
+        unit: pJ
+        value: 12.0
+      wavelength:
+        unit: nm
+        value: 254.0
+    pulse_fraction: 0.8
     pulse_frequency:
       unit: kHz
-      value: 250
-    pulse_mode: laser
+      value: 250.0
+    pulse_mode: laser_and_voltage
   reflectron_applied: true
-  specimen_monitoring_detection_rate: 0.6
+  specimen_monitoring_detection_rate: 0.8
F438
   specimen_monitoring_initial_radius:
     unit: nm
-    value: 30
+    value: 12.0
   specimen_monitoring_shank_angle:
     unit: °
-    value: 5
+    value: 5.0
   stage_lab_base_temperature:
     unit: K
-    value: 30
+    value: 20.0
   status: success
 entry:
-  attr_version: nexus-fairmat-proposal successor of 9636feecb79bb32b828b1a9804269573256d7696
-  definition: NXapm
-  end_time: '2022-09-22T20:00:00+00:00'
-  experiment_description: some details for nomad, ODS steel precipitates for testing
-    a developmental clustering algorithm called OPTICS.
-  experiment_identifier: R31-06365-v02
+  experiment_description: '<p>Normal</p>
+
+    <p><strong>Bold</strong></p>
+
+    <p><em>Italics</em></p>'
+  experiment_identifier: Si test
+  start_time: '2023-06-11T11:20:00+00:00'
+  end_time: '2023-06-11T11:20:00+00:00'
+  run_number: '2121'
   operation_mode: apt
-  program: IVAS
-  program__attr_version: 3.6.4
-  run_number: '6365'
-  start_time: '2022-09-20T20:00:00+00:00'
 ranging:
   program: IVAS
-  program__attr_version: 3.6.4
+  program__attr_version: 3.6.8
 reconstruction:
   crystallographic_calibration: n/a
-  parameter: kf = 1.8, ICF = 1.02, Vat = 60 at/nm^3
+  parameter: kf = 1.8, icf = 3.3
   program: IVAS
-  program__attr_version: 3.6.4
-  protocol_name: cameca
+  program__attr_version: 3.6.8
+  protocol_name: bas
+sample:
+  composition:
+  - Mo
+  - Al 12 +- 3
+  - B 50 ppm +- 12
+  - C 3.6
+  grain_diameter:
+    unit: µm
+    value: 200.0
+  grain_diameter_error:
+    unit: µm
+    value: 50.0
+  heat_treatment_quenching_rate:
+    unit: K / s
+    value: 150.0
+  heat_treatment_quenching_rate_error:
+    unit: K / s
+    value: 10.0
+  heat_treatment_temperature:
+    unit: K
+    value: 600.0
+  heat_treatment_temperature_error:
+    unit: K
+    value: 20.0
 specimen:
-  atom_types:
-  - Fe
-  - Cr
-  - Y
-  - O
-  description: ODS steel, i.e. material with Y2O3 dispersoids
-  name: ODS-Specimen 1
-  preparation_date: '2022-09-12T20:01:00+00:00'
-  sample_history: undocumented
-  short_title: ODS
+  alias: Si
+  description: '<p>normal</p>
+
+    <p><em>bold</em></p>
+
+    <p><em>italics</em></p>'
+  is_polycrystalline: true
+  name: usa_denton_smith_si
+  preparation_date: '2023-06-11T12:51:00+00:00'
 user:
-- name: Jing Wang
-- name: Daniel Schreiber
+- {}
+- {}
diff --git a/examples/em_nion/Write.NXem_nion.Example.1.ipynb b/examples/em_nion/Write.NXem_nion.Example.1.ipynb
@@ -88,7 +88,15 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "! wget https://www.zenodo.org/record/7986279/files/ger_berlin_haas_nionswift_multimodal.zip\n",
+    "! wget https://www.zenodo.org/record/7986279/files/ger_berlin_haas_nionswift_multimodal.zip"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
     "zp.ZipFile(\"ger_berlin_haas_nionswift_multimodal.zip\").extractall(path=\"\", members=None, pwd=None)"
    ]
   },
@@ -240,7 +248,7 @@
    "metadata": {},
    "source": [
     "### Contact person for the em_nion reader and related examples in FAIRmat:\n",
-    "Markus Kühbach, 2023/05<br>\n",
+    "Markus Kühbach, 2023/08/31<br>\n",
     "\n",
     "### Funding\n",
     "<a href=\"https://www.fairmat-nfdi.eu/fairmat\">FAIRmat</a> is a consortium on research data management which is part of the German NFDI.<br>\n",
@@ -271,7 +279,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.10.6"
+   "version": "3.10.12"
   }
  },
  "nbformat": 4,