Reader for native SPECS SLE format #110
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Pull Request Test Coverage Report for Build 5142075054
💛 - Coveralls |
Initial commit for XPS reader for SLE files (native format of SPECS Prodigy) - Add sle_parsers which read the schedule (as xml) as well as the data from the SLE datafiles (which are SQLlite databases) - Implement interface in dataconverter/xps/reader_utils for SLE readers for different versions of .sle files Currently missing: - mapping from output format of parse_file method of the different SleParsers to get_dict method of parse_xml of XML reader -> SleParser output cannot be mapped to Nxmpes at the moment
The output of the SleParsers is a flattened list of dictionaries, which each dict containing the data and metadata of one spectrum (including the individual channel scans). This list is mapped to a dictionary that has a similar structure as the output of the XmlSpecs parser, which can then be used to write the nxmpes structure using config files.
- Individual channel scans are stored in the data field of the spectrum dictionaries - Fix some data type errors during reading - Read out transmission data. Note: Channel readout and calibration is currently not working ( see method _get_calibrated_data). Therefore, the "calibrated" data is currently just the scan of the first channel.
- Restructure CONVERT_DICT - Add "NXmpes_xps" to supported NXDLs - Config files: either the config files in the xps folder are used (based on the file extension of the input) or a config file can be explicitly passed during the reading
- Read both TF data and the file it was stored in.
Since there is no NXDL for the NXmpes_xps application definition, the pytests are failing. Temporarily remove it until NXDL is available.
Different SLE parsers have different xml schedules. Implement abstractmethod for abstract SleParser base class.
sanbrock
reviewed
May 30, 2023
RubelMozumder
pushed a commit
that referenced
this pull request
Jun 28, 2023
* Add reader for .sle XPS files (SPECS Prodigy) Initial commit for XPS reader for SLE files (native format of SPECS Prodigy) - Add sle_parsers which read the schedule (as xml) as well as the data from the SLE datafiles (which are SQLlite databases) - Implement interface in dataconverter/xps/reader_utils for SLE readers for different versions of .sle files Currently missing: - mapping from output format of parse_file method of the different SleParsers to get_dict method of parse_xml of XML reader -> SleParser output cannot be mapped to Nxmpes at the moment * Pylinting, minor naming updates * Map from SleParser to common format The output of the SleParsers is a flattened list of dictionaries, which each dict containing the data and metadata of one spectrum (including the individual channel scans). This list is mapped to a dictionary that has a similar structure as the output of the XmlSpecs parser, which can then be used to write the nxmpes structure using config files. * Small updates to SleParsers - Individual channel scans are stored in the data field of the spectrum dictionaries - Fix some data type errors during reading - Read out transmission data. Note: Channel readout and calibration is currently not working ( see method _get_calibrated_data). Therefore, the "calibrated" data is currently just the scan of the first channel. * Small changes to reader.py, multiple config files - Restructure CONVERT_DICT - Add "NXmpes_xps" to supported NXDLs - Config files: either the config files in the xps folder are used (based on the file extension of the input) or a config file can be explicitly passed during the reading * Fix error in transmission data parsing - Read both TF data and the file it was stored in. * Allow kwargs passing to the SleParsers * Small update to config file parsing * Pycodestyle linting * Temporarily remove NXmpes_xps from supported_nxdls Since there is no NXDL for the NXmpes_xps application definition, the pytests are failing. Temporarily remove it until NXDL is available. * Implement abstractmethod for flattening XML Different SLE parsers have different xml schedules. Implement abstractmethod for abstract SleParser base class. * pylinting * Linting * some more linting * Mypy changes
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Implement reader for different versions of .sle file.
spectrum (including the individual channel scans).
Mapping from parser output to a dictionary that has a similar structure as the output of the XmlSpecs parser, which can then be used to write the nxmpes structure using config files.
Different config files for xml and sle parsing are read automatically based on the file extension.
Keyword arguments can be passed from convert to the individual parsers. For SLE files, one can e.g. remove the initial position alignment scans which improves the reading time.
Not working: