8000 Add parsing of Scienta text export by lukaspie · Pull Request #129 · FAIRmat-NFDI/pynxtools · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

8000 Add parsing of Scienta text export #129

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 14 commits into from
Jun 19, 2023
Merged

Add parsing of Scienta text export #129

merged 14 commits into from
Jun 19, 2023

Conversation

lukaspie
Copy link
Collaborator
  • Parse exported TXT files from Scienta vendor
  • Mapping from scienta_txt_parser to NXmpes]
  • Add the possibility of having multiple XPS data files such that each file can have multiple entries in the resulting NeXus file

lukaspie added 6 commits June 15, 2023 10:28
@lukaspie
Add parser for Scienta TXT export
73df6ed 
- Extract all lines from Scienta .txt export file
- Read out data and region meta data.
- Store all in a flat list of dictionaries, with each dict 
8000
containing one spectrum.
@lukaspie
Add mapping from scienta_txt_parser to NXmpes
5d5a219 
- Add class ScientaTxtSpecs for mapping from scienta_txt_parser output to NXmpes-ready dictionary
- Move methods for key construction to function level (reused in all Specs classes)
- Add new reader to the XpsDataFileParser readers
- Update error messages
@lukaspie
Allow the reader to read scienta txt file
19c83e4 
- Change allowed file extensions in XPS reader
- Add config file for Scienta txt reader
@lukaspie
Add default channel 0 data for raw detector data
05f0fde 
- Since in the TXT export, the data for each channel is not given, we use the exported counts as the raw detector data.
@lukaspie
Small change to SLE reader
a31b762
@lukaspie
Enable parsing of multiple data files
5942ed0 
 - In the read method of the XPSReader, if there are two data files (e.g. with file extensions ".sle", ".xml", ".txt"), ensure that existing keys are not overwritten (especially 'data').
 - Instead, the existing values of existing keys are extended, dependent on the type of the value.
@github-actions
Copy link
Contributor

Pull Request Test Coverage Report for Build 5280291390

  • 42 of 241 (17.43%) changed or added relevant lines in 3 files are covered.
  • 4 unchanged lines in 3 files lost coverage.
  • Overall coverage decreased (-0.7%) to 49.37%
Changes Missing Coverage Covered Lines Changed/Added Lines %
pynxtools/dataconverter/readers/xps/reader.py 7 16 43.75%
pynxtools/dataconverter/readers/xps/reader_utils.py 16 93 17.2%
pynxtools/dataconverter/readers/xps/scienta_txt_parsers.py 19 132 14.39%
Files with Coverage Reduction New Missed Lines %
pynxtools/dataconverter/readers/em_nion/concepts/swift_handle_nx_concepts.py 1 11.86%
pynxtools/nyaml2nxdl/nyaml2nxdl_forward_tools.py 1 79.42%
pynxtools/nexus/nexus.py 2 73.67%
Totals Coverage Status
Change from base Build 5275838012: -0.7%
Covered Lines: 5371
Relevant Lines: 10879
💛 - Coveralls

@coveralls
Copy link

Pull Request Test Coverage Report for Build 5281037287

  • 46 of 241 (19.09%) changed or added relevant lines in 3 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage decreased (-0.6%) to 49.425%
Changes Missing Coverage Covered Lines Changed/Added Lines %
pynxtools/dataconverter/readers/xps/reader.py 6 15 40.0%
pynxtools/dataconverter/readers/xps/reader_utils.py 21 94 22.34%
pynxtools/dataconverter/readers/xps/scienta_txt_parsers.py 19 132 14.39%
Totals Coverage Status
Change from base Build 5275838012: -0.6%
Covered Lines: 5373
Relevant Lines: 10871
💛 - Coveralls

@coveralls
Copy link
coveralls commented Jun 15, 2023 •
edited
Loading

Pull Request Test Coverage Report for Build 5297723828

  • 46 of 241 (19.09%) changed or added relevant lines in 3 files are covered.
  • No unchanged relevant lines lost coverage.
  • Overall coverage decreased (-0.6%) to 49.425%
Changes Missing Coverage Covered Lines Changed/Added Lines %
pynxtools/dataconverter/readers/xps/reader.py 6 15 40.0%
pynxtools/dataconverter/readers/xps/reader_utils.py 21 94 22.34%
pynxtools/dataconverter/readers/xps/scienta_txt_parsers.py 19 132 14.39%
Totals Coverage Status
Change from base Build 5275838012: -0.6%
Covered Lines: 5373
Relevant Lines: 10871
💛 - Coveralls

lukaspie
lukaspie commented Jun 16, 2023
View reviewed changes
lukaspie
lukaspie commented Jun 16, 2023
View reviewed changes
@lukaspie lukaspie marked this pull request as ready for review June 16, 2023 05:55
8000
@lukaspie lukaspie marked this pull request as draft June 16, 2023 10:47
@lukaspie
Update function for concatenating two raw data files
b4cfa0e 
Dicts are merged and every other object is appended to a list.
@lukaspie lukaspie marked this pull request as ready for review June 16, 2023 10:48
RubelMozumder
RubelMozumder reviewed Jun 16, 2023
View reviewed changes
@RubelMozumder RubelMozumder self-requested a review June 16, 2023 15:59
@lukaspie lukaspie merged commit 610a3be into master Jun 19, 2023
@lukaspie lukaspie deleted the KIT_xps branch June 19, 2023 07:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@RubelMozumder RubelMozumder RubelMozumder approved these changes

@sanbrock sanbrock Awaiting requested review from sanbrock

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@lukaspie @coveralls @RubelMozumder
0