Closed
Description
This issue is part of a JOSS review, tracked in-project with issue #479 (paper) and #478 (library and features).
To streamline reviewing, I'm creating this meta-issue that addresses all of the documentation and paper criteria for a JOSS publication. Where needed, dedicated issues will be created that link back to this mete-issue.
Once this meta-issue closes, all documentation and paper requirements for the JOSS review will satisfy criteria for acceptance, and thus the entire submission (func, doc, paper) will be recommended for acceptance.
Documentation
- A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
- Installation instructions: Is there a clearly-stated list of dependencies? Ideally these should be handled with an automated package management solution.
- In part due to
ginkgo's nature as a high performance library, not all dependencies can pragmatically be automated with package management solutions. However, requirements are clearly and accurately stated, and my tests found them to be sufficient.
- In part due to
- Example usage: Do the authors include examples of how to use the software (ideally to solve real-world analysis problems).
- An excellent set of examples are included with large coverage and real problems. Very well done aspect of the project.
- Functionality documentation: Is the core functionality of the software documented to a satisfactory level (e.g., API method documentation)?
- Extensive and multi-modal documentation is available, much of it automated and integrated into code development. Could use some improvements/updates here and there, but overall robust (especially for core functionality).
- Automated tests: Are there automated tests or manual steps described so that the functionality of the software can be verified?
- Extensive unit and integration test are provided, along with performance benchmarking setups along with public data giving performance results from various machines and configurations.
ginkgo's testing and automation is robust.
- Extensive unit and integration test are provided, along with performance benchmarking setups along with public data giving performance results from various machines and configurations.
- Community guidelines: Are there clear guidelines for third parties wishing to 1) Contribute to the software 2) Report issues or problems with the software 3) Seek support
- Yes, both the GitHub/repo documentation and the website documentation provide guidelines on contributing. My experience reporting issues and getting support from the developers indicates the team backs up the channels given for support and contributing.
Software paper
- Summary: Has a clear description of the high-level functionality and purpose of the software for a diverse, non-specialist audience been provided?
- The summary is clear for a specialist audience. But I suspect the JOSS audience includes people with less familiarity with the use of and current state of high performance linear algebra and the kinds of computation enabled by performant sparse solvers.
The paper would be stronger and accessible to more of the JOSS audience if the summary section zoomed out a bit and gave non-specialist audiences a bit more context on the state of the field/technologies and the value that ginkgo brings.
Edit: context and ginkgo's role were made clearer for broad audience.
- The summary is clear for a specialist audience. But I suspect the JOSS audience includes people with less familiarity with the use of and current state of high performance linear algebra and the kinds of computation enabled by performant sparse solvers.
- A statement of need: Do the authors clearly state what problems the software is designed to solve and who the target audience is?
- State of the field: Do the authors describe how this software compares to other commonly-used packages?
- Quality of writing: Is the paper well written (i.e., it does not require editing for structure, language, or writing quality)?
- Paper is to-the-point and perhaps more accessible to specialist audiences than those not in relevant areas of HPC/scientific computing, but it is clear and makes clear to me the value of ginkgo and its place in the ecosystem.
- References: Is the list of references complete, and is everything cited appropriately that should be cited (e.g., papers, datasets, software)? Do references in the text use the proper citation syntax?
- Paper may be using bib instead of md? https://github.com/ginkgo-project/ginkgo/tree/joss-paper/doc/joss 5246 . Can we confirm the references are compliant with JOSS syntax?
Edit: issues have been addressed. Doc and paper review criteria are satisfied.