8000 Fixed Score coordinates and phantom subtomo tests by DavidHerreros · Pull Request #167 · I2PC/scipion-em-xmipptomo · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Fixed Score coordinates and phantom subtomo tests #167

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 14 commits into from
Nov 6, 2023

Conversation

DavidHerreros
Copy link
Contributor

Some checks to be done:

  • Check if score coordinate test is robust enough or extra changes are needed

  • Check micrograph cleaner scores (good particles should have a score closer to 0 or 1?)

  • Check if phantom subtomo tests runs fins (locally I am getting an error when doing a --fourier wedge filter with Xmipp, but it might be a local installation problem).

# Conflicts:
#	xmipptomo/tests/test_protocols_xmipptomo.py
@albertmena
Copy link
Contributor
  • It does not look robust enough (each machine different output coordinate number)
  • The micrograph cleaner scores maybe is inverted (@danielmarchan3 ?)
  • These warnings look suspicious:

WARNING: MetaData: Error parsing column 'goodRegionScore' value.
WARNING!!! valueIn.failed = True, binding NULL
WARNING!!! valueIn.failed = True, binding NULL
WARNING!!! valueIn.failed = True, binding NULL

< 8000 div data-show-on-forbidden-error hidden>

Uh oh!

There was an error while loading. Please reload this page.

@DavidHerreros
Copy link
Contributor Author
  • It does not look robust enough (each machine different output coordinate number)
  • The micrograph cleaner scores maybe is inverted (@danielmarchan3 ?)
  • These warnings look suspicious:

WARNING: MetaData: Error parsing column 'goodRegionScore' value.
WARNING!!! valueIn.failed = True, binding NULL
WARNING!!! valueIn.failed = True, binding NULL
WARNING!!! valueIn.failed = True, binding NULL

I have relaxed XmippTomoScoreCoordinates test's checks so it is more robust. Still, the dataset used for this test is not the best choice to evaluate the performance of the carbon score. Probably it should be checked in the future.

Regarding the warnings, I am not being able to replicate them on my machine. I guess they are related to the empty metadata generated by the tests (currently, one of the two metadata generated in the executions will be always empty). If this is the case, the warning should be expected. Again, the warning issue should be also solved with a better testing dataset.

@albertmena
Copy link
Contributor
albertmena commented Oct 31, 2023

The error buildbot report (the same I had in my mahcine) is:
/bin/sh: 1: xmipp_deep_micrograph_cleaner: not found
xmipp is not exported, I solve it with source xmipp/build/xmipp.bashrc
but scipion should do it

@DavidHerreros
Copy link
Contributor Author
DavidHerreros commented Oct 31, 2023

The error buildbot report (the same I had in my mahcine) is: /bin/sh: 1: xmipp_deep_micrograph_cleaner: not found xmipp is not exported, I solve it with source xmipp/build/xmipp.bashrc but scipion should do it

Hi @albertmena,

I am trying on my laptop locally and I can't reproduce this issue... the tests run ok on my local installation with no Xmipp added to the bash. The changes in this PR should fix this issue (found on __init__.py --> getTensorflowEnv --> line 42). Are the tests running on the right branches and is deepLearningToolkit being installed?

@albertmena
Copy link
Contributor

The error buildbot report (the same I had in my mahcine) is: /bin/sh: 1: xmipp_deep_micrograph_cleaner: not found xmipp is not exported, I solve it with source xmipp/build/xmipp.bashrc but scipion should do it

Hi @albertmena,

I am trying on my laptop locally and I can't reproduce this issue... the tests run ok on my local installation with no Xmipp added to the bash. The changes in this PR should fix this issue (found on __init__.py --> getTensorflowEnv --> line 42). Are the tests running on the right branches and is deepLearningToolkit being installed?

ok, we can consider this point as fixed.

But the test is not robust (4 machines 4 fails) we may change the test data

@DavidHerreros
Copy link
Contributor Author

The error buildbot report (the same I had in my mahcine) is: /bin/sh: 1: xmipp_deep_micrograph_cleaner: not found xmipp is not exported, I solve it with source xmipp/build/xmipp.bashrc but scipion should do it

Hi @albertmena,
I am trying on my laptop locally and I can't reproduce this issue... the tests run ok on my local installation with no Xmipp added to the bash. The changes in this PR should fix this issue (found on __init__.py --> getTensorflowEnv --> line 42). Are the tests running on the right branches and is deepLearningToolkit being installed?

ok, we can consider this point as fixed.

But the test is not robust (4 machines 4 fails) we may change the test data

I have modified the test and the protocol to improve its performance:

  • Score coordinates protocol now allows choosing the GPU to be used
  • Score coordinates protocol now saves the predicted carbon closeness mask in extra folder (useful for validating the selected parameters in the form)
  • Score coordinates tests executed on an updated and more robust dataset (PySeg dataset)

@@ -113,6 +121,7 @@ def detectOutliers(self):
self.scoreOutliers[idn][1] = z_scores[idn]

def detectCarbonCloseness(self, coordinates):
pwutils.makePath(self._getExtraPath('carbonMask'))
self.scoreCarbon = []
for tomoName in self.tomoNames:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The class has not self.tomoNames definition

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @albertmena,

The attribute self.tomoNames is initialized in the method computeParams in line 101, which is called as a step inside the _insertAllSteps method (line 94). This step is called before detectCarbonCloseness, are you getting any error related to it? In my local installation, the test runs without issues. Thanks!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks like some local issue, the test works on galileo_shadow

@albertmena
Copy link
Contributor

I think we are close to the solution!

Copy link
sonarqubecloud bot commented Nov 6, 2023

Kudos, SonarCloud Quality Gate passed!    Quality Gate passed

Bug A 0 Bugs
Vulnerability A 0 Vulnerabilities
Security Hotspot A 0 Security Hotspots
Code Smell A 3 Code Smells

No Coverage information No Coverage information
0.0% 0.0% Duplication

@albertmena albertmena merged commit ed7a96e into devel Nov 6, 2023
@albertmena albertmena deleted the fix_test_score_coordinates branch November 6, 2023 09:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants
0