8000 Driver fails to download large data files · Issue #7 · areaDetector/ADEiger · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Driver fails to download large data files #7

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
GDYendell opened this issue Aug 23, 2017 · 2 comments
Closed

Driver fails to download large data files #7

GDYendell opened this issue Aug 23, 2017 · 2 comments

Comments

@GDYendell
Copy link
Contributor

Hi Bruno,

We are testing the frame drop rate of the internal file writer to compare to the performance issues we are seeing on the Stream interface. We are finding that if we do an acquisition of 500+ frames into a single file (NumImages and FWNImagesPerFile to 500) that the only the master file is downloaded and not the data file. The data file remains on the detector storage and the master is deleted (FWAutoRemove = Yes). We can then curl to download the data file without issue. We also found with an acquisition of 10,000 frames with 1,000 frames per file, the first 8 were downloaded OK, but the final 2 were not.

Have you seen this problem?

@brunoseivam
Copy link
Member

Hi Gary,

Yes, I saw that before, the driver was failing to download the last few files. This commit (2e17935) was supposed to solve that. Could you try enabling ASYN_TRACE_FLOW mask to see the sequence of events?

Could you also try to play with MAX_RETRIES in that bit of code to see if it helps?

My guess is that the problem stems from the FileWriter reporting it is done while still writing files.

@GDYendell
Copy link
Contributor Author

Ah, yes I remember seeing that. I didn't realise it was only on dev. We are on master (+ b3ec2b0), so it's quite likely that is the problem.

I think we have done enough testing to convince Dectris of the problem, so this won't be much of an issue for now. I will update to dev if we do need the fix.

Cheers!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants
0