You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Based on my knowledge of the issue, I'd say the important part is the input FASTQ files have about 8 billion pairs, so it's probably too large to share here.
The error happens at a certain point of the stdout output:
Where all lines after that point all report negative assignment percentages. The output files appear normal.
This means, the between these two lines, the number of read pairs processed happens to caused an integer overflow if int32 is used as the numerator, since int32 has the range of [-2147483648 : 2147483647].
The output here relates to the following lines of ReadProcessor::processBuffer():
I suspect the problem has something to do with nummapped, which got overflowed when more than 2,147,483,647 read pairs get mapped. Since NovaSeq 6000 instruments can generate 20b clusters per run, and NovaSeq X instruments can generate 50b, I can foresee this particular overflow eventually become rather common if the code wasn't changed. I wonder if there're more places in the code affected by this?
The text was updated successfully, but these errors were encountered:
Thanks! Yeah, I actually noticed that strange output a while back (I was processing some large datasets myself) but hadn't had a chance to investigate further (I suspected it was an int overflow somewhere). I will fix this in the next release (in the next week or so).
Other parts of the code should be fine (and the functionality itself will be unaffected). It really only affects whenever we're counting the reads in a FASTQ file one-by-one for the entire duration of the run (i.e. numreads and nummapped -- which are just used for statistics / progress); otherwise, ints should never reach such a high value. But I'll go through the codebase and see where the 32-bit ints should be changed to 64-bit ints.
Uh oh!
There was an error while loading. Please reload this page.
Hi,
I was running splitcode v 1.29 with the following arguments:
Based on my knowledge of the issue, I'd say the important part is the input FASTQ files have about 8 billion pairs, so it's probably too large to share here.
The error happens at a certain point of the stdout output:
Where all lines after that point all report negative assignment percentages. The output files appear normal.
This means, the between these two lines, the number of read pairs processed happens to caused an integer overflow if
int32
is used as the numerator, sinceint32
has the range of [-2147483648 : 2147483647].The output here relates to the following lines of
ReadProcessor::processBuffer()
:I suspect the problem has something to do with
nummapped
, which got overflowed when more than 2,147,483,647 read pairs get mapped. Since NovaSeq 6000 instruments can generate 20b clusters per run, and NovaSeq X instruments can generate 50b, I can foresee this particular overflow eventually become rather common if the code wasn't changed. I wonder if there're more places in the code affected by this?The text was updated successfully, but these errors were encountered: