Replies: 1 comment
-
|
Good question. If you are sure it's not your function messing with the random seed, then it must be some function called by BADS internally. This might be not impossible to solve with modern AI coding assistants, but it still requires some understanding of the codebase to avoid that the suggested changes would be nonsense or break the code in other ways. I might give it a try at some point, but given it's not high-priority on my to-do list, AGI is likely to come first... Since BADS is open-source, other people are welcome to contribute and submit a PR to fix this! |
Beta Was this translation helpful? Give feedback.
Uh oh!
There was an error while loading. Please reload this page.
-
Hi
I am using BADS to fit a model with four parameters that contains a noisy objective function. While my pattern of results seemed overall very consistent with 24 restarts when I set the seeds once before the first restart to the default value, I noted some minor discrepancies in the parameter estimates.
To ensure exact reproducibility, I am resetting now the seed below before each restart to the restart number j:
rng(j) % reset seed to based on restart number
paramsStart = plb + (pub-plb).*rand(1,numel(plb)) % initialize starting parameters taking bounds into account [maximum_likelihood_parameters(j,:),fval(j),exitflag(j),output(j).output] = bads(@(params)fstr(params, data, modelInfo,sim),paramsStart,lb,ub,plb,pub,[],OPTIONS)
Below are the correlations for parameter values from two separate fitting runs with 8 restarts for 316 subjects, using the code set up above with resetting the seed before each restart. While the values highly correlated (and indeed perfectly correlate for the two plots at the top), you can see that that for at least three subjects, the parameter estimates are not exactly the same in the two bottom plots.
In the BADS wiki, it says multiple times, that setting the seed should lead to reproducible results (and this is also on line with general understanding).
However, how can the below discrepancy still occur? And what I can do to resolve it?
Thanks for your advice!
Beta Was this translation helpful? Give feedback.
All reactions