Tags: usnistgov/trec_eval
Tags
Version 9.0a3 Always dangerous to think about things overnight! More widespread changes that clean things up internally quite a bit, making it easier to add new measures and cleanup after them. Values of measures are the same, only visible change is there is now a "runid" measure that prints the results file runid when requested.
Version 9.0a2 OK, I think I'm almost done with trec_eval for now! Here's the latest version: widespread changes but the ndcg algorithm remains the same. Could you check ndcg is getting reasonable responses? Also, I'd probably like to replace ndcg with ndcg_p (the parameterized version) if you think appropriate. Unless you have other comments, that's the only change before I'll release it as a beta in a few days (SIGIR reviews come first.)
Version 9.0a1 Here's the revamped trec_eval. Much cleaner in spots, and can be incorporated in other systems. So you can discard the original alpha. Not entirely backward compatible any more. Going from float to double changes even one of the small trec_eval test set values by 5% (it's the only value changed on that test set- the integer cutoff for ircl_prn.30 for one query changes from 24 to 23 docs, with consequences). Is it worthwhile doing a massive name change now? I don't know how many abbreviations for precision are used, and how many times '.', '-', and '_' are used interchangeably! If it's going to be changed ever, now is probably the time. Note I found at least one bug that existed in Version 8 (an un-initialized malloc'd value that happens to be 0 with normal mallocs); I'm not planning to do anything about it for Version 8. I've started to add citations, if you know of any that should be included, let me know.
PreviousNext