This repository contains the data for ThrowBench. Data is given in JSONL format. Records have the following fields:
bug_id: The RunBugRun bug idcode: Program codeexception_type: The name of the exception thrown, orno_exceptionif none thrown. This is the target label.exception_message: The full message of the exception thrown`language: The langugue, eitherc_sharp,java,pythonorrubyinput: The program input that triggers the exceptioninputs: Other triggering inputs (inputwas randomly selected from this list)locs: Program length (in lines of code)
Model outputs can be found in the results directory. Files have the following fields:
model: Model namepredicted_exception: Answer given by modelactual_exception: Ground-truth answerbug_id: See aboveoutput: Full model outputlanguage: See above
The evaluation script can be found in run.py