Description
Prerequisites
Before submitting your question, please ensure the following:
- [ O ] I am running the latest version of PowerInfer. Development is rapid, and as of now, there are no tagged versions.
- [ O ] I have carefully read and followed the instructions in the README.md.
- [ O ] I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
Question Details
I reviewed the entire repository and noticed that the threshold parameter is not actually used.
In powerinfer-py/powerinfer/main.py, there is an argument named threshold, which seems to correspond to C_l in Equation 7 of the paper.
However, in llama.cpp, at line 3126, the PowerInfer Python module is called without passing the threshold argument.
Additionally, I could not find any implementation of the computation constraint described in Equations 4 and 5 of Section 6.3.1.
Was it intentional to omit the computation constraint?
Additional Context
Please provide any additional information that may be relevant to your question, such as specific system configurations, environment details, or any other context that could be helpful in addressing your inquiry.
Thanks, in advance