8000 How to convert PyTorch model to FP16 TFLite? · Issue #169 · quic/ai-hub-models · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

How to convert PyTorch model to FP16 TFLite? #169

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
tumuyan opened this issue Feb 25, 2025 · 1 comment
Open

How to convert PyTorch model to FP16 TFLite? #169

tumuyan opened this issue Feb 25, 2025 · 1 comment
Labels
question Please ask any questions on Slack. This issue will be closed once responded to.

Comments

@tumuyan
Copy link
tumuyan commented Feb 25, 2025

The TFLite model supports FP16 quantization, which is known toreduce the application size. Different with INT8 and INT16 quantization, FP16 does not require calibration.
However, Qualcomm's documentation does not provide the example how to optimize for FP16.
Could you provide some suggestions on how to convert a PyTorch model to an FP16 TFLite model? pytorch -> onnx -> fp16 onnx -> tflite or others way?
Or FP16 quantization is not recommended?
@mestrona-3 mestrona-3 added the question Please ask any questions on Slack. This issue will be closed once responded to. label Feb 26, 2025
@kory
Copy link
Contributor
kory commented Mar 6, 2025 •
edited
Loading

Do you need weight quantization?

If not, we run your model with fp16 activations for GPU and NPU by default. No explicit quantization is needed; rather it's just a set of runtime flags (that AI Hub enables by default). See our default settings here:
https://github.com/quic/ai-hub-apps/blob/main/apps/android/tflite_helpers/TFLiteHelpers.java

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
question Please ask any questions on Slack. This issue will be closed once responded to.
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@tumuyan @kory @mestrona-3
0