KanTV("Kan", aka English "watch") , an open source project focus on study and practise on-device AI technology in real scenario(such as perform online-TV playback and realtime transcription and online-TV record at the same time) on Android phone:
-
Watch online TV and local media by customized
. this project is derived from original
(that project has stopped maintenance since 2021), with much enhancements and new features. source code of customized FFmpeg 6.1 could be found in external/ffmpeg according to FFmpeg's license. source code of FFmpeg 6.1's all dependent libraries could be found in external/ffmpeg-deps .
-
Watch online TV by customized
, source code of customized Exoplayer2.15.1 could be found in android/kantvplayer-exo2 .
-
Record online TV to local file on phone.
-
2D graphic performance benchmark.
-
AI subtitle(real-time English subtitle for English online-TV(aka OTT TV) via the great & excellent & amazing whisper.cpp ).
-
Well-maintained turn-key / self-contained workbench for AI experts/researchers whom focus on highly-value on-device AI R&D activity on Android. some on-device AI R&D activities (AI algorithm validation and AI model validation and performance benchmark with ASR/Text2Image/LLM on Android) could be done via this project easily.
-
Well-maintained turn-key / self-contained workbench for AI beginners to learning on-device AI technology on Android.
-
Built-in Google's Gemma3-4B(multimodal text + image), Google's Gemma3-12B(multimodal text + image) , Alibaba's Qwen1.5-1.8B, Alibaba's Qwen2.5-3B, Alibaba's Qwen3-4B, Alibaba's Qwen3-8B, Nvidia's Llama-3.1-Nemotron-Nano-4B, Microsoft's Phi-4-mini-reasoning, Huggingface's SmolVLM2-256M(highly-optimized multimodal for realtime-video-recognition), Alibaba's Qwen2.5-Omni-3B(multimoda text + audio), DeepSeek's DeepSeek-R1-0528-Qwen3-8B, Xiaomi's MiMo-VL-7B supportive and runs entirely offline(no Internet required). these supported LLM models can be downloadded in the Android APK directly without manually preparation. APK's users can compare the real experience of these LLM models on the Android phone. developers can add other LLM models manually in source code KANTVAIModelMgr.java#L284.
-
Text2Image on Android phone via the amazing stable-diffusion.cpp.
-
The ggml-hexagon(original name is ggml-qnn) in this project is probably the first open-source reference implementation of a specified llama.cpp backend for Qualcomm Hexagon NPU on Android phone.
- Clone this repository and build locally, see how to build
- Download pre-built Android APK from https://github.com/kantv-ai/kantv/releases
- Download pre-built Android APK from Github CI-build: https://github.com/kantv-ai/kantv/actions/
- Android 8.0(2017.08) --- Android 15(2024.10) and higher version with ANY mainstream arm64 mobile SoC.
- Android smartphone equipped with ANY mainstream high-end mobile SoC is highly recommented for realtime AI-subtitle feature otherwise unexpected behavior would happen.
- Android smartphone equipped with one of below Qualcomm mobile SoCs(Qualcomm's state-of-the-art high-end mobile SoC Snapdragon 8Gen3 series and Snapdragon 8Elite series are highly recommended) is required for verify/running ggml-hexagon backend on Android phone:
Snapdragon 8 Gen 1
Snapdragon 8 Gen 1+
Snapdragon 8 Gen 2
Snapdragon 8 Gen 3
Snapdragon 8 Elite
here is a short video to demostrate realtime AI subtitle by running the great & excellent & amazing whisper.cpp on an Android phone equipped with Qualcomm Snapdragon 8Gen3 mobile SoC - fully offline, on-device.
realtime-subtitle-by-whispercpp-demo-on-xiaomi14-finetune-20240324.mp4
a screenshot to demostrate multi-modal inference by running the magic llama.cpp on an Android phone equipped with Qualcomm Snapdragon 8Elite mobile SoC - fully offline, on-device.
a screenshot to demostrate realtime-video-recognition via MTMD from llama.cpp + a lightweight multimodal model SmolVLM2-256M from Huggingface on an Android phone equipped with Qualcomm Snapdragon 8Elite mobile SoC - fully offline, on-device.
some other screenshots
a screenshot to demostrate ASR inference by running the excellent whisper.cpp on an Android phone equipped with Qualcomm Snapdragon 8Gen3 mobile SoC - fully offline, on-device.
a screenshot to demostrate Text-2-Image inference by running the amazaing stable-diffusion.cpp on an Android phone equipped with Qualcomm Snapdragon 8Elite mobile SoC - fully offline, on-divice.
a screenshot to demostrate download LLM model in APK.
- How to build
- How to customize tv.xml for personal needs
- How to download supported AI models in the APK
- Authors
- Acknowledgement
- ChangeLog
- ggml-hexagon:history of ggml-hexagon
- ggml-hexagon:high-level data path of ggml-hexagon
- Roadmap
Report issue in Android phone equipped with mainstream mobile SoC or submit PR to this project is greatly welcomed.
We use GitHub issues for tracking feature requests and issue reports, please see how to submit issue in this project .
- AI inference framework
- AI application engine
- ASR engine whisper.cpp
- LLM engine llama.cpp
- Text2Image engine stable-diffusion.cpp
- CV engine opencv-mobile
- MTMD(multimodal) engine MTMD subsystem in llama.cpp