Disclamer: this code was almost ENTIRELY written by me, AI was poorly involved in this project (only used it for some permission issues while deleting git folders, because git clone is a buggy command for security purposes). Yes, it's fun to code manually and optimize code naturally :))
Unlock Your AI's Full Potential with Flawless Code Understanding.
Large Language Models (LLMs) are revolutionizing software development. However, their efficacy is often limited by the quality and completeness of the provided context. Manually feeding your AI with relevant information from your codebase can be a tedious, error-prone process, especially for large projects constrained by context window limitations.
Lumen is engineered to address this challenge.
Lumen is an intelligent Command Line Interface (CLI) tool that automatically scans, structures, and formats your entire codebase into a meticulously crafted prompt suitable for any LLM. Eliminate the friction of manual copy-pasting and context constraints. With Lumen, you provide your AI with the deep, structured understanding necessary to generate truly insightful and accurate results.
Elevate your AI interactions from challenging to highly effective. Power up with Lumen.
- Effortless Context Generation: Automatically gathers and structures your entire project, removing manual drudgery.
- Optimized Performance: Experience significantly faster context generation. Internal optimizations, especially in configuration handling, deliver substantial speed improvements, particularly noticeable on large-scale projects.
- Intelligent File Handling:
- Reads a wide variety of file types, including
.ipynbnotebooks. - Employs a smart encoding strategy: defaults to UTF-8 for speed and reliability, with an intelligent fallback to detect encoding only if an initial read encounters issues. This ensures correct rendering of special characters and improves overall reading performance.
- Reads a wide variety of file types, including
- Optimized for AI: Delivers a standardized, AI-friendly output format, including consistent file separators and an introductory message, ensuring maximum LLM comprehension.
- GitHub Repository Analysis: Seamlessly analyze public GitHub repositories with a single command. Lumen handles the cloning and subsequent cleanup.
- 100% Private & Secure: Processes everything locally. For local projects, your code never leaves your machine during context generation.
- Token Insights: Utilize the
-lor--leaderboardoption to identify the most token-heavy files in your project (top 20 by default), aiding in the optimization of large contexts. - Reliable & Tested: Backed by a comprehensive test suite to ensure stability and correctness.
Before installing Lumen, ensure you have the following installed and correctly configured on your system. Lumen is a Python tool and relies on standard development environments.
-
Python (3.7 or higher):
- How to Check: Open your terminal or command prompt and type
python --versionorpython3 --version. - Installation & Environment Setup:
- Windows: Download the installer from python.org. Crucially, during installation, ensure you check the box that says "Add Python to PATH".
- macOS: Python 3 is often pre-installed or easily available via Homebrew (
brew install python). - Linux (Debian/Ubuntu):
sudo apt update && sudo apt install python3 python3-pip - Linux (Fedora/CentOS/RHEL):
sudo dnf install python3 python3-pip(oryum)
- Pip: Python's package installer, usually installed with Python 3.7+.
- How to Check:
pip --versionorpip3 --version. - Upgrade (Recommended):
python -m pip install --upgrade pip.
- How to Check:
- How to Check: Open your terminal or command prompt and type
-
Git: (Required only if you plan to use the GitHub repository feature (
-gflag)).- How to Check:
git --version. - Installation: Download from git-scm.com or use your system's package manager (e.g.,
brew install git,sudo apt install git).
- How to Check:
Install Lumen easily using pip:
pip install pylumen
To upgrade to the latest version:
pip install pylumen --upgrade
Lumen is designed for ease of use from your command line (lum).
1. Generate Full Context (Clipboard):
Navigate to your project's root and run:
lum
(The complete, structured prompt is copied to your clipboard. For very large codebases, consider the -t option for better performance.)
2. Analyze a Specific Path:
lum /path/to/your/project
3. Save to File:
lum -t my_project_context
(Creates my_project_context.txt in the project's root.)
4. Analyze a Public GitHub Repository:
(Requires Git installed!)
lum -g https://github.com/user/repo-name
5. Identify Token-Heavy Files:
See the top 20 (default) most token-consuming files:
lum -l
Or specify a different number (e.g., top 10):
lum -l 10
(This also generates and copies/saves the full context as per other commands.)
6. Manage Configuration:
- Edit your settings:
lum -c(Opens~/.lum/config.jsonin your default editor.) - Reset to defaults:
lum -r
For a full list of commands and options, Lumen features a clear and consistent help section:
lum --help
Tailor Lumen to your exact needs by editing its configuration file (~/.lum/config.json). Use lum -c to open it.
Key settings include:
intro_text: Customize the introductory message for your prompts.title_text: Define the format for file titles (e.g.,--- FILE : {file} ---).skipped_folders: A comprehensive list of folder names to ignore. This supports two types of matching:- Exact Match: A string like
"build"will only skip folders named exactlybuild. - Ends-With Match: A string prefixed with
*, like"*.log"or".cache", will skip any folder whose name ends with.log(e.g.,app.log,server.log) or.cache(e.g.,.pytest_cache,.mypy_cache).
- Exact Match: A string like
skipped_files: A list of specific file names to exclude from context (e.g.,package-lock.json,.DS_Store).allowed_file_types: Specify which file extensions Lumen should process.
Automatic Updates: Lumen's configuration is designed to be future-proof. If new configuration options are added in an update, your config.json will be intelligently updated to include them with their default values, preserving your existing customizations. You can always reset to the latest defaults with lum -r.
Lumen is actively evolving! Here's a glimpse of planned features and directions:
- IDE Integration: Development of a VS Code extension is underway to allow "Lumen: Copy Context" functionality directly from your editor.
- Web Interface (Exploratory): We are exploring the potential for a web-based interface to further enhance usability and accessibility.
- Lumen v1.0: The next major milestone will be Lumen v1.0, focusing on delivering a highly stable, polished, and production-ready experience with further refinements and potential feature enhancements.
Stay tuned for these and more improvements!
If Lumen helps you supercharge your AI workflows, please consider starring the project on GitHub! It's a great way to show your support and helps others discover Lumen.
We welcome contributions, issues, and feature requests! Check out the issues page or submit a pull request. See CONTRIBUTING.md for more details. Let's make Lumen even better, together!
This project is licensed under the MIT License - see the LICENSE file for details.
Developed by Far3k
- GitHub: Far3000-YT
- Email: far3000yt@gmail.com
- Discord: @far3000
- X (Twitter): @0xFar3000