Zeptaframe is an AI-native video editor that allows you to create precise AI-generated movies with a visual-first approach, minimizing the need for text prompts. It implements the Go-with-the-flow technique for precise video generation. Try it at CVPR 2025.
This repository only contains the front-end interface. The backend is hosted in cloud GPU instances loaded with ComfyUI, running custom ComfyUI nodes and pipelines.
The site is live at https://zeptaframe.com.
Zeptaframe is a open-source web-based platform that enables users to create AI-generated videos through an intuitive visual interface. Unlike traditional video editors or AI video generators that rely heavily on text prompts, Zeptaframe allows you to upload images and directly manipulate objects on a canvas to create animations.
The backend is hosted in cloud GPU instances loaded with ComfyUI, running custom ComfyUI nodes and pipelines. The respective Go-with-the-Flow LoRA is used in conjuction with either CogVideoX or Skyreels. Wan2.1 support is coming soon.
- Upload or generate images and animate any object on the canvas
- Segment objects automatically and instantly using the Segment Anything Model (SAM)
- Define object trajectories visually rather than through text descriptions
- Create complex videos using multiple workbenches (canvases)
- Seamlessly navigate between workbenches
- Each workbench has it own context and controls
- Generate videos from your visual compositions
- Multiple quality modes: Flash, Normal, and Ultra
- Support for various video models
- Shape creation and manipulation
- Text editing with font customization
- Image import and filtering
- Drawing tools
- Object animation controls
- Camera movement controls
- History of generated videos
- Export options
- Video playback controls
- Next.js and React for the UI
- Fabric.js for canvas manipulation
- NextAuth.js for authentication
- Drizzle ORM for database operations
- Hono for API routes
- Stripe for payment processing
- Segment Anything Model (SAMv2) for object segmentation
- ComfyDeploy for GPU instances loaded with ComfyUI.
- The back-end is using a finetuned version of SkyreelsV1/CogVideoXwith Go-with-the-flow. This allows the model to generate videos of the subject traveling as per the users delineation through the interface.
This README was generated with Deepwiki from Devin.