Supercharge your codebase for LLM processing
A powerful CLI tool to convert and tokenize codebases for LLM usage. Perfect for preparing your project for AI analysis!
npx llm-tokenize .Key Features
Everything you need to prepare your codebase for LLM processing
Recursive Processing
Handles entire directory structures with ease
Smart Ignores
Automatically skips unnecessary files and respects .gitignore
Token Optimization
Calculates and optimizes token count for LLM context
Command Examples
Explore the various ways to use LLM Tokenize
Process specific project with custom output
npx llm-tokenize ./my-project gpt-4 ./docs/codebase.mdProcess with different LLM model
npx llm-tokenize . gpt-3.5-turboGlobal installation
npm install -g llm-tokenize