Archive your Twitter/X bookmarks to markdown with AI-powered analysis. Supports Claude Code and OpenCode for multi-model flexibility. Like a dragon hoarding treasure, Smaug collects the valuable things you bookmark.
Based on repository activity, growth velocity and community engagement.
39
Growth7/30
Activity5/25
Popularity12/25
Trust15/20
829
Stars
High
Sentiment
Votes
829
README.md
Smaug š
Archive your Twitter/X bookmarks (and/or optionally, likes) to markdown. Automatically.
Like a dragon hoarding treasure, Smaug collects the valuable things you bookmark and like.
Multi-model support: Smaug works with Claude Code (default) and OpenCode, giving you access to a wide range of AI models. Results may vary depending on the model you choose ā test carefully and find what works best for your workflow. See AI CLI Integration for setup details.
# 1. Install bird CLI (Twitter API wrapper)
# See https://github.com/steipete/bird for installation
# 2. Clone and install Smaug
git clone https://github.com/alexknowshtml/smaug
cd smaug
npm install
# 3. Run the setup wizard
npx smaug setup
# 4. Run the full job (fetch + process with Claude)
npx smaug run
The setup wizard will:
Create required directories
Guide you through getting Twitter credentials
Create your config file
Manually Getting Twitter Credentials
Smaug uses the bird CLI which needs your Twitter session cookies.
If you don't want to use the wizard to make it easy, you can manually put your session info into the config.
X/Twitter long-form articles (full content via bird CLI)
Quote tweets and reply threads (full context)
Invokes Claude Code to analyze and categorize each tweet
Saves to markdown organized by date with rich context
Files to knowledge library - GitHub repos to knowledge/tools/, articles to knowledge/articles/
Running Manually
# Full job (fetch + process with Claude)
npx smaug run
# Fetch from bookmarks (default)
npx smaug fetch 20
# Fetch ALL bookmarks (paginated - requires bird CLI from git)
npx smaug fetch --all
npx smaug fetch --all --max-pages 5 # Limit to 5 pages
# Fetch from likes instead
npx smaug fetch --source likes
# Fetch from both bookmarks AND likes
npx smaug fetch --source both
# Process already-fetched tweets
npx smaug process
# Force re-process (ignore duplicates)
npx smaug process --force
# Check what's pending
node -e "console.log(require('./.state/pending-bookmarks.json').count)"
Fetching All Bookmarks
By default, Twitter's API returns ~50-70 bookmarks per request. To fetch more, use the --all flag which enables pagination:
npx smaug fetch --all # Fetch all (up to 10 pages)
npx smaug fetch --all --max-pages 20 # Fetch up to 20 pages
Note: This requires bird CLI built from git (not the npm release). See Troubleshooting for installation instructions.
Cost warning: Processing large bookmark backlogs can consume significant Claude tokens. Each bookmark with content-heavy links (long articles, GitHub READMEs, etc.) adds to the context. Process in batches to control costs:
npx smaug run --limit 50 -t # Process 50 at a time with token tracking
Use the -t flag to monitor usage. See Token Usage Tracking for cost estimates by model.
Categories
Categories define how different bookmark types are handled. Smaug comes with sensible defaults, but you can customize them in smaug.config.json.
š Note: Transcription is flagged but not yet automated. PRs welcome!
X/Twitter Long-Form Articles
X articles (x.com/i/article/*) are Twitter's native long-form content format. Smaug extracts the full article text using bird CLI:
Direct extraction: If the bookmarked tweet is the article author's original post, content is extracted directly
Search fallback: If you bookmark someone sharing/quoting an article, Smaug searches for the original author's tweet and extracts the full content from there
Metadata fallback: If search fails, basic metadata (title, description) is captured
Example X article bookmark:
## @joaomdmoura - Lessons From 2 Billion Agentic Workflows
> [Full article content extracted]
- **Tweet:** https://x.com/joaomdmoura/status/123456789
- **Link:** https://x.com/i/article/987654321
- **Filed:** [lessons-from-2-billion-agentic-workflows.md](./knowledge/articles/lessons-from-2-billion-agentic-workflows.md)
- **What:** Deep dive into patterns from scaling CrewAI to billions of agent executions.
Actions
file: Create a separate markdown file with rich metadata
capture: Add to bookmarks.md only (no separate file)
transcribe: Flag for future transcription (auto-transcription coming soon! PRs welcome)
| Option | Default | Description |
|--------|---------|-------------|
| source | bookmarks | What to fetch: bookmarks (default), likes, or both |
| includeMedia | false | EXPERIMENTAL: Include media attachments (photos, videos, GIFs) |
| archiveFile | ./bookmarks.md | Main archive file |
| timezone | America/New_York | For date formatting |
| cliTool | claude | AI CLI to use: claude or opencode |
| autoInvokeClaude | true | Auto-run Claude Code for analysis |
| claudeModel | sonnet | Model to use (sonnet, haiku, or opus) |
| autoInvokeOpencode | true | Auto-run OpenCode for analysis |
| opencodeModel | opencode/glm-4.7-free | OpenCode model (see OpenCode docs) |
| claudeTimeout | 900000 | Max processing time (15 min) |
| parallelThreshold | 8 | Min bookmarks before parallel processing kicks in |
| webhookUrl | null | Discord/Slack webhook for notifications |
Environment variables also work: AUTH_TOKEN, CT0, SOURCE, INCLUDE_MEDIA, ARCHIVE_FILE, TIMEZONE, CLI_TOOL, CLAUDE_MODEL, OPENCODE_MODEL, etc.
Experimental: Media Attachments
Media extraction (photos, videos, GIFs) is available but disabled by default. To enable:
# One-time with flag
npx smaug fetch --media
# Or in config
{
"includeMedia": true
}
When enabled, the media[] array is included in the pending JSON with:
type: "photo", "video", or "animated_gif"
url: Full-size media URL
previewUrl: Thumbnail (smaller, faster)
width, height: Dimensions
videoUrl, durationMs: For videos only
ā ļø Why experimental?
Requires bird with media support - PR #14 adds media extraction. Until merged, you'll need a fork with this PR or wait for an upstream release. Without it, --media is a no-op (empty array).
Workflow still being refined - Short screengrabs (< 30s) don't need transcripts, but longer videos might. We're still figuring out the best handling.
AI CLI Integration
Smaug supports multiple AI CLI tools for intelligent bookmark processing:
Claude Code (default) - Anthropic's Claude CLI
OpenCode - Alternative AI CLI with support for multiple models
For large batches (8+ bookmarks by default), Smaug spawns parallel subagents. By default, these use Haiku instead of Sonnet, which cuts costs nearly in half:
Your Twitter cookies may have expired. Get fresh ones from your browser.
Processing is slow
Try haiku model instead of sonnet in config for faster (but less thorough) processing
Make sure you're not re-processing with --force (causes edits instead of appends)
Only ~50-70 bookmarks fetched
The npm release of bird CLI (v0.5.1) doesn't support pagination. To fetch all bookmarks, install bird from git:
# Clone and build bird from source
cd /tmp
git clone https://github.com/steipete/bird.git
cd bird
pnpm install # or: npm install -g pnpm && pnpm install
pnpm run build:dist
# Link globally (may need sudo or --force)
npm link --force
# Verify
bird --version # Should show a newer commit hash
bird bookmarks --help # Should show --all flag
Then use npx smaug fetch --all to fetch all bookmarks with pagination.