CreatorJot
An AI content automation platform that turned YouTube transcripts into structured, platform-ready content through a multi-LLM pipeline (GPT, Claude, Gemini) running on background job queues.
Designed and built an AI content automation platform that turned YouTube video transcripts into structured content for multiple publishing channels: hooks, summaries, and key moments tailored to each platform. A multi-LLM pipeline orchestrated GPT, Claude, and Gemini through background job queues across a 5-node Railway deployment. CreatorJot is now closed.
LLMs collapsed JSON structure, fabricated timestamps, and missed extraction targets when processing long-form transcripts — requiring a principled approach to prompt engineering and output validation.
Designed a custom 'Transcript Checkpoints' algorithm that partitions transcripts into discrete segments, each scored 1–10 by the model for relevance, automatically selecting the highest-signal angle. Paired with a pg-boss distributed job pipeline enforcing forbidden phrases, verbatim evidence usage, and platform-specific formatting constraints.
Score-and-select transcript segmentation over single-pass prompting
WhyFull-transcript prompting collapsed JSON structure, fabricated timestamps, and missed extraction targets on long-form input.
AlternativeA single prompt over the entire transcript
ApproachThe 'Transcript Checkpoints' algorithm partitions transcripts into discrete segments, each scored 1–10 by the model for relevance, automatically selecting the highest-signal angle.
TradeoffAdds a scoring/selection pass and more pipeline complexity than a single call.
OutcomeIn production — structural prompt constraints eliminated JSON collapse and timestamp fabrication.
Distributed pg-boss job pipeline over inline request handling
WhyAI transcript extraction and content generation run long enough that handling them inline would block API responses.
ApproachA pg-boss distributed job pipeline separates the API, transcript workers, and generation workers into independent processes, with progress reported back over Supabase Realtime/SSE.
Tradeoff5 independent services to deploy and operate instead of one.
Outcome97+ Lighthouse score across the multi-service Railway deployment.
Schema-based context compression for LLM calls
WhyRunning GPT, Claude, and Gemini over long transcripts made token usage the main cost to control.
ApproachEach stage writes structured, schema-versioned intermediate data that later calls reuse as compressed context, with per-call token and cost tracking.
OutcomeRoughly 85% lower token usage per call.
- →~85% lower per-call token usage via schema-based context compression
- →Multi-LLM pipeline (GPT, Claude, Gemini) across a 5-node Railway deployment
- →Job processing, schema versioning, and per-call token and cost tracking
- →Structural prompt constraints eliminating JSON collapse and timestamp fabrication
- →Real-time progress tracking via Supabase Realtime/SSE
- →97+ Lighthouse score across the multi-service deployment