Designed and built an end-to-end AI content repurposing system that extracts high-signal angles from YouTube transcripts and generates hooks, summaries, and key moments tailored to each platform.
LLMs collapsed JSON structure, fabricated timestamps, and missed extraction targets when processing long-form transcripts — requiring a principled approach to prompt engineering and output validation.
Designed a custom 'Transcript Checkpoints' algorithm that partitions transcripts into discrete segments, each scored 1–10 by the model for relevance, automatically selecting the highest-signal angle. Paired with a pg-boss distributed job pipeline enforcing forbidden phrases, verbatim evidence usage, and platform-specific formatting constraints.
Score-and-select transcript segmentation over single-pass prompting
WhyFull-transcript prompting collapsed JSON structure, fabricated timestamps, and missed extraction targets on long-form input.
AlternativeA single prompt over the entire transcript
ApproachThe 'Transcript Checkpoints' algorithm partitions transcripts into discrete segments, each scored 1–10 by the model for relevance, automatically selecting the highest-signal angle.
TradeoffAdds a scoring/selection pass and more pipeline complexity than a single call.
OutcomeIn production — structural prompt constraints eliminated JSON collapse and timestamp fabrication.
Distributed pg-boss job pipeline over inline request handling
WhyAI transcript extraction and content generation run long enough that handling them inline would block API responses.
ApproachA pg-boss distributed job pipeline separates the API, transcript workers, and generation workers into independent processes, with progress reported back over Supabase Realtime/SSE.
Tradeoff5 independent services to deploy and operate instead of one.
Outcome97+ Lighthouse score across the multi-service Railway deployment.
- →97+ Lighthouse score across multi-service Railway deployment
- →5 independent nodes: API, workers, frontend
- →Real-time progress tracking via Supabase Realtime/SSE
- →Structural prompt constraints eliminating JSON collapse and timestamp fabrication
- →Concurrent AI transcript extraction and content generation via distributed job pipeline