Case Study: Branch
Scaling political topic modeling from manual annotation to a fine-tuned NLP pipeline
When Branch needed to replace manual human annotation with automated topic modeling across a growing volume of political campaigns, Rosenblatt built and shipped a two-model NLP pipeline: a relevancy classifier and a contrastive-learning topic matcher. We took topic-matching accuracy from under 40% with off-the-shelf models to 92% on our internal test set and 88% on real-world campaign data, and deployed both models to production — the relevancy classifier through an AWS Lambda pipeline and the topic-matcher to a SageMaker endpoint supporting batch and real-time inference.
Initial Consultation
Branch Politics is a free, nonpartisan app that walks voters through the elections and candidates in their area — breaking down who each candidate is, what they believe, and why it matters. Before working with Rosenblatt, Branch relied on a team of human data annotators to manually match candidate quotes to campaign topics. As Branch's revenue grew, that manual process couldn't keep pace with the volume of new campaigns and candidates, so they came to Rosenblatt to automate it.
Automated Political Topic Modeling
Rosenblatt's first approach was a fully unsupervised pipeline built on the BERTopic library: tokenize sentences from Branch's scraped candidate-page content, embed them, deduplicate semantically, reduce the embeddings, cluster them, and label each cluster using Llama 3 8B. Two things went wrong. The scraped web data was full of irrelevant boilerplate (privacy policies, terms of service), and density-based clustering kept collapsing into just a handful of clusters that didn't match how Branch's own team already organized political topics.
That led the team to a two-model supervised approach instead. First, a relevancy classifier: to fix a dataset skewed toward a handful of over-represented candidates, Rosenblatt built a custom sampling function, then worked with Branch’s own annotation team to label 1,000 sampled sentences as relevant or irrelevant. Because Branch needed to avoid ever dropping a relevant sentence, the team optimized for recall rather than raw accuracy — the final model hit over 90% accuracy and 90% recall, deployed via a pipeline of AWS Lambda functions and a SageMaker endpoint.
Second, a topic-matching model: given a relevant sentence, which political topic does it belong to? Early attempts using a stock Hugging Face model and a traditional single-label classification design (plus triplet loss) capped out below 40% accuracy — real campaign sentences often relate to more than one topic, which a single-label design can’t capture. The breakthrough was contrastive learning: fine-tuning the model on positive and negative sentence-topic pairs so it learns to maximize the contrast between matching and non-matching topics. That pushed accuracy to 92% on the internal test set and 88% on brand-new, real-world campaign data — deployed to a SageMaker endpoint supporting both batch and real-time inference.
Both models were handed off production-ready: the relevancy classifier running through its AWS Lambda pipeline, and the fine-tuned topic-matcher deployed to a SageMaker endpoint supporting both batch and real-time inference against new campaign data as it comes in.
About Rosenblatt
MIT reports that partnering externally on AI-first products brings them live 6 months sooner and twice as successfully as internal builds. Rosenblatt specializes in AI transformation for mid-market, tech-enabled, professional services. We bring together AI leaders from big 4 consulting firms and founders from exited AI startups to lead companies from fast pilots to complete AI transformations. Click below to schedule a 30-minute meeting with our CEO and founder.