What you can build, whether it's worth building, and what you can't build at all, layer by layer.
| Source | How | Caveats | Worth it? |
|---|---|---|---|
News sites | Google News RSS, GDELT (free, global), paid news APIs | Google News RSS is unofficial and can break. GDELT is noisy. Paywalled articles give only headline and snippet. Commercial news APIs cost real money. | YesCore source, cheap, high PR value |
Blogs & websites | RSS feeds, a web search API (e.g. Brave), your own crawler for a curated list | You won't match a huge crawled index. A curated list of 200–2,000 sites is realistic. Bing's search API was retired in 2025. | YesIf you curate the list for your niche |
Reddit | Official API | Free tier is non-commercial with rate limits. Commercial use needs Reddit's terms or a paid agreement. Deleted content must be removed from your data. | YesHigh-signal for B2B and tech. Check licensing first. |
YouTube | YouTube Data API v3 | 10,000 units/day by default and a search costs 100, so about 100 searches/day. You get titles, descriptions and comments, but not what's said in the video without transcripts. | YesTitles and comments. Spoken content only for a short list. |
Hacker News | Algolia HN API (free) | Developer audience only | YesIf tech or dev audiences matter |
Bluesky | Open firehose and search API | Smaller audience, but fully open | YesCheap to add, moderate value |
Mastodon | Per-server public APIs | Fragmented, with no global search | MaybeOnly if your niche is there |
X / Twitter | Paid API | Pricing has ranged from about $200/mo to $5,000+/mo and keeps changing. Lower tiers have tight read caps and a short search lookback. | With budgetUseful volume is expensive |
Threads | Official API | Some public keyword search has opened up, with tight rate limits. Check the current terms. | Test itMaybe the one usable Meta-owned source |
Telegram | Telegram's own API, public channels only | Needs a user account. No global search, so you monitor channels you pick. | NicheCrypto, Eastern Europe, some news |
App store reviews | Apple's public review RSS per app. Google Play through scraping libraries. | Apple returns only recent reviews. Google Play has no official API for other companies' apps, so scraping is grey under its terms. | YesIf you or competitors have apps |
Review sitesG2, Capterra, Trustpilot, TripAdvisor | Their own APIs cover your own listings only. Competitors require scraping. | Scraping breaks their terms and they actively block it | Own onlyYour own reviews yes. Competitors' reviews, mostly no. |
Podcasts | RSS feed, then speech-to-text, then keyword search | Transcription costs grow with hours of audio | Curated10–30 shows: yes. Broad coverage: no. |
Your own social accountsIG, FB, LinkedIn page | Official APIs for accounts you own | You see @mentions and comments on your posts only, not wider conversations | YesEngagement tracking, not listening |
| Feature | How | Caveats | Worth it? |
|---|---|---|---|
Sentiment #6 | Claude classifies each mention, Haiku-class for cost | Handles sarcasm better than older models but not perfectly. Spot-check about 50 items per month. | YesLikely better than most vendors' |
Intent #14 | Claude classifies into question, complaint, purchase intent, comparison, promo, etc. | You define the categories, which is a plus for B2B | YesVery high value for sales leads |
Topic analysis #13 | Embeddings, clustering, then Claude names each cluster | Clusters drift between runs, so fix the topic list once it settles | Yes |
Semantic search #15 | Embeddings in a vector store (pgvector or SQLite-vec) | None worth mentioning | Yes |
Context & word cloud #9, #29 | Counts of terms that appear together, or Claude-extracted themes | None | CheapMostly useful as a visual |
AI Insights briefing #11 | A scheduled Claude job writes a weekly summary | Only as good as your data coverage | YesWhat managers actually read |
AI Brand Assistant #10 | Claude plus an MCP server over your mentions database | None. Very natural to build with Claude. | Yes |
Spike detection #12 | Rolling z-score or seasonal decomposition on daily counts, then Claude explains the spike | Needs about 4–8 weeks of history before it's reliable | Yes |
Dedup & relevance filtering | Embedding similarity plus a Claude relevance check | Essential because keyword matching is noisy. Commercial tools have the same problem. | RequiredNot optional |
| Feature | How | Caveats | Worth it? |
|---|---|---|---|
LLM brand monitoring #16 | Send a set of prompts to the OpenAI, Perplexity, Gemini and Claude APIs on a schedule, then parse mentions, rank and sentiment | API answers can differ from the consumer apps (personalization, memory, different search). Answers vary run to run, so sample each prompt 3–5 times and average. | Best ROINo data access problem |
Prompt & competitor gaps #17 | Same data, compared across competitors | Choosing the prompts is the real work. Base them on Semrush keywords and personas. | YesFeeds straight into content planning |
Cited sources tracking | Parse the citations in Perplexity and Gemini answers | Not every model returns citations | YesShows which sites to get coverage on |
Google AI Overviews | Search-results APIs (SerpApi, DataForSEO) | Paid per query | YesAt a modest budget |
Copilot & consumer ChatGPT app | Only by scraping the web UI | Breaks their terms and is fragile | SkipUse the APIs as a proxy |
| Feature | Caveats | Worth it? |
|---|---|---|
Volume, trends, share of voice #5, #7 | Only as complete as your sources. Share of voice is relative, so gaps affect all brands similarly. | Yes |
Reach estimate | You invent the formula: followers, views and upvotes for social, site traffic for news (your Semrush connector can provide traffic). It won't be comparable with vendors' numbers. | YesKeep it simple and documented |
Influence Score & influencers #8, #26 | Only for authors on sources you collect. No Instagram or TikTok influencers. | B2B onlyFine for Reddit, YouTube, blogs, journalists. Weak for consumer brands. |
Hashtag tracking #4 | Works on Bluesky, Mastodon, YouTube, and X if you pay. Not on Instagram or TikTok, where hashtags matter most. | Low valueWithout those platforms |
Geolocation #25 | Very little source data. Mostly inferred from language or the publication's country. | Approximate |
Language filter #24 | Easy to detect | Yes |
| Feature | Caveats | Worth it? |
|---|---|---|
Slack / Teams alerts #3, #30, #31 | Tune the thresholds or people start ignoring the alerts | Yes |
Scheduled email reports #20 | Needs an email-sending service | Yes |
PDF / Excel / PPT exports #18, #19, #21 | Can be branded with your SBS brand skill | YesYou already have these skills |
Dashboard | Streamlit or Next.js, or an Artifact dashboard for internal use | Yes |
Filters, Boolean, tags, saved filters #22, #23, #27, #28 | Standard app work | Yes |
Own API & MCP server #32, #33 | Lets you query everything from Claude | YesCosts very little extra |
Notion export | Fits your existing workflows | YesNatural fit for you |
Storing posts and author names counts as processing personal data. You need a legitimate-interest basis, a retention policy and a way to handle deletion requests. Get this signed off internally if it's for SBS.
Your history starts the day you switch it on. Most APIs only look back days or weeks.
APIs change, scrapers break and quotas move. Expect a few hours a month of upkeep. Claude Code makes the fixes fast, but someone has to notice the breakage.
LLM classification is cheap per mention with small models. Paid APIs (X, news, search results) are the real cost. Many sources still cost nothing.
| What | Why not | Can vendors do it? |
|---|---|---|
Instagram listeningpublic posts mentioning you | Meta's API only shows your own account, @mentions of you, and a capped hashtag search (about 30 hashtags/week). It has no keyword search, and scraping breaks Meta's terms, which Meta enforces. | PartlyThrough licensed data partners |
Facebook listening | Public page content access is restricted. CrowdTangle was shut down in 2024, and Meta's Content Library is for approved researchers only. | PartlyLicensed |
TikTok listening | The Research API is limited to academic and non-profit researchers. No commercial access to search public posts. | PartlyLicensed |
LinkedIn listening | No public post-search API. LinkedIn sues scrapers (it won against hiQ in 2022). | Very limited |
Private spacesFB groups, private accounts, Discord, WhatsApp, Slack communities, closed forums | No legal access | NoVendors can't either |
A vendor-scale source indextens of millions of sources | The code isn't the obstacle. It takes years of crawler infrastructure, data licensing contracts and operations. Claude Code writes software; it doesn't sign data deals. | YesThat's their business |
Full, affordable X coverage | Possible in principle but cost-prohibitive. Large volumes need enterprise contracts. | YesThrough a commercial agreement |
Historical data before you start | APIs don't offer deep history, so you can't backfill a year | YesYears of archives |
Vendor-compatible numbersreach, proprietary scores, benchmarks | Their models are proprietary and calibrated on their own data. You can build equivalents, not exact copies. | n/a |
Spoken content at scaleall podcasts, video audio, Reels | Transcription cost at that volume, and no access to TikTok or IG media | PartialFor them too |
Exact consumer-app AI answersChatGPT with memory, Copilot | The only route is scraping the web UI, which breaks their terms and gives personalized results | ApproximateThey approximate as well |
Competitor reviews at scaleG2, Capterra, Trustpilot | Their APIs cover your own listings only. Scraping breaks terms and is blocked. | Partly |
Use Semrush keywords to choose the prompts. Track ChatGPT, Perplexity, Gemini, Claude and AI Overviews.
News, blogs, Reddit, YouTube, HN and reviews, with Claude sentiment, intent and topics, a weekly briefing and Slack alerts.
Instagram, TikTok, LinkedIn and Facebook listening, or vendor-scale crawling.