In the good old pre-AI days, publishers didn’t mind crawlers taking their content because they got traffic in return. AI search generates very little publishers traffic, so publishers don’t want AI crawlers taking their content without compensation. BidSwitch is tackling the issue with the Dynamic Content Ledger, which allows AI crawlers to programmatically bid for page-level access.

Meta has announced agreements with USA Today, People Inc., and other news publishers to pay for access to their content and link to their articles in its AI chatbot. It’s a sharp contrast to Perplexity, which the Chicago Tribune and New York Times are separately suing for allegedly scraping content and bypassing paywalls. Despite this deal, Meta apparently still opposes compensating publishers for news links in social media posts.