Is web scraping legal? What US, UK and EU law says in 2026

Scraping publicly available data is not automatically illegal in the US, UK or EU, but it can still breach a site's terms, copyright, database rights or data-protection law. Risk rises when you log in, bypass blocks, collect personal data or reuse content to train AI. This is general information, not legal advice.

Not legal advice. This guide summarises public court decisions and regulator guidance, checked on 24 Oct 2026. Laws differ by country and change with new cases. Your counsel confirms what applies to your project.

Many posts on this question present hiQ v. LinkedIn as a clean win for scrapers, quote it without its ending, or get basic dates wrong. Below, each point links to the court decision, a law-firm summary of it, or the regulator's own guidance, so you can read the source.

The short answer by question

United States: the CFAA after Van Buren and hiQ

The Computer Fraud and Abuse Act (CFAA) makes it a federal offence to access a computer "without authorization" or to "exceed authorized access". For years, sites argued that scraping in breach of their terms did both.

Van Buren v. United States (3 Jun 2021). The Supreme Court, by six votes to three, read "exceeds authorized access" narrowly. It described the test as a "gates-up-or-down inquiry": either you're entitled to access a part of a system or you aren't. Misusing information you're allowed to reach isn't the same as breaking in.

hiQ Labs v. LinkedIn. hiQ scraped public LinkedIn profiles to sell workforce analytics. In April 2022, after Van Buren, the Ninth Circuit again upheld an injunction in hiQ's favour, finding hiQ had raised "at least serious questions" that scraping publicly available profiles, even after LinkedIn revoked access, didn't violate the CFAA. That's the part most posts quote.

The ending matters as much. As Proskauer reported, the district court found in November 2022 that hiQ had breached LinkedIn's User Agreement through scraping and the use of fake accounts. On 6 Dec 2022 the parties filed a stipulated consent judgment: a $500,000 judgment against hiQ and a permanent injunction barring it from scraping LinkedIn in breach of the User Agreement, "whether logged in to a LinkedIn account or not", and requiring it to delete the scraped data and the code built on it. The lesson: winning on the hacking law didn't protect hiQ from contract claims.

Contracts and terms of service

Because the CFAA route narrowed, sites now lean on contract law. Two 2024 decisions in the Northern District of California went the scraper's way, on specific facts:

• Meta v. Bright Data (23 Jan 2024). Judge Edward Chen granted summary judgment to Bright Data on Meta's breach-of-contract claim. As Farella Braun + Martel summarised, Meta's terms governed a user's "use" of its products, and Bright Data did not "use" Facebook and Instagram when it scraped public data while logged off. Logged-in scraping was a different matter.
• X Corp. v. Bright Data (May 2024). The court dismissed X's claims. Skadden's summary (21 May 2024) explains that the court held X's contract claims over scraping public posts were preempted by the Copyright Act, because enforcing them would give X rights beyond copyright over content its users own. Claims based on harm to X's servers failed because X hadn't adequately pleaded the damage.

Neither decision makes terms of service irrelevant. They turn on whether you agreed to the terms (logging in usually means you did), what the terms actually cover, and who owns the content. Other courts may see it differently.

Copyright and AI training

Scraping for analysis is one thing; copying expression and reusing it, especially to train AI, is where the biggest cases now sit.

• Thomson Reuters v. Ross Intelligence (11 Feb 2025). A federal judge rejected Ross's fair-use defence for using Westlaw's headnotes to build a competing legal research tool, as Venable reported on 13 Feb 2025. The tool wasn't generative AI, so the ruling's reach is debated, but it shows fair use isn't a given.
• Bartz v. Anthropic. On 25 Sep 2025, Bloomberg Law reported that a judge had preliminarily approved a $1.5 billion settlement between Anthropic and authors over books used in AI training. A settlement isn't a ruling on the law, but its size shows the stakes of how training data is sourced.

If you plan to train or fine-tune models on scraped content, check licences first and keep a record of where each item came from.

UK and EU: data protection

In the UK and EU, the first question isn't "is the page public?" but "does the data identify people?" Names, profile photos, reviews with usernames and contact details are all personal data, even when published openly. The GDPR applies based on where the people are and where you operate, not on their citizenship.

• Lawful basis. For scraping, the realistic basis is usually legitimate interests under Article 6(1)(f), which needs a documented balancing test.
• ICO (December 2024). In its consultation response on generative AI, the UK regulator said legitimate interests is "the sole available lawful basis" for training generative AI on web-scraped personal data under current practices, and that developers must show why other ways of collecting data aren't suitable and be far more transparent.
• CNIL (19 Jun 2025). France's regulator published guidance on legitimate interest and web scraping (in French): in summary, limit collection to freely accessible data, define what you collect in advance, and exclude sites that clearly object to scraping, for example through robots.txt or CAPTCHAs.
• EU AI Act. On 10 Jul 2025 the European Commission published the General-Purpose AI Code of Practice, a voluntary code covering transparency, copyright and safety for providers of general-purpose AI models. If you build or fine-tune such models for the EU market, read it alongside the AI Act itself.

Database rights and terms in the EU

The EU's Database Directive protects databases through copyright (for original selection or arrangement) and a separate sui generis right (for substantial investment). In Ryanair v PR Aviation (C-30/14, 15 Jan 2015), a price-comparison site had scraped Ryanair's flight data. The Court of Justice held that where a database is protected by neither right, the Directive doesn't stop its owner from setting contractual limits on its use, subject to national law. MediaLaws and Osborne Clarke summarise the case. In practice, EU site terms can matter even for data that isn't otherwise protected.

Robots.txt: polite, not permission

Robots.txt was standardised as RFC 9309 in September 2022. The RFC is explicit that its rules "are not a form of access authorization". So ignoring robots.txt isn't, by itself, a crime, and obeying it doesn't give you a licence.

It still matters. Courts and regulators look at whether you respected a site's clear signals; the CNIL names robots.txt as one of them. Infrastructure providers enforce it too: Cloudflare began blocking AI crawlers by default on 1 Jul 2025 for crawling without permission, and on 15 Sep 2026 replaced its single "Block AI Bots" switch with three separate controls for search, AI training and AI agents. A crawler that identifies itself honestly and respects these signals is both easier to defend and more reliable to run.

A practical checklist for lawful scraping

Before you build a scraper, work through these questions with your counsel:

1. Is there an official API, data feed or licence? Use it first. It's usually more stable and removes most legal questions.
2. Is the data public, without a login? Avoid scraping behind logins unless it's your own account data and the terms allow it.
3. What do the site's terms say, and have you agreed to them?
4. Does robots.txt allow your crawler? Respect it, and identify your crawler with an honest user agent and contact URL.
5. Are you bypassing anything? Don't defeat CAPTCHAs, IP blocks or other technical barriers.
6. Is any of it personal data? If so, document a lawful basis, a legitimate-interests assessment and, where needed, a data protection impact assessment. Collect the minimum.
7. Are you copying expression or just facts? Prices and specifications differ from articles, photos and reviews.
8. Will the data train or fine-tune AI? Check licences and the ICO, CNIL and EU AI Act material above.
9. Is your rate polite? Throttle requests so you don't load the site's servers.
10. Can you show where each record came from? Keep the source URL and timestamp on every row.

Our web scraping services follow this checklist: API or licence first, robots.txt respected, no login walls or CAPTCHA bypass, and personal data only with a lawful basis.

What changed and when

Checked 24 Oct 2026. We'll review this guide in January 2027.

How Elgrith can help

Elgrith has delivered 1,000+ projects for 250+ clients since 2015, from our New Delhi HQ and client-partnership centres in New York, London and Shanghai. Our web scraping and data extraction team builds polite, identified crawlers for public data, delivers clean JSON, CSV or database feeds with a source on every row, and monitors for site changes. Your crawler code, schedules and data stay in your accounts. Feeds can land in your warehouse through our data engineering work, or power retrieval for your own AI tools through LLM integration. We don't give legal advice; we build to the rules your counsel sets. You get a fixed written quote after a free call. Book a call or start a project.

General guidance from public sources, not legal advice; your counsel confirms what applies to you.