A Comprehensive Legal Guide to Web Scraping in the US

//

Date of Last Update: January 12, 2022

TL;DR

This is a long article about a complicated issue involving more than a dozen different laws. But you can read the bolded sections in just a few minutes and you’ll get the gist of it.

The good news for web scrapers is that the overall trend has been toward greater permissiveness with web scraping, even if that trend has wavered somewhat in the last couple of years. The bad news is that the overall history of web-scraping litigation has not been kind to scrapers.

Note: The laws that govern web scraping are complicated and evolving. New legal decisions are being published frequently on web scraping issues, and many important cases are pending and yet unresolved. Many of these have the potential to change the legal landscape with web scraping. Some case law may have been published since the last update of this guide. In short, this guide reflects the state of the law as of the date of the last update. It is for informational purposes only and may not reflect the most current state of the law on the issues presented or reflect the full state of the law as it pertains to your situation.

  1. Introduction

There are a few websites online that purport to answer the question of “whether web scraping is legal.” And way too many of those websites, with unwavering confidence and a complete absence of caution, provide clear and concise answers to that question that are laughably and dangerously false.

One such website claims to have a “three-part test” to determine whether web scraping is legal.

But a web scraper could follow that test and still violate dozens of state and federal laws (and that’s before we get into the hundreds, if not thousands, of legal jurisdictional issues associated with web scraping). That blog post would be certifiable legal malpractice—except it wasn’t written by a lawyer in the first place.

With the increasing importance of data collection from privately owned websites, web scraping has grown from a niche enterprise to a bona fide industry in the last decade.

As a lawyer (and a python programmer) who has worked with dozens of clients in the web-scraping industry, from Y Combinator startups to Fortune 50 companies, I figured the internet was long overdue for a practitioner’s guide to web-scraping law that was grounded in the law. With that, I took the time to read every web-scraping case and scholarly journal on the subject published in the last ten years.

What you see here is the end-product of that research.

The rest of this article is available only via PDF – please provide your email below to receive a copy. We will not share your information with anyone for any reason, other than to send you future emails about web scraping legal issues.

This blog post is for informational purposes only. The views expressed in this post may not reflect the opinions of other attorneys at the firm or the firm’s clients.

  • This field is for validation purposes and should be left unchanged.

Leave a Comment

This site uses Akismet to reduce spam. Learn how your comment data is processed.