One of the main reasons I first became interested in the law of web scraping was because of how hard it was to find reliable information about it. Years ago, I had a client with a big web-scraping project, and they asked me to research whether what they were planning on doing was legal or … Read More
Every time I drive my car, I follow the same algorithm for how fast I drive. If I’m driving on a highway or an open road, I drive between 6 and 9 miles over the speed limit. If I’m driving in a neighborhood or near a school, I drive 3 to 4 miles over the … Read More
Date of Last Update: April 25, 2022 TL;DR This is a long article (nearly 13,000 words!) about a complicated subject. It provides substantial amounts of historical context and legal analysis. You might not care about any of that. But you can read the bolded sections and the infographics, and you’ll get gist of it. One … Read More
On April 18th, the Ninth Circuit issued its opinion in hiQ Labs, Inc. v. LinkedIn. Corp. on remand from the Supreme Court. My quick take: This is perhaps the most important case in the history of web-scraping litigation. It is about limiting the scope of a federal criminal statute with civil applications in a civil … Read More
Web Scraping and Antitrust Law In 2019, in what was at the time the most important legal precedent in the United States with web scraping, the Ninth Circuit Court of Appeals wrote: We agree with the district court that giving companies like LinkedIn free rein to decide, on any basis, who can collect and use … Read More
The first rule of the law of web scraping is this: Details matter. As I’ve written before, there are dozens of laws associated with web scraping. As such, no single “three-part test” can tell you whether what you’re doing is legal or not. The only way to properly assess that is to look into the … Read More
Web scraping has recently come under scrutiny from regulatory agencies that investigate securities laws. Increasingly, investment firms have been using web scraping as an alternative form of data collection. To bolster more traditional data sets like SEC filings and financial statements, investment firms have been going directly to websites and online resources to get information … Read More
Facebook aggressively monitors web scraping on its platforms. And they aren’t shy about sending out cease and desist letters when they believe someone is violating their terms of service. After the Ninth Circuit’s decision in hiQ Labs, Inc. v. LinkedIn Corp. in 2019, however, there might have been some question whether a court (and, in … Read More
I recently wrote a post at Professor Eric Goldman’s Law and Technology Blog about an important web scraping case out of Florida from earlier this year. If you’re in the web-scraping business, you’ll definitely want to check that out here. This blog post is for informational purposes only. The views expressed in this post may … Read More
Date of Last Update: July 20, 2023 When I first published this guide, in the summer of 2020, I wrote that the recent trend had been toward greater permissiveness with web scraping. That sentence isn’t as true today as the day when I first wrote it. Recent policy trends have wavered. The few jurisdictions where … Read More