> For the complete documentation index, see [llms.txt](https://codingmace.gitbook.io/masterward/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://codingmace.gitbook.io/masterward/software/work-in-progress/web-scraper-wallpaper-edition.md).

# Web Scraper - Wallpaper Edition

### Origin

This thought came while making Tobias and I could have implemented it, I felt it would be wasted effort as that was an API based program. In addition, I wanted to test another website and see if they had protection. This meant it couldn't be a major website and couldn't be just a few hits, it needed to be big.

### Overview

Spidering through a website we grab the URL of each page and download the list of images. Saving by default name and in a folder based on the index of the image.

### Step 1: Crawling