Bulk Image Downloader: Crawl Sites, Dedupe Photos, Export ZIP

Devansh Tiwari · @devanshtiwari365

Turn any page or list of image links into deduplicated image files with one metadata row per image.

Developer toolsAutomation

About

Bulk Image Downloader: Crawl Sites, Dedupe Photos, Metadata Export

Download every image from a public page, or from a list of image links, and get one clean row per image: source page, size, dimensions, format, SHA-256 and how the image was found. Duplicates are skipped, tiny tracking pixels can be filtered out, and each failed image keeps a row that says why.

Start a 7-day free trial and run a small list first. No card is needed for the trial.

🖼️ What you get

  • Images from pages and direct links. Paste webpages, image links, or a mix. The Actor reads img tags, srcset and picture sources, Open Graph and Twitter Card images, and lazy-load attributes.
  • A bounded crawl. Follow links from a starting page to the depth and page limits you set, and keep it on the right sections with URL patterns.
  • Duplicates removed. Each image body gets a SHA-256 fingerprint, so the same photo found on many pages is saved once.
  • Filters. Minimum width, height and file size, file extensions, MIME types and a per-link image cap.
  • Cleaning and conversion. Strip EXIF metadata from JPEGs and convert WebP to PNG or PNG to JPG.
  • Full source tracking. Every row says whether the image came from srcset, a picture tag, an Open Graph tag, a data attribute or a direct link.
  • A status row for every attempt. Failed images keep their row with the HTTP status and error.
  • Your format. Download the metadata as Excel, CSV or JSON, or pull it through the API.

👥 Who it is for

  • AI engineers: "I need every image deduped and tagged with how it was found, not a folder of pictures."
  • Developers with a catalog scraper: "I hand over a list of image links and get the files and a metadata table back."
  • E-commerce operators: "I crawl competitor product pages on a schedule and compare image fingerprints to catch photo swaps."
  • Researchers and archivists: "I pull every image from a story or site in one run."

🚀 How it works

  1. Add your URLs. Paste webpages, direct image links, or both.
  2. Set your options. Choose depth, size filters, de-duplication and any cleaning.
  3. Run it and download. Read the metadata table, and fetch the image files from the run's storage.

📥 Input

FieldWhat it does
URLsWebpages or direct image links, mixed freely.
ModeAuto-detect, force page parsing, or force direct download.
Crawl depth and max pagesHow far to follow links from each starting page.
URL patternsWhich links the crawl may follow.
Min width, height, sizeSkip small images and tracking pixels.
Extensions and MIME typesKeep only the formats you want.
Max images per URL, max URLsCaps that keep a run bounded.
De-duplicate by hashSkip images already seen.
Strip EXIF, convert formatClean JPEGs, convert WebP or PNG.
File name patternName files with the slug, hash, extension, index or source.

📋 Output

One row per image attempt.

FieldWhat it holds
source_url, image_urlThe page and the image link
filename, kv_store_keyThe saved file name and its key in the run's storage
status, error, http_statusWhether it worked and why not
content_type, format, size_bytesWhat the file is
width, heightPixel dimensions
sha256, is_duplicateFingerprint and duplicate flag
from_srcset, from_og_tag, from_direct_url and similarHow the image was found

💰 Pricing

$19 a month after a 7-day free trial. Compute is billed from your prepaid credits during the trial and after it.

💡 Good to know

  • Only public pages and image links are read. The Actor does not log in.
  • It does not run JavaScript, so pages that build their images with scripts need their image links pasted in directly.
  • Image files are saved as generic binary. Use the extension in each row's file name.
  • The Actor is independent and not affiliated with any site it reads.

FAQ

Does it run JavaScript on the page?
No. It reads the page's HTML, including srcset and lazy-load attributes, Open Graph and Twitter Card images. Pages that build their image list with JavaScript return little or nothing, so for those, collect the image links first and paste them in as direct links.
How are duplicate images handled?
Each image body gets a SHA-256 fingerprint. With de-duplication on, an image whose fingerprint was already seen is flagged as a duplicate and not saved again, so the same photo found on ten pages is stored once.
Can it crawl a whole website?
It follows links from your starting page up to the depth and page limits you set, and a URL pattern list keeps it on the sections you want. By default it stays on the pages you supply. Per-link caps (1,000 images per URL, 10,000 URLs per run) keep a run bounded.
What does each result row tell me?
One row per image: the page it came from, the image link, how it was found (srcset, picture tag, Open Graph, direct link and others), HTTP status, size in bytes, width, height, format, SHA-256 and whether it succeeded. A failed image keeps its row with the error, so you can see what failed and why.
Where do the image files go?
Each image is saved to the run's storage under a key that starts with IMG- and ends with the file name, and the row names that key. Files are saved as generic binary, so give each file the extension shown in its row's file name when you download it.

Related

More Actors from the same publisher.

View all from @devanshtiwari365