Shopify Products Scraper
codemaster devops · @codemasterdevops
Export public Shopify prices, SKUs, stock status, options, and images in one row per variant. Compare catalogs across stores.

About
What does Shopify Products Scraper do?
Shopify Products Scraper exports public catalog data from one or more Shopify storefronts. It reads the store's catalog feed when available and falls back to the sitemap and individual product endpoints. It saves one row per variant for catalog exports, competitor research, and recurring price and stock snapshots.
What Shopify data can you extract?
- Product details: title, product ID, URL, plain-text description, vendor, product type, tags, and published timestamps when available.
- Variant details: SKU, price, color, size, material, and any other options the store supplies. If a merchant SKU is missing, the Actor uses the variant ID.
- Stock and media: availability, inventory count where published, and image URLs. Currency, barcodes, weight, and shipping flags are included when the storefront provides them.
How to use Shopify Products Scraper
- Paste one or more Shopify storefront homepages into Storefront URLs.
- Set Maximum products to visit. Start with 3–10 for a quick sample.
- Run the Actor and open its dataset. Export JSON or CSV, or read the dataset through the Bytewells API. JSON preserves nested metadata and image lists for downstream scripts.
{
"startUrls": [{ "url": "https://kith.com/" }],
"maxRequestsPerCrawl": 3,
"maxConcurrency": 1,
"maxRequestRetries": 2,
"checkForBanner": false,
"proxyConfig": { "useApifyProxy": false }
}
The cap applies to unique product pages across the run. A product with four variants can yield four rows. The Actor checks storefronts in input order, so an early store can use the entire cap. Set the cap to 0 to visit all discovered products.
Example output
This shortened example uses fictional product data. A full row also includes the description, product type, timestamps, and other available metadata.
{
"url": "https://example.com/products/weekender",
"title": "Weekender Bag",
"id": "1234567890123",
"sku": "WB-FOREST-M",
"price": 49,
"currency": null,
"availability": "in stock",
"brand": "Example Outfitters",
"color": "Forest",
"size": "M",
"images_urls": ["https://example.com/images/weekender.jpg"],
"additional": {
"variant_title": "Forest / M",
"stock_count": null,
"tags": ["travel"]
}
}
Coverage and troubleshooting
The store needs a readable public catalog feed or a public sitemap and readable product JSON endpoints. Password protection, robots rules, and request blocking can reduce coverage. Currency, inventory counts, and availability are null when the source does not establish them. Each run captures the current public catalog.
If a known Shopify store uses a custom robots.txt without a Shopify marker, turn off Check Shopify marker. Product requests still follow robots rules. Public feeds require no Shopify account or Admin API key. The Actor connects directly by default and accepts optional custom proxy URLs. The STATS record reports selected products, products read from the feed, failures, skips, and rows saved. Successful rows remain in the dataset if some requests fail. For support, share the affected storefront and run ID through the listing’s support channel.
FAQ
- What does Shopify Products Scraper collect?
- It exports public product and variant data, including titles, SKUs, prices, options, vendor details, and image URLs. Stock and other optional fields depend on what each storefront publishes.
- What URL should I enter?
- Enter the Shopify storefront homepage in startUrls, such as https://kith.com/. A storefront URL starts a catalog scan; a product or collection path does not restrict the scan to that product or collection.
- Can I scrape multiple Shopify stores in one run?
- Yes. Add one storefront per entry in startUrls. Stores are checked in input order, and they share the same product cap.
- Does the product limit equal the number of output rows?
- No. maxRequestsPerCrawl caps unique product URLs selected for crawling. A product can produce several variant rows, and inaccessible products can produce none.
- How does the scraper find more products?
- It reads the public catalog feed page by page when available. If the feed is unavailable, it follows the product sitemaps. Use maxRequestsPerCrawl: 0 to remove the product cap.
- Does the scraper show exact inventory and currency?
- Only when the public source provides them. Missing inventory counts, currency, or stock status are null. A null stock status does not mean out of stock.
- Why did a Shopify store return few or no results?
- Check the run log and STATS for blocked feeds or product requests, missing sitemaps, and robots exclusions. If a known Shopify store uses a custom robots.txt without a Shopify marker, try checkForBanner: false.
- Do I need Shopify credentials or a proxy?
- No Shopify credentials or Admin API key are required. Direct access is the default. You can supply custom proxy URLs; this build does not use Apify-managed proxies.
- How can I export the Shopify catalog?
- Download JSON or CSV from the run dataset, or read the dataset through the Bytewells API. JSON is best for keeping image lists and nested variant metadata.
- Can I use it for price monitoring?
- Yes. Schedule recurring runs and compare snapshots by product URL and SKU. Each run captures the current public catalog; the Actor does not calculate changes or recover past prices.
Input
Preview of the input form. Rent the Actor to fill it in and start runs from the console.
Fields
startUrlsPublic Shopify storefront URLs. The Actor reads the catalog feed when available and otherwise discovers product pages from the sitemap.
maxRequestsPerCrawlGlobal cap on unique product pages across all stores. Each product can produce several variant rows. Use 0 for no product cap.
checkForBannerRequire the storefront robots.txt to mention Shopify. Turn off only for a known Shopify store with a customized robots.txt.
proxyConfigDirect requests by default. Use custom proxy URLs if needed. Apify-managed proxies are unavailable in this build.
maxConcurrencyUpper limit on concurrent product requests.
maxRequestRetriesRetry attempts after a retryable request failure.
debugLogEnable extra diagnostic log messages.
API
Bytewells speaks the Apify API. Existing integrations keep working: point apify-client at https://bytewells.com/api and use your Bytewells API token (Console → Settings → API access). The Actor ID is codemasterdevops/shopify-products-scraper (or codemasterdevops~shopify-products-scraper in URLs).
JavaScript · apify-client
import { ApifyClient } from 'apify-client';
const client = new ApifyClient({
token: process.env.BYTEWELLS_TOKEN,
baseUrl: 'https://bytewells.com/api',
});
const input = {
"startUrls": [
{
"url": "https://kith.com/"
}
],
"maxRequestsPerCrawl": 10,
"checkForBanner": false,
"proxyConfig": {
"useApifyProxy": false
},
"maxConcurrency": 10,
"maxRequestRetries": 3,
"debugLog": false
};
const run = await client.actor('codemasterdevops/shopify-products-scraper').call(input);
const { items } = await client.dataset(run.defaultDatasetId).listItems();
console.log(items);Python · apify-client
import os
from apify_client import ApifyClient
client = ApifyClient(os.environ["BYTEWELLS_TOKEN"], api_url="https://bytewells.com/api")
run_input = {
"startUrls": [
{
"url": "https://kith.com/"
}
],
"maxRequestsPerCrawl": 10,
"checkForBanner": False,
"proxyConfig": {
"useApifyProxy": False
},
"maxConcurrency": 10,
"maxRequestRetries": 3,
"debugLog": False
}
run = client.actor("codemasterdevops/shopify-products-scraper").call(run_input=run_input)
items = client.dataset(run["defaultDatasetId"]).list_items().items
print(items)HTTP · curl
curl -X POST "https://bytewells.com/api/v2/acts/codemasterdevops~shopify-products-scraper/runs" \
-H "Authorization: Bearer $BYTEWELLS_TOKEN" \
-H "Content-Type: application/json" \
-d '{"startUrls":[{"url":"https://kith.com/"}],"maxRequestsPerCrawl":10,"checkForBanner":false,"proxyConfig":{"useApifyProxy":false},"maxConcurrency":10,"maxRequestRetries":3,"debugLog":false}'Runs are billed as compute from your credit balance. Paid Actors need an active rental or trial; otherwise the API answers 402 rental-required.
Related
More Actors from the same publisher.

Best Buy Products Scraper
codemaster devops · @codemasterdevops
Extract Best Buy product names, SKUs, available prices, images, and US reviews for catalog research and price tracking.
E-commerceAutomationDeveloper tools$19.99/mo3-day free trial
Pinterest Scraper – Extract Profiles & Pin Data
codemaster devops · @codemasterdevops
Scrape public Pinterest profiles and pins. Extract image URLs, descriptions, board details, source links, and available engagement stats.
Social mediaMarketingE-commerce$19.99/mo3-day free trial
Website Screenshot Generator
codemaster devops · @codemasterdevops
Capture full-page or viewport PNG screenshots from a list of URLs. Export page URLs and image links for visual QA, monitoring, and archives.
Developer toolsAutomation$19.99/mo3-day free trial
ZOOT Product Scraper
codemaster devops · @codemasterdevops
Extract ZOOT product names, prices, sizes, availability, and image URLs from category and product pages for fashion catalog research.
E-commerceAutomation$19.99/mo3-day free trial
Bing Scraper - Search Results & SERP Data
codemaster devops · @codemasterdevops
Scrape Bing search results, titles, links, and snippets. Run multiple queries with market, language, and page controls for SEO and competitor research.
SEO toolsMarketingDeveloper toolsUsage-based
Google Play Reviews Scraper - App Ratings & Feedback
codemaster devops · @codemasterdevops
Extract public Google Play app reviews, ratings, dates, helpful votes, and available developer replies for app research and competitor analysis.
MarketingSEO toolsDeveloper tools$19.99/mo3-day free trial