ScrapeOps to Google Sheets, Walmart prices tracked
You pull Walmart prices for a report, then a stakeholder asks, “Can we refresh this tomorrow too?” Suddenly you’re stuck in a loop of tabs, copy-paste, and spreadsheet cleanup that never really ends.
This hits e-commerce operators first, but market analysts and growth marketers feel it too. Walmart price tracking automation turns the messy part into a repeatable feed so you can monitor changes without babysitting a scraper.
This workflow runs on a schedule, grabs Walmart search results through ScrapeOps, cleans the fields, and appends tidy rows into Google Sheets. You’ll see what it automates, what results to expect, and what you need to set it up safely.
How This Automation Works
The full n8n workflow, from trigger to final output:
n8n Workflow Template: ScrapeOps to Google Sheets, Walmart prices tracked
flowchart LR
subgraph sg0["Schedule Flow"]
direction LR
n0@{ icon: "mdi:cog", form: "rounded", label: "ScrapeOps Parser", pos: "b", h: 48 }
n1@{ icon: "mdi:cog", form: "rounded", label: "ScrapeOps Proxy", pos: "b", h: 48 }
n2@{ icon: "mdi:swap-vertical", form: "rounded", label: "Add Metadata", pos: "b", h: 48 }
n3@{ icon: "mdi:database", form: "rounded", label: "Save to Google Sheets", pos: "b", h: 48 }
n4@{ icon: "mdi:swap-vertical", form: "rounded", label: "Format Product Data", pos: "b", h: 48 }
n5["<div style='background:#f5f5f5;padding:10px;border-radius:8px;display:inline-block;border:1px solid #e0e0e0'><img src='https://flowpast.com/wp-content/uploads/n8n-workflow-icons/code.svg' width='40' height='40' /></div><br/>Split Products Array"]
n6@{ icon: "mdi:swap-vertical", form: "rounded", label: "Set Search Parameters", pos: "b", h: 48 }
n7@{ icon: "mdi:play-circle", form: "rounded", label: "Schedule Trigger", pos: "b", h: 48 }
n8@{ icon: "mdi:swap-horizontal", form: "rounded", label: "Filter Valid Products", pos: "b", h: 48 }
n9["<div style='background:#f5f5f5;padding:10px;border-radius:8px;display:inline-block;border:1px solid #e0e0e0'><img src='https://flowpast.com/wp-content/uploads/n8n-workflow-icons/slack.svg' width='40' height='40' /></div><br/>Send Slack Summary"]
n2 --> n5
n1 --> n0
n7 --> n6
n0 --> n2
n4 --> n8
n5 --> n4
n8 --> n3
n8 --> n9
n3 --> n9
n6 --> n1
end
%% Styling
classDef trigger fill:#e8f5e9,stroke:#388e3c,stroke-width:2px
classDef ai fill:#e3f2fd,stroke:#1976d2,stroke-width:2px
classDef aiModel fill:#e8eaf6,stroke:#3f51b5,stroke-width:2px
classDef decision fill:#fff8e1,stroke:#f9a825,stroke-width:2px
classDef database fill:#fce4ec,stroke:#c2185b,stroke-width:2px
classDef api fill:#fff3e0,stroke:#e65100,stroke-width:2px
classDef code fill:#f3e5f5,stroke:#7b1fa2,stroke-width:2px
classDef disabled stroke-dasharray: 5 5,opacity: 0.5
class n7 trigger
class n8 decision
class n3 database
class n5 code
classDef customIcon fill:none,stroke:none
class n5,n9 customIcon
The Problem: Walmart prices change, but your spreadsheet doesn’t
Tracking Walmart search results manually sounds simple until you do it more than once. Titles aren’t consistent, sponsored listings sneak into your data, and prices come through with weird formatting that breaks formulas. Then Walmart changes a page element and your “quick scrape” turns into an hour of rework. Meanwhile your report is already stale, so you’re making decisions from yesterday’s numbers (or last week’s). Honestly, the worst part is the mental load: you never trust the sheet because you know how fragile the process is.
The friction compounds fast. Here’s where it usually breaks down.
- You spend about 1–2 hours a week just collecting and cleaning rows for a single keyword.
- Small copy-paste mistakes slip in, which means you end up double-checking everything anyway.
- When the page renders differently, your results go missing and you don’t notice until a meeting.
- Your “source of truth” lives in someone’s browser tabs instead of a repeatable pipeline.
The Solution: scheduled Walmart discovery into clean Google Sheets rows
This n8n workflow runs on a timer (the default is every 4 hours) and pulls a Walmart search results page for your chosen keyword. Instead of relying on a brittle scraper, it uses ScrapeOps Proxy API to fetch the page with JavaScript rendering and scrolling handled for you. Then ScrapeOps Parser API extracts structured fields like title, price, rating, reviews count, image URL, product URL, and a sponsored flag. After that, the workflow normalizes the records, filters out empty or bad price rows, and appends clean entries into Google Sheets for reporting. Optionally, it posts a quick summary to Slack so you know the refresh happened and where to look.
The workflow starts with a scheduled trigger and a simple “search inputs” step that builds the Walmart query URL. ScrapeOps fetches and parses the page, n8n cleans and validates the product records, and Google Sheets becomes your living dataset. If you enable Slack, your team gets a lightweight pulse check without opening the sheet.
What You Get: Automation vs. Results
| What This Workflow Automates | Results You’ll Get |
|---|---|
|
|
Example: What This Looks Like
Say you track 3 Walmart keywords and review the top results for each, twice a day. Manually, it’s maybe 20 minutes per keyword once you copy titles, clean prices, and remove sponsored items, so that’s about 2 hours a day. With this workflow, you change the keyword once in n8n, let the schedule run every 4 hours, and review the sheet when you want. Your “work” becomes a 10-minute scan for anomalies, not an endless collection job.
What You’ll Need
- n8n instance (try n8n Cloud free)
- Self-hosting option if you prefer (Hostinger works well)
- ScrapeOps Proxy API to fetch rendered Walmart pages
- ScrapeOps Parser API to extract structured product fields
- Google Sheets for storing and sharing the results
- ScrapeOps API key (get it from your ScrapeOps dashboard)
Skill level: Beginner. You’ll mostly paste credentials, edit a keyword, and confirm the Google Sheets destination.
Don’t want to set this up yourself? Talk to an automation expert (free 15-minute consultation).
How It Works
A schedule kicks things off. The workflow uses a Scheduled Run Trigger set to run every 4 hours by default. You can change that to daily, hourly, or even specific business hours if your reporting cadence is different.
The search URL gets built from your keyword. In “Configure Search Inputs,” you set the term you want to monitor and the workflow constructs a Walmart search URL. Keep it simple at first, then refine it with category filters once you know the output looks right.
ScrapeOps fetches and parses the results. The ScrapeOps Fetch Proxy node requests the page with rendering and scrolling so the product list actually loads. Next, the ScrapeOps Parse Engine turns that HTML into structured product data (titles, prices, ratings, reviews, image URLs, product links, and a sponsored flag).
n8n cleans the records and writes them where you need them. The workflow explodes the product list into individual items, normalizes the fields, then uses an “If” check to drop empty rows or bad prices. Finally, it appends the validated results to Google Sheets and can post a short Slack summary with a link to your sheet.
You can easily modify the keyword inputs to track multiple terms, or switch the destination from Google Sheets to Excel 365 based on your needs. See the full implementation guide below for customization options.
Step-by-Step Implementation Guide
Step 1: Configure the Scheduled Run Trigger
Set the workflow’s schedule so the scrape runs automatically at your desired interval.
- Add and open Scheduled Run Trigger.
- Set Rule → Interval to
hourswithhoursIntervalset to4. - Confirm the trigger connects directly to Configure Search Inputs.
Step 2: Connect ScrapeOps for Fetching and Parsing
Configure the proxy fetch and parser nodes that retrieve and parse Walmart search results.
- Open ScrapeOps Fetch Proxy and set URL to
=https://www.walmart.com/search?q= {{ $json.search_term }}. - Credential Required: Connect your
scrapeOpsApicredentials in ScrapeOps Fetch Proxy. - Open ScrapeOps Parse Engine and set API Type to
parserApi. - Set Parser URL to
=https://www.walmart.com/search?q= {{ $('Configure Search Inputs').item.json.search_term }}. - Set Parser HTML to
={{ $json }}and Parser Domain towalmart. - Credential Required: Connect your
scrapeOpsApicredentials in ScrapeOps Parse Engine.
Step 3: Set Up Product Processing
Define the search term, build context metadata, split product lists, and normalize fields for clean output.
- Open Configure Search Inputs and set search_term to
Raspberry Pi. - In Append Context Details, map the fields using these expressions: timestamp →
{{ new Date().toISOString() }}, search_term →{{ $('Configure Search Inputs').item.json.search_term }}, total_products_found →{{ $json.data.searchMetadata.totalResults || 0}}, products →{{ $json.data.products }}, url_scraped →{{ $json.url }}, data_coverage →{{ $json.data_coverage_percentage }}. - Keep the JavaScript in Explode Product List as provided to split the product array into individual items.
- In Normalize Product Fields, keep the field mappings such as product_name →
{{ $json.name || $json.title || 'N/A' }}and price →{{ $json.price || 'N/A' }}. - Set scraped_at to
{{ $('Configure Search Inputs').item.json.Month }}/{{ $('Configure Search Inputs').item.json["Day of month"] }}/{{ $('Configure Search Inputs').item.json.Year }}and search_term_used to{{ $('Append Context Details').item.json.search_term }}.
⚠️ Common Pitfall: Normalize Product Fields references Month, Day of month, and Year from Configure Search Inputs. If you don’t add these fields, scraped_at will be blank. Add them if you want date stamping.
Step 4: Configure Validation and Output Destinations
Filter out invalid products, append clean data to Google Sheets, and send a Slack summary.
- In Validate Product Records, confirm the conditions: product_name notEquals
N/Aand price notEqualsN/A. - Open Append to Sheets and set Operation to
append. - Set Document ID to
https://docs.google.com/spreadsheets/d/[YOUR_ID]and Sheet Name to[YOUR_ID]. - Keep the column mappings, including Price Display →
${{ Number(($json.price.match(/(?:current price|now)[^\d]*(\d+\.\d+)/i) || ["","0"])[1]) }}and Popularity Level →{{ parseInt($json.review_count ) > 1000 ? 'High' : (parseInt($json.review_count || 0) > 100 ? 'Medium' : (parseInt($json.review_count || 0) > 10 ? 'Low' : 'Very Low')) }}. - Credential Required: Connect your
googleSheetsOAuth2Apicredentials in Append to Sheets. - Open Post Slack Summary and set Text to
🛒 Scrape Complete! | Results Summary: https://docs.google.com/spreadsheets/d/[YOUR_ID]. - Select the Slack channel in Post Slack Summary and keep Include Link To Workflow enabled.
- Credential Required: Connect your
slackApicredentials in Post Slack Summary.
⚠️ Common Pitfall: Replace every [YOUR_ID] placeholder in Append to Sheets and Post Slack Summary with your actual Google Sheet ID and Slack channel.
Step 5: Test and Activate Your Workflow
Run a manual test to verify data flows through each node, then activate the automation.
- Click Execute Workflow to run the flow from Scheduled Run Trigger.
- Verify that Append to Sheets adds new rows with product details and that Post Slack Summary posts the summary message.
- If results are missing, inspect Explode Product List and Normalize Product Fields for empty outputs.
- When the test is successful, toggle the workflow to Active for scheduled production runs.
Common Gotchas
- Google Sheets credentials can expire or need specific permissions. If things break, check the n8n Credentials panel and confirm the connected Google account still has edit access to the destination Sheet.
- If you rely on JS render + scroll via ScrapeOps, processing times vary. If the parser returns partial results, increase the render/scroll/wait settings in the ScrapeOps Fetch Proxy node before assuming the parser is “wrong.”
- AI nodes can be tempting for “cleaning,” but default prompts are generic. If you use the AI Agent/OpenAI Chat Model in this workflow, add strict formatting rules (currency, decimals, and allowed characters) or you will be fixing outputs by hand.
Frequently Asked Questions
About 10 minutes if your ScrapeOps key and Google Sheet are ready.
No. You’ll edit a keyword, connect accounts, and test one run.
Yes. n8n has a free self-hosted option and a free trial on n8n Cloud. Cloud plans start at $20/month for higher volume. You’ll also need to factor in ScrapeOps usage based on your quota and how often you run the schedule.
Two options: n8n Cloud (managed, easiest setup) or self-hosting on a VPS. For self-hosting, Hostinger VPS is affordable and handles n8n well. Self-hosting gives you unlimited executions but requires basic server management.
Yes, but you’ll want to do it intentionally. The simplest option is to duplicate the workflow per keyword and keep each Google Sheet tab separate. If you prefer one workflow, you can turn “Configure Search Inputs” into a list of keywords and loop through them (Split in Batches), then append rows with an extra “keyword” column from the “Append Context Details” step. Many teams also customize the Slack message in “Post Slack Summary” to include the keyword and row count so it’s obvious what refreshed.
Usually it’s an API key issue or quota. Regenerate your ScrapeOps key (or confirm it didn’t get rotated), then update the ScrapeOps credentials in n8n. If the key is fine, check the ScrapeOps Fetch Proxy node settings for render/scroll values that are too aggressive, which can increase cost and trigger limits. It can also be Walmart blocking certain request patterns, so switching proxy settings or adding a bit more wait time often stabilizes runs.
Practically, it depends on how many items your Walmart search returns and how far you scroll. On n8n Cloud Starter you’re capped by monthly executions, while self-hosting has no execution limit (your server becomes the limit). In most cases this workflow is used for the first page or two of results per keyword, refreshed a few times a day, which stays manageable and keeps the sheet clean.
Often, yes, because the hard part here isn’t “send data to Sheets,” it’s fetching and cleaning data reliably. n8n is better when you need filtering (“Validate Product Records”), formatting (“Normalize Product Fields”), and looping through a batch of items without paying extra for every branch. It also supports self-hosting, which matters once you run this every 4 hours across multiple keywords. Zapier or Make can be fine for lightweight workflows, but they’re not built for scraping-style pipelines. If you’re torn, Talk to an automation expert and we’ll map it to your volume and reporting needs.
Once this is running, your sheet stays fresh on autopilot. You’ll spend your time spotting changes, not collecting them.
Need Help Setting This Up?
Our automation experts can build and customize this workflow for your specific needs. Free 15-minute consultation—no commitment required.
