🔓 Unlock all 10,000+ workflows & prompts free Join Newsletter →
✅ Full access unlocked — explore all 10,000 AI workflow and prompt templates Browse Templates →
Home › n8n › Workflow
January 22, 2026

Web pages to Google Sheets, competitor data captured

Web pages to Google Sheets, competitor data captured - n8n workflow automation template
Lisa Granqvist Partner Workflow Automation Expert
Web pages to Google Sheets, competitor data captured - n8n workflow automation template
Get a free AI assessment → Get a free AI assessment → ⬇️ Use template

Competitor tracking sounds simple until you’re copying numbers from five tabs into a sheet and realizing one “Followers” value came from the wrong page. Again. That’s how messy “manual research” turns into decisions built on bad data.

Marketing managers feel this when weekly reporting comes around. Agency owners get it when clients ask, “Can we track this for 10 brands?” And solo operators just want competitor data automation that doesn’t steal their afternoons.

This workflow uses Selenium plus AI to pull the exact numbers you care about from almost any webpage and send them into Google Sheets (or Excel). You’ll see how it works, what you need, and where people usually get stuck.

How This Automation Works

Here’s the complete workflow you’ll be setting up:

n8n Workflow Template: Web pages to Google Sheets, competitor data captured

Why This Matters: Clean competitor numbers without busywork

If you track competitors, you already know the pain: the numbers you need are buried in different layouts, loaded by scripts, and sometimes behind a login. So you open a site, scroll, copy a value, paste it into a sheet, then repeat for the next brand. After a few rounds, your “quick check” becomes a recurring task that quietly eats a couple hours a week. Worse, it’s easy to paste the right number into the wrong row, or pull yesterday’s value from a cached page and not notice until someone asks why the chart looks weird.

It adds up fast. Here’s where it usually breaks down.

  • You end up re-checking the same pages because you don’t fully trust what you pasted last time.
  • Modern sites render key stats dynamically, so simple “scrapers” miss the value or grab the wrong element.
  • Login-required pages force awkward workarounds, which usually means someone does it manually anyway.
  • When the page layout changes, your tracking process silently fails and nobody notices for days.

What You’ll Build: A webpage-to-sheet competitor tracker that adapts

This template is built for real-world webpages, not perfect static HTML. You send a request (usually via a webhook) with a target URL and a list of the fields you want, like “Followers” or “Total Stars.” The workflow then spins up a Selenium browser session, navigates to the page, and captures screenshots so the AI can “see” what’s on the screen. OpenAI handles the interpretation and extraction, turning what it finds into structured results you can store and trend over time. If you need authenticated access, you can pass session cookies so the workflow can load logged-in pages too. Finally, the workflow returns a clean response and can log the results into Google Sheets or Excel for reporting.

The flow starts with your input (URL + the data points you care about). Selenium loads the page and produces visual evidence (screenshots), which means dynamic pages are fair game. Then AI extracts the requested values and pushes them into a spreadsheet-ready structure.

What You’re Building

Expected Results

Say you track 8 competitors and you collect 5 numbers from each site every Monday. Manually, it’s easy to spend about 10 minutes per competitor between loading pages, finding the right section, and pasting values, so you lose roughly 80 minutes weekly. With this workflow, you submit one request per competitor (or batch them) and wait for the run to finish, which is usually more like 10 minutes of active time total. The time isn’t the only win. You stop second-guessing the sheet.

Before You Start

  • n8n instance (try n8n Cloud free)
  • Self-hosting option if you prefer (Hostinger works well)
  • Selenium setup to load and screenshot web pages.
  • OpenAI API to interpret screenshots and extract fields.
  • Google Sheets to store competitor metrics over time.
  • OpenAI API key (get it from your OpenAI dashboard).

Skill level: Intermediate. You won’t write much code, but you will connect accounts and follow a setup guide for the Selenium side.

Want someone to build this for you? Talk to an automation expert (free 15-minute consultation).

Step by Step

A webhook receives your tracking request. You send the target URL (or a subject the workflow can search for) plus a list of “Target data” fields you want pulled from the page.

The workflow chooses the best URL to visit. If you provided a direct URL, it uses that. If not, it can run a search query, pull a likely match, and validate it before moving forward.

Selenium loads the page like a real browser. The automation starts a session, resizes the window, refreshes when needed, and captures multiple screenshots to increase the chance of catching the right element on the page.

AI extracts the exact fields you requested. The screenshots are sent through OpenAI-powered vision and extraction steps, then normalized into a clean response you can store in Google Sheets or Excel.

You can easily modify the target fields to track different KPIs based on your needs. See the full implementation guide below for customization options.

Step-by-Step Implementation Guide

Step 1: Configure the Webhook Trigger

Start by setting up the webhook that initiates the scraping request and captures input fields.

  1. Add or open Incoming Webhook Trigger and set HTTP Method to POST.
  2. Set Path to 67d77918-2d5b-48c1-ae73-2004b32125f0 and Response Mode to responseNode.
  3. Open Map Input Fields and map Subject to {{ $json.body.subject }} and Website Domaine to {{ $json.body.Url }}.

Step 2: Connect the URL Discovery Path

This workflow searches for a URL when the target URL is missing, then validates it before scraping.

  1. In Check Target URL Empty, keep the empty check on {{ $('Incoming Webhook Trigger').item.json.body['Target Url'] }}.
  2. Configure Search Engine Query with URL set to https://www.google.com/search?q=site:{{ $json['Website Domaine'] }}+{{$json.Subject}}&oq=site&gs_lcrp=EgZjaHJvbWUqCAgAEEUYJxg7MggIABBFGCcYOzIICAEQRRgnGDsyBggCEEUYOzIRCAMQRRg5GEMYyQMYgAQYigUyBggEEEUYQDIGCAUQRRg9MgYIBhBFGD0yBggHEEUYPdIBCDEwNTRqMGo3qAIAsAIA&sourceid=chrome&ie=UTF-8.
  3. In Pull First URL Match, keep the HTML extraction selector set to =a[href*="https://"][href*="{{ $('Map Input Fields').item.json['Website Domaine'] }}"].
  4. Set Extract URL Candidate Text to {{ $json['Url Find '][1] }}{{ $json['Url Find '][2] }}{{ $json['Url Find '][3] }} and keep the system prompt as configured.
  5. In Validate URL Output, keep the conditions that treat empty or NA as invalid and route to Return URL Not Found.

Step 3: Configure Selenium Session Startup

These nodes launch Selenium, size the browser, and apply stealth settings before navigation.

  1. Set Start Selenium Session URL to http://selenium_chrome:4444/wd/hub/session and keep the JSON body with the Chrome options provided.
  2. Configure Resize Browser Window with URL http://selenium_chrome:4444/wd/hub/session/{{ $json.value.sessionId }}/window/rect and JSON body {"width":1920,"height":1080,"x":0,"y":0}.
  3. In Stealth Driver Cleanup, keep URL set to http://selenium_chrome:4444/wd/hub/session/{{ $('Start Selenium Session').item.json.value.sessionId }}/execute/sync and preserve the stealth JavaScript in JSON Body.

Step 4: Configure Cookie Validation and Injection

This section verifies cookie presence, checks domain matching, and inserts cookies into Selenium.

  1. In Check Cookies Present, keep the not-empty check on {{ $('Incoming Webhook Trigger').item.json.body.cookies }}.
  2. In Domain Match Check, keep the contains condition between {{ $('Incoming Webhook Trigger').item.json.body.cookies[0].cookie.domain }} and {{ $('Incoming Webhook Trigger').item.json.body.Url }}.
  3. Keep the JavaScript in Custom Cookie Parser unchanged so it normalizes sameSite values.
  4. Set Insert Session Cookie URL to http://selenium_chrome:4444/wd/hub/session/{{ $('Start Selenium Session').item.json.value.sessionId }}/cookie and keep the JSON body that maps cookie fields from {{ $json.* }}.
⚠️ Common Pitfall: If the cookies don’t match the target domain, Terminate Session F will fire and return Return Cookie Error. Ensure the webhook payload includes valid cookies for the site.

Step 5: Configure Navigation and Screenshot Capture

The workflow chooses a URL path and captures screenshots for AI analysis.

  1. In Choose URL Source, keep the not-empty condition on {{ $('Incoming Webhook Trigger').item.json.body['Target Url'] }} to decide between direct or extracted URL navigation.
  2. Set Navigate Direct URL and Navigate Target URL JSON Body to {"url":"{{ $('Incoming Webhook Trigger').item.json.body['Target Url'] }}"} .
  3. Set Navigate Extracted URL and Navigate Found URL JSON Body to {"url":"{{ $('Extract URL Candidate').item.json.output.Good_url_for_etract_information }}"} .
  4. Confirm Capture Screenshot A, Capture Screenshot B, and Capture Screenshot C use http://selenium_chrome:4444/wd/hub/session/{{ $('Start Selenium Session').item.json.value.sessionId }}/screenshot.
  5. Keep Convert Image File A, Convert Image File B, and Convert Image File C set to Operation toBinary and Source Property value.

Step 6: Set Up AI Analysis and URL Extraction

OpenAI models analyze screenshots and extract structured data or detect blocks.

  1. Open Vision Analysis A and Vision Analysis B, and keep the analysis prompt with {{ $('Incoming Webhook Trigger').item.json.body.subject }} in the text.
  2. Credential Required: Connect your openAiApi credentials in Vision Analysis A and Vision Analysis B.
  3. Open Extract Info Set A and Extract Info Set B and keep Text set to {{ $('Vision Analysis A').item.json.content }} and {{ $('Vision Analysis B').item.json.content }}.
  4. AI Chat Model Mini A is connected as the language model for Extract Info Set A — ensure credentials are added to AI Chat Model Mini A.
  5. AI Chat Model Mini B is connected as the language model for Extract Info Set B — ensure credentials are added to AI Chat Model Mini B.
  6. AI Chat Model Core is connected as the language model for Extract URL Candidate — ensure credentials are added to AI Chat Model Core with model gpt-4o.
Tip: If AI responses are empty or malformed, confirm your OpenAI credentials are active and your models are available in your account.

Step 7: Configure Block Detection and Branching

The workflow detects blocked pages and routes to the appropriate response nodes.

  1. In Block Check Branch, keep the condition {{ $json.content }} equals BLOCK.
  2. In Block Content Check, keep the contains condition for BLOCK in {{ $json.content }}.
  3. Confirm the branch output links to Terminate Session B → Return Block Notice and Terminate Session D → Return Block Notice 2.

Step 8: Configure Output Responses

Set the workflow’s responses for successful and unsuccessful runs.

  1. In Return Success Output, keep Response Key set to {{ $json.output }} and Response Code to 200.
  2. In Respond Success Cookie, keep the default Response Code 200 for cookie-based success.
  3. Confirm fallback responses are configured: Return URL Not Found, Return Cookie Error, Return Page Crash, Return Session Error, and Return Server Error.

Step 9: Add Error Handling

Although there is no dedicated error handler node, the workflow uses terminate-and-respond patterns to handle failures.

  1. Review all termination nodes (e.g., Terminate Session A through Terminate Session I) to ensure they target http://selenium_chrome:4444/wd/hub/session/{{ $('Start Selenium Session').item.json.value.sessionId }}.
  2. Verify that each termination node routes to a corresponding response node (e.g., Return Server Error, Return Page Crash).
  3. Group review: The workflow contains many httpRequest and respondToWebhook nodes; ensure all are correctly connected to terminate-and-respond branches.
⚠️ Common Pitfall: If any Selenium node fails to return a sessionId, the terminate nodes will fail too. Validate Start Selenium Session is returning {{ $json.value.sessionId }}.

Step 10: Test & Activate Your Workflow

Run a full test to confirm navigation, AI analysis, and response paths before enabling production usage.

  1. Use Incoming Webhook Trigger’s test URL and send a POST body containing subject, Url, optional Target Url, and cookies.
  2. Manually execute the workflow and verify that a successful run returns data via Return Success Output or Respond Success Cookie.
  3. Check that blocked or invalid paths return Return Block Notice, Return URL Not Found, or other error responses as expected.
  4. When results look correct, switch the workflow to Active to enable production use.
🔒

Unlock Full Step-by-Step Guide

Get the complete implementation guide + downloadable template

Troubleshooting Tips

  • OpenAI credentials can expire or be restricted by billing. If things break, check your OpenAI API key status and usage limits first.
  • If you’re using Wait nodes or external rendering, processing times vary. Bump up the wait duration if downstream nodes fail on empty responses.
  • Default prompts in AI nodes are generic. Add your brand voice early or you’ll be editing outputs forever.

Next Steps

If your team also needs a second source for research, ScrapeGraphAI + Google Sheets: research feed ready pairs nicely because it’s built for turning broad web findings into a sheet you can filter and prioritize.

For teams doing both competitor tracking and internal release monitoring, GitHub to Slack, safer releases without manual imports keeps product updates visible without somebody playing messenger all week.

When your workflow is “collect data, then summarize it for a quick read,” OpenAI + NocoDB, news summaries logged for you is a strong follow-on so you can store summaries alongside raw competitor metrics.

Need scrapers that don’t fall over the moment a site tweaks its layout? Gemini + Google Sheets: scrapers that fix themselves is worth adding when you want more resilience and less babysitting.

Finally, if you want a lightweight “daily touch” channel to stay on top of what competitors are doing socially, X replies with Apify + Telegram, stay active daily covers the engagement side while this workflow covers the numbers.


Quick reference:

Quick Answers

What’s the setup time for this competitor data automation?

Plan on about an hour if Selenium is new to you.

Is coding required for this competitor tracking automation?

No. You’ll mostly connect accounts and follow the Selenium setup instructions. The workflow already contains the logic for URL checks, screenshots, and extraction.

Is n8n free to use for this competitor data automation workflow?

Yes. n8n has a free self-hosted option and a free trial on n8n Cloud. Cloud plans start at $20/month for higher volume. You’ll also need to factor in OpenAI API costs, which are usually a few cents per run depending on screenshot size and prompt length.

Where can I host n8n to run this automation?

Two options: n8n Cloud (managed, easiest setup) or self-hosting on a VPS. For self-hosting, Hostinger VPS is affordable and handles n8n well. Self-hosting gives you unlimited executions but requires basic server management.

Can I modify this competitor data automation workflow for different use cases?

Yes, and you should. The easiest change is editing the incoming request fields (the “Target data” list) so the extractor looks for different metrics, like “Price,” “Plan name,” or “Employee count.” If a site blocks bots, you can adjust the block-check logic and add cookies via the cookie parsing step so Selenium loads an authenticated page. You can also swap the destination from Google Sheets to Excel 365 if that’s where your reporting lives. And if you want to track multiple pages per brand, you can run the webhook request in a loop and merge the results before writing a single row.

Why is my Selenium connection failing in this workflow?

Usually the Selenium service isn’t running, isn’t reachable from n8n, or the session terminates early. Confirm your Selenium endpoint and make sure the workflow can create a session, then check the “block notice” responses because some sites detect automation and serve a dead page.

What volume can this competitor data automation workflow process?

If you self-host, there’s no execution limit, but Selenium sessions are the bottleneck, so think in terms of “a few pages at a time” rather than hundreds at once.

Is this competitor data automation better than using Zapier or Make?

For this workflow, yes, most of the time. Zapier and Make are great for SaaS-to-SaaS automations, but browser-based scraping with branching logic, error handling, and session cleanup is where they start feeling cramped (and expensive). n8n lets you control the full flow: URL selection, cookie handling, block detection, multi-shot screenshots, and structured extraction. You can also self-host, which matters if you’re running lots of checks. If you only need something like “copy a value from an API into a sheet,” Zapier or Make might be quicker. Talk to an automation expert if you want a recommendation for your exact use case.

Once this is running, competitor checks stop being a weekly scramble. The workflow pulls the numbers, your sheet stays clean, and you can finally focus on what the data means.

Need Help Setting This Up?

Our automation experts can build and customize this workflow for your specific needs. Free 15-minute consultation—no commitment required.

Talk to Automation Expert →

Lisa Granqvist

Workflow Automation Expert

Expert in workflow automation and no-code tools.

×

Use template

Get instant access to this n8n workflow Json file

Copy and import json workflow into your n8n workflow

💬
Get a free quote today!
Get a free quote today!

Tell us what you need and we'll get back to you within one working day.

Get a free quote today!
Get a free quote today!

Tell us what you need and we'll get back to you within one working day.