Step 2: Connect the Primary Request and Search Parameters

Define the Etsy search URL and send the initial scraping request through Bright Data.

1. In Define Search URL, set url to https://www.etsy.com/search?q=wall+art+for+mum&order=date_desc&page=1&ref=pagination and zone to web_unlocker1.
2. In Primary Etsy Request, set URL to https://api.brightdata.com/request and Method to POST.
3. In Primary Etsy Request, enable Send Body and Send Headers.
4. Set body parameters in Primary Etsy Request:
   • zone to {{ $json.zone }}
   • url to {{ $json.url }}
   • format to raw
   • data_format to markdown
5. Credential Required: Connect your httpHeaderAuth credentials in Primary Etsy Request.

Step 3: Set Up Pagination Parsing and Page Iteration

Use Gemini to extract pagination URLs, split the results, and iterate over each page.

1. Connect Gemini Chat Engine to Parse Pagination Set as the language model; credentials are added on Gemini Chat Engine, not the extractor node.
2. Credential Required: Connect your googlePalmApi credentials in Gemini Chat Engine.
3. In Parse Pagination Set, set Text to Analyze and Extract the below content. Make sure to produce a unique resultset. Exclude page_numbers which are not numbers. {{ $json.data }}.
4. In Split Output Items, set Field To Split Out to output.
5. Keep Iterate Pages as the batch iterator to drive paginated requests.

Step 4: Configure Paginated Requests and Product Extraction

Request each paginated page and extract product details using Gemini as the model.

1. In Paginated Etsy Request, set URL to https://api.brightdata.com/request and Method to POST.
2. Set body parameters in Paginated Etsy Request:
   • zone to web_unlocker1
   • url to {{ $json.url }}
   • format to raw
   • data_format to markdown
3. Credential Required: Connect your httpHeaderAuth credentials in Paginated Etsy Request.
4. Connect Gemini Product Model to Extract Product Details as the language model; credentials are added on Gemini Product Model, not the extractor node.
5. Credential Required: Connect your googlePalmApi credentials in Gemini Product Model.
6. In Extract Product Details, set Text to Extract the product info in JSON {{ $json.data }} and keep Schema Type as fromJson with the provided example.

Step 5: Route Summary and File Output in Parallel

After extraction, the workflow sends a summary webhook and writes a local JSON file at the same time.

Extract Product Details outputs to both Post Summary Webhook and Build Binary Payload in parallel.

1. In Post Summary Webhook, set URL to https://webhook.site/[YOUR_ID] and enable Send Body.
2. Set the summary body parameter in Post Summary Webhook to {{ $json.output }}.
3. In Build Binary Payload, keep the Function Code as provided to convert JSON into base64 binary data.
4. In Save Scrape File, set Operation to write and File Name to =d:\Esty-Scraped-Content-{{ $('Iterate Pages').item.json.page_number }}.json.
5. Ensure Post Summary Webhook reconnects to Iterate Pages to continue pagination.

Step 6: Optional OpenAI Pagination Parsing

The workflow includes an OpenAI-based pagination parser that can be used as an alternative or for testing.

1. Connect OpenAI Chat Engine to Parse Pages via OpenAI as the language model; credentials are added on OpenAI Chat Engine, not the extractor node.
2. Credential Required: Connect your openAiApi credentials in OpenAI Chat Engine.
3. In Parse Pages via OpenAI, set Text to Analyze and Extract the below content. Make sure to produce a unique resultset. Exclude page_numbers which are not numbers. {{ $json.data }} if you plan to use this branch.

Step 7: Test & Activate Your Workflow

Validate end-to-end scraping, extraction, webhook posting, and file writing before enabling the workflow.

1. Click Execute Workflow on Manual Launch Trigger to run the workflow manually.
2. Confirm that Primary Etsy Request returns data and Parse Pagination Set outputs a list of page URLs.
3. Verify that Post Summary Webhook receives a payload and Save Scrape File writes a file for each page.
4. When results are correct, toggle the workflow to Active for production use.