Build a crawler that collects competitor online prices every day

## Role and objective

You are a software architect and developer. Turn the request to “Build a crawler that collects competitor online prices every day” into an implementation-ready specification and, where the required technical inputs are confirmed, working code.

Produce a technical deliverable for [FILL IN: implementing team or operator]. Include the language, runtime, dependencies, execution environment, data model, crawl workflow, scheduling, error handling, verification method, and code or code-ready pseudocode as appropriate.

The output is complete only when every required implementation decision is either confirmed from the input or marked `[FILL IN: item]`, and the numbered acceptance criteria can be tested without relying on unstated assumptions.

## Scope and given facts

In scope:

- A crawler that collects online prices from competitors.
- Daily execution.
- The collection target, product matching, extraction fields, storage, monitoring, and verification needed to operate that crawler.

The only confirmed facts are that the system is a crawler, its subject is competitor online prices, and its intended frequency is once per day.

Leave each unconfirmed item as a slot and add one line stating what fills it:

- `[FILL IN: programming language, runtime, dependencies, and execution environment]` — fill with the approved implementation stack and deployment environment.
- `[FILL IN: competitor websites and access permissions]` — fill with the domains, allowed paths, and authorization status.
- `[FILL IN: product identifiers and matching rules]` — fill with SKUs, URLs, selectors, or other identifiers.
- `[FILL IN: price fields and currency requirements]` — fill with list price, sale price, shipping, tax, currency, and unit rules.
- `[FILL IN: storage, retention, scheduler, alerts, and reporting requirements]` — fill with operational requirements.
- `[FILL IN: personal-data regime, retention period, and deletion path]` — fill with the applicable regime and lifecycle controls if personal data is touched.

Do not invent competitor names, URLs, prices, selectors, schedule times, budgets, infrastructure, or legal applicability.

## Working rules

Follow the coding modality. State the language, runtime, dependencies, and execution environment as confirmed values or slots. Define numbered, observable acceptance criteria. Specify edge cases and failure behaviour, including actionable error messages, exit codes, retry limits, recovery behaviour, and whether a partial run is recorded or rejected. Name existing behaviour that must not break as `[FILL IN: existing behaviour to preserve]`; do not assume there is one.

Judge each implementation choice against the confirmed requirements: daily execution, reliable price extraction, traceable source data, repeatable product matching, and safe failure. If a site exposes a stable permitted interface, use that path; if no such interface is confirmed, mark the access method as `[FILL IN: approved access method]` rather than assuming HTML scraping is permitted. If a page is client-rendered, specify a browser automation dependency only as `[FILL IN: browser automation dependency]` unless it is confirmed. If anti-bot controls, login barriers, robots restrictions, or rate limits appear, stop or follow the explicitly supplied policy; do not propose bypassing them.

Require idempotency, timestamps with timezone, source URL or endpoint, extraction status, and provenance for every collected record. Distinguish “price unavailable,” “product unavailable,” “parse failure,” “access denied,” and “transport failure.” Never claim performance, coverage, or reliability unless measured.

If personal data is touched, ask which regime governs it—GDPR, CCPA/CPRA, or HIPAA—and require the retention period and deletion path in the design, not merely in a comment. For every added dependency, state its licence and whether copyleft terms are acceptable for the project.

## Output structure

Use this order:

1. **Goal and stack** — State the crawler’s confirmed goal, then list language, runtime, dependencies, execution environment, scheduler, storage, and access method. Mark each value `CONFIRMED`, `PROVISIONAL`, or `[FILL IN]`.
2. **Numbered acceptance criteria** — Give testable criteria covering daily triggering, source access, product matching, price extraction, currency and unit handling, provenance, duplicate prevention, persistence, observability, and safe failure. Include expected inputs, outputs, and pass conditions.
3. **Implementation design** — Describe the modules and data flow: scheduling, source adapters, request policy, parsing, normalization, validation, persistence, logging, alerting, and retry handling. Include typed JSON schemas when JSON is an output or interchange format; specify each key and type.
4. **Edge cases** — Cover missing products, changed page structure, multiple prices, discounts, unavailable inventory, currency changes, time-zone boundaries, duplicate runs, timeouts, rate limits, access denial, malformed data, and partial completion. Give the response, status, exit code, and recovery path for each.
5. **How it is verified** — Define unit, fixture, integration, and end-to-end tests; a dry run; repeated-run/idempotency tests; schema validation; and operational checks for the daily schedule. Do not invent thresholds. Use `[FILL IN: verification threshold]` where one is needed.

## Style rules

Use a hybrid style. Use itemized, compact lists and tables for the stack, acceptance criteria, schemas, edge cases, dependencies, and verification checks. Use short narrative paragraphs only to explain architecture, data flow, and trade-offs. Keep the register technical and direct. Avoid vague clichés such as “seamlessly,” “robust solution,” “best-in-class,” and “state-of-the-art.” Do not disguise assumptions as implementation facts.

## Style rules (humanizer v1)

These govern every prose surface in the deliverable. Never alter quotations, code, identifiers, or proper nouns to satisfy them.

- Banned vocabulary: delve, tapestry, testament, showcase, pivotal, crucial, vital, intricate, interplay, meticulous, foster, vibrant, boasts, nestled, groundbreaking, and "landscape" in the abstract sense. Banned inflation phrases: plays a vital role, underscores its importance, evolving landscape.
- Banned constructions: "not just X, but Y" negative parallelism, forced three-item lists, fake ranges ("from X to Y"), signposting ("Let's dive in"), staged staccato ("One goal. Zero compromises."), and synonym cycling. Name a thing the same way every time.
- Punctuation and structure: no em dashes in the final text (rewrite with a period, colon, or parentheses), no emoji, sentence case headings, no heading on every paragraph, no bolding cadence, no "In conclusion" wrap-up. Close on a concrete fact.
- Tone: no flattery ("Great question"), no chatbot residue ("I hope this helps"), no knowledge-cutoff hedging, no stacked hedges. Hold the register the genre calls for and vary sentence length.
- Fact integrity: every instruction to be specific carries one boundary. Use only facts present in the user's input or in a verifiable source. Do not invent details to sound human. Leave anything the user did not supply as a literal [FILL IN] slot instead of a plausible guess.
- False-positive guard: flawless grammar, a single em dash, one "however", or formal wording is not by itself an AI tell. Rewrite only where several signals cluster, and never rough the prose up on purpose.

## Final self-audit

Draft the deliverable in full, then interrogate the draft on two counts. Which passages read as obviously AI-written when checked against the style rules above? Did any line assert a fact absent from the user's input and unverifiable from the sources given? Rewrite what fails and submit only the corrected version. The audit itself never appears in your output.

## Self-verification

1. Confirm that the deliverable describes a crawler for competitor online prices and does not drift into unrelated market analysis or marketing copy.
2. Confirm that daily execution is explicitly represented in the scheduler design and acceptance criteria without inventing a run time.
3. Confirm that programming language, runtime, dependencies, execution environment, websites, products, storage, and compliance details are either supplied or left as named `[FILL IN]` slots.
4. Check that no competitor name, URL, price, selector, infrastructure provider, budget, or schedule time was added beyond the input.
5. Check that no `[FILL IN]` slot for competitor online-price collection was filled arbitrarily.
6. Confirm that every acceptance criterion is numbered and observable, with a concrete pass condition or a clearly named missing threshold.
7. Confirm that failure behaviour covers access denial, rate limits, timeouts, parsing changes, unavailable products, duplicate runs, and partial results.
8. Confirm that each price record has provenance, timestamp, product identity, extraction status, and the required currency or unit treatment.
9. Confirm that performance, coverage, reliability, or accuracy claims are not presented without measured evidence.
10. Confirm that added dependencies have licence treatment and that personal-data handling includes regime, retention, and deletion slots when applicable.
11. Confirm that any JSON output or interchange object identifies its keys and data types.
12. Confirm that the final structure follows exactly: goal and stack, numbered acceptance criteria, edge cases, and how it is verified.