Build a crawler that collects competitor online prices every day

## Role and objective

<instructions>
You are a software architect and developer. Design and specify a crawler that collects competitor online prices every day. Produce an implementation-ready technical response for the person who will build, operate, and verify the crawler.

Your output must be a development specification ordered as: goal and stack, numbered acceptance criteria, edge cases, and how it is verified. Completion means that each acceptance criterion is observable through a test, log, stored result, or explicit operational check, and that no unconfirmed implementation choice is presented as settled fact.

Before the conclusion, provide concise reasoning steps that explain how you resolved the supplied requirements, identified missing inputs, and selected each proposed design decision. Do not expose private chain-of-thought; provide only a brief decision rationale and verifiable basis.
</instructions>

## Scope and given facts

<instructions>
Treat only the following as confirmed:

- The system is a crawler.
- Its subject is competitor online prices.
- Collection must occur every day.

In scope are the collection workflow, scheduling, extraction, normalization, storage, validation, monitoring, and verification needed to satisfy those facts. Do not silently add product categories, competitors, countries, currencies, websites, APIs, authentication methods, database systems, cloud providers, notification channels, legal permissions, or freshness guarantees.

Leave each unconfirmed value as a slot and add one line explaining what supplies it:

- [FILL IN: programming language, runtime, dependencies, and execution environment] — fill this with the approved technical stack and deployment context.
- [FILL IN: competitor websites, product identifiers, and permitted collection method] — fill this with the authorized sources and matching rules.
- [FILL IN: schedule time zone and daily run time] — fill this with the operational schedule.
- [FILL IN: price fields, currency handling, tax, shipping, and promotion rules] — fill this with the required price definition.
- [FILL IN: storage target and retention period] — fill this with the persistence and lifecycle requirements.
- [FILL IN: alert recipients and notification channel] — fill this with the monitoring destination.

Do not fill the competitor websites, daily schedule, or price definition with plausible examples.
</instructions>

## Working rules

<instructions>
Judge every design choice against four criteria: whether it can collect the required competitor prices daily, whether the result can be reproduced and audited, whether failures are visible rather than silently accepted, and whether the choice is supported by supplied facts or an explicitly marked slot.

Use the following rules:

1. State language, runtime, dependencies, and execution environment as confirmed values only if supplied; otherwise retain the corresponding slots.
2. Define product matching before extraction. If stable identifiers are supplied, use them; if not, require [FILL IN: product matching rule] rather than guessing that names or URLs are sufficient.
3. Define the price unit explicitly. If tax, shipping, discounts, availability, currency conversion, or regional pricing are unspecified, preserve them as slots and store the raw observed value separately from any normalized value.
4. Record source URL, collection timestamp, schedule date, raw price text, parsed price, currency, product identifier, and extraction status when those fields are confirmed or required by the design. Mark any additional field as provisional unless supplied.
5. Make daily execution idempotent: rerunning the same source and schedule date must not create unexplained duplicate records. State the deduplication key as [FILL IN: deduplication key] if it is not supplied.
6. Define branches. If a source exposes an authorized structured endpoint, use it only when that endpoint and access permission are confirmed. Otherwise specify HTML extraction only for permitted pages. If a page is blocked, changed, incomplete, or ambiguous, record a failure or review status; never invent a price.
7. Specify retry limits, timeout behaviour, rate limiting, robots and terms-of-use review, parser versioning, and error messages or exit codes. Do not claim performance or coverage without measurements.
8. If the crawler touches personal data, ask which regime governs it: [FILL IN: GDPR, CCPA/CPRA, HIPAA, or other regime]. Require the retention period and deletion path in the design, not merely in a comment. For every added dependency, state its licence and whether copyleft terms are acceptable.
</instructions>

## Output structure

<output_format>
Order the response exactly as follows:

1. **Goal and stack** — State the daily competitor-price collection goal, then list confirmed stack values and unresolved stack slots. Describe the execution flow from scheduler to source fetch, extraction, validation, persistence, and reporting. Allocate approximately 20% of the response to this part.
2. **Numbered acceptance criteria** — Provide numbered, observable criteria covering daily scheduling, authorized source access, product matching, price parsing, raw-value preservation, normalization, timestamps, idempotency, persistence, logging, monitoring, and recovery. For each criterion, state the test input or inspection method and the expected result. Allocate approximately 35%.
3. **Edge cases** — Cover unavailable pages, HTTP failures, rate limits, changed markup, missing prices, multiple prices, out-of-stock products, currency or tax ambiguity, duplicate runs, partial completion, dependency failure, and storage failure. For each case, specify status handling, retry or recovery behaviour, and whether the run is failed, partial, or successful. Allocate approximately 25%.
4. **How it is verified** — Give a test plan with unit, fixture-based parser, integration, scheduler, idempotency, failure-injection, data-quality, security, and operational checks. Include the evidence retained for each check and identify all unresolved slots that must be confirmed before deployment. Allocate approximately 20%.

Include code only when the language, runtime, interfaces, and execution environment are confirmed or explicitly slotted. Do not present placeholder values as working production configuration.
</output_format>

## Style rules

Use a hybrid style. Present stack values, acceptance criteria, edge cases, fields, statuses, commands, and verification checks in itemized form. Explain architecture decisions, branch conditions, and operational trade-offs in short narrative paragraphs. Use a precise engineering register. Avoid crawler-specific clichés such as “seamless,” “robust solution,” “at scale,” or “simply scrape”; replace them with measurable behaviour or a clearly marked unresolved requirement.

## Style rules (humanizer v1)

These govern every prose surface in the deliverable. Never alter quotations, code, identifiers, or proper nouns to satisfy them.

- Banned vocabulary: delve, tapestry, testament, showcase, pivotal, crucial, vital, intricate, interplay, meticulous, foster, vibrant, boasts, nestled, groundbreaking, and "landscape" in the abstract sense. Banned inflation phrases: plays a vital role, underscores its importance, evolving landscape.
- Banned constructions: "not just X, but Y" negative parallelism, forced three-item lists, fake ranges ("from X to Y"), signposting ("Let's dive in"), staged staccato ("One goal. Zero compromises."), and synonym cycling. Name a thing the same way every time.
- Punctuation and structure: no em dashes in the final text (rewrite with a period, colon, or parentheses), no emoji, sentence case headings, no heading on every paragraph, no bolding cadence, no "In conclusion" wrap-up. Close on a concrete fact.
- Tone: no flattery ("Great question"), no chatbot residue ("I hope this helps"), no knowledge-cutoff hedging, no stacked hedges. Hold the register the genre calls for and vary sentence length.
- Fact integrity: every instruction to be specific carries one boundary. Use only facts present in the user's input or in a verifiable source. Do not invent details to sound human. Leave anything the user did not supply as a literal [FILL IN] slot instead of a plausible guess.
- False-positive guard: flawless grammar, a single em dash, one "however", or formal wording is not by itself an AI tell. Rewrite only where several signals cluster, and never rough the prose up on purpose.

## Final self-audit

Draft the deliverable in full, then interrogate the draft on two counts. Which passages read as obviously AI-written when checked against the style rules above? Did any line assert a fact absent from the user's input and unverifiable from the sources given? Rewrite what fails and submit only the corrected version. The audit itself never appears in your output.

## Self-verification

1. Confirm that the response treats the subject as a daily competitor-price crawler rather than a general data platform.
2. Confirm that the deliverable follows the required order: goal and stack, numbered acceptance criteria, edge cases, and how it is verified.
3. Check that language, runtime, dependencies, and execution environment are confirmed values or `[FILL IN]` slots.
4. Check that competitor websites, product identifiers, collection permission, schedule, price definition, storage, and notification details are not invented.
5. Check that every proposed price field distinguishes observed raw data from parsed or normalized data where ambiguity exists.
6. Check that changed markup, blocked pages, missing prices, multiple prices, currency ambiguity, duplicate runs, partial completion, and storage failure have explicit handling.
7. Check that acceptance criteria are observable through tests, logs, stored records, exit codes, or inspection rather than vague quality claims.
8. Check that personal-data handling asks for the governing regime, retention period, and deletion path if personal data may be touched.
9. Check that each added dependency has a licence check and copyleft-acceptance decision.
10. Identify every fact added beyond the input “Build a crawler that collects competitor online prices every day,” and replace unsupported facts with slots.
11. Confirm that no slot—especially the competitor websites, daily schedule, or price definition—was filled arbitrarily.
12. Confirm that the response does not drift into unrelated features such as competitor strategy analysis, automated purchasing, or marketing recommendations.
13. Count the checks: there are twelve. Do not deliver if any check fails.