Question
Closed · Answered

How do you check that Buda has collected every page of a web table?

Jen

I'd like Buda to build a table from a website I'm allowed to access, where it needs to set filters and browse several pages.

Alongside the CSV, I'd want the filters used, the number of pages actually checked, the number of records collected, and any pages it couldn't read.

How would you brief this task? If a page doesn't open, I'd want the gap recorded rather than the first few pages presented as a complete collection.

Report
Kelly
Team
· edited

Treat pages read, row counts, and gaps as part of the deliverable. If a page fails, log the gap — don’t present the first pages as a complete collection.

(Diagram: filter snapshot → page through → gap log → CSV + completeness report.)

(Local demo: agent session that can run scripts)

(Local demo: agent chat session)

Suggested steps

  1. Freeze the filters up front

Have Buda record site, filters, sort, and the access scope you allow — and echo them in the report.

  1. Page through with counters

Log page numbers opened, rows per page, and running totals.

  1. Register failures explicitly

Page number + short error; no silent skips; no “complete” label when gaps exist.

  1. Ship a pack

CSV plus a short report: filters, pages read, rows collected, gap list, completeness (complete / incomplete).

How to confirm

  • Pages read match the site total under those filters (or the report says total unknown and why).
  • Any gap forces an “incomplete” conclusion with the missing pages listed.
  • Spot-check: CSV rows ≈ sum of successful page counts.

Boundaries

  • Redesigns, expired logins, and captchas belong in the gap log.
  • Stay inside pages you’re allowed to open.
  • Large tables go to files; chat keeps the summary only.
Report

Your reply

Sign in to join the discussion. Reading is always open; you only need an account to post.

Sign in