A Python parser for open tender and procurement notices: buyer, subject, price and deadline in a table

Example result

def product_links(list_url: str) -> Iterator[str]:
    """Product page links from every page of the list, following 'Next'."""
    url: str | None = list_url
    while url:
        page = fetch(url)
        for card in page.select("div.product-card a.title"):
            yield urljoin(url, card["href"])
        next_link = page.select_one("a.pagination-next")
        url = urljoin(url, next_link["href"]) if next_link else None


def fetch(url: str) -> BeautifulSoup:
    """One open page, at a polite pace: a pause between requests and a few tries on errors."""
    for attempt in range(3):
        time.sleep(settings.delay_seconds)
        response = session.get(url, timeout=30)
        if response.status_code == 200:
            return BeautifulSoup(response.text, "html.parser")
        log.warning("%s answered %s, try %d of 3", url, response.status_code, attempt + 1)
    raise ScraperError(f"{url} did not open after 3 tries")

About this service

Suppliers who bid for contracts use this parser to see open notices in one table: notice number, subject, buyer organisation, region, procedure type, starting price and currency, publication date, deadline, status and link. Run a search on the procurement portal with your keywords or codes, then send that link, a notice page and one or two pages saved from your browser. You get Python code, a settings file and a README and run it on your computer each morning or as you need. Contact persons are left out: no names, emails or phone numbers. It does not log in, download papers behind an account, get around captchas or protection, or submit bids, and it keeps a polite pace. You decide whether the portal's terms allow it; a later change of its layout needs an update.

What is included

• What you getA ZIP archive with the Python code, a settings file and a README with step-by-step instructions; the code is also shown on the order page.
• Deadlines you can sortDates in one format, ready to sort by deadline
• Starting price as a numberWith the currency in its own column
• Your keywordsStarts from the search you set up on the portal
• No contact personsThe buyer organisation only, no names or phones
• Polite paceA pause between pages, set in the settings

How it works

1. 1
   SearchYou set up the search and send its link and a saved notice
2. 2
   ParserWe write the code and the guide to running it
3. 3
   RunYou run it and check today's notices against the portal
4. 4
   ChangesYou ask for changes within your package and send any error message

What we need from you

The procurement portal and a link to a search with your keywords, codes or regions, one notice page and roughly how many notices the search shows. The fields in the order of your table, e.g. number, subject, buyer, region, starting price, currency, published, deadline, link. One or two of those pages saved from your browser (File > Save page as) or screenshots. CSV or Excel.

We will ask you:

• Which portal and search?
• Keywords, codes or regions
• The fields you need
• How often will you run it?

Compare packages

Questions and answers

Did not find your answer? Ask us