A Python parser that saves open exchange and crypto rates into a table with the time of each reading

Example result

def product_links(list_url: str) -> Iterator[str]:
    """Product page links from every page of the list, following 'Next'."""
    url: str | None = list_url
    while url:
        page = fetch(url)
        for card in page.select("div.product-card a.title"):
            yield urljoin(url, card["href"])
        next_link = page.select_one("a.pagination-next")
        url = urljoin(url, next_link["href"]) if next_link else None


def fetch(url: str) -> BeautifulSoup:
    """One open page, at a polite pace: a pause between requests and a few tries on errors."""
    for attempt in range(3):
        time.sleep(settings.delay_seconds)
        response = session.get(url, timeout=30)
        if response.status_code == 200:
            return BeautifulSoup(response.text, "html.parser")
        log.warning("%s answered %s, try %d of 3", url, response.status_code, attempt + 1)
    raise ScraperError(f"{url} did not open after 3 tries")

About this service

Exchange offices, shops that price in several currencies and bookkeepers use this parser to save rates from open pages into a table: currency or coin, buy and sell rate, official rate, change for the day when shown, the source and the time of the reading. Send the pages you take rates from, such as a bank's public rates table, and one or two of them saved from your browser. You get Python code, a settings file and a README and run it on your computer when you need a reading. Rates are copied as the page shows them at that moment: it is not a live feed and gives no financial advice or forecasts. Where a source has an official open data feed, we say so. It reads open pages at a polite pace and never logs in or gets around captchas or protection. You decide whether the site's terms allow it; a later layout change needs an update.

What is included

• What you getA ZIP archive with the Python code, a settings file and a README with step-by-step instructions; the code is also shown on the order page.
• Buy and sell apartSeparate columns, as numbers
• Time of each readingDate and time in every row
• Source in a columnSo rates from different pages stay apart
• Polite paceNot every second: a pause you set
• Runs on your computerWindows, macOS or Linux, with Python

How it works

1. 1
   SourcesYou give the rate pages and the currencies or coins
2. 2
   ParserWe write the code and the guide to running it
3. 3
   RunYou run it and compare the first reading with the site
4. 4
   ChangesYou ask for changes within your package and send any error message

What we need from you

The pages you take rates from, with links, and the currencies or coins you need. The fields in the order of your table, e.g. currency, buy, sell, official rate, change, source, time. One or two of those pages saved from your browser (File > Save page as) or screenshots. How often you need a reading, the number format (dot or comma), and CSV or Excel.

We will ask you:

• Which pages?
• Currencies or coins
• The fields you need
• How often will you run it?

Compare packages

Questions and answers

Did not find your answer? Ask us