← All articles

Topic

rag & agents

  1. Convert any web page to Markdown for an LLM with one request (format=markdown)

    Add format=markdown to /v1/scrape: headings, lists, tables, code and absolute links come back, scripts and forms do not. Made for prompts and RAG chunks.

    · 2 min read

  2. Wikipedia summary and infobox as JSON, in any language edition, for RAG and agents

    Lead paragraphs as plain text, the thumbnail and the infobox as key/value pairs, in any language edition, from one request. Stored 7 days.

    · 2 min read

  3. Web scraping API vs writing your own scraper: which one for a side project, an agent, or a data pipeline

    Fetching pages yourself versus calling a scraping API: headers, JavaScript rendering, blocked sites, parsing, cost, and the cases where each one wins.

    · 3 min read