https://webscrapingapi.dev/blog Practical notes on getting data out of web pages: site endpoints, caching, scraping trade-offs. en Tue, 22 Sep 2026 00:00:00 GMT https://webscrapingapi.dev/blog/convert-any-web-page-to-markdown-for-llm https://webscrapingapi.dev/blog/convert-any-web-page-to-markdown-for-llm Tue, 22 Sep 2026 00:00:00 GMT Add format=markdown to /v1/scrape: headings, lists, tables, code and absolute links come back, scripts and forms do not. Made for prompts and RAG chunks. generic scrape rag & agents https://webscrapingapi.dev/blog/hacker-news-front-page-and-comments-as-json https://webscrapingapi.dev/blog/hacker-news-front-page-and-comments-as-json Tue, 22 Sep 2026 00:00:00 GMT Front page stories in rank order with points and comment counts, or one thread with its comments flattened and a depth on each. Two GET requests. site endpoints media & communities https://webscrapingapi.dev/blog/css-selectors-to-json-extract-parameter https://webscrapingapi.dev/blog/css-selectors-to-json-extract-parameter Tue, 22 Sep 2026 00:00:00 GMT Name fields as CSS selectors and /v1/scrape returns them as JSON: attributes, lists, inner HTML, absolute links, up to 50 fields, no parsing code. generic scrape getting started https://webscrapingapi.dev/blog/github-repo-stars-forks-topics-without-token https://webscrapingapi.dev/blog/github-repo-stars-forks-topics-without-token Tue, 22 Sep 2026 00:00:00 GMT GitHub's REST API allows 60 anonymous requests an hour. This endpoint reads the public repo page: stars, forks, watchers, topics, license, releases. site endpoints caching & credits https://webscrapingapi.dev/blog/scraping-api-error-codes-explained https://webscrapingapi.dev/blog/scraping-api-error-codes-explained Tue, 22 Sep 2026 00:00:00 GMT Every error is a status plus a short code and a sentence. The full list: the cause of each one, whether a retry helps, and which ones refund the credit. reliability how it works caching & credits https://webscrapingapi.dev/blog/read-open-graph-json-ld-metadata-from-any-url https://webscrapingapi.dev/blog/read-open-graph-json-ld-metadata-from-any-url Tue, 22 Sep 2026 00:00:00 GMT /v1/metadata returns title, description, canonical, favicon, language, Open Graph and Twitter tags and every JSON-LD block from the head alone. One credit. generic scrape getting started https://webscrapingapi.dev/blog/wikipedia-summary-and-infobox-api-any-language https://webscrapingapi.dev/blog/wikipedia-summary-and-infobox-api-any-language Tue, 22 Sep 2026 00:00:00 GMT Lead paragraphs as plain text, the thumbnail and the infobox as key/value pairs, in any language edition, from one request. Stored 7 days. site endpoints rag & agents https://webscrapingapi.dev/blog/amazon-product-data-json-without-product-advertising-api https://webscrapingapi.dev/blog/amazon-product-data-json-without-product-advertising-api Mon, 21 Sep 2026 00:00:00 GMT Turn an ASIN into JSON with one GET: title, price, availability, rating, images, bullets and the spec table. No Amazon developer account or affiliate approval. site endpoints shopping & apps reliability https://webscrapingapi.dev/blog/app-store-google-play-app-data-json https://webscrapingapi.dev/blog/app-store-google-play-app-data-json Mon, 21 Sep 2026 00:00:00 GMT Two endpoints, one shape: an iOS app by id or an Android app by package name, returning name, developer, rating, price, category, screenshots and installs. site endpoints shopping & apps https://webscrapingapi.dev/blog/what-max-age-means-in-a-scraping-api https://webscrapingapi.dev/blog/what-max-age-means-in-a-scraping-api Mon, 21 Sep 2026 00:00:00 GMT Site endpoints store answers and share them. max_age sets how old a stored copy may be; every response says whether it was live or stored, and how old it is. caching & credits how it works https://webscrapingapi.dev/blog/web-scraping-api-vs-writing-your-own-scraper https://webscrapingapi.dev/blog/web-scraping-api-vs-writing-your-own-scraper Mon, 21 Sep 2026 00:00:00 GMT Fetching pages yourself versus calling a scraping API: headers, JavaScript rendering, blocked sites, parsing, cost, and the cases where each one wins. getting started how it works rag & agents https://webscrapingapi.dev/blog/youtube-video-metadata-without-api-key https://webscrapingapi.dev/blog/youtube-video-metadata-without-api-key Mon, 21 Sep 2026 00:00:00 GMT Title, channel, views, likes, length, publish date, thumbnails and keywords for any public video, from one request. And when the official Data API is better. site endpoints media & communities