'/%3E%3Cpath fill='%23ffffff' transform='translate(256 256) scale(0.6) translate(-256 -256)' d='M501.1 395.7L384 278.6c-23.1-23.1-57.6-27.6-85.4-13.9L192 158.1V96L64 0 0 64l96 128h62.1l106.6 106.6c-13.6 27.8-9.2 62.3 13.9 85.4l117.1 117.1c14.6 14.6 38.2 14.6 52.7 0l52.7-52.7c14.5-14.6 14.5-38.2 0-52.7zM331.7 225c28.3 0 54.9 11 74.9 31l19.4 19.4c15.8-6.9 30.8-16.5 43.8-29.5 37.1-37.1 49.7-89.3 37.9-136.7-2.2-9-13.5-12.1-20.1-5.5l-74.4 74.4-67.9-11.3L334 98.9l74.4-74.4c6.6-6.6 3.4-17.9-5.7-20.2-47.4-11.7-99.6.9-136.6 37.9-28.5 28.5-41.9 66.1-41.2 103.6l82.1 82.1c8.1-1.9 16.5-2.9 24.7-2.9zm-103.9 82l-56.7-56.7L18.7 402.8c-25 25-25 65.5 0 90.5s65.5 25 90.5 0l123.6-123.6c-7.6-19.9-9.9-41.6-5-62.7zM64 472c-13.2 0-24-10.8-24-24 0-13.3 10.7-24 24-24s24 10.7 24 24c0 13.2-10.7 24-24 24z'/%3E%3C/svg%3E){kind=small}
How to Extract All URLs & Links from a Website (5 Methods)
A list of a site's URLs is the raw material for most practical web work. SEO specialists extract links to audit internal linking and find orphan pages. Teams planning a site migration export every old URL to build a redirect map. Content researchers harvest a competitor's outbound references. QA engineers pull links to hunt for 404s. In each case the first step is identical: turn a page — or a whole site — into a clean, deduplicated list of URLs you can sort, filter, and export.
1. What Counts as a Link: Internal, External, Relative & Nofollow
Before extracting, it helps to know what you are collecting. On an HTML page, a link is an anchor element — <a href="..."> — and its href can take several forms that a good extractor normalizes for you.
Internal links stay on the same domain; external links leave it. The split matters for SEO: internal links route authority within your site.
/about is relative; https://site.com/about is absolute. Extractors resolve relatives against the page URL.
A rel="nofollow" hint tells search engines not to pass ranking signal. The URL is still a real link.
mailto:, tel: and #anchor hrefs are usually filtered out of a page-URL list.
2. Five Methods to Extract URLs, Compared
| Method | Scope | Best For |
|---|---|---|
| Online URL extractor | One page | Fast, no-setup page link lists |
| View-source / DevTools | One page | Manual spot-checks, debugging |
| sitemap.xml | Whole site | Owner-declared canonical URL list |
| Crawler | Whole site | Full discovery when no sitemap exists |
| Script (fetch + parse) | Custom | Automation and pipelines |
3. Extracting Links from a Single Page
For one page, the quickest route is a dedicated extractor. Paste the URL into the Website URL Extractor; it fetches the HTML, parses every anchor, resolves relative paths to absolute URLs, deduplicates, and splits the result into internal and external lists you can copy or export.
Prefer to see the raw markup? Open the page, press Ctrl/Cmd + U for view-source, or run this one-liner in the browser console to list every link on the rendered page:
// Paste into the browser DevTools console
[...document.querySelectorAll('a[href]')]
.map(a => a.href)
.filter((v, i, arr) => arr.indexOf(v) === i);View-source shows only the initial HTML. On sites that render links with JavaScript, the console snippet (which reads the live DOM) catches links that view-source misses.
4. Whole-Site Extraction: Sitemaps & Crawling
A single page only reveals the links it contains. To get every URL on a site, start with the sitemap — the owner's own declared list of indexable pages. Locate it with the Sitemap Finder, which checks common paths and robots.txt, then expands any nested sitemap index into the full URL set.
If no sitemap exists or it looks incomplete, a crawler is the fallback: it starts from the homepage, extracts that page's internal links, visits each, and repeats until the whole site is mapped. Crawling respects robots.txt and should be rate-limited so it does not overload the server. For images specifically, the Website Image Scraper applies the same idea to <img> sources.
5. What to Do with the List of URLs
A URL list is a means, not an end. The most common next steps:
Internal-link audit: find orphan pages and over-linked hubs to rebalance crawl equity.
Migration redirect map: pair every old URL with its new destination before a relaunch.
Broken-link check: request each URL and flag any 404 or 5xx responses.
Competitive research: map a site's structure and outbound references at a glance.
6. Frequently Asked Questions
What is the fastest way to extract all links from a web page?+
For a single page this is far quicker than manually scanning view-source. For an entire website with many pages, a sitemap or a crawler is the better tool because a single page only exposes the links it directly contains.
What is the difference between internal and external links?+
A good extractor labels each URL as internal or external automatically by comparing its host to the source page's domain, so you can filter the list for whichever analysis you are doing.
Can I get every URL of an entire website, not just one page?+
Start with the sitemap: it is the site owner's declared list of canonical URLs and is instant to read. Use the Sitemap Finder to locate it, then crawl only if the sitemap is missing or incomplete.
Is it legal to extract links from a website?+
Extracting your own site's links for an audit, or a handful of pages for research, is routine. Large-scale automated scraping of a third-party site can breach its terms of service, so scale and intent matter — keep requests modest and purpose-driven.
Extract Every Link from Any Page — Instantly
Paste a URL and get a clean, deduplicated list of every link on the page, split into internal and external and ready to export. Fast, free, and no signup.