Website URL Extractor
Crawl public website pages and sitemaps, classify internal and external links, and export a deduplicated URL inventory.
Rate Website URL Extractor
FeedbackSave a personal rating in this browser.
How to use Website URL Extractor
- 1Input Webpage Target URL: Paste the full HTTP or HTTPS address of any webpage into the input field.
- 2Start the Website Crawl: Click 'Extract Website URLs'. The crawler follows same-host links, pagination and nested XML sitemaps. Keep the tab open while it works.
- 3Filter Internal vs. Outbound Links: Toggle between all links, internal site navigation, or external citations, and use the real-time search box to filter by keyword or directory.
- 4Export to CSV: Export all or filtered URLs to CSV, including the first source page. Review skipped and failed resources, before treating the inventory as complete.
Key Features & Highlights
- •Recursive Page and Sitemap Discovery: Follows discovered public pages without a fixed page-count limit. Parses standard HTML5 <a href> destinations, relative paths, and query string parameters.
- •Automated Domain Categorization: Segregates internal navigation routes from external outbound citations based on the target hostname.
- •Anchor Text Mapping: Correlates each discovered hyperlink with its visible clickable anchor text to audit keyword relevance and spot generic 'click here' labels.
- •Real-Time Substring Filtering: Instantly filter hundreds of URLs by path (e.g. /blog/, /products/), keyword, or security scheme (http vs https).
- •Spreadsheet-Ready CSV Export: Download clean, structured CSV files compatible with Microsoft Excel, Google Sheets, and technical SEO audit tools.
- •Pause and Resume: Crawl progress and extracted results are saved in this browser. Fetched HTML is processed in server memory rather than stored in an application database.
To inventory a section before a migration, crawl its public URLs, filter the table to that directory and export the results. Keep the source page column for link updates. Cross-check skipped resources and JavaScript-only links before using the CSV as a complete inventory.
Extract URLs across a website and export its links
This free website link extractor starts from your submitted URL and the site homepage, follows internal links and reads XML sitemaps. It keeps working until its discovered queue is empty, with no fixed limit on the total pages. Filter unique URLs or export CSV while crawling. Login-only pages, bot protection, JavaScript-only navigation and unlinked pages absent from sitemaps can leave gaps; review the crawl coverage note for failed or skipped resources.
How does the website crawl work?
Paste a public website URL. Each fetched page adds new internal links to a queue. The crawler also checks robots.txt sitemap declarations and common sitemap paths, then expands nested sitemap indexes. External destinations are listed but their pages are not crawled.
To inspect sitemap files separately, use the Sitemap Finder. A sitemap may omit pages, and a crawler can still miss unlinked or restricted URLs. Compare the methods in our guide to extracting website URLs. To check destination statuses and find the pages that need edits, use the Broken Link Checker.
Webpage Hyperlink Extractor & DOM Link Auditor
The Website URL Extractor reads the HTML returned by public pages and extracts link destinations and anchor text. JavaScript that adds links after the page loads can leave gaps in the report.
Whether you need to audit internal link equity across a key landing page, verify partner citations, check affiliate link attributes, or gather research references for content curation, BrivTools provides a browser-accessible solution without software installations or paywalls.
The BrivTools Website URL Extractor is a recursive public website crawler. It resolves relative HTML links against each page URL or its base element, follows same-host pages and reads XML sitemaps. It deduplicates destination URLs, records the first source found, and supports filtering, CSV export, pause, resume and resource reporting.
How to Extract Links from a Webpage
Extracting link inventories from individual webpages is simple and requires no command-line tools or scraping scripts:
Step 1: Input Webpage Target URL
Enter the complete destination web address (including https://) into the input box. The serverless engine retrieves the HTML markup for analysis.
Step 2: Parse DOM Anchor Tags
Click "Extract Website URLs". The parser scans the HTML markup, identifies active <a href> tags, resolves relative paths into absolute URLs, and associates visible anchor text.
Step 3: Filter Internal vs. External Links
Use the tab controls to separate internal site links from external citations. The live search box allows filtering by keyword, path slug (such as /blog/), or protocol.
Step 4: Export to CSV or Clipboard
Click "Export CSV" to download the extracted links in spreadsheet format with columns for anchor text, URL, and link type, ready for auditing in Excel or Google Sheets.
HTML Link Crawling and Sitemap Discovery
HTML links and XML sitemaps provide complementary discovery paths. This tool combines both; you can inspect sitemap files separately when you need their declared URL inventory:
Recursive HTML Discovery (Website URL Extractor)
Starts from your submitted page and the homepage, then visits newly discovered internal links. URLs on the bare hostname and its www variant count as internal; other subdomains are outside the crawl. Pagination query parameters are preserved, while fragments and common tracking parameters are removed.
Domain-Wide URL Discovery (Sitemap Finder & XML Sitemaps)
To discover all indexed URLs across an entire root domain, use our Sitemap Finder to inspect robots.txt and XML sitemaps, which can catalog thousands of published pages across a domain.
Architectural Comparison: Link Extraction Approaches
Different extraction methodologies provide varying levels of depth and speed:
| Evaluation Criteria | BrivTools URL Extractor | Raw Regex Extractors | Recursive Site Crawlers |
|---|---|---|---|
| Analysis Scope | Public pages plus XML sitemaps | Raw Text Strings | Entire Domain Architecture |
| Anchor Text Mapping | First discovered anchor text (up to 1,000 characters) | None (URLs Only) | Full Link Architecture |
| Internal vs External Separation | Automatic Hostname Classification | Manual User Sorting | Configurable Rule Sets |
| Execution Speed | Depends on website size; minutes to hours | Instant (Local Text) | Minutes to Hours |
| Data Export Format | CSV with the first source page | Plain Text Dump | Comprehensive Database / CSV |
Practical Use Cases for Web Auditing
🔗 Internal Navigation & Link Auditing
Audit how links are distributed across critical landing pages. Extract internal links from key hubs to verify that important subpages receive appropriate navigational visibility.
🚨 Outbound Citation & Affiliate Review
Export external links to verify outbound references, ensure partner tracking parameters are properly formatted, and confirm appropriate link relationships.
🎯 Anchor Text Relevance Review
Review anchor text pairings to spot unhelpful or generic labels (such as "click here" or "read more") and replace them with descriptive, user-friendly copy.
📋 Status Code Verification via CSV
Download the CSV export and run URLs through status-checking tools to detect 404 broken links, unexpected redirect chains, or protocol inconsistencies.
Related SEO & Discovery Utilities
Connect your workflow with complementary BrivTools utilities:
- Sitemap Finder — Discover XML sitemaps and robots.txt files across entire domains.
- Sitemap Generator — Generate standard sitemaps.org XML for discovered pages.
- Website Image Scraper — Crawl public website pages and discover image URLs, responsive variants and alt text.
- How to Extract All URLs & Links Guide — In-depth guide detailing browser DevTools, sitemaps, and automated scraping workflows.
Processing and privacy
This tool uses our server to fetch public URLs. Access restrictions and request limits can affect results.
Frequently Asked Questions
Common questions about Website URL Extractor usage, file handling, security, and limits
Can I extract all links from a website with this URL extractor?+
It recursively follows discovered internal HTML links, pagination and nested XML sitemaps. There is no fixed total page-count limit. It cannot guarantee every URL: JavaScript-only navigation, bot protection, login requirements and orphan pages absent from sitemaps can leave gaps.
What is the difference between an internal link and an external link?+
For this crawler, internal links use the same hostname, including its www variant (for example, navigating from the homepage to a blog post). Other subdomains count as external and are not crawled. They define website navigation and distribute search crawler visibility. External links (outbound links) point toward third-party domains to cite sources or provide external resources. This tool automatically classifies each discovered link based on the target hostname.
How do I extract links from a single webpage?+
Paste the target URL and click 'Extract Website URLs'. Results appear as pages are read. The tool then follows internal links and sitemaps across that hostname. Pause the crawl if you want to inspect the current results before continuing.
Can this tool extract every URL across an entire website domain?+
It crawls discoverable public pages on the same hostname, including www, and reads sitemap indexes recursively. It respects robots.txt, reports skipped or failed resources. Queue completion means all discovered resources were attempted, not that hidden or restricted URLs were found.
How do I export the extracted links to Excel or Google Sheets?+
After the extraction table renders, click 'Export CSV'. The resulting file can be opened directly in Microsoft Excel, Apple Numbers, or Google Sheets with dedicated columns for the destination URL, visible anchor text, and link type (internal vs external).
Can this tool extract links generated dynamically by client-side JavaScript?+
The extractor parses the initial HTML response of each crawled page. Standard HTML pages, CMS platforms (WordPress, Shopify, Webflow), and server-rendered applications (Next.js SSR) are extracted accurately. For single-page applications that construct links exclusively via post-load client-side JavaScript, only links present in the initial server response are captured.
Are submitted URLs or extracted links stored on your servers?+
Crawl progress and extracted URLs are saved locally in this browser so you can resume after reopening the tool. Our server fetches public pages and parses them in memory; fetched HTML and results are not saved to an application database. Browser storage availability and capacity vary.
Learn the workflow
Extract URLs from one page or a whole websiteRelated Free Utilities
Tools for related tasks
Website Broken Link Checker
Find dead links and redirects, see where they appear on your website, and export a CSV report.
LLMs.txt Generator
Create an llms.txt link index from a public page or sitemap, then edit and download the Markdown.
Sitemap Generator
Crawl any website and generate a ready-to-submit XML sitemap.
Website Image Scraper
Find images across public website pages, export image and source URLs to CSV, or download the files as a ZIP.