{"slug":"sitemap-analyzer","title":"Sitemap Analyzer — Page Inventory and Site Architecture Mapping","tags":["seo","technical-seo","sitemap","site-architecture","indexing"],"agent_summary":"Extracts root domain, crawls sitemap index, catalogs every URL with title and purpose, identifies site structure patterns, and surfaces architecture gaps for SEO planning.","trigger_phrases":["sitemap analysis","site structure","map the site","page inventory","URL inventory","site architecture audit","sitemap audit","site mapping"],"runnable":true,"markdown":"\n## Purpose\n\nProduce a complete, structured inventory of every page on a site from its sitemap. Used before content planning, internal linking strategy, or SEO audits to understand what the site actually contains vs. what it should contain.\n\n## Step 1: Extract Domain and Sitemap Location\n\nFrom any URL on the target site:\n1. Identify root domain: `domain.com`\n2. Check standard sitemap locations in order:\n - `domain.com/sitemap.xml`\n - `domain.com/sitemap_index.xml`\n - `domain.com/sitemaps/sitemap.xml`\n - Check robots.txt for `Sitemap:` directive if standard locations fail\n\n## Step 2: Crawl the Sitemap\n\nIf the primary sitemap is a sitemap index (a sitemap of sitemaps):\n1. Parse all child sitemap URLs\n2. Crawl each child sitemap\n3. Extract all URLs across the entire sitemap index\n\nIf a single XML sitemap: extract all `` entries directly.\n\nFor each URL record:\n- Full URL\n- `` date if present\n- `` if present\n- `` if present\n\n## Step 3: Build Page Inventory Table\n\nFor each URL, determine:\n- **Page title** (from `` tag or sitemap if included)\n- **Page type** (homepage, service page, location page, blog article, category, product, about, contact, legal, etc.)\n- **Purpose** (one sentence describing what the page does for the business)\n\nOutput as a structured table:\n\n| URL | Page Type | Title | Purpose |\n|-----|-----------|-------|---------|\n| domain.com/ | Homepage | [title] | [purpose] |\n| domain.com/services/roofing/ | Service | [title] | [purpose] |\n\n## Step 4: Site Structure Summary\n\nAfter the inventory, provide:\n\n**Total pages by type:**\n- Homepage: 1\n- Service pages: X\n- Location pages: X\n- Blog articles: X\n- Other: X\n\n**Content siloing assessment:**\n- Are service pages grouped under /services/ or scattered?\n- Do location pages have their own section?\n- Is there a clear pillar → supporting page hierarchy?\n- Are there orphaned sections with no path from homepage?\n\n**Content gap indicators:**\n- Services mentioned on homepage but no dedicated service page\n- Cities/locations served but no location page\n- Blog posts with no internal links to/from service pages\n\n## Step 5: Architecture Recommendations\n\nBased on inventory:\n\n**Silo violations:** Pages that belong in a silo but are in the wrong location (e.g., location pages nested under blog/)\n\n**Missing pages:** Services or locations with no dedicated page\n\n**Depth violations:** Money pages more than 3 clicks from homepage\n\n**URL structure issues:** Inconsistent slugs, unnecessary parameters in URLs, overly deep nesting (4+ levels)\n\n## Using Sitemap Analysis with LibreCrawl\n\nFor large sites or when sitemap is incomplete, supplement with LibreCrawl (`D:/ClaudeDev/00_GITHUB/LibreCrawl/`):\n- Run `python main.py --local` to start crawler at `http://localhost:5000`\n- LibreCrawl finds pages that may not be in the sitemap (discovered via internal links)\n- Cross-reference LibreCrawl output with sitemap output → delta reveals unsubmitted or orphaned pages\n\n## Cross-References\n\n- `crawl-budget-optimizer` — sitemap accuracy is a direct crawl budget input\n- `internal-linking-optimizer` — link depth issues discovered in sitemap analysis\n- `local-seo-silo-generator` — rebuilding silo structure after architecture audit\n- `duplicate-content-risk-scanner` — checking URL patterns for duplication signals\n\n#seo-sop #seo #technical-seo #sitemap #site-architecture #indexing\n","html":"<h2>Purpose</h2>\n<p>Produce a complete, structured inventory of every page on a site from its sitemap. Used before content planning, internal linking strategy, or SEO audits to understand what the site actually contains vs. what it should contain.</p>\n<h2>Step 1: Extract Domain and Sitemap Location</h2>\n<p>From any URL on the target site:</p>\n<ol>\n<li>Identify root domain: <code>domain.com</code></li>\n<li>Check standard sitemap locations in order:\n<ul>\n<li><code>domain.com/sitemap.xml</code></li>\n<li><code>domain.com/sitemap_index.xml</code></li>\n<li><code>domain.com/sitemaps/sitemap.xml</code></li>\n<li>Check robots.txt for <code>Sitemap:</code> directive if standard locations fail</li>\n</ul>\n</li>\n</ol>\n<h2>Step 2: Crawl the Sitemap</h2>\n<p>If the primary sitemap is a sitemap index (a sitemap of sitemaps):</p>\n<ol>\n<li>Parse all child sitemap URLs</li>\n<li>Crawl each child sitemap</li>\n<li>Extract all URLs across the entire sitemap index</li>\n</ol>\n<p>If a single XML sitemap: extract all <code><loc></code> entries directly.</p>\n<p>For each URL record:</p>\n<ul>\n<li>Full URL</li>\n<li><code><lastmod></code> date if present</li>\n<li><code><priority></code> if present</li>\n<li><code><changefreq></code> if present</li>\n</ul>\n<h2>Step 3: Build Page Inventory Table</h2>\n<p>For each URL, determine:</p>\n<ul>\n<li><strong>Page title</strong> (from <code><title></code> tag or sitemap if included)</li>\n<li><strong>Page type</strong> (homepage, service page, location page, blog article, category, product, about, contact, legal, etc.)</li>\n<li><strong>Purpose</strong> (one sentence describing what the page does for the business)</li>\n</ul>\n<p>Output as a structured table:</p>\n<p>| URL | Page Type | Title | Purpose |\n|-----|-----------|-------|---------|\n| domain.com/ | Homepage | [title] | [purpose] |\n| domain.com/services/roofing/ | Service | [title] | [purpose] |</p>\n<h2>Step 4: Site Structure Summary</h2>\n<p>After the inventory, provide:</p>\n<p><strong>Total pages by type:</strong></p>\n<ul>\n<li>Homepage: 1</li>\n<li>Service pages: X</li>\n<li>Location pages: X</li>\n<li>Blog articles: X</li>\n<li>Other: X</li>\n</ul>\n<p><strong>Content siloing assessment:</strong></p>\n<ul>\n<li>Are service pages grouped under /services/ or scattered?</li>\n<li>Do location pages have their own section?</li>\n<li>Is there a clear pillar → supporting page hierarchy?</li>\n<li>Are there orphaned sections with no path from homepage?</li>\n</ul>\n<p><strong>Content gap indicators:</strong></p>\n<ul>\n<li>Services mentioned on homepage but no dedicated service page</li>\n<li>Cities/locations served but no location page</li>\n<li>Blog posts with no internal links to/from service pages</li>\n</ul>\n<h2>Step 5: Architecture Recommendations</h2>\n<p>Based on inventory:</p>\n<p><strong>Silo violations:</strong> Pages that belong in a silo but are in the wrong location (e.g., location pages nested under blog/)</p>\n<p><strong>Missing pages:</strong> Services or locations with no dedicated page</p>\n<p><strong>Depth violations:</strong> Money pages more than 3 clicks from homepage</p>\n<p><strong>URL structure issues:</strong> Inconsistent slugs, unnecessary parameters in URLs, overly deep nesting (4+ levels)</p>\n<h2>Using Sitemap Analysis with LibreCrawl</h2>\n<p>For large sites or when sitemap is incomplete, supplement with LibreCrawl (<code>D:/ClaudeDev/00_GITHUB/LibreCrawl/</code>):</p>\n<ul>\n<li>Run <code>python main.py --local</code> to start crawler at <code>http://localhost:5000</code></li>\n<li>LibreCrawl finds pages that may not be in the sitemap (discovered via internal links)</li>\n<li>Cross-reference LibreCrawl output with sitemap output → delta reveals unsubmitted or orphaned pages</li>\n</ul>\n<h2>Cross-References</h2>\n<ul>\n<li><code>crawl-budget-optimizer</code> — sitemap accuracy is a direct crawl budget input</li>\n<li><code>internal-linking-optimizer</code> — link depth issues discovered in sitemap analysis</li>\n<li><code>local-seo-silo-generator</code> — rebuilding silo structure after architecture audit</li>\n<li><code>duplicate-content-risk-scanner</code> — checking URL patterns for duplication signals</li>\n</ul>\n<p>#seo-sop #seo #technical-seo #sitemap #site-architecture #indexing</p>\n"}