XML Sitemaps for Small Sites: What to Include and Submit

Three requests for /sitemap.xml returned a 404 on this site during the 14.8 days of logs we studied. Our robots.txt points crawlers to sitemap-index.xml, the file name our generator writes, so whatever asked for the conventional name was guessing, and guessing wrong. The fix was one line in our build script. The lesson is wider than a file name: a sitemap that crawlers cannot find, or that lists URLs you would not want indexed, helps nobody.

What Google does with a sitemap

Google calls a sitemap “a simple hint”, not a command. It does not promise to crawl or index every URL you list, and it finds most pages through links anyway. On a small site where every page is linked from another page, a sitemap is not what makes crawling possible. It is what gives you a list to check the site against, and a place where Google reads your URLs and their modification dates.

The format is simple. A sitemap can be XML, an RSS or Atom feed, or a plain text list of URLs, and each file may hold up to 50,000 URLs or 50 MB uncompressed. URLs must be absolute and the file must be UTF-8. Of the optional fields, Google uses lastmod only when it is consistently accurate, and ignores priority and changefreq entirely, so do not spend time tuning those two.

A sitemap is also the weakest of the signals Google lists for choosing between duplicate URLs. Redirects and rel=canonical rank above it. List the canonical version of every page, and fix the duplicates elsewhere: canonical tags and duplicate URLs covers how.

A minimal valid file

This is the whole of what a small site needs:

<?xml version="1.0" encoding="UTF-8"?>
<urlset xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
  <url>
    <loc>https://example.com/</loc>
    <lastmod>2026-10-07</lastmod>
  </url>
  <url>
    <loc>https://example.com/guides/xml-sitemap-guide/</loc>
    <lastmod>2026-10-13</lastmod>
  </url>
</urlset>

Set lastmod to the date the content last changed in a way a reader would notice, not the date the file was generated. A generator that stamps every URL with today’s date makes the field useless, and Google only uses it when it proves reliable. If a URL contains an ampersand, write it as &, as in any XML file.

Put the file at the root of the site. The sitemap protocol limits a file to URLs at its own path or below, so /catalog/sitemap.xml cannot list pages under /news/, while a file at the root can list everything.

What goes in, what stays out

Our rule is short: only canonical, indexable pages that answer 200.

Once the file is clean, its length becomes a useful number. If your sitemap lists 40 pages and Search Console reports far fewer indexed, the gap is your to-do list. A bloated sitemap hides that signal.

The mistakes that waste a sitemap

The wrong protocol or host. A site served on https://example.com whose sitemap lists http:// or www URLs is listing redirects. Generators often build URLs from a configured site address, so check that setting first.

A trailing slash that disagrees with the site. If /guides/xml-sitemap-guide redirects to /guides/xml-sitemap-guide/, the sitemap must contain the second form, the one the server finally serves.

Redirected, deleted or noindex pages that stayed in. Pages get moved, and the sitemap is the last thing anyone updates. Generating it on every build, from the same source as the pages, prevents the whole class.

A lastmod that never changes, or always changes. Both teach Google to disregard it.

A sitemap nobody can find. It is not enough to upload the file. It has to be declared in robots.txt, submitted in Search Console, or sit where a crawler looks.

Our own case: a sitemap with the wrong name

Our generator writes sitemap-index.xml, which points to the child file sitemap-0.xml. Our robots.txt declares the index. Crawlers that read the Sitemap: line find it. The 3 requests for /sitemap.xml came from clients that skipped the line and tried the usual name, and they got a 404.

We fixed it with a build step that copies the index to /sitemap.xml, so both addresses answer with the same file. It is a small change, and we found it only because we read our logs. The AdSense checker follows the same order as a careful crawler: it uses the Sitemap: lines in robots.txt first, and only when there are none does it try /sitemap.xml and /sitemap_index.xml. A site with a differently named file and no declaration in robots.txt fails that check even though the sitemap exists.

Submit it, then check it

1. Add an absolute Sitemap: line to robots.txt: Sitemap: https://example.com/sitemap.xml. It works anywhere in the file, you can list several, and it does not belong to any User-agent group.
2. In Search Console, open the Sitemaps report, enter the path and submit it. A status of Success means Google fetched and parsed the file, which is not the same as indexing every URL in it.
3. Test the entries yourself. This loop requests every URL in the sitemap and prints the ones that do not answer 200, which means redirects, errors and anything else you should not have listed:

curl -s https://example.com/sitemap.xml | grep -o '<loc>[^<]*</loc>' | sed -e 's/<[^>]*>//g' -e 's/&amp;/\&/g' |
while read -r url; do
  printf '%s %s\n' "$(curl -s -o /dev/null -w '%{http_code}' "$url")" "$url"
done | grep -v '^200 '

No output means every entry answers 200. For a count of entries, use curl -s URL | grep -o '<loc>' | wc -l. The loop makes one request per URL, which is fine for a site under a few hundred pages; for a sitemap index, run it on each child file.

Then let the tools check the rest. The robots.txt tester flags a missing Sitemap: line and any declared sitemap that does not answer with a success status, and the SEO checker tells you whether the page you test is listed in your sitemap. For the file that points crawlers here, read robots.txt mistakes that block Google and AdSense.