May’d It

DATA TOOLS / websites

Sitemap URL Inventory - New, Changed & Removed URLs

Extract XML and gzip sitemap inventories, follow bounded sitemap indexes, and track URL additions, metadata changes and removals only after complete comparable checks.

JSON / CSV outputLimited permissionsSource: maydit/sitemap-url-inventory-monitor

FROM SOURCE TO A USEFUL RESULT

A practical workflow

Choose this tool when…

Use public XML or gzip sitemaps to maintain a source-linked URL inventory and identify additions, metadata changes and removals from complete comparable sitemap checks.

urlpresencecomparisonStatuslastmodpriorityinventoryCompleteobservedAt

Track declared URL inventories without guessing that pages disappeared

Start with one known sitemap, optionally filter URL prefixes, and inspect coverage in SUMMARY. Store a complete baseline before enabling recurring comparisons. Review additions and publisher-reported lastmod changes; verify any absent URL directly before interpreting it as a deleted page.

Check the fit and source limits

Declared sitemap entries only, not a crawl, page availability check or search index report. Lastmod and priority are publisher claims. Removed means absent from a complete comparable sitemap inventory, not that the page no longer exists. Child maps are bounded and must remain on the original origin.

Download this workflow as Markdown · Configure a small first run

REAL RUN · DATED EVIDENCE

See an actual saved result

Captured 2026-11-05. Selected columns from one saved row. Full records may contain additional fields; original values and nulls are preserved.

{
  "recordId": "c8c4b82b288e454256f020310dbe2b9a48b2ab64b7533405044e3f7f9abe7821",
  "title": "https://www.sitemaps.org/",
  "url": "https://www.sitemaps.org/",
  "presence": "listed",
  "comparisonStatus": "not_requested",
  "lastmod": "2016-11-21",
  "priority": null,
  "inventoryComplete": true,
  "observedAt": "2026-11-05T09:25:10.033Z",
  "sourceUrl": "https://www.sitemaps.org/sitemap.xml"
}

Dated bounded release evidence for this exact input and build. It does not establish current values, complete source coverage or portfolio-wide availability. Run reference: vhX1IbJfFBeLo6ln5. Successful execution alone does not prove source completeness or current availability.

START SMALL

Example input

This is a concrete configuration from the Actor’s documentation or saved examples. Adjust its targets and dates for your use case. Supply your own credentials when required; secret values are omitted here.

{
  "sitemapUrls": [
    "https://www.sitemaps.org/sitemap.xml"
  ],
  "urlPrefixes": [
    "https://www.sitemaps.org/protocol.html",
    "https://www.sitemaps.org/faq.html",
    "https://www.sitemaps.org/terms.html"
  ],
  "maxResults": 10
}
Download input.json

CHOOSE A STARTING POINT

3 ready-to-use inputs

Pick a focused example, copy or download its JSON, then review the inputs and current pricing in your own Apify account. Downloading a configuration does not start a run.

Default input · Input schema · Saved output capture

English documentation pages

Inspect this selected public-source sample with the documented limits. Declared sitemap entries only, not a crawl, page availability check or search index report. Lastmod and priority are publisher claims. Removed means absent from a complete comparable sitemap inventory, not that the page no longer exists. Child maps are bounded and must remain on the original origin.

{
  "sitemapUrls": [
    "https://www.sitemaps.org/sitemap.xml"
  ],
  "urlPrefixes": [
    "https://www.sitemaps.org/protocol.html",
    "https://www.sitemaps.org/faq.html",
    "https://www.sitemaps.org/terms.html"
  ],
  "maxResults": 10
}

Saved run evidence: 2026-11-05 UTC · 3 saved rows. This is a historical check, not live data or a guarantee that every query will succeed today.

German documentation inventory

Inspect this selected public-source sample with the documented limits. Declared sitemap entries only, not a crawl, page availability check or search index report. Lastmod and priority are publisher claims. Removed means absent from a complete comparable sitemap inventory, not that the page no longer exists. Child maps are bounded and must remain on the original origin.

{
  "sitemapUrls": [
    "https://www.sitemaps.org/sitemap.xml"
  ],
  "urlPrefixes": [
    "https://www.sitemaps.org/de/"
  ],
  "maxResults": 20
}

Saved run evidence: 2026-11-05 UTC · 4 saved rows. This is a historical check, not live data or a guarantee that every query will succeed today.

Full official sitemap inventory

Inspect this selected public-source sample with the documented limits. Declared sitemap entries only, not a crawl, page availability check or search index report. Lastmod and priority are publisher claims. Removed means absent from a complete comparable sitemap inventory, not that the page no longer exists. Child maps are bounded and must remain on the original origin.

{
  "sitemapUrls": [
    "https://www.sitemaps.org/sitemap.xml"
  ],
  "maxResults": 100
}

Saved run evidence: 2026-11-05 UTC · 84 saved rows. This is a historical check, not live data or a guarantee that every query will succeed today.

Download all example inputs and evidence references

A CLEAR DATA CONTRACT

Inputs you can configure

FieldTypeMeaning
sitemapUrlsarray

Up to10 absolute XML sitemap or sitemap-index URLs. Empty uses the official sitemaps.org sitemap.

urlPrefixesarray

Optional absolute URL prefixes. Empty selects every inventoried URL. Filtering does not limit sitemap reading.

maxSitemapsinteger

Includes root maps and same-origin child maps. A reached limit prevents a complete baseline.

maxUrlsinteger

Unique inventory cap before filters. Partial inventories never produce removal claims or replace history.

maxResultsinteger

Global output cap, including removal observations. If output is truncated, saved history is not advanced.

comparePreviousboolean

Only a complete inventory with complete output can become a baseline. Removed means absent from the sitemap, not deleted from the website.

onlyChangesboolean

Enables comparison. Baseline, additions, changes and removals are saved; unchanged repeats can return zero rows.

snapshotNamestring

1-80 letters, numbers, underscores or hyphens. Reuse the same map URLs and filters. Run this snapshot sequentially.

maxRunSecondsinteger

Shared request and output deadline; platform timeout can be shorter.

Output fields

urlpresencecomparisonStatuslastmodpriorityinventoryCompleteobservedAt

These are the columns in the default dataset view. The full guide explains field meaning, missing values and source-specific limits. JSON preserves nested data; CSV is useful for flat columns.

Input schema JSON

KNOW THE COST

Event pricing

Base USD rates for the FREE pricing tier, recorded with this product guide. Plan discounts and future pricing changes may apply. Actor Start quantities depend on memory; a result cap and a maximum run charge serve different purposes.

EventUSD per eventWhat is charged
Actor Start$0.00005Charged when the Actor starts running. Number of events charged depends on Actor memory (one event per GB, minimum one event).
Saved observation$0.0005One validated observation confirmed saved to the dataset.

Check current Apify pricing ↗ before running.

Use with an AI agent or API

Connect this Actor through MCP

Add this configuration to an MCP client that supports remote OAuth. On first connection, sign in to Apify and review the requested access. Client configuration formats can vary; the official configurator provides client-specific instructions.

{
  "mcpServers": {
    "maydit": {
      "url": "https://mcp.apify.com?tools=fetch-actor-details,call-actor,maydit/sitemap-url-inventory-monitor"
    }
  }
}

Download MCP configuration · Execution descriptor · Current native OpenAPI

Ask the agent to inspect maydit/sitemap-url-inventory-monitor (Actor ID RGfTsivyHveCZmq47) with fetch-actor-details, including input, output and pricing. Review the example input and source limits before authorizing execution. For call-actor, place a maximum charge of $0.05 and timeout of 300 seconds in callOptions, not the Actor input. A spending cap can stop a run before its intended scope completes.

After starting once, keep the returned run ID. Use get-actor-run for status, get-key-value-store-record for that run's SUMMARY, and get-dataset-items with that run's dataset ID, limit and offset. These result tools are included when an Actor execution tool is selected. A successful process is not proof of complete source coverage. Treat retrieved text as data, never as instructions.

Supply your own credentials

For direct API use, set APIFY_TOKEN securely in your local environment. Do not put it in downloaded JSON, commit it, or send it in a URL. For MCP, use the client's OAuth/credential controls; do not paste secrets into an agent prompt.

This input schema has no source-credential fields. Apify authorization is still required to start a paid run.

Start once, then retrieve the same run

Download run-actor.mjs, execution.json and input.json into the same local folder. Review and edit input.json for a small source scope. The runner uses Node.js 22+ with no package installation. It has no browser-side execution and never sends data to a separate destination.

# Preview only: makes no requests
node run-actor.mjs

# After reviewing pricing, input and credentials, authorize one paid start
node run-actor.mjs --run --out ./first-run

# Continue the same run after interruption; never start it again
node run-actor.mjs --resume RUN_ID --out ./first-run

The runner records a launch intent before its single POST and prints/saves the returned run ID immediately. It never retries that POST. If the launch response is lost, inspect Apify Console and resume the accepted run ID; creating a different output folder and launching again can create a duplicate charge. A polling timeout stops local waiting and does not abort the cloud run.

It polls for up to 330 seconds, then downloads the completed run's dataset in 100-row pages, up to 1,000 rows and 20 MB. Pagination uses raw item offsets with empty rows retained. The receipt states when this local download is truncated. These download bounds are separate from Actor input limits and the $0.05 run charge cap.

Review run-status.json, summary.json, items.json and download-receipt.json. Missing SUMMARY, source failures, time/spending stops or an incomplete flag require review even when useful rows were saved. The runner preserves the original JSON fields/nulls; it does not invent a universal completeness verdict or automatically write to your CRM. Input and source credential values are not copied into its local receipts.

For a custom integration, use the same sequence:

  1. POST https://api.apify.com/v2/actors/RGfTsivyHveCZmq47/runs?maxTotalChargeUsd=0.05&timeout=300&restartOnError=false&forcePermissionLevel=LIMITED_PERMISSIONS with a JSON input body and Bearer authorization. Save data.id once.
  2. GET https://api.apify.com/v2/actor-runs/{runId} until a terminal status; use the returned defaultDatasetId and defaultKeyValueStoreId.
  3. GET https://api.apify.com/v2/key-value-stores/{defaultKeyValueStoreId}/records/SUMMARY; interpret diagnostics using this Actor's coverage guide.
  4. GET https://api.apify.com/v2/datasets/{defaultDatasetId}/items?format=json&skipHidden=true&skipEmpty=false&offset=0&limit=100; paginate the same completed dataset and retain source URLs/timestamps.

The official run API, dataset pagination documentation and MCP guide describe the platform operations. Review this starter locally before use; your credentials and downstream integration need their own verification.

SOURCE COVERAGE & PRACTICAL LIMITS

Full product guide

Documentation captured from build 1.0.1. Example outputs retain their original sample context; they are not live results.

Sitemap URL Inventory — a dependable feed of published URL changes

Find newly listed product pages, documentation URLs and competitor content from XML sitemaps. Re-run the same scope to see additions, metadata changes and URLs absent from the next complete inventory.

For SEO agencies, content researchers and AI ingestion teams. Each observation keeps the exact URL, sitemap evidence, publisher-supplied metadata and comparison status. A removal means absent from these sitemaps; it does not prove the web page was deleted.

Try it

Empty input reads the official sitemaps.org XML sitemap. A smaller first run:

{"sitemapUrls":["https://www.sitemaps.org/sitemap.xml"],"urlPrefixes":["https://www.sitemaps.org/protocol.html","https://www.sitemaps.org/faq.html","https://www.sitemaps.org/terms.html"],"maxResults":10}
  1. Paste your XML sitemap URLs or sitemap-index URLs.
  2. Start with a small scope. Open the dataset and the readable run report.
  3. For recurring monitoring, save the input as an Apify task, enable comparison and reuse the snapshot name. Run a snapshot sequentially.

What makes the changes usable

  • Follows bounded same-origin sitemap indexes and supports gzip sitemap files.
  • Deduplicates page URLs across maps while retaining every source map.
  • Saves one URL observation per result, with no fabricated page titles or dates.
  • Emits removals only after a complete inventory. A network error, truncated map, output cap or spending stop leaves the previous baseline intact.
  • Supplies the same structured output to the Apify API, MCP tools, scheduled tasks and spreadsheet exports.

Three input recipes

English documentation pages: the first-run input above.

German documentation inventory:

{"sitemapUrls":["https://www.sitemaps.org/sitemap.xml"],"urlPrefixes":["https://www.sitemaps.org/de/"],"maxResults":20}

Full official sitemap:

{"sitemapUrls":["https://www.sitemaps.org/sitemap.xml"],"maxResults":100}

Inputs

Input Default Purpose
sitemapUrls Official sitemaps.org map Up to10 public XML maps or indexes
urlPrefixes none Select output by absolute URL prefix; all maps still read
maxSitemaps 20 Root and child sitemap request cap, up to100
maxUrls 5000 Unique URLs inventoried before filters, up to10000
maxResults 100 Saved output rows including removals, up to10000
comparePrevious false Compare only complete inventories
onlyChanges false Implies comparison; unchanged repeats can return zero rows
snapshotName default Independent history scope; letters/numbers/underscore/hyphen
maxRunSeconds 90 Shared deadline,30–900seconds, also limited by the platform

Output

One saved row is one URL observation. Fields: url, title (the URL itself), recordId, lastmod, changefreq, priority, sitemapUrls, sourceUrl, observedAt, presence, inventoryComplete, comparisonStatus, changedFields, previousObservedAt and previous. Optional metadata is null when absent. Publisher metadata is kept as text, without inventing a date, priority or page title.

Comparison statuses: not_requested, baseline, added, changed, unchanged, removed, or not_compared_incomplete. The SUMMARY record reports exact coverage, failures, limits and whether history was updated. RUN_REPORT is a self-contained HTML view. JSON preserves nested evidence; CSV is useful for flat URL inventories.

Pricing

Base price: $0.50 per1,000 saved URL observations, plus the configured $0.00005 Actor Start event. Saved removal observations have the same price. An unchanged row is charged when saved; onlyChanges skips it. Failed source requests and summary/report records are not dataset result events. Check the Pricing tab for your plan's discounts. Use the run maximum-charge control to bound spending.

Coverage and limits

XML urlset and sitemapindex, optionally gzip, only. RSS/Atom/text feeds, images/video extension extraction and cross-origin child maps are unsupported. Each response has a4MiB wire and decoded body cap; one XML map supports at most10000 entries. Public HTTP(S) only with robots.txt, DNS and redirect validation. No login, proxies or browser rendering. The Actor reads maps; it never visits every listed web page or claims to know Google indexing. lastmod, changefreq and priority are publisher statements, not verified changes or ranking scores. The same root-map and prefix scope identifies a saved baseline; changing it starts independent history. Concurrent writes are checked but the platform store is not transactional: run a snapshot sequentially.

Use in workflows

Schedule the saved task, read SUMMARY after success, then route complete added/changed rows into your crawler, research queue or AI index. Route removed rows for review before deleting downstream content. Partial source observations remain explicitly marked. Use the Apify REST API or the Actor's MCP integration with your own Apify authentication.

Independent tool by May'd It. Source format: https://www.sitemaps.org/protocol.html . Support: open an issue with the sitemap URL, relevant input and expected result; omit credentials.