Getting inventory crawled while the car is still on the lot
This is the technical problem at the center of automotive SEO, and the one most consistently misunderstood by agencies who arrived from other verticals.
A dealership does not have a website with some pages that change. It has a website where the majority of pages exist for a few weeks and then should stop existing, replaced by a different set of pages that also exist for a few weeks.
Search engines are not built around that assumption, and neither is most SEO advice. This guide is the long version of what actually happens, why it goes wrong, and which parts you can fix.
Why this is a scale problem, not a tidiness problem
- Typical franchise stock
- 400 to 800 units
- Turn
- Roughly monthly
- Pages created a year
- Thousands
- Useful life of a VDP
- 3 to 8 weeks
- SRP filter permutations
- Often thousands
- Crawl budget
- Fixed, unpurchasable
Take a store carrying 600 units turning over monthly. That is roughly 7,200 vehicle detail pages created and retired in a year, each with a useful life measured in weeks. Add the search results pages, and every filter combination of make, model, price band, mileage, color and body style, and the crawlable surface runs into the tens of thousands.
Against that you have a crawl budget you cannot increase by paying for it. So the entire discipline is about allocation. Every request spent on a car that sold in March is a request not spent on the one that landed yesterday, and only one of those still has gross attached.
You cannot buy more crawl budget. You can stop wasting the budget you have.
How a unit should move through the index
- 1. Unit lands and the page is createdThe feed publishes a VDP. At this moment nothing links to it except, if you are fortunate, an SRP that is itself crawled regularly. If the only path to it is a JavaScript filter with no linkable URL, it is effectively invisible and the rest of this list never happens.
- 2. The page is discoveredDiscovery comes from internal links, sitemaps and, if you have it, indexing APIs. A sitemap that regenerates promptly and is genuinely accurate is the cheapest improvement available to most stores.
- 3. The page is crawled while it is liveThis is the step that either produces value or does not. Crawled in week one, the page can capture searches for that exact vehicle. Crawled in week nine, the car has usually gone.
- 4. The unit sellsThe moment that changes everything. From here the page has no commercial value and every further crawl of it is waste.
- 5. The page is retired cleanlyEither a 301 to a relevant SRP or comparable unit, or a 410 that removes it deliberately. What must not happen is a page that keeps returning a success status with no vehicle on it, because search engines will keep coming back to check.
The five failures we find on almost every store
- Soft 404s on sold inventoryThe most common and most expensive. The page returns 200 with a "vehicle no longer available" message. Search engines keep crawling it for months because nothing told them not to.
- Inventory behind an unlinkable filterStock reachable only through a JavaScript filter that produces no distinct URL. If a page has no address, it cannot be crawled, linked to, or ranked.
- Parameter explosion on SRPsEvery combination of filters generating a crawlable URL, producing thousands of near-identical pages competing with each other and consuming the budget the VDPs needed.
- Sitemaps that lag realityA sitemap regenerated weekly on a store with monthly turn is wrong most of the time. It advertises sold cars and omits new arrivals, which is worse than having none.
- Identical descriptions across every storeFeed copy repeated verbatim on every dealer carrying the same unit type. Search has no reason to prefer your page and generally picks the domain with more authority instead.
- SRPs treated as disposableThe pages that should be the long-term ranking assets left thin and untitled, while effort goes into VDPs that will not exist next month.
Two page types, two completely different jobs
A search results page serves someone browsing: used trucks under thirty thousand, certified sedans in a city. That search is repeated by different people every month and the page answering it can persist for years. It is where lasting ranking is built.
A vehicle detail page serves someone who has narrowed to one car. The intent is far higher and the page will not exist in two months. It is a capture surface, not an asset.
Most dealer sites get this backwards, pouring effort into individual vehicle pages and leaving the browse pages thin and generically titled. The correct allocation is to make SRPs genuinely good, stable and well targeted, and to make VDPs fast to discover and clean to retire.
What you can actually do if you cannot edit anything
Diagnosis is always available and needs no access whatsoever. How much of your stock is indexed, what is happening to sold units, whether your sitemap matches reality, and how many parameter variants are crawlable can all be established from outside.
Requests to your provider cover most of the fixes: status codes on retired units, canonical handling on filtered pages, sitemap regeneration frequency. These succeed far more often when written in the language the provider's support queue expects, with the specific behavior and the specific URLs attached.
A few things are genuinely unavailable on a mandated platform. Wholesale URL restructuring, custom templates per model, and anything needing server configuration. When we reach one of those we say so and move the effort to authority, content or the service side rather than billing to attempt it.
- Always available, no access neededIndexed stock counts, sold-unit behavior, sitemap accuracy, parameter sprawl, and time from listing to first crawl. All measurable from outside.
- Available through your providerStatus codes on retired units, canonical handling on filtered pages, and sitemap regeneration frequency. Requests succeed far more often when they name the URLs and the behavior wanted.
- Genuinely unavailableWholesale URL restructuring, custom templates per model, server configuration. We say so rather than billing you to attempt it.
Three numbers worth watching
- Indexed stock as a share of live stockIf you carry 600 units and 180 are indexed, you know the size of the problem without any further analysis. This is the single most useful number in this whole guide.
- Time from listing to first crawlMeasured in days. Under a week is healthy. Over three weeks means most units are selling before they are ever findable, and the pages are producing nothing.
- Share of crawl going to expired pagesIf a meaningful share of crawl requests are hitting sold inventory, that is the budget your new arrivals needed, and retirement handling is the fix.
Common questions
What is a VDP?
A VDP is a vehicle detail page: the page for one specific car, carrying its VIN, mileage, price, photos and features. It is distinct from an SRP, a search results page, which lists many vehicles matching a filter.
What is crawl budget and does it really affect dealers?
Crawl budget is the rough allocation of crawling a search engine gives your domain, based on size, update frequency, response speed and perceived importance. It is irrelevant for a forty-page site and central for a dealer site publishing and retiring hundreds of pages a month.
What should happen to a VDP when the vehicle sells?
It should stop returning a success status with no vehicle on it, because that is a soft 404 and it wastes crawl indefinitely. The two defensible options are a 301 redirect to a relevant SRP or comparable unit, or a 410 that removes it cleanly. Which is right depends on the page and your platform.
Why do our new arrivals take weeks to appear in search?
Usually because crawl budget is being consumed by expired inventory, filter permutations and parameter variants, so new pages sit in a queue behind pages that no longer sell anything. Fixing retirement usually improves discovery of new stock as a side effect.
Should SRPs or VDPs be the ranking target?
SRPs, for anything you want to hold long term. They persist, they match browse intent, and they survive inventory turnover. VDPs capture specific, immediate intent and disappear with the car, so they are a capture surface rather than a ranking asset.
Can any of this be fixed on a mandated OEM platform?
Diagnosis needs no access at all and is always available. Some fixes require your provider and can be requested. A few, such as wholesale URL restructuring, are genuinely unavailable, and an honest agency will tell you which is which rather than billing you to attempt the impossible.
Find out what your inventory is doing right now.
The free analysis includes everything we can determine about your inventory indexation from the outside: how much stock is actually findable, what is happening to sold units, and where crawl budget is going. No platform access required.