Twenty-seven ways to write an expiry date

A domain in our own fleet was twenty-six hours from expiring, and every dashboard was calm. Three weeks ago we wrote about dead domains — how a loopback probe keeps calling a site healthy after its name has left the internet — and that post ended with a fix built on RDAP, the structured successor to WHOIS. This is the sequel, about the half of the internet where the successor never arrived.

The map with holes in it

RDAP fixed the right problem: JSON instead of free text, real timestamps, and a bootstrap file at IANA that says which server speaks for which TLD. For generic TLDs it is mandatory, and port-43 WHOIS is being switched off there entirely. Country-code registries, though, join if and when they feel like it. Many never did.

The one that almost burned us was .us. Not an exotic island TLD — plain American .us, absent from the RDAP bootstrap file. Our sweep asked RDAP, the map had no entry, the lookup returned nothing, and nothing produced no metric. No metric means no rule can fire. Absence of data read as health — the oldest lie in monitoring — while the registration ran out its last day.

Back to 1982

So we went where the data actually lives: WHOIS, a protocol older than DNS itself. Open TCP port 43, send a name, read whatever text comes back. No schema, no required fields, no language guarantee. IANA's server refers you to the registry's server, and from there you are on your own.

The first instinct is a pile of regular expressions, and it is wrong — the pile only grows and every new registry breaks it. What holds up is parsing by field name: take each line's label, strip everything that is not a letter, and look it up in a dictionary. Registry Expiry Date. Expires On. paid-till. Valid Until. Record expires on. Date d'expiration. Fecha de vencimiento. The registrar's own expiration field sits deliberately last in that dictionary, because when a record carries both, the registry's date is the one that decides whether the name lives.

Then the value. We are at twenty-seven date layouts and counting: 4.7.2027 10:15:55, 2027-Aug-25. with a trailing dot, September 5 2027 with two spaces, Mon Mar 22 23:59:00 2027, 20270606, 2027. 03. 02., and my favorite, 2030-02-08 21:00:00 CLST — a bare timezone abbreviation that no standard library resolves for you. And then the sentinels: Never, N/A, and expiry years past 9000. Those are not parse failures. Some registries genuinely renew forever, and "this domain has no expiry" is an answer, not an error.

One more courtesy of the old world: .jp answers in Japanese unless you append /e to the query string. The protocol has no version negotiation, but it has that.

An answer that is not an answer

Three kinds of non-answer come back from port 43, and each needs its own treatment. A rate limit — "quota exceeded", "too many requests" — is transient: retry in an hour, and keep the previous dates meanwhile, because a registry blinking is not the same as a domain vanishing. A "not found" is real and cheap to remember: negative-cache it for a week. The dangerous one is the third: a genuine record whose expiry field we simply do not recognize.

The tempting move is to fail the check red. That is wrong in a way that matters. A registry quietly rewording its text output is a monitoring vendor's parsing problem, and a parsing problem must never page a customer. So an unrecognized format becomes a quiet per-domain marker — "no expiry data" — plus a retry tomorrow, and the raw shape lands on our desk to be added as layout twenty-eight. The customer's console never turns red over prose drift.

Some registries publish no dates at all, as policy — .lv, .eu, .de, .nl among them. Those we never query in the first place, and the console says honestly, per TLD, that expiry data does not exist there. A few more are half-open: .kr answers only in Korean, .ro's public data stops in 2018. Where the world publishes nothing, we display nothing — an invented reassurance is how you end up explaining an outage you were paid to predict.

Manners

Before asking anyone anything, you have to know what to ask about. shop.example.co.uk is not a domain you can renew; example.co.uk is. That resolution comes from the public suffix list — the ICANN section only, because private suffixes like github.io are not registrations you control either — plus a short skip list for wildcard-DNS conveniences, so a throwaway 10-0-0-1.nip.io style name never sends us to interrogate a registry about something nobody registered.

Results are cached on disk with a ladder tied to urgency: a domain with two months of runway is checked weekly, under sixty days daily, under fourteen days every six hours. The metric is emitted on every pass from cache; the network is touched only when the ladder says so. That detail is load-bearing: it means upgrading or restarting an entire fleet of agents does not stampede anyone's registry, because the cache travels and the TTLs hold. And when we do dial out, it is one request in flight per registry host with a polite gap between them.

The box that cannot ask

Then there are the servers that cannot ask at all. Hardened hosts often have no outbound port 43 — nobody writes a firewall exception for a 1982 protocol. For those, the lookup relays through the platform over the connection the agent already holds, and the courier is deliberately deaf: the agent names only the domain, and the platform decides whom to dial from IANA's own data, never from anything the agent sent. Size caps, a daily budget of three hundred lookups per agent, and one rule that matters more than the rest — a 429 from a registry is an answer, not a network failure, and answers are never relayed around.

We did not have to stage a test. The first evening this shipped, the relay engaged on its own: one of our own boxes turned out to have port 43 closed, and we had never noticed — because until that day, nothing had ever tried.

Twenty-six hours

Seven minutes after rollout, the quiet domain surfaced: just over a day of registration left, warning raised, renewed with room to spare. The date had existed all along, printed in plain text on a server that was never going to volunteer it.

The takeaway is the same one the dead-domains post ended with, one layer deeper. The structured future is real, but it is unevenly distributed, and the protocol it replaced is still load-bearing for half the map. If you go down to fetch that data yourself: parse by field name, treat sentinels as answers, treat "cannot parse" as your bug and never the customer's outage — and when a registry publishes nothing, say "nothing" out loud. It reads worse on a dashboard and much better at 3am.