Over the last couple of weeks, I’ve been working on a project where I needed to determine the creation dates of various domains. Little did I know about the complexities involved in achieving such an apparently simple task. After all, ICANN has a WHOIS Data Reminder Policy (WDRP), thus all domains must have this information readily available via a simple WHOIS query.
WHOIS is a query-based protocol that is used for obtaining information about the registered users of various Internet resources, such as domain names and IP address blocks.
My first hurdle was realizing that I first had to determine which WHOIS server to query for a specific domain. Luckily, whois-servers.net provides DNS alias records for top level domain (TLD) WHOIS servers. Unfortunately it is far from comprehensive. Whois.iana.org, however, provides information for each TLDs WHOIS server. Excellent; so now I know who to query for a given a TLD. I used an amalgam of the two methods and that seemed to do the trick.
Next hurdle: parsing the response from the appropriate WHOIS server. There is no standard enforced for the output of each individual WHOIS server. The only requirement seems to be that the output is human readable. Each server (there are now over 500) requires its own dedicated parser! Ruby Whois is a project under the MIT license that has courageously taken on this task. I needed a C# implementation, however, so I decided to code my own solution, an ongoing process which I think will remain ongoing for a long time.
I finally started to get some results and then realized that there were even more hurdles. Some of the official TLD WHOIS servers don’t even respond to queries. They timeout or are not even running. Furthermore, the data that the functional ones spit out don’t even necessarily have the creation date! There are all sorts of privacy agreements between certain registrars and registrants (see Domain Privacy). Even the language of output is not necessarily English: try making a query to whois.jprs.jp. How exactly can one build an automated system to accurately get the creation date of domains?
Spammers register domains in abundance, and getting information about them and their domains in an automated fashion seems to be unnecessarily painful. Worse, spammers pay registrars who then pay ICANN (see registrar accreditation). ICANN is supposed to enforce the WDRP on the various registrars. Isn’t that like biting the hand that feeds you? Check out this blog by Garth Bruen of Knujon on whats driving spam and domain fraud.
Leave a Comment