with WHOIS XML API
in the New gTLD Era: a Threat Crying for Urgent Solution
Sometimes certain comfortable and seemingly innocent protocols can introduce significant security risks, especially when the system's environment changes. The present document describes such a case.
The WPAD (Web Proxy Autodiscovery) protocol is prevalently used to configure the web proxy settings of end systems such as desktops and other devices belonging to an administrative domain, e.g. a corporate network. The benefit of this solution is that system administrators can deploy local web proxy settings essentially without any user interaction. Due to a very progressive change in the domain registration policies, the otherwise very useful WPAD protocol has introduced the possibility of a new and very dangerous man-in-the-middle attack. In 2016 the researchers estimated  that at least 6.6 million end-users are at a serious risk.
There is no perfect protection against this threat yet. However, there do exist some efficient remediation strategies, and those which are the easiest to deploy rely highly on domain registration data.
In order to facilitate the enhancement of innovation, competition and consumer choice, the Internet Corporation for Assigned Names and Numbers (ICANN) has announced the new gTLD program  in 2012, enabling the largest expansion of the domain name system. Beside the few legacy generic TLDs such as .com, more than 1,200 new gTLDs have been delegated since.
The idea of the WPAD protocol is to obtain a proxy configuration file for the client’s browser via a HTTP request. The proxy file location is inferred from this name and fetched using HTTP request to an URL deduced from the client’s internal domain. This latter domain is deduced from a DHCP or a DNS query. E.g. if the internal domain is company.ntld, the file will be searched for at the URL http://wpad.company.ntld/wpad.dat, involving a DNS query for wpad.company.ntld. And here lies the problem: since the advent of the new GTLD era, the company’s ntld can also be registered as a new gTLD, thus a name collision occurs. Assuming that the adversary can register hosts under the given gTLD (either by delegating the domain for the purpose of the attack or realizing this opportunity later on), the following man-in-the-middle attack becomes feasible:
- From the leaked DNS query of the WPAD protocol the adversary deduces the URL of the proxy configuration file.
- Based on this information, he sets up the appropriate resource at his server.
- The client will download the proxy file prepared by the adversary.
- All the web traffic of the client will now be sent through the adversary’s proxy.
A recent detailed study in Ref.  is pioneering in the analysis of this threat and quantifying the attack surface. The analysis is partly based on WhoisXML API data. As already mentioned, a dramatically large number of users are at risk because of this vulnerability. And although in 2016 the researchers “did not find strong evidence of adversaries actively registering attack surface domains, but do observe potential blind attack registrations” , this lucky situation may, however, change anytime.
According to Ref.  there is no perfect remediation strategy at the moment. An obvious solution is to treat the problem at the client’s side by disabling the WPAD service, upgrading operating systems and revising their settings, or filtering device-level leaks. Owing to the huge number of potentially affected clients, however, this approach has serious deployment issues. The other extreme level would be at the new gTLD registry, scrutinizing the registration of the union set of highly-vulnerable domains, which would be an efficient approach. It requires, however, a collective effort.
The most viable strategy thus seems to be at the enterprise level. The filtering of highly vulnerable domains can lead to an efficiency of 97.4%. This is feasible by purchasing accurate WHOIS data, and using Ref.  as a guideline.
Qi Alfred Chen, Eric Osterweil, Matthew Thomas, and Z. Morley Mao. Mitm attack by name collision: Cause analysis and vulnerability assessment in the new gtld era 2016 IEEE Symposium on Security and Privacy (SP), May 2016.
 Internet Corporation for Assigned Names and Numbers (ICANN). The new generic top level domain program.
https://newgtlds.icann.org/en, visited on 2017.10.06., 2012.