[solved] curl rabbit hole

Links: DNS server list Note: the workaround described here is probably only necessary during this temporary era of IPv4 -> IPv6 transition. I have little doubt this curl bug will eventually be resolved.

---

Currently (7/2013), when we enter a URL via curl at the command line, curl's first action is a DNS resolution query to resolve the address. By default, curl queries in both IPv4 and IPv6 formats. The rub is that, unless the DNS server responds in both formats, A and AAAA, curl spawns an error (see below) and exits. Most major sites are returned in both formats but many sites, including repositories needed by rpm/yum, are not resolved in IPv6.

$ curl pkgs.repoforge.org
curl: (6) Could not resolve host: pkgs.repoforge.org; Cannot allocate memory

We can overcome the problem with the "--ipv4" switch.

$ curl --ipv4 pkgs.repoforge.org
[page loads normally]

The more substantial problem is rpm/yum reliance upon curl for network repository access. Calls to curl from within rpm/yum are made without any switches available to the user. Accordingly, curl makes such DNS queries in a default IPv4 + IPv6 format. This means rpm/yum fail and exit unless curl receives both A and AAAA responses.

# rpm -ivh http://pkgs.repoforge.org/rpmforge-release/rpmforge-release-0.5.3-1.el6.rf.i686.rpm
Retrieving http://pkgs.repoforge.org/rpmforge-release/rpmforge-release-0.5.3-1.el6.rf.i686.rpm
curl: (6) Could not resolve host: pkgs.repoforge.org; Cannot allocate memory

troubleshoot - tcpdump

Tcpdump information confirms that it is curl's insistence on both IPv4 and IPv6 information that causes the failure:

# rpm -ivh http://pkgs.repoforge.org/rpmforge-release/rpmforge-release-0.5.3-1.el6.rf.i686.rpm
curl: (6) Could not resolve host: pkgs.repoforge.org; Cannot allocate memory
[tcpdump information]
192.168.1.20.57866 > 192.168.1.254.domain: 41495+ A? pkgs.repoforge.org. (36)
192.168.1.20.57866 > 192.168.1.254.domain: 25356+ AAAA? pkgs.repoforge.org. (36)
192.168.1.254.domain > 192.168.1.20.57866: 41495 1/0/0 pkgs.repoforge.org. A 78.46.17.228 (52)
192.168.1.254.domain > 192.168.1.20.57866: 25356- 0/0/0 (36)

Curl has received valid DNS resolution (emboldened), but only to the IPv4 query. Curl's failure inside of an rpm request thereby causes rpm to also fail. Although we know so far that curl has a design flaw requiring it receive A and AAAA query responses (instead of either), and although we know we can't control curl inside of rpm, we need more information about why no AAAA (IPv6) data is being returned. We might be able to control that.

troubleshoot - nslookup, dig

$ nslookup pkgs.repoforge.org
Server: 192.168.1.254
Address: 192.168.1.254#53

Non-authoritative answer:
Name: pkgs.repoforge.org
Address: 78.46.17.228

$ nslookup type=AAAA pkgs.repoforge.org
nslookup -type=aaaa pkgs.repoforge.org
Server: 192.168.1.254
Address: 192.168.1.254#53
Non-authoritative answer:
*** Can't find pkgs.repoforge.org: No answer

$ dig +short pkgs.repoforge.org A
78.46.17.228

$ dig +short pkgs.repoforge.org AAAA

Perhaps an AAAA record (PTR or zone) was never created for the sourceforge repository (consider eg.this article). It's also possible ATT does not update its DNS files often, or there is an incorrect AAAA record in their zone. Too many upstream variables to determine reliably. At this point, unless we work for the NSA, we're stuck with the information we have. What is a feasible solution?

strategy 1

Write a patch and recompile curl (or pfsense) to succeed with either IPv4 or IPv6 information. The most reliable solution --- except that I'm not a programmer.

strategy 2 (inelegant but successful)

Provide BIND with an AAAA /etc/hosts entry for pkgs.repoforge.org. Some good /etc/hosts IPv6 information is available here. The excellent site http://ip-lookup.net/conversion.php provided a set of conversion options for 78.46.17. The IPv6 address which appeared best for tricky operations such as the current curl release (operating in IPv4 mode, but needing IPv6 info!) appeared to be "IPv4-mapped address". This is 0:0:0:0:0:ffff:4e2e:11e4, written as ::ffff:4e2e:11e4.

# nano /etc/hosts
::ffff:4e2e:11e4 pkgs.repoforge.org
78.46.17.228 pkgs.repoforge.org

# nano /etc/host.conf
order hosts,bind

(Other helpful links were this article and this IPv4-6 translator). Before the successful curl run, I tried the URL successfully in Chromium, entering http://[::ffff:4e2e:11e4] in the URLbar.

strategy 3

Try other DNS servers, ones likely to have the most up-to-date zones. Google provides solid instructions in this document describing how to point to their DNS servers. There is also this DNS list. Alternatively, I could simply add "nameserver" entries in /etc/resolv.conf using "supersede" to prevent overwriting by dhcpcd as described in comments here.

strategy 4

A common way to manage IPv4 vs IPv6 confusion in the past has been locking-out IPv6. I even showed how to do this in a prior post. However, since curl now requires requires A and AAAA records to be returned, shutting out IPv6 is no longer a sensible confusion-stopper. Unless a person has no need to contact software repositories.