Showing posts with label ipv6. Show all posts
Showing posts with label ipv6. Show all posts

Tuesday, June 12, 2018

pacman failure "404", duid (dhcpcd)

There's a mix of ipv4, 6, and duid stuff in here. On a home network -- our home LAN -- why would we ever need NSA level DUID (DHCP Unique IDentifier) fingerprinting of our system? If it's configured, but there's some conflict, our dhcpcd will time out, or make a connection but fail to browse (no DNS), and so on. This is just another level of possible failure on a home system; why would we ever want duid in a million years? DUID settings are further down, but they seem to be set no matter what a person does to remove.

pacman

One time, I began receiving 404 failures during # pacman -Syu. I quickly dug deeper using # pacman -Syu --debug 2>&1 |tee debugfile.txt and found that either curl itself (used by pacman to retrieve packages), or the way in which pacman was using curl, was the source. This is not always a bed of roses to determine and repair since they are often random glitches in what's become a decades-long, intermittent, annoying ipv6/ipv4 transition mess. This can involve tens of kernel and application settings and confusing conf files that often interact badly. For example, no longer can one edit /etc/resolv.conf directly, since it will be overwritten by resolvconf during dhcpcd initialization. One must now edit /etc/resolvconf.conf to indirectly get at /etc/resolv.conf, and resolvconf.conf has its own syntax, different from resolv.conf, so that both file syntaxes must be verified following any change. And so on.

which ipv is working?

The most obvious start is to verify both ipv4 and ipv6 curl function. Then, one can disable or activate the failed half or, possibly... force pacman to use the ip version that *is* working, (although I'm not sure if ipv type can be specified during a pacman operation).

First, the curl ipv functionality. Take a URL from one's mirror list and ping it to see that DNS is working. Then...
# curl aprs.ele.etsmtl.ca
Bienvenue | Welcome [curl works]
# curl --ipv4 aprs.ele.etsmtl.ca
Bienvenue | Welcome
[curl, and therefor pacman, is working ipv4]
# curl --ipv6 aprs.ele.etsmtl.ca
curl: (6) Could not resolve host: aprs.ele.etsmtl.ca
[curl, and therefore pacman, is failing ipv6]

In this case, it appears pacman is requesting both an ipv6 (AAAA) and ipv4 (A) handshake via curl, but, at various points in its development, pacman could only accomplish ipv4. Strace shows futher that IPV6 is not even able to open a socket.

Another way to check the kernel network stuff is to do a # sysctl -A, which will reveal all the kernel settings, but then grep it for, "net", "ipv4", or "ipv6" settings.

Users could also run # ip addr list and look to see there's an ipv6, and ipv4, or both.

~/.curlrc

As part of the curl man page, we can see some settings for the configuration file.

In the case above, I was getting proper ipv6 configuration, but still getting a curl fail on ipv6. It turns out that I was only able to find a single post on this, here. The post indicates that it's really a glibc issue, that could only be resolved by slowing down DNS resolution via manipulations inside /etc/resolvconf.conf to manipulate /etc/resolv.conf. We want /etc/resolv.conf to say "options single-request"
and so, to achieve this, we write...
# nano /etc/resolvconf.conf
# Solve ipv4/v6 fails
resolv_conf_options="single-request"
For a list of all resolvconf.conf options, see this man page.
Disabling IPV6 in Arch is no bed of roses -- one has either to put a boot line into their GRUB, or to create an effective /etc/sysctl.d/40-ipv6.conf (see bottom). Further, we may instead need to enable ipv6 more thoroughly across all applications, not further cripple it. It's a typical ipv4/ipv6 trial and error mess due to opaque reliance upon multiple programs (eg. curl)

old fix

This one used to work, but nowadays with built-in ipv6, doesn't always work. Add a couple lines in your blacklist file, eg...
# nano /etc/modprobe.d/blacklist
# stop pacman failure when encounter ipv6 sites
blacklist nf_conntrack_ipv6
blacklist nf_defrag_ipv6

grub

Use nano or whatever to add

/etc/sysctl.d/40-ipv6.conf

Here's one example, not guaranteed to work.
# Disable IPv6
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6=1
net.ipv6.conf.lo.disable_ipv6 = 1
net.ipv6.conf.wlan0.disable_ipv6 = 1

duid

I've removed all duid references but I still get some duid when I connect with dhcpcd. Be sure too to delete all old "leases" in /var/lib/dhcpcd. Note also that they put a duplicate version of dhcpcd.conf inside /var/lib/dhdpcd/etc/, and that this is the one used by the system. You could probably even delete /etc/dhcpcd.So "duid" needs to be commented out at that location.

# rm /etc/dhcpcd.duid
# rm /etc/dhdpcd.secret
# rm /var/lib/dhdpcd/duid
# nano /etc/dhdpcd.conf [comment duid and clientid lines]

So then, here's a truncated version of etc/dhcpcd.conf. But even with al duid disabled, the router is apparently coaxed into collecting one.

# nano /etc/dhcpcd.conf
# A sample configuration for dhcpcd.
# See dhcpcd.conf(5) for details.

# Inform the DHCP server of our hostname for DDNS.
hostname

# duid

# Persist interface configuration when dhcpcd exits.
persistent

option rapid_commit

# A list of options to request from the DHCP server.
option domain_name_servers, domain_name, domain_search
option classless_static_routes

# Get the hostname
option host_name

# Most distributions have NTP support.
option ntp_servers

# Respect the network MTU. This is applied to DHCP routes.
option interface_mtu

# A ServerID is required by RFC2131.
require dhcp_server_identifier

# Prevent timeouts for ipv6 failures
noipv6
noipv6rs

Wednesday, January 10, 2018

Latitude D520 Arch install

Problem: your grandparent or a teacher's assistant needs a work-a-day laptop, but you're not a first responder, professional athlete, or an investment banker. If you can carry one (5lbs), 2007 Latitude D520's are available on EBay for about $60. They are 64bit 2 core processors. Load 'em up with 2 Gigs of RAM, Arch Linux and IceWM: within an $80 budget, you've got a laptop that'll do common functions without much lag. It'll show movies, edit photos and audio. High CPU tasks like rendering Blender output, or other video editing, takes ages (so just avoid video editing).

CD (10 mins)

Download an ISO and check the MD5. Put in a fresh disc. I found it's best to burn using root -- some systems bark, even with group permissions configured, at cdrecord at user level. I typically add "eject" as a simple completion notification, but it's not needed.
$ cdrecord -scanbus
# cdrecord -dao -eject dev=0,0,0 install.iso
$ md5sum /dev/sr0

base CLI (runlevel 2) configuration :: 20 mins

There's no UEFI or other boot complexities on older systems. But if you have a newer system, get rid of UEFI. Format the disk with "fdisk -t" to create an MBR. 0x04. The only real question is whether to make multiple partitions. The instructions below include multiple partitions. If multiple partitions aren't desired, just make two primary partitions in cfdisk: a linux (83) SDA1 and a linux swap (82) SDA2, and jump from the step mounting on /mnt to the step with mkdir /mnt/etc
# mkswap /dev/sdb3
# swapon /dev/sdb3
# free -m [check swap is on]
# mount -rw -t ext3 /dev/sdb2 /mnt
# mkdir /mnt/home
# mount -rw -t ext3 /dev/sda1 /mnt/home
# mkdir /mnt/boot
# mount -rw -t ext3 /dev/sdb1 /mnt/boot
# mkdir /mnt/etc/
# genfstab -p /mnt >> /mnt/etc/fstab
# pacstrap /mnt base base-devel linux linux-firmware
# arch-chroot /mnt
# ln -s /usr/share/zoneinfo/US/[zone] /etc/localtime
# mkinitcpio -p linux
# passwd
# pacman -Syu grub
# mkdir /boot/grub
# grub-mkconfig -o /boot/grub/grub.cfg
# grub-install /dev/sda
# exit
# reboot

useradd :: 5 mins

User 500, name "foo", home directory of "foo", using bash shell.
# useradd -G wheel,lp,audio -u 500 -s /bin/bash -m foo

aur :: 30 mins

Used to be so simple, so I'm including it again here in case it returns some day:
# nano /etc/pacman.conf
[archlinuxfr]
SigLevel = Never
Server = http://repo.archlinux.fr/$arch

# pacman -Sy yaourt
However, even in past, one had to have base-devel installed, and to add oneself to the sudoers file. Unfortunately, editing sudoers is done through the visudo command, which uses those stupid vim commands:
# visudo
foo ALL=(ALL) ALL
However, yaourt hasn't been maintained since 2018, so the (only) way to get it is to build it from scratch out of AUR source, as in olden times. The AUR website has aur source. Be sure you've got wget installed (# pacman -S wget), retrieve the package, untar it, morph it into a pacman version ("makepkg"), then install it using pacman with a "U" flag. The "U" flag has pacman looking into the current directory instead of out over the Net. Prior to building any AUR helper (yaourt, aurman), you'll need to first build "package-query", also from the AUR.

Also there is a wikipedia page and reddit conversations. In spite of its weaknesses, I continued to use yaourt. I downloaded aurman, but was unable to compile it.
$ mkdir pkgs
$ cd pkgs
$ wget https://aur.archlinux.org/cgit/aur.git/snapshot/package-query.tar.gz
$ wget https://aur.archlinux.org/cgit/aur.git/snapshot/yaourt.tar.gz
$ tar xzvf package-query.tar.gz
$ cd package-query
$ makepkg -s
# pacman -U package-query[version]tar.xz
And then the same with yaourt thereafter.

dhcpcd timeouts/limit journalctl size :: 5 mins

If you're having dhcpcd timeout problems, they can show up in pacman and curl, and are usually due to IPV6. This ipv6 request mod has to be re-accomplished after every dhcpcd package update.
# nano /etc/dhcpcd.conf
# custom to stop ipv6 service request timeouts
noipv6rs
Systemd will log GB's and GB's of data if not limited
# nano /etc/systemd/journald.conf
SystemMaxUse=200K

X install :: 20 mins

These Dells have the Intel integrated graphics 945GM; the correct xf86 driver is the Intel VA driver.
# pacman -S xf86-video-intel
This driver now uses DRI3 as the default Direct Rendering
Infrastructure. You can try falling back to DRI2 if you run
into trouble. To do so, save a file with the following
content as /etc/X11/xorg.conf.d/20-intel.conf :
Section "Device"
Identifier "Intel Graphics"
Driver "intel"
Option "DRI" "2" # DRI3 is now default
#Option "AccelMethod" "sna" # default
#Option "AccelMethod" "uxa" # fallback
EndSection
# pacman -S xorg-server xorg-apps xorg-xinit xorg-xrandr
If you scroll down this Arch page discussing Xorg, we can see that we'll want the mesa for Open GL and lib32-mesa for older apps. Also, Intel chips, as we know (scroll down to about item 13), do not support VDPAU, viz:
"Intel Embedded Graphics Drivers do not support VDPAU. VDPAU stands for video decode and presentation API for UNIX*. VDPAU is an open source library and API originally designed by NVIDIA that provides an interface to support hardware-accelerated video decode."
... and so Intel sez libVA is correct. More specific to Arch, there's additional information in their video acceleration page, if you like to read. The only problem with the VA-API is it can't decode MP4 and FLASH containers, but it does all other common formats, and all codecs, including H264 and the new VP8 and 9. I just use MKV and AVI containers and set the VLC codec to VA (instead of its default VDPAU). In spite of inefficiencies on the Intel hardware, some may wish to overlay VDPAU functionality onto their Intel chip, which is an installation beyond this post. If a person does that, any mistakes will defeat X working properly -- no harm, just revert to runlevel 2 and reconfigure until Xorg is working well. FYI, one of the tweaks I've seen for VIDPAU overlaid onto VA, is adding "export VDPAU_DRIVER=r600"in one's ~/.xinitrc file. Anyway, back to pure libVA...
# pacman -S libva libva-intel-driver libva-utils libva-mesa-driver
... then check the install with "$ vainfo".

window manager :: 10 mins

On an old system, I don't waste memory with display managers, instead I login and "startx" from runlevel 2. I like Ice Window Manager, a light interface with simple text configuration, wallpaper, and menu files (look inside ~/.icewm ). Efficient on older systems: perhaps 150M usage after logging-in, connecting to network, and starting X. In Arch, the template files are inside /usr/share/icewm/, including the themes. See the main Arch file.
# pacman -S icewm
$ cp /etc/X11/xinit/xinitrc .xinitrc
$ nano .xinitrc
exec dbus-launch icewm-session
$ startx
I also looked here to get the names of additional drivers, for example to solve the pesky touchpad problem. I couldn't stop the Dell touchpad with synclient TouchPadOff=1 until I # pacman -S xf86-input-synaptics.

wifi digression :: 60 mins

Some of these old Latitudes have the dreaded Broadcom 43 series card. For example, if users run lspci and the wifi readout line is this or similar...
Network controller: Broadcom Limited BCM4311 802.11b/g WLAN (rev 01)
... then it's unlikely the typical fix for older cards, # pacman -S wireless_tools will have effect. It's also unlikely anything will appear in either of these two verification steps...
# iw dev
# cat /proc/net/wireless
If this is the case, the easiest move is installing yaourt and building a b43 driver, but take note:
"BCM4306 rev.3, BCM4311, BCM4312 and BCM4318 rev.2 have been noticed to experience problems with b43-firmware. Use b43-firmware-classicAUR for these cards instead."
During that b43-firmware-classicAUR install, the following warning was displayed...
Please add $VISUAL to your environment variables
for example:
export VISUAL="vim" (in ~/.bashrc)
(replace vim with your favorite editor)
...and you may also encounter
~/.config/aurvote must have username and password. Run: aurvote --configure
...if you installed the yaourt rating app but hadn't configured it. You can use any username and pass you like. Reboot after this install, and something like this is normal:
# iw dev
phy#0
Interface wlan0
ifindex 3
wdev 0x1
etc..

QT or Gtk

I try to stick with one or the other to keep a smaller install and shorter update. Sometimes both become necessary. I prefer GTK (except gvfs), but VLC requires QT. QT is about 400MB, and typically pulls in PyQT. But since I'm a fan of VLC ... QT became my baseline API.
# pacman -S qt4
However, you're going to see that udiskie (to avoid gvfs) brings in about 80MB of shit, including basic Gtk.

sound

I avoid PulseAudio as much as I can. See my post from 2016. ALSA is now built-in, so that all that's required is alsamixer in order to control the sound levels (unmute, etc)
# pacman -S alsa-utils
Done.

rc.local

A consolidated place for random startup shit too lazy to configure individually. It's like an initrc in X, but for runlevel 3.
# nano /etc/rc.local
#!/bin/bash
wpa_supplicant etc
dhcpd etc
exit 0
# systemctl enable rc-local.service should make it happen next boot, but you also have to create the service file before enabling it.
# nano /etc/systemd/system/rc-local.service

# This file is part of systemd.
#
# systemd is free software; you can redistribute it and/or modify it
# under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.

[Unit]
Description=/etc/rc.local Compatibility
ConditionPathExists=/etc/rc.local

[Service]
Type=forking
ExecStart=/etc/rc.local start
TimeoutSec=0
StandardOutput=tty
RemainAfterExit=yes
SysVStartPriority=99

[Install]
WantedBy=multi-user.target

Monday, July 22, 2013

more ipv6 / ipv4 fun

Links: ipv6 over adsl :: Linksys E1000 knowledge base :: Linksys list of IPv6 enabled

www.southparkstudios.com is a useful site for checking IPv6 functionality. YouTube, probably due its fallback capacities (PPAP, HTML 5) is much more tolerant. So it was of interest to recently that I could access South Park at the gf's place, but not at home.

home setup

At home, we have ATT ADSL split between roommates. Service was initiated during the years when ATT provided Linksys E1000 routers in tandem with Motorola 2210-02 modems. This equipment is questionable; the Motorola overheats if it's breathed on, and the Linksys has no IPv6 support. This post is about the latter.

To set-up IPv6, we only want one IPv6 choke-point, so we want the modem to pass everything (bridge mode) and do the PPoE inside the router. But can we get the E1000 to do IPv6? Appears not easily.

Step 1 - ISP

Contact with ATT today noted that their DNS servers are resolving in IPv6.

Step 2 - Modem and Router

Modem is not in bridge mode; it's providing DHCP downstream to the LAN (E1000 Router). Unclear whether that's already degraded or filtered to IPv4-only coming from modem. Also it appears that the modem cannot be set to bridge mode without physical modification.

The router itself is not inherently IPv6 capable, so it may be worthwhile to start there. ATT sells the IPv6 Motorola NVG510 modem/router combo for $100. That one's not been getting good reviews. Another option is say, the less expensive D-Link G604T ($60). Outdated, but OpenWRT or DD-WRT should handle things.

E3000 v.2.1 (dd-wrt)

In an attempt to enable IPv6, installed dd-wrt software. Apparently the E3000 has a small (4MB) memory capacity so we're only looking for K2.6 builds, which run about 3.3MB. It was also suggested to keep Tx power down around 45mW for best throughput. I'll probably set my MTU filter at 1500 or 1400. We'll see.

Couple of useful links:
  • page 4 describes 16964 as first build supporting the router, then the 16968 build as stable, then 16994 (nokaid) having ipv6. 19519 may be most recent, but appears best to start with 16968 and then flash up to 19519 (nokaid).



  • firewall code








  • Wednesday, July 10, 2013

    [solved] curl rabbit hole

    Links: DNS server list Note: the workaround described here is probably only necessary during this temporary era of IPv4 -> IPv6 transition. I have little doubt this curl bug will eventually be resolved.

    Currently (7/2013), when we enter a URL via curl at the command line, curl's first action is a DNS resolution query to resolve the address. By default, curl queries in both IPv4 and IPv6 formats. The rub is that, unless the DNS server responds in both formats, A and AAAA, curl spawns an error (see below) and exits. Most major sites are returned in both formats but many sites, including repositories needed by rpm/yum, are not resolved in IPv6.
    $ curl pkgs.repoforge.org
    curl: (6) Could not resolve host: pkgs.repoforge.org; Cannot allocate memory

    We can overcome the problem with the "--ipv4" switch.
    $ curl --ipv4 pkgs.repoforge.org
    [page loads normally]

    The more substantial problem is rpm/yum reliance upon curl for network repository access. Calls to curl from within rpm/yum are made without any switches available to the user. Accordingly, curl makes such DNS queries in a default IPv4 + IPv6 format. This means rpm/yum fail and exit unless curl receives both A and AAAA responses.
    # rpm -ivh http://pkgs.repoforge.org/rpmforge-release/rpmforge-release-0.5.3-1.el6.rf.i686.rpm
    Retrieving http://pkgs.repoforge.org/rpmforge-release/rpmforge-release-0.5.3-1.el6.rf.i686.rpm
    curl: (6) Could not resolve host: pkgs.repoforge.org; Cannot allocate memory

    troubleshoot - tcpdump

    Tcpdump information confirms that it is curl's insistence on both IPv4 and IPv6 information that causes the failure:
    # rpm -ivh http://pkgs.repoforge.org/rpmforge-release/rpmforge-release-0.5.3-1.el6.rf.i686.rpm
    curl: (6) Could not resolve host: pkgs.repoforge.org; Cannot allocate memory
    [tcpdump information]
    192.168.1.20.57866 > 192.168.1.254.domain: 41495+ A? pkgs.repoforge.org. (36)
    192.168.1.20.57866 > 192.168.1.254.domain: 25356+ AAAA? pkgs.repoforge.org. (36)
    192.168.1.254.domain > 192.168.1.20.57866: 41495 1/0/0 pkgs.repoforge.org. A 78.46.17.228 (52)
    192.168.1.254.domain > 192.168.1.20.57866: 25356- 0/0/0 (36)
    Curl has received valid DNS resolution (emboldened), but only to the IPv4 query. Curl's failure inside of an rpm request thereby causes rpm to also fail. Although we know so far that curl has a design flaw requiring it receive A and AAAA query responses (instead of either), and although we know we can't control curl inside of rpm, we need more information about why no AAAA (IPv6) data is being returned. We might be able to control that.

    troubleshoot - nslookup, dig

    $ nslookup pkgs.repoforge.org
    Server: 192.168.1.254
    Address: 192.168.1.254#53

    Non-authoritative answer:
    Name: pkgs.repoforge.org
    Address: 78.46.17.228

    $ nslookup type=AAAA pkgs.repoforge.org
    nslookup -type=aaaa pkgs.repoforge.org
    Server: 192.168.1.254
    Address: 192.168.1.254#53
    Non-authoritative answer:
    *** Can't find pkgs.repoforge.org: No answer

    $ dig +short pkgs.repoforge.org A
    78.46.17.228

    $ dig +short pkgs.repoforge.org AAAA
    Perhaps an AAAA record (PTR or zone) was never created for the sourceforge repository (consider eg.this article). It's also possible ATT does not update its DNS files often, or there is an incorrect AAAA record in their zone. Too many upstream variables to determine reliably. At this point, unless we work for the NSA, we're stuck with the information we have. What is a feasible solution?

    strategy 1

    Write a patch and recompile curl (or pfsense) to succeed with either IPv4 or IPv6 information. The most reliable solution --- except that I'm not a programmer.

    strategy 2 (inelegant but successful)

    Provide BIND with an AAAA /etc/hosts entry for pkgs.repoforge.org. Some good /etc/hosts IPv6 information is available here. The excellent site http://ip-lookup.net/conversion.php provided a set of conversion options for 78.46.17. The IPv6 address which appeared best for tricky operations such as the current curl release (operating in IPv4 mode, but needing IPv6 info!) appeared to be "IPv4-mapped address". This is 0:0:0:0:0:ffff:4e2e:11e4, written as ::ffff:4e2e:11e4.
    # nano /etc/hosts
    ::ffff:4e2e:11e4 pkgs.repoforge.org
    78.46.17.228 pkgs.repoforge.org

    # nano /etc/host.conf
    order hosts,bind
    (Other helpful links were this article and this IPv4-6 translator). Before the successful curl run, I tried the URL successfully in Chromium, entering http://[::ffff:4e2e:11e4] in the URLbar.

    strategy 3

    Try other DNS servers, ones likely to have the most up-to-date zones. Google provides solid instructions in this document describing how to point to their DNS servers. There is also this DNS list. Alternatively, I could simply add "nameserver" entries in /etc/resolv.conf using "supersede" to prevent overwriting by dhcpcd as described in comments here.

    strategy 4

    A common way to manage IPv4 vs IPv6 confusion in the past has been locking-out IPv6. I even showed how to do this in a prior post. However, since curl now requires requires A and AAAA records to be returned, shutting out IPv6 is no longer a sensible confusion-stopper. Unless a person has no need to contact software repositories.

    Tuesday, July 9, 2013

    [solved] dns, yum, rpm, curl, ping

    Links: yum variables :: IPv4 address conversion :: yum commands
    NB: This is complicated post. It first addresses IPv6 (mostly successfully), but a second problem is revealed specific to Fuduntu, that I could not circumvent. Since Fuduntu is defunct, I'm disregarding and posting "solved" above. Hopefully, there's plenty of info below for others working on what might be a similar Fedora flavor of the Fuduntu release problem.
    Consider the following problem -- if I can ping, I should be equally able to curl, but I'm not:
    $ ping www.websense.com
    PING www.websense.com (204.15.67.11) 56(84) bytes of data.
    64 bytes from www.websense.com (204.15.67.11): icmp_seq=1 ttl=49 time=27.9 ms
    64 bytes from www.websense.com (204.15.67.11): icmp_seq=2 ttl=49 time=27.7 ms
    ^C
    $ curl www.websense.com
    curl: (6) Couldn't resolve host 'www.websense.com'
    This did more than just raise my curiosity; rpm/yum relies on curl during access to repositories. First I checked for proxy and IPv6 settings. All looked normal: no proxy, IPv6 set to ignore, but not to block or forced resolution. Let's look under the hood.

    tcpdump

    Here are portions of dumps for the successful ping and struggling curl:
    # tcpdump -nlieth0 -s0 udp port 53 -vvv
    [during ping]
    192.168.1.20.34097 > 192.168.1.254.53: [udp sum ok] 11891+ A? www.websense.com. (34)
    192.168.1.254.53 > 192.168.1.20.34097: [udp sum ok] 11891 q: A? www.websense.com. 1/0/0 www.websense.com. [5s] A 204.15.67.11 (50)
    192.168.1.20.58651 > 192.168.1.254.53: [udp sum ok] 51147+ PTR? 11.67.15.204.in-addr.arpa. (43)
    192.168.1.254.53 > 192.168.1.20.58651: [udp sum ok] 51147 q: PTR? 11.67.15.204.in-addr.arpa. 1/0/0 11.67.15.204.in-addr.arpa. [9h53m39s] PTR www.websense.com. (73)

    [during curl]
    192.168.1.20.41050 > 192.168.1.254.53: [udp sum ok] 26082+ A? www.websense.com. (34)
    192.168.1.20.41050 > 192.168.1.254.53: [udp sum ok] 54668+ AAAA? www.websense.com. (34)
    192.168.1.254.53 > 192.168.1.20.41050: [udp sum ok] 26082 q: A? www.websense.com. 1/0/0 www.websense.com. [5s] A 204.15.67.11 (50)
    192.168.1.254.53 > 192.168.1.20.41050: [udp sum ok] 54668- q: AAAA? www.websense.com. 0/0/0 (34)
    192.168.1.20.58040 > 192.168.1.254.53: [udp sum ok] 47978+ A? www.websense.com.localdomain. (46)
    192.168.1.20.58040 > 192.168.1.254.53: [udp sum ok] 42568+ AAAA? www.websense.com.localdomain. (46)
    192.168.1.254.53 > 192.168.1.20.58040: [udp sum ok] 47978 NXDomain- q: A? www.websense.com.localdomain. 0/0/0 (46)
    192.168.1.254.53 > 192.168.1.20.58040: [udp sum ok] 42568 NXDomain- q: AAAA? www.websense.com.localdomain. 0/0/0 (46)
    Ping only queries the DNS server in IPv4 (A?) and has success. Curl initially requests in both IPv4(A?) and IPv6 (AAAA?). Although curl receives a proper response (204.15.67.11) to its IPv4 request, nothing is returned for IPv6 request. Apparently due to some bug, curl ignores the IPv4 resolution and requests a second time in both formats. It also mysteriously appends "localdomain" onto its query(!).

    solution - /etc/hosts + release awareness

    Links: IPv4 address conversion :: yum concerns :: cleaning old yum info

    We should write a patch for curl and recompile it, but that's for programmers. I only know how to supply curl with the IPv6 information it wants. The site www.websense.com may not have an AAAA record in its DNS zone file, but I can still manually enter IPv6 info into /etc/hosts and force curl to use that.
    # nano /etc/hosts
    ::ffff:cc0f:430b www.websense.com
    204.15.67.11 www.websense.com

    # nano /etc/host.conf
    order hosts,bind

    $ curl www.websense.com
    [page loads normally]
    Problem 1 solved. However, there is a second problem, one specific to Fuduntu, not curl. Fuduntu is a hybrid. It accordingly doesn't have typical Fedora values in its rpm variables, eg $releasever.
    $ rpm - q fedora-release
    package fedora-release is not installed

    $ rpm -q fuduntu-release
    fuduntu-release-2013-3.noarch

    $ ls /etc/*release
    ls: cannot access /etc/release*: No such file or directory

    $ yum list fedora-release
    Loaded plugins: fastestmirror, langpacks, presto, refresh-packagekit
    Adding en_US to language list
    Determining fastest mirrors
    Could not retrieve mirrorlist http://packages.fuduntu.org/repo/mirrors/fuduntu-stable-rpms-2013 error was
    14: PYCURL ERROR 6 - ""
    Error: Cannot find a valid baseurl for repo: fuduntu
    Glitches also cause this with Fedora users when version conflicts arise. In the case of Fuduntu however, the repos no longer exist -- one strategy might be to spoof Fuduntu version checking as if were being upgraded when it accesses third-party repos. If we eliminate the locally-stored repo files and the rpm release file, we might be able to override with third party information. First let's do a debug dump with the current info (in case we need it later), then remove local information.
    $ yum-debug-dump
    Output written to: /home/~/yum_debug_dump-local-2013-07-12_20:50:30.txt.gz

    $ rpm -q fuduntu-release
    fuduntu-release-2013-3.noarch

    # yum remove fuduntu-release-2013-3.noarch
    [screens and screens of removal]

    $ rpm -q fuduntu-release
    fuduntu-release-2013-3.noarch

    # ls /etc/yum.repos.d
    dropbox.repo fuduntu.repo
    # rm /etc/yum.repos.d/fuduntu.repo
    # ls /etc/pki/rpm-gpg/
    RPM-GPG-KEY-fuduntu RPM-GPG-KEY-fuduntu-i386
    RPM-GPG-KEY-fuduntu-2013-primary RPM-GPG-KEY-fuduntu-x86_64
    # rm /etc/pki/rpm-gpg/*

    # yum clean all

    $ rpm -q fuduntu-release
    fuduntu-release-2013-3.noarch


    $releasever


    Links:replace $releasever using sed :: yum variables
    The orphaned Fuduntu release has no access to Fuduntu repositories because they no longer exist. Fuduntu must rely on third-party repos to move forward. Fedora-related repositories are arranged with "f[$releasever]-[$basearch]". In Fuduntu this variable was "2013-i386". This special variable worked in Fuduntu repos, but fails in 3rd party repos -- $releasever, needs to be changed to something standard, such as "17", to create a more Fedora-standard "f17-i386" variable.

    But nothing worked. Not exporting the variable $releasever=17 to the kernel, not changing /etc/yum.conf, not swapping out the value of "2013" for "17" in each /etc/*release. Nothing I could find globally changed this variable. Eventually, after a couple of lost days on the project, I gave up and brute forced the repo files individually. Before modifying, I eliminated the old cache and backed-up all the unmodified repos into a new directory I called "default". Then I modified the repos, exchanging "$releasever" for "17" in each file.
    # rm -r /var/tmp/*

    # mkdir /etc/yum.repos.d/default
    # cp /etc/yum.repos.d/* /etc/yum.repos.d/default/


    # sed -i 's/$releasever/17/g' ./etc/yum.repos.d/*
    The repos finally loaded.

    (non)solution - remove IPv6 functionality

    Link: Disabling IPv6

    This is not a solution, because curl/rpm/yum needs IPv6 and IPv4 information and does not get what it needs with the process below. I'm including this info however, because it provides insight into how other TCP clients (ie, not strictly curl/rpm) can be assisted in a mixed A/AAAA environment. Some readers are interested in a Chromium,etc.
    # nano /etc/sysconfig/network
    NETWORKING_IPV6=no

    # nano /etc/modprobe.d/blacklist.conf
    blacklist nf_conntrack_ipv6
    blacklist nf_defrag_ipv6


    Appendix 1 - wireshark

    A good link for using wireshark to check DNS problems. The wireshark GUI is more elegant than my CLI approach above, and arguably more user-friendly for those working on IPv4 /IPv6 solutions.

    Appendix 2 - strace

    Straces are too long to regurgitate here; let's look at the 14 relevant lines where ping succeeds and curl is unsuccessful. Of possible interest here is that, from inside the LAN, the DNS server, via DHCP, is simply the gateway at "192.168.1.254".
    $ strace ping www.websense.com
    socket(PF_INET, SOCK_DGRAM|SOCK_NONBLOCK, IPPROTO_IP) = 3
    connect(3, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("192.168.1.254")}, 16) = 0
    gettimeofday({1373430779, 7870}, NULL) = 0
    poll([{fd=3, events=POLLOUT}], 1, 0) = 1 ([{fd=3, revents=POLLOUT}])
    send(3, "\346\f\1\0\0\1\0\0\0\0\0\0\3www\10websense\3com\0\0\1"..., 34, MSG_NOSIGNAL) = 34
    poll([{fd=3, events=POLLIN}], 1, 5000) = 1 ([{fd=3, revents=POLLIN}])
    ioctl(3, FIONREAD, [50]) = 0
    recvfrom(3, "\346\f\201\200\0\1\0\1\0\0\0\0\3www\10websense\3com\0\0\1"..., 1024, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("192.168.1.254")}, [16]) = 50
    close(3) = 0
    socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 3
    connect(3, {sa_family=AF_INET, sin_port=htons(1025), sin_addr=inet_addr("204.15.67.11")}, 16) = 0
    getsockname(3, {sa_family=AF_INET, sin_port=htons(53553), sin_addr=inet_addr("192.168.1.20")}, [16]) = 0
    close(3)

    And for the failing curl :
    $ strace curl www.websense.com
    socket(PF_INET, SOCK_DGRAM|SOCK_NONBLOCK, IPPROTO_IP) = 3
    connect(3, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("192.168.1.254")}, 16) = 0
    gettimeofday({1373429425, 713068}, NULL) = 0
    poll([{fd=3, events=POLLOUT}], 1, 0) = 1 ([{fd=3, revents=POLLOUT}])
    sendmmsg(3, {{{msg_name(0)=NULL, msg_iov(1)=[{"\215s\1\0\0\1\0\0\0\0\0\0\3www\10websense\3com\0\0\1"..., 34}], msg_controllen=0, msg_flags=0}, 34}, {{msg_name(0)=NULL, msg_iov(1)=[{"\34\305\1\0\0\1\0\0\0\0\0\0\3www\10websense\3com\0\0\34"..., 34}], msg_controllen=0, msg_flags=0}, 34}}, 2, MSG_NOSIGNAL) = 2
    poll([{fd=3, events=POLLIN}], 1, 5000) = 1 ([{fd=3, revents=POLLIN}])
    ioctl(3, FIONREAD, [34]) = 0
    recvfrom(3, "\34\305\200\0\0\1\0\0\0\0\0\0\3www\10websense\3com\0\0\34"..., 2048, 0, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("192.168.1.254")}, [16]) = 34
    close(3) = 0
    socket(PF_INET, SOCK_DGRAM|SOCK_NONBLOCK, IPPROTO_IP) = 3
    connect(3, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("192.168.1.254")}, 16) = 0

    Curl never seems to leave port 53, and it also appears curl may have actually received the IP of the DNS server in response to its query to that selfsame DNS server. Perhaps this is due to curl embedding its request inside a more complex sendmmsg routine, as opposed to ping's simpler send routine. Additionally, ping uses a getsockname process not used by curl.

    More information: while Epiphany is running, we check to see what calls are creating errors. Get its PID, open a terminal, and let strace run for several seconds while attempting to surf to an address in Epiphany. Then CTRL-C out and examine the data,eg....
    $ strace -c -p 13881
    Process 3811 attached
    ^CProcess 3811 detached
    % time seconds usecs/call calls errors syscall
    ------ ----------- ----------- --------- --------- ----------------
    54.08 0.000384 0 2167 writev
    14.79 0.000105 5 20 munmap
    13.24 0.000094 0 1526 clock_gettime
    13.24 0.000094 0 6690 4567 recv
    4.65 0.000033 0 4583 poll
    0.00 0.000000 0 1 restart_syscall
    0.00 0.000000 0 80 9 read
    0.00 0.000000 0 31 write
    0.00 0.000000 0 36 open
    0.00 0.000000 0 36 close
    0.00 0.000000 0 4 unlink
    0.00 0.000000 0 18 access
    0.00 0.000000 0 8 rename
    0.00 0.000000 0 262 gettimeofday
    0.00 0.000000 0 1 clone
    0.00 0.000000 0 14 _llseek
    0.00 0.000000 0 21 mmap2
    0.00 0.000000 0 58 46 stat64
    0.00 0.000000 0 84 8 lstat64
    0.00 0.000000 0 36 fstat64
    0.00 0.000000 0 2 1 madvise
    0.00 0.000000 0 70 6 futex
    0.00 0.000000 0 1 statfs64
    ------ ----------- ----------- --------- --------- ----------------
    100.00 0.000710 15749 4637 total
    Blog formatting squishes the data a little, but we see significant errors(4567 of them) on "recv" calls, as well as some on "stat64" and a few others.

    Wish I could write in C and recompile curl.

    Saturday, July 6, 2013

    fuduntu - yum/rpm stuff

    links: yum setup :: yum command examples

    why yum/rpm?

    In the previous post, I noted an install of Fuduntu, a recently orphaned distro. The idea was to establish a simple system which might retain its stability against the tide of media player updates. An expected side benefit was the opportunity to learn yum. Specifically, it's necessary to modify Fuduntu's default yum files because Fuduntu's yum URL's are now invalid. Note: the final Fuduntu release appears to be based on Fedora 17 (or roughly Enterprise Linux 6), plus some Ubuntu features.

    /etc/yum.conf

    No notable differences between Fedora and Fuduntu in the global /etc/yum.conf file. Standard.

    repos to remove - /etc/yum.repos.d/

    The helpful System > Administration > Software Sources window:


    We can see the Fuduntu repository entries are there, though they no longer point anywhere. To follow good housekeeping, toggle enable to "enable=0" inside each undesired repo entry in /etc/yum.repos.d/. To be more thorough, we could delete unwanted repo files(in /etc/yum.repos.d/) and gpg keys (in /etc/pki/gpg-keys/). In my simple world, I renamed all the repos with a .bak extension and kept them -- I might want to look inside one later. This also allows me to change the extension back to ".repo" if I wanted to reactivate one for some reason.
    # cd /etc/yum.repos.d/
    # rename .repo .bak *.repo

    repos to add

    As noted at the top, I assumed Fuduntu's nearest releases were F17 and EL6.
    (1) REPOFORGE (formerly RPM Forge)
    # rpm -ivh http://pkgs.repoforge.org/rpmforge-release/rpmforge-release-0.5.3-1.el6.rf.i686.rpm
    However, rpm for some reason could not resolve the host and was providing the following error
    Retrieving http://pkgs.repoforge.org/rpmforge-release/rpmforge-release-0.5.3-1.el6.rf.i686.rpm
    curl: (6) Could not resolve host: pkgs.repoforge.org; Cannot allocate memory
    error: skipping http://pkgs.repoforge.org/rpmforge-release/rpmforge-release-0.5.3-1.el6.rf.i686.rpm - transfer failed
    This is not a small problem, and requires a separate post on IPv4 and IPv6 addressing, which will be my next post. Essentially, curl does the DNS and downloading for rpm/yum actions. Users can specify "--ipv4" in direct CLI curl requests, but there are no such switches when rpm calls curl as a subroutine, out of sight of the user. If it doesn't receive IPv6 information, curl fails, causing rpm to fail.

    So, for now, downloaded the rpm repo file directly via my browser, and then...
    # yum --nogpgcheck localinstall rpmforge-release-0.5.3-1.el6.rf.i686.rpm
    ...and finally
    # yum install --enablerepo=rpmforge-extras
    Looking at its file, it performs a gpg check on the downloaded packages. There are many settings available, a simple example is the one above -- to toggle a repo in a file on or off, change the state in the "enabled" line.

    (2) ATRPMS Below is the repair for curl IPv6 finickiness. Then one can directly download and install the file using rpm.
    [verify]
    # nano /etc/host.conf
    order hosts,bind

    [enter]
    # nano /etc/hosts
    ::ffff:a02d:fe1a packages.atrpms.net
    160.45.254.26 packages.atrpms.net
    ::ffff:a02d:fe16 dl.atrpms.net
    160.45.254.22 dl.atrpms.net

    [test]
    # curl --ipv6 packages.atrpms.net

    [install key]
    # rpm --import http://packages.atrpms.net/RPM-GPG-KEY.atrpms

    [install]
    # rpm -ivh http://dl.atrpms.net/all/atrpms-repo-17-6.fc17.i686.rpm
    Retrieving http://dl.atrpms.net/all/atrpms-repo-17-6.fc17.i686.rpm
    Preparing... ########################################### [100%]
    1:atrpms-repo ########################################### [100%]

    # ldconfig

    # yum clean all

    (3) RPMFUSION
    [verify]
    # nano /etc/host.conf
    order hosts,bind

    [enter]
    # nano /etc/hosts
    ::ffff:c11c:eb3c download1.rpmfusion.org
    193.28.235.60 download1.rpmfusion.org

    [test]
    # curl --ipv6 download1.rpmfusion.org

    [install]
    # rpm -ivh http://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-17.noarch.rpm
    Retrieving http://download1.rpmfusion.org/free/fedora/rpmfusion-free-release-17.noarch.rpm
    warning: /var/tmp/rpm-tmp.QSSeST: Header V3 RSA/SHA256 Signature, key ID 8296fa0f: NOKEY
    Preparing... ########################################### [100%]
    1:rpmfusion-free-release ########################################### [100%]

    # ldconfig

    # yum clean all

    manual repo pointing

    Let's create a new repo file, fedora1.repo. Permissions of the file should be 644...
    # chown 644 /etc/yum.repos.d/fedora1.repo
    ...it should appear in System > Administration > Software Sources:


    quick test - yum repolist

    Fast way to check if everything is operating correctly


    NOTES: gpg keys

    On first cut, the new repository did not work. Note that in the above repo listing, we requested a gpg check and provided the folder to find it...
    gpgcheck=1
    gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-$basearch
    ... but then of course we only had Fuduntu keys in that folder, no Fedora keys. Further, the Fedora project has moved on, it's no longer operating repos for Fedora 17. So we could use rpm to import the keys and automatically create softlinks... /etc/pki/rpm-gpg/...
    # rpm --import RPM-GPG-KEY-fedora-17-primary
    # rpm --import RPM-GPG-KEY-fedora-17-secondary
    ...but Fedora does not maintain older packages, so we'd have the keys to nothing. We could even add the old F17 keys manually...
    # gedit /etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-17-primary
    ...post the key block from the link into that file, then made a softlink for housekeeping...
    # ln -s /etc/pki/rpm-gpg/RPM-GPG-KEY-fedora-17-primary /etc/pki/rpm-gpg/RPM-GPG-KEY-fedora
    ...but again this would be pointless since no F17 software is available at Fedora.

    There are also these sorts of possible yum problems.