Tuesday, June 12, 2018

pacman failure "404"

I suddenly began receiving 404 failures during # pacman -Syu. I quickly dug deeper using # pacman -Syu --debug 2>&1 |tee debugfile.txt and found that either curl itself (used by pacman to retrieve packages), or the way in which pacman was using curl, was the source. This is not always a bed of roses to determine and repair since they are often random glitches in what's become a decades-long, intermittent, annoying ipv6/ipv4 transition mess. This can involve tens of kernel and application settings and confusing conf files that often interact badly. For example, no longer can one edit /etc/resolv.conf directly, since it will be overwritten by resolvconf during dhcpcd initialization. One must now edit /etc/resolvconf.conf to indirectly get at /etc/resolv.conf, and resolvconf.conf has its own syntax, different from resolv.conf, so that both file syntaxes must be verified following any change. And so on.

which ipv is working?

The most obvious start is to verify both ipv4 and ipv6 curl function. Then, one can disable or activate the failed half or, possibly... force pacman to use the ip version that *is* working, (although I'm not sure if ipv type can be specified during a pacman operation).

First, the curl ipv functionality. Take a URL from one's mirror list and ping it to see that DNS is working. Then...
# curl aprs.ele.etsmtl.ca
Bienvenue | Welcome [curl works]
# curl --ipv4 aprs.ele.etsmtl.ca
Bienvenue | Welcome
[curl, and therefor pacman, is working ipv4]
# curl --ipv6 aprs.ele.etsmtl.ca
curl: (6) Could not resolve host: aprs.ele.etsmtl.ca
[curl, and therefore pacman, is failing ipv6]

In this case, it appears pacman is requesting both an ipv6 (AAAA) and ipv4 (A) handshake via curl, but of course pacman can only accomplish ipv4. Strace shows futher that IPV6 is not even able to open a socket.

Another way to check exactly what's happening with any of the kernel stuff is to do a # sysctl -A, which will reveal all the ipv4 and ipv6 settings.

And yet another way to check if ipv4 and ipv6 are both working is to run # ip addr list and look to see there's an ipv6 line for the address.

~/.curlrc

As part of the curl man page, we can see some settings for the configuration file.

In the case above, I was getting proper ipv6 configuration, but still getting a curl fail on ipv6. It turns out that I was only able to find a single post on this, here. The post indicates that it's really a glibc issue, that could only be resolved by slowing down DNS resolution via manipulations inside /etc/resolvconf.conf to manipulate /etc/resolv.conf. We want /etc/resolv.conf to say "options single-request"
and so, to achieve this, we write...
# nano /etc/resolvconf.conf
# Solve ipv4/v6 fails
resolv_conf_options="single-request"
For a list of all resolvconf.conf options, see this man page.
Disabling IPV6 in Arch is no bed of roses -- one has either to put a boot line into their GRUB, or to create an effective /etc/sysctl.d/40-ipv6.conf (see bottom). Further, we may instead need to enable ipv6 more thoroughly across all applications, not further cripple it. It's a typical ipv4/ipv6 trial and error mess due to opaque reliance upon multiple programs (eg. curl)

old fix

This one used to work, but nowadays with built-in ipv6, doesn't always work. Add a couple lines in your blacklist file, eg...
# nano /etc/modprobe.d/blacklist
# stop pacman failure when encounter ipv6 sites
blacklist nf_conntrack_ipv6
blacklist nf_defrag_ipv6

grub

Use nano or whatever to add

/etc/sysctl.d/40-ipv6.conf

Here's one example, not guaranteed to work.
# Disable IPv6
net.ipv6.conf.all.disable_ipv6 = 1
net.ipv6.conf.default.disable_ipv6=1
net.ipv6.conf.lo.disable_ipv6 = 1
net.ipv6.conf.wlan0.disable_ipv6 = 1

Sunday, June 10, 2018

[solved] pacman issues, keys

Recently attempted a pacman -Syu and had some of the typical failures of breaking dependencies and so forth, repaired by identifying orphan applications...
# pacman -Qdt
and then removing them...
# pacman -Rs [the package]
But then, attempting to do an update to my install, I found that there was not enough disk space. Something had to be accumulating logs or whatever.
$ df
Filesystem 1K-blocks Used Available Use% Mounted on
dev 1884252 0 1884252 0% /dev
run 1891812 760 1891052 1% /run
/dev/sda1 27867324 25491112 960636 97% /
tmpfs 1891812 5880 1885932 1% /dev/shm
tmpfs 1891812 0 1891812 0% /sys/fs/cgroup
tmpfs 1891812 8 1891804 1% /tmp
/dev/sda2 681201384 114146820 532451556 18% /home
So, shit, yah. Not good, 25GB of 27GB used? Let's try the usual suspect /var/pacman/cache.
# du -sh /var/cache/pacman
18G /var/cache/pacman
# pacman -Sc
Packages to keep:
All locally installed packages

Cache directory: /var/cache/pacman/pkg/
:: Do you want to remove all other packages from cache? [Y/n] y
removing old packages from cache...

Database directory: /var/lib/pacman/
:: Do you want to remove unused repositories? [Y/n] y
removing unused sync repositories...
# du -sh /var/cache/pacman
1.5G /var/cache/pacman
Pretty nice to get rid of 16GB of unused shit

install some local package

If you're in this mess, you might also have to CLI mount a USB drive. I first checked which partitions I had on the USB drive. I plugged in the USB and noted it was assigned "/dev/sdg, so I then...
# lsblk /dev/sdg
... and see there's only one partition "sdg1", so...
# mount -t vfat -o rw,users /dev/sdg1 /home
... then when done with copying the files, I umount the device.
Now you have a local package and want to install it using pacman
#
# pacman -U

signatures

We'd like pacman to verify signatures, but occasionally, there are error messages if it's been a long time since the update. Often times, I've found that simply reinstalling the keyring works.
# pacman -S archlinux-keyring

/etc/pacman.d/mirrorlist

Occasionally, mirrors go offline so that, if it's been a while since an update, pacman occasionally can't locate the previously enabled mirror. Go into this file and uncomment some other mirror or get the latest available list at the pacman mirrorlist generator.
# pacman -Syy
# pacman -Syu

ffmpeg dependencies

Another potential long period update fail, is a pacman exit due to any of several ffmpeg dependencies. This may look like...
error: failed to prepare transaction (could not satisfy dependencies)
:: ffmpeg2.8: installing libvpx (1.7.0-1) breaks dependency 'libvpx.so=4-64'
:: ffmpeg2.8: removing libx264 breaks dependency 'libx264.so=148-64'
:: ffmpeg2.8: installing x265 (2.8-1) breaks dependency 'libx265.so=146-64'
Fix...
# pacman -Rs vlc
Sometimes, this leads to a second dependency issue with phonon. If so, uninstall phonon, then vlc. Next try to update normally with the -Syu flags. This usually works, so then just re-install vlc after the update.

Pacman Key failures

Sometimes a pacman update fails due to key errors. All your apps have PGP keys now and sometimes pacman won't update due to some error in that database or so forth. Usual cure
# pacman -Sy archlinux-keyring -