Printing the full output of apt-smart --list-mirrors to a file

[Sadi58]

I'd like to automate the process of selecting the best mirror via a cron job script. Probably the biggest problem is that I can't print the output of apt-smart --list-mirrors to a file fully. For example, the command apt-smart --list-mirrors > /tmp/mirrors.txt contains only the names of mirror servers - without rank, last updated and bandwidth columns, which I also need so that I can better optimize the choice of the best server, as currently apt-smart --find-best-mirror unfortunately chooses the most up-to-date server, although it may be far too slow compared with another which is several times faster while only a couple of hours behind its last update time.

[Fleshgrinder]

You can use script to fool Python’s isatty() function to think that it is connected to a TTY:

script --command 'apt-smart --list-mirrors' --return >mirrors.txt

[Sadi58]

@Fleshgrinder , thanks a million! This works perfect. Now it's time to change how to prioritize servers giving more weight to speed than the default behavior.

[Fleshgrinder]

I'm sure this can be simplified, but I think it does what we are looking for.

• Use 2>/dev/null to make it silent.
• Use head -10 to limit the mirrors.

#!/usr/bin/env bash
set -Eeuo pipefail

script --command 'apt-smart --list-mirrors' --return >&2
trap 'rm -f typescript' EXIT

declare -A urls=()
while read -r line; do
    urls["${line%%:*}"]="${line##*: }"
done < <(grep --extended-regexp '^[0-9]+:' typescript)

mirrors=
while read -r line; do
    line=${line,,}
    if [[ $line =~ .*(unknown|days?|weeks?|months?|years?).* ]]; then
        continue
    fi
    bandwidth=$(cut --delimiter='|' --fields=4 <<<"$line" | xargs)
    case $bandwidth in
    *kb/s) unit=k ;;
    *mb/s) unit=M ;;
    *gb/s) unit=G ;;
    *) unit= ;;
    esac
    bandwidth=${bandwidth%% *}
    bandwidth=${bandwidth%%.*}
    bandwidth="$bandwidth$unit"

    lag=$(cut --delimiter='|' --fields=3 <<<"$line" | xargs)
    case $lag in
    'up to date') lag=0 ;;
    *hours*) lag=$((${lag%% *} * 60)) ;;
    *) lag=${lag%% *} ;;
    esac

    url=$(cut --delimiter='|' --fields=2 <<<"$line" | xargs)
    [[ $url != *...* ]] || url="${urls["$(cut --delimiter='|' --fields=1 <<<"$line" | xargs)"]}"

    mirrors+="$bandwidth $lag $url"$'\n'
done < <(cut --delimiter='|' --fields=2,3,6,7 --only-delimited typescript | tail --lines=+4)
mirrors=${mirrors:0:-1}

sort --key=1,1hr --key=2,2n <<<"$mirrors" | cut --delimiter=' ' --fields=3

[Sadi58]

@Fleshgrinder , this looks like what I was looking for!
Does it filter out any servers that have not been updated in days or longer, and then sort out the remaining ones based on speed?
If so, maybe we could filter out those that not been updated in 10 or more (two-digit) hours as well?

[Fleshgrinder]

Sure, no problem. You can remove the | cut ... at the end of the script to see the bandwidth and lag of the mirrors that have been selected.

#!/usr/bin/env bash
set -Eeuo pipefail

script --command 'apt-smart --list-mirrors' --return >&2
trap 'rm -f typescript' EXIT

declare -A urls=()
while read -r line; do
    urls["${line%%:*}"]="${line##*: }"
done < <(grep --extended-regexp '^[0-9]+:' typescript)

mirrors=
while read -r line; do
    line=${line,,}
    if [[ $line =~ .*(unknown|days?|weeks?|months?|years?).* ]]; then
        continue
    fi

    lag=$(cut --delimiter='|' --fields=3 <<<"$line" | xargs)
    case $lag in
    'up to date') lag=0 ;;
    *hours*) lag=$((${lag%% *} * 60)) ;;
    *) lag=${lag%% *} ;;
    esac
    ((lag < 600)) || continue
    
    bandwidth=$(cut --delimiter='|' --fields=4 <<<"$line" | xargs)
    case $bandwidth in
    *kb/s) unit=k ;;
    *mb/s) unit=M ;;
    *gb/s) unit=G ;;
    *) unit= ;;
    esac
    bandwidth=${bandwidth%% *}
    bandwidth=${bandwidth%%.*}
    bandwidth="$bandwidth$unit"

    url=$(cut --delimiter='|' --fields=2 <<<"$line" | xargs)
    [[ $url != *...* ]] || url="${urls["$(cut --delimiter='|' --fields=1 <<<"$line" | xargs)"]}"

    mirrors+="$bandwidth $lag $url"$'\n'
done < <(cut --delimiter='|' --fields=2,3,6,7 --only-delimited typescript | tail --lines=+4)
mirrors=${mirrors:0:-1}

sort --key=1,1hr --key=2,2n <<<"$mirrors" | cut --delimiter=' ' --fields=3

[Sadi58]

Thanks a lot!
Then a cron script like the one below can update the servers at a set interval:

#!/usr/bin/env bash
# First check connection, waiting 3 sec for response, retry every 10 sec, and quit after 15 min
until ping -w 900 -W 3 -c 1 google.com >& /dev/null; do
	sleep 10
done
best="$(apt-smart.sh 2>/dev/null | head -1)"
current="$(apt-smart --find-current-mirror)"
if [[ ! "$current" == "$best" ]]; then
    cp "/etc/apt/sources.list" "/etc/apt/sources.list.bak"
    sed -i -e "s@$current@$best@g" "/etc/apt/sources.list"
    apt update
fi