Fish responsiveness in interactive mode is slow

[thegreyshadow]

• Have you checked if problem occurs with fish 2.3.1?
• Tried fish without third-party customizations (check sh -c 'env HOME=$(mktemp -d) fish')?

fish version installed (fish --version):
2.3.1

OS/terminal used:
Slackware Linux, 64-bit, -current series

Output of "uname -a":

Linux [hostname] 4.4.23 #1 SMP Fri Sep 30 21:14:07 CDT 2016 x86_64 Intel(R) Core(TM) i7-4510U CPU @ 2.00GHz GenuineIntel GNU/Linux

Terminal: yakuake, using konsolepart5 (KDE Plasma)

Talk about the the issue here.

I like fish and made it the default shell for my user.
The experience so far it's ok but I find it very slow in its responsiveness in interactive use.

Most of the time, response between a typed command and its response is measured in seconds.

Even clearing out the terminal screen by issuing Ctrl-L takes two seconds (!)

This is even despite the fact that I have a reasonably fast CPU (see uname output, above) and 12 GB RAM. In this machine bash flies, but fish does not.

Reproduction steps

1. step one
2. …

(not necessary)

Thanks again for developing fish. I think it's a fantastic product. Hope you guys keep improving it.

[faho]

This seems really unusual. Do you have anything weird in your configuration, particularly any event handlers? Any "frameworks" - OMF or fisherman or so?

Can you test this in another terminal?

Is it just when launching a command or when typing?

[faho]

Also, are you using NFS or any other filesystem that might be slow?

[thegreyshadow]

Hi @faho, thanks for answering.

1. I don't have any weird config. I just use fish as vanilla as I can. And Slackware is as close to the bare metal as possible. No OMF or fisherman, either.
2. Tested in xterm, mate-terminal, xfce4-terminal, QTerminal, rxvt, sakura and cool-retro-term. Same behavior observed. Even when I just press a carriage return at the prompt (i.e., no command typed, just a carriage return), the prompt sometimes takes between 1 or 2 seconds to reappear. In rxvt (and only on it) I observed a "tofu" character inserted after I pressed carriage return and before fish shows the prompt again.
3. When launching a command. That is, the slow response happens to responses that fish should display after the operator has pressed the carriage return.
4. No, I'm not using NFS. I'm using ext4 on a SSD drive. Not really slow and in fact, when using bash or ksh this slowness does not show at all.

Thanks again for the kind response.

[floam]

Could you make a screen recording on asciinema.org? Or otherwise?

[faho]

The most likely thing is that the prompt is slowing this down - you don't happen to have your $HOME in a hg repository, perchance?

This can be tested by redefining the fish_prompt function, like function fish_prompt; echo ">"; end.

If that's not it, do you have vi-mode enabled?

Also a recording (text or gif) of a fish -d5 session might be helpful.

[thegreyshadow]

Will try.
Check this: https://asciinema.org/a/f52wznwl2pxrzfb2m68a4qoc7

[thegreyshadow]

This is a record of fish -d5: https://asciinema.org/a/5xix93qsbkisvokwy4ge4guio

[thegreyshadow]

hg repository: no idea of what's that.

Now, redefining fish_prompt makes it faster and responsiveness looks close or equal to normal. Bingo!

[thegreyshadow]

So it appears that the prompt is responsible. I just chose one of the suggested prompts in fish_config prompt. You can see in my recordings of asciinema.org the one I chose.

[ridiculousfish]

Nice detective work. I wonder if mercurial is like git, which walks all the way up the parent directory hierarchy looking for a repository.

[faho]

@ridiculousfish: I don't think it's mercurial, though that can be rather slow. (The main cost is usually just starting it, which is why I've written custom code to determine if you're inside a repo)

[faho]

@thegreyshadow: Which prompt did you pick? Can you run fish --profile prompt.prof -c 'fish_prompt' and upload "prompt.prof"?

[thegreyshadow]

@faho I chose this one: [eduardo@loli ~]$ '. It's one of the suggestions after typingfish_config prompt`. I don't even want the hostname to be displayed. I'm still in the very early phase of the learning curve (in plain English: i'm a total n00b).

prompt.prof uploaded after converting it to txt due to github's restrictions
prompt.prof.txt
.

[thegreyshadow]

Also I did this:

[eduardo@loli ]$ type fish_prompt
fish_prompt is a function with definition
function fish_prompt --description 'Write out the prompt'
set -l home_escaped (echo -n $HOME | sed 's///\//g')
set -l pwd (echo -n $PWD | sed "s/^$home_escaped//" | sed 's/ /%20/g')
set -l prompt_symbol ''
switch $USER
case root toor
set prompt_symbol '#'
case '*'
set prompt_symbol '$'
end
printf "[%s@%s %s%s%s]%s " $USER (hostname -s) (set_color $fish_color_cwd) $pwd (set_color normal) $prompt_symbol
end
[eduardo@loli ~]$

[faho]

Eureka!

I've actually seen this one before! sort -nk 2 prompt.prof.txt shows the lines sorted by sum-time, and it shows this:

55336   55424   --> hostname -s
196     55873   -> printf "[%s@%s %s%s%s]%s " $USER (hostname -s) (set_color $fish_color_cwd) $pwd (set_color normal) $prompt_symbol
112     61290   > fish_prompt

hostname -s. Do you not have your hostname in /etc/hosts?

[faho]

See #2854.

[thegreyshadow]

hostname -s brings the text string "loli"
Yes, I have my hostname in /etc/hosts.

[thegreyshadow]

# For loopbacking.
127.0.0.1               localhost
127.0.0.1               loli.sombragris.org loli

[faho]

Does /etc/hostname agree with /etc/hosts on your hostname? Do you have any hostname resolver setup?

The data says that hostname -s accounts for 90% of your prompt time.

# For loopbacking. 127.0.0.1 localhost 127.0.0.1 loli.sombragris.org loli

I'm assuming that's not on the same line?

[thegreyshadow]

Sorry. Will try to put it correctly:

# For loopbacking.
127.0.0.1               localhost
127.0.0.1               loli.sombragris.org loli

Sorry for the inconvenience.

[faho]

What does /etc/HOSTNAME say? Apparently, that's the slackware file.

[thegreyshadow]

Output:

[eduardo@loli ~]$ cat /etc/HOSTNAME 
loli.sombragris.org
[eduardo@loli ~]$ 

[faho]

Okay, I'm not actually sure how slack has this setup, so it'd be better if you ask on a slackware forum (I believe linuxquestions.org does a lot of slackware?).

I think we've cleared up why your prompt is slow, so the fish part of this is solved.

[thegreyshadow]

@faho I concur. I even don't need my hostname to be displayed so I will see how I could get rid of it.
However, the fact remains that in bash I don't think this delay happens. Slackware's default bash prompt shows the hostname but there is no delay like in fish. But anyway, is a matter of little importance.

I will look on how to get rid of the hostname in my fish prompt.
Thanks for your patience! I think you may close the issue.

[faho]

However, the fact remains that in bash I don't think this delay happens. Slackware's default bash prompt shows the hostname but there is no delay like in fish. But anyway, is a matter of little importance.

It probably doesn't do the equivalent of hostname -s, or maybe slackware's hostname command is just slow and bash has another implementation?

[thegreyshadow]

@faho just found two things now:

1. Bash has the hostname as an escape sequence which could be used in the prompt: \h -- Check http://www.tldp.org/HOWTO/Bash-Prompt-HOWTO/bash-prompt-escape-sequences.html
2. I found that hostname -s is way slower than just hostname but both commands give the same output in my system.

Thanks again!

[thegreyshadow]

@faho I just want to say that I solved my particular problem getting rid of the hostname:

function fish_prompt --description 'Write out the prompt'
    set -l home_escaped (echo -n $HOME | sed 's/\//\\\\\//g')
    set -l pwd (echo -n $PWD | sed "s/^$home_escaped/~/" | sed 's/ /%20/g')
    set -l prompt_symbol ''
    switch $USER
        case root toor
            set prompt_symbol '#'
        case '*'
            set prompt_symbol '$'
    end
    printf "[%s %s%s%s]%s " $USER (set_color $fish_color_cwd) $pwd (set_color normal) $prompt_symbol
end

This gives me this prompt:

[eduardo ~]$ 

which is exactly what I wanted in the first place.
Again, thanks for your help and patience in sorting this out.

[floam]

Maybe we should offer a $hostname electric var.

[krader1961]

Maybe we should offer a $hostname electric var.

Fish currently only fetches the hostname for naming the universal var file. And the consensus is we should stop doing that. Also, it's pretty unlikely that any gethostname() call or equivalent done by fish will be faster than running the hostname command (ignoring the couple hundred microseconds needed to launch an external command). Too, some prompt performance problems have been in other areas such as fetching SCM data (e.g., from Mercurial).

Every "responsiveness" issue I can recall being opened in the past year has involved the prompt. So I'm wondering if perhaps we shouldn't just emit a one time warning if constructing the prompt takes a long time. Say more than 300 ms.

[floam]

I think it's certain that we can get the hostname faster on our own than by forking an external process. Also consider that hostname -s or hostname doesn't really work everywhere.

[thegreyshadow]

Well, in Slackware hostname is provided by net-tools 1.6x. In my particular case, invoking hostname without any switch gives the same result than invoking hostname -s, but the execution is much faster.

Asciinema link showing the above: https://asciinema.org/a/2al3brwb31migfkkwd6sg1hwx

[krader1961]

Of course we can get the hostname faster by avoiding the overhead of launching an external command. Slow down and re-read what I wrote. That overhead is normally inconsequential. Too, the speedup by avoiding that overhead is completely irrelevant if there is a problem that makes host name resolution slow.

[ridiculousfish]

Why is hostname slow? Is it doing reverse DNS or something wacky?

[thegreyshadow]

In my system hostname isn't slow but hostname -s is.

This might hint at a reason for it: http://www.linuxquestions.org/questions/slackware-14/hostname-vs-hostname-s-4175592010/#post5621658

[faho]

@thegreyshadow: What happens when you add another line with just the short name?

I.e.

127.0.0.1 loli

In any case, it's rather weird that hostname -s would be doing a dns lookup here.

[thegreyshadow]

@faho I did it and there is no difference. hostname -s still takes much longer than just hostname OR bash's \h escape sequence.

[faho]

@thegreyshadow: Can you try the inetutils version of hostname?

[thegreyshadow]

@faho since it's such a low level system utility I would be reluctant to do so. But I'll check it out and see what happens.

[krader1961]

It's really weird that hostname -s is slower than just hostname. All the -s modifier is supposed to do is strip the domain from the hostname. An operation that should add no more than a microsecond to the run time. I don't see any justification for modifying fish to workaround whatever is going on with the O.P.'s system. There are simple solutions people can use in these types of situations; e.g., create a ~/bin/hostname script that calls the real hostname without the -s flag.

[thegreyshadow]

tested inetutils-1.9.4 compiled locally

Output:

[eduardo: ~/Downloads/inetutils/inetutils-1.9.4/tests]$ ./hostname.sh 
loli
hostname: skipping tests to set host name

the hostname appears rapidly but then the script (since it's a script, hostname.sh appears to take a little more time doing tests, then prints that skipping tests... part and exits.

[thegreyshadow]

@krader1961 I know next to nothing about network utilities (or any systems programming for that matter. I'm just a n00b and a user). You might be right, but the fact remains that this is something that is perceived in fish but not in bash. I don't know how bash implements its \h escape sequence but is faster than hostname -s.

[krader1961]

Bash implements \h by calling gethostname(). Although in actuality it calls the function exactly once when it starts and caches the value. Note that the hostname command should be using the same function to get the host name. So in all likelihood bash just seems faster because it only fetches the host name once when it starts running. Note that most (all?) of the prompts that we package with fish use this pattern to avoid calling hostname more than once since DNS lookups can be slow (from share/functions/fish_prompt.fish):

# Just calculate this once, to save a few cycles when displaying the prompt
if not set -q __fish_prompt_hostname
        set -g __fish_prompt_hostname (hostname|cut -d . -f 1)
end

Then it uses $__fish_prompt_hostname in the prompt string.

[floam]

This, plus things like hostname behavior on Solaris, and Cygwin, all makes to me a compelling case for an electric var.

[thegreyshadow]

@krader1961 In net-tools 1.6x hostname also just calls gethostbyname(), but hostname-s does a lot of other things and calls a lot of programs. Fish could just use hostname instead of hostname -s or implement what you propose, setting $__fish_prompt_hostname.

[krader1961]

but hostname -s does a lot of other things and calls a lot of programs.

That deserves a "WTF?" response. I have a hard time believing that's true.

Fish could just use hostname instead of hostname -s

No, because the two are not equivalent. The -s strips the domain. For example, on my system where I'm typing this hostname => macbook.skepticism.us and hostname -s => macbook.

or implement what you propose, setting $__fish_prompt_hostname.

That's what fish_prompt.fish and fish_fallback_prompt.fish already do. As do most, but not all, of the sample prompts. I'll make a change to fix the prompts which run hostname every time the prompt is displayed.

For the record the prompt you chose is from share/tools/web_config/sample_prompts/user_host_path.fish and is obviously one of the sample prompts that is doing it the slow way.

[thegreyshadow]

@krader1961

but hostname -s does a lot of other things and calls a lot of programs.
That deserves a "WTF?" response. I have a hard time believing that's true.

Well, check for yourself at the linuxquestions.org thread I quoted above. Wish I could comment more on the merits or flaws of this but the whole issue is way above my head.

For the record the prompt you chose is from share/tools/web_config/sample_prompts/user_host_path.fish and is obviously one of the sample prompts that is doing it the slow way.

Indeed. Anyway, I didn't need the hostname and thankfully I learned how to get rid of it in the fish_prompt function.

[faho]

From that linuxquestions.org thread:

Just by comparing the strace output of both "hostname" and "hostname -s", a few things happen when using the "-s" parameter, such as opening /etc/resolv.conf, /etc/nsswitch.conf, /etc/host.conf, /etc/hosts and a handful of library calls. Perhaps that's the reason.

That's basically what it needs to do to resolve that name. This is probably the result of some misconfiguration. Usually, it should be able to figure it out from /etc/hosts alone.

[krader1961]

Closing since this has been resolved and I've opened a new issue to remind me to fix all the prompts we package with fish.

[thegreyshadow]

@krader1961 Agreed, thanks for the help and patience of all members of the fish-shell in understanding and solving the issue.