'breakindent' seems to break gqap

[chrisbra]

Steps to reproduce

Reported by Gary on the vim-dev mailinglist

When 'breakindent' is set and the window width is less than
'textwidth', the gqap command reformats indented paragraphs to less
than 'textwidth'.

Steps to reproduce

1. Create a file containing a single paragraph of arbitrary text,
   indented by one 8-space tab stop and formatted for a textwidth
   of 78. The text below, indented by a tab stop, will do.

Lorem ipsum dolor sit amet, consectetur adipiscing elit. Etiam luctus
lectus sodales, dictum augue vel, molestie augue. Duis sit amet
rhoncus justo. Nullam posuere risus semper magna commodo scelerisque.
Duis et venenatis sem. In rhoncus augue sed tempor mattis. Mauris id
aliquet odio.

2. Open that file in vim in an 80-column terminal with this command
   (on one line):

   $ vim -N -u NONE -i NONE --cmd 'set ai tw=78 breakindent' -c vnew -c 'wincmd w' lorem_ipsum.txt
   

3. Execute

   gqap
   Ctrl-W |
   

4. Note that the paragraph has been reformatted so that the longest
   line is 70 columns. (This new text width varies with the window
   width in ways I haven't investigated.)

(Seems also to happen up to v7.4.338)

Expected behaviour

The paragraph should be formatted for a text width of
'textwidth', regardless of the setting of 'breakindent' or the
window width. It works as expected with 'breakindent' off.

Version of Vim

v9.1.366

Environment

Operating system: Ubuntu 22.04
Terminal: XTerm(389)
Value of $TERM: xterm-256color

Shell: bash 5.1.16(1)-release

Logs and stack traces

No response

[chrisbra]

Hm, we already disable 'linebreak' option when formatting text. I think we may also need to disable 'breakindent' so that counting the line length is not influenced by the additional indent from the breakindent option.

Something like this:

diff --git a/src/textformat.c b/src/textformat.c
index 41ec2cfe4..d380899c8 100644
--- a/src/textformat.c
+++ b/src/textformat.c
@@ -59,9 +59,11 @@ internal_format(
     int                safe_tw = trim_to_int(8 * (vimlong_T)textwidth);
 #ifdef FEAT_LINEBREAK
     int                has_lbr = curwin->w_p_lbr;
+    int                has_bri = curwin->w_p_bri;

     // make sure win_lbr_chartabsize() counts correctly
     curwin->w_p_lbr = FALSE;
+    curwin->w_p_bri = FALSE;
 #endif

     // When 'ai' is off we don't want a space under the cursor to be
@@ -475,6 +477,7 @@ internal_format(

 #ifdef FEAT_LINEBREAK
     curwin->w_p_lbr = has_lbr;
+    curwin->w_p_bri = has_bri;
 #endif
     if (!format_only && haveto_redraw)
     {