Make gen_helptags.lua call help.lua instead of keeping its own tag
parser. Builds and :helptags now use the same code.
Allow the shared code to run without Neovim, and fail the build when tag
generation reports an error.
AI-assisted
Problem: Generating helptags builds and queries a full vimdoc syntax
tree. This is much slower than the previous scanner and also happens
before discovering that the tags file cannot be written.
Solution: Extract tags with an LPeg grammar, reading each file once and
skipping example blocks and ordinary text. Open the output first so
unwritable directories can return immediately.
This restores legacy handling of tags inside inline code spans. On an
arm64 Mac, the 5.36 MB runtime corpus takes 24 ms instead of 604 ms,
with identical generated tags.
AI-assisted
Enable checks for unused code, deprecated calls, return types, and
annotations.
Fix the warnings and replace the deprecated highlight function. Keep
exceptions where old APIs are still needed or the checker gets it wrong.
AI-assisted
Problem: K on generic and nullable type annotations can jump to the
wrong help tag. Generated See references repeat the type names.
Solution: Strip generic arguments and nullable suffixes when resolving
help tags. Remove the redundant generated See references.
AI-assisted
Problem:
The vimdoc parser does not treat "\r" as whitespace, so in a CRLF
helpfile the codeblock rule fails and "*" pairs inside examples become
tags. On Windows this generates duplicate "." and "/" tags:
D:/a/neovim/neovim/build/bin/nvim.exe -u NONE -i NONE -e --headless -c "helptags ++t doc" -c "exe 'cquit' !empty(v:errmsg)""
Error in command line:
E154: Duplicate tag "." in gui.txt and repeat.txt
E154: Duplicate tag "/" in pattern.txt and usr_08.txt
E154: Duplicate tag "/" in usr_08.txt and pattern.txt
Note: The old C parser was unaffected because it read helpfiles in text
mode (`os_fopen(…, "r")`).
Solution:
Strip "\r" before parsing.
Problem:
Parent commit regressed some behavior of the old C helptags-gen impl:
- helpfiles in sub-directories are named by basename, so :help fails
- "help-tags" always names "tags", never "tags-nl"
- E150 E151 E152 E153 are never reported
- existing tags file is not overwritten if no tags were found
- a duplicate tag aborts the run, skipping the remaining directories
- `*.TXT` and `*.FRX` (uppercase) are not recognized as helpfiles
- tree-sitter-vimdoc accepts tags that the C parser rejected: `*a|b*`
(breaks |links|) and unterminated `*tag`
- PUC Lua "<" compares with (localized) `strcoll()`, so the tags file is
not sorted by byte value (E432)
- requiring `vim.treesitter` at load time breaks :help itself, not just
:helptags, where the module is unavailable
- `vim.pack` runs :helptags for plugins that have no "doc/" directory,
so every install/update emits E150
Solution:
- Fail the build if generating helptags reports any `v:errmsg`.
- Restore old behavior: tags generated for runtime/doc are now identical
to those from the C implementation. Errors are non-fatal messages
instead of exceptions, so all directories are still processed.
- Load treesitter lazily, report a plain error if parser is missing.
Problem:
Tags are manually parsed in C which is not flexible and prone to errors.
Extending the help system to allow for other formats (e.g. Markdown)
would require a large rewrite in the C core, while with Treesitter it
only needs a query update.
Solution:
Use the power of treesitter to extract the tags from helpfiles.
- build: set `$VIMRUNTIME` when generating helptags, like
`cmake/Util.cmake` already does for other generators.
- fix(help): only accept tags delimited by whitespace. The old C parser
only accepted a `*tag*` preceded by start-of-line or whitespace and
followed by whitespace or end-of-line. The vimdoc parser also captures
tags followed by other text, e.g. `*$XDG_STATE_HOME*/.../logs` in
starting.txt, which caused an E154 duplicate tag error for docs that
were previously fine.
* fix(help): preserve multi-letter CTRL modifiers in :help subject
Problem: After PR 39537 normalized `CTRL-U-default` by excluding
`-` from the trigger char class, the regex `(CTRL%-[^{])([^-_\\])`
also fires between `CTRL-S` and the `H` of the `SHIFT` modifier
in tags like `c_CTRL-SHIFT-V`, splitting it into the non-existent
`c_CTRL-S_HIFT-V`.
Solution: Exclude uppercase letters from the char class so a
multi-letter modifier following `CTRL-X` (e.g. `SHIFT`, `ALT`) is
no longer split. The `-` exclusion from 39537 is preserved so
`i_CTRL-U-default` still works.
* fix(help): escape ~ in expr-=~? exception to keep regex valid
Problem: The `expr-=~?` exception in the tag alias table maps to
`=~?`, but in Vim's regex engine `~` refers to the previous
substitute pattern. With no prior substitute the pattern fails to
compile, so `:help expr-=~?` returns E149 even though the tag exists.
Solution: Escape `~` as \\\~` so the alias expands to a valid
regex (`=\\\~?`) that substring-matches the `expr-=~?` tag. Other
`expr-X?` entries are unaffected since they don't contain `~`.
* fix(help): don't insert _ before CTRL- when preceded by -
Problem: The rule `([^_])CTRL%- -> %1_CTRL-` inserts an underscore
between any non-underscore char and `CTRL-`, which wrongly rewrites
tags like `map-CTRL-C` and `telnet-CTRL-]` (where `CTRL-` is
literal text in the tag, not a key chord).
Solution: Exclude `-` from the trigger char class so a hyphen
immediately before `CTRL-` is preserved. The existing behavior for
`iCTRL-GCTRL-J` -> `i_CTRL-G_CTRL-J` is unaffected because the
`G` before `CTRL-J` is not `-`.
* fix(help): only treat ^X as caret notation for valid control chars
Problem: The regex `%^([^_])` converts any `^X` to `CTRL-X`,
including `:set^=` where `^=` is literal text in the tag, not a
caret-notation control character.
Solution: Restrict the transformation to caret-notation chars
(`%a` or one of `?@[\\]^{}`). This matches the C reference
behavior where `^X` is only converted when X is alphanumeric or one
of the control-notation punctuation chars. `{` is retained so
`^{char}` still maps to the `CTRL-{char}` placeholder tag.
Problem:
The argument to `:help` is normalized to fit the general tag format.
I.e. i^U-default, iCTRL-U-default and i_CTRL_U-default should all point
to the i_CTRL_U-default tag. Our normalization adds an underscore around
the CTRL keycode, e.g. iCTRL-GCTRL-J becomes i_CTRL-G_CTRL-J. That's not
necessary if the following part starts with a dash, like the case of
iCTRL-U-default.
Solution:
Do not insert an underscore if the following character is a dash/minus
(-).
Problem:
`K` in help files may fail in some noisy text. Example:
(`fun(config: vim.lsp.ClientConfig): boolean`)
^cursor
Solution:
- `:help!` (bang, no args) activates DWIM behavior: tries `<cWORD>`,
then trims punctuation until a valid tag is found.
- Set `keywordprg=:help!` by default.
- Does not affect `CTRL-]`, that is still fully "tags" based.
Problem:
Descriptions of plugins often contain taglinks which are generally
concealed. This misaligns them by 2 characters with descriptions that
don't have a taglink in them.
Solution:
Don't count "bar" characters (`|`) for the description width.
Example:
Actual buffer content:
```
myplugin.txt |lsp| is cool
myplugin.txt this is a nice plugin
```
Rendered as:
```
myplugin.txt lsp is cool
myplugin.txt this is a nice plugin
```
Problem:
- ~200 line function of hard-to-maintain C code.
- Local Addition section looks messy because of the varying description
formats.
Solution:
- Move code to Lua.
- Have a best-effort approach where short descriptions are right
aligned, giving a cleaner look. Long descriptions are untouched.
Problem:
Escaping logic for {subject} in ex cmd `:help {subject}` is done in a
messy 200+ lines C function which is hard to maintain and improve.
Solution:
Rewrite in Lua. Use `string.gsub()` instead of looping over characters
to improve clarity and add many more tests to be able to confidently
improve current code later on.
Lewis Russell