A new 'diff.<driver>.process' configuration has been introduced to
allow a long-running external process to act as a hunk provider,
enabling external tools to control which lines Git considers changed
while leaving all output formatting (word diff, color, blame, etc.) to
Git's standard pipeline.
* mm/diff-process-hunks:
diff: consult oid-only hunk providers via diff.<driver>.process
userdiff: add diff.<driver>.process config
sub-process: add a gentle status read
sub-process: separate process lifecycle from hashmap management
blame: read precomputed hunks
diff: read precomputed hunks for stat output
diff: record precomputed hunks during stat output
diff-hunks: add the store format, library, and command
diff: introduce a hunk provider interface
gitattributes: document how external diff drivers relate to diff features
The provider chain so far holds the diff-hunks store in front of the
terminal builtin computation. Open it to external processes: a pair on
a path whose driver configures diff.<driver>.process is answered by a
long-running process speaking a pkt-line protocol (following the filter
process protocol), registered at the head of the chain and consulted
before the store and before any blob is loaded.
The protocol starts with the smallest request that can carry an answer:
object names alone. A request is the pathname and the pair's
old-oid/new-oid, with no content. The process answers with hunk lines,
with a zero-hunk success that asserts the blobs equivalent (trailing
newlines included), or with status=need-content, on which the pair
falls through to the builtin answer. This serves the two shapes that
need no content pushed to them: a cache keyed on the blob pair, and a
process that fetches the blobs itself (for example over "git cat-file
--batch"). A pair whose side is not a stored blob carries a NULL id;
the provider sends no request and passes it. Because Git holds no
content for the exchange, the answer is used as sent: hunks are
validated for order, overlap, lockstep alignment, and magnitude, then
replayed without the normalization xdiff applies to diffs it computes
itself. The magnitude bound is the blobs' sizes, read from the object
database without loading content: a blob of N bytes holds at most N
lines.
Because the process's answer is authoritative, it outranks the store,
and its head-of-chain position says so. A pair the process answers
never reaches the store and is never recorded, so nothing it produces
enters the store, which holds the builtin answer only. A request it
does not answer, whether need-content, a missing capability, or a
missing id, passes down the chain to the builtin answer, which is what
the store serves, so the store may serve such a pair and a warming run
may record it. Entries recorded before a process was configured are
not purged; a pair the process answers ignores them, and "git diff-hunks
clear" discards them.
The provider gates itself per request. The driver is looked up by the
old-side path, so a renamed file resolves to the same driver, and by
the repository-relative path, so a diff.relative run from a
subdirectory names the pair the same way. Options the process is never
told about select no process: the whitespace-ignoring options, -I,
--anchored, and an algorithm forced by option or configuration (blame
routes its algorithm through xdl_opts, so --histogram is covered). The
request gains its last field, the path; the consumers change only by
filling it, and neither names the process.
The provider's state is its repository's pool of running processes,
keyed by the configured command, so drivers sharing a command share a
process, a submodule speaks to its own, and releasing the provider
(from repo_clear()) stops them. The pool owns a copy of each command
string, so an entry outlives a config re-read. A command that fails
stays as an entry that is not retried: its request and every later one
pass, so the store may serve the path for the rest of the command.
A protocol error in a response never kills the command. The response
is read through a packet reader gentle about framing, so an error takes
one path: a single warning, the process stopped and marked failed, and
the builtin diff for the rest of the command. That covers garbage
bytes, a truncated response, an empty packet, a bare status, and an
unrecognized status. Semantically invalid coordinates cost only their
pair: the response is drained, the pair is computed, and the process
stays alive. A path the protocol cannot carry (an embedded newline, or
one too long for a packet) falls back per path rather than costing the
command its process. The handshake keeps one fatal check: a process
that announces a capability Git did not request aborts the command, as
the long-running filter protocol does.
Consulting is allowed per command, following the allow_textconv
precedent. "git diff", "git log" and "git show", and "git blame" set
allow_diff_process; the plumbing diff commands and the interactive-patch
machinery never set it, so scripted and staging output stays builtin.
The options adjust the flag:
- --no-ext-diff clears it and --ext-diff sets it;
- --diff-process and --no-diff-process set and clear it alone, leaving
external diff drivers as they were;
- format-patch clears it unconditionally, so a generated patch applies
for recipients without the process;
- range-diff passes --no-ext-diff to the "git log" it compares.
git blame and the summary formats consult the process. For blame, a
pair reported equivalent emits no hunks, so the whole commit passes to
its parent. In the stat formats such a pair sums to a zero-count entry,
which the "nothing changed" rule omits, as under -w. The subprocess is
long-running: one startup cost across a traversal, one round-trip per
consulted pair. Answers travel in struct xdl_hunk, new in
xdiff-interface.h, holding xdiff's 1-based coordinates; nothing feeds
them back to xdiff, since only coordinate consumers consult.
A content-carrying request is the natural extension: it would serve
sides that are not stored blobs and processes that want content pushed
to them, and bring patch output and log -L's range tracking to the same
answer. As it stands, a process's answers show in blame and the summary
formats while patch output stays builtin.
t4080 exercises the protocol, the per-command gate, and the error paths:
- each adversarial response shape warns and falls back to builtin, the
request log proving which failures disable the process and which keep
it alive (a malformed hunk line, coordinates past the blob size, a
count overflowing strtol(), overlapping or misaligned hunks, an
unrecognized status, a bare status, an empty packet, a mid-response
crash, and raw garbage);
- a capability-less process and status=abort degrade without noise, and
a failed start warns once and returns the path to the store;
- a trailing token on a hunk line is ignored, pinning field
appendability;
- positive consults for git diff, git show, and diff-tree under
--ext-diff and --diff-process; textconv output and gitlink sides are
never identified; a diff.relative run consults by the repo-relative
path;
- the equivalence answer is pinned from both consumers, and a warming
run past a deferring process records the pair for a later read.
Helped-by: Johannes Schindelin <johannes.schindelin@gmx.de>
Signed-off-by: Michael Montalbo <mmontalbo@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Replace [verse] with [synopsis] in the SYNOPSIS block and remove
single-quote formatting from the command name.
Backtick-quote all option terms in the OPTIONS section, convert
standalone placeholders to _<placeholder>_ form, and convert
single-quoted commands and tools in prose to backtick form.
Also update the included files:
- diff-options.adoc: backtick-quote the git-format-patch-specific
option terms (-p, --no-stat, --max-depth=<depth>)
- format-patch-caveats.adoc: convert patch(1) to `patch`(1)
Signed-off-by: Jean-Noël Avila <jn.avila@free.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The documentation for "--word-diff" has been extended with a bit of
implementation detail of where these different words come from.
* mm/doc-word-diff:
doc: clarify that --word-diff operates on line-level hunks
The --word-diff documentation describes the output modes and
word-regex mechanics but does not explain that word-diff operates
within the hunks produced by the line-level diff rather than
performing an independent word-stream comparison. This can
surprise users when the line-level alignment causes word-level
changes to appear even though the words in both files are
identical.
Add an implementation note explaining the two-stage relationship
and that the output may change if Git acquires a different
implementation in the future.
Signed-off-by: Michael Montalbo <mmontalbo@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git diff -U<num>" was too lenient in its command line parsing and
took an empty string as a valid <num>.
* ty/doc-diff-u-wo-number:
diff: document -U without <n> as using default context
The documentation for '-U<n>' implies that the numeric value '<n>' is
mandatory. However, the command line parser has historically accepted
'-U' without a number.
Strictly requiring a number for '-U' would break existing tests
(e.g., in 't4013') and likely disrupt user scripts relying on this
undocumented behavior.
Hence we retain this fallback behavior for backward compatibility, but
document it as such.
Signed-off-by: Tian Yuchen <cat@malon.dev>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We cannot split single words like what we did in the previous
commit. That is because the doc translations are processed in
bigger chunks.
Instead write the two paragraphs with the only variations being this
configuration variable.
Reported-by: Jean-Noël Avila <jn.avila@free.fr>
Signed-off-by: Kristoffer Haugsbakk <code@khaugsbakk.name>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
git-format-patch(1) uses `format.noprefix` and ignores `diff.noprefix`.
The configuration variable `format.prefix` was added as an “escape
hatch”, and “it’s unlikely that anybody really wants format.
noprefix=true in the first place.”[1] Based on that there doesn’t
seem to be a need to widely advertise this configuration variable.
But in any case: the documentation for this option should not claim
that it overrides a config that is always ignored.
† 1: 8d5213de (format-patch: add format.noprefix option, 2023-03-09)
Signed-off-by: Kristoffer Haugsbakk <code@khaugsbakk.name>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The diff algorithm used in 'git-blame(1)' is set to 'myers',
without the possibility to change it aside from the `--minimal` option.
There has been long-standing interest in changing the default diff
algorithm to "histogram", and Git 3.0 was floated as a possible occasion
for taking some steps towards that:
https://lore.kernel.org/git/xmqqed873vgn.fsf@gitster.g/
As a preparation for this move, it is worth making sure that the diff
algorithm is configurable where useful.
Make it configurable in the `git-blame(1)` command by introducing the
`--diff-algorithm` option and make honor the `diff.algorithm` config
variable. Keep Myers diff as the default.
Signed-off-by: Antonin Delpeuch <antonin@delpeuch.eu>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Doc lint updates to encourage the newer and easier-to-use
`synopsis` format, with fixes to a handful of existing uses.
* ja/doc-lint-sections-and-synopsis:
doc lint: check that synopsis manpages have synopsis inlines
doc:git-for-each-ref: fix styling and typos
doc: check for absence of the form --[no-]parameter
doc: check for absence of multiple terms in each entry of desc list
doc: check well-formedness of delimited sections
doc: test linkgit macros for well-formedness
For better searchability, this commit adds a check to ensure that parameters
expressed in the form of `--[no-]parameter` are not used in the
documentation. In the place of such parameters, the documentation should
list two separate parameters: `--parameter` and `--no-parameter`.
Signed-off-by: Jean-Noël Avila <jn.avila@free.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When you are doing a tree-diff, there are basically two options: do not
recurse into subtrees at all, or recurse indefinitely. While most
callers would want to always recurse and see full pathnames, some may
want the efficiency of looking only at a particular level of the tree.
This is currently easy to do for the top-level (just turn off
recursion), but you cannot say "show me what changed in subdir/, but do
not recurse".
This patch adds a max-depth parameter which is measured from the closest
pathspec match, so that you can do:
git log --raw --max-depth=1 -- a/b/c
and see the raw output for a/b/c/, but not those of a/b/c/d/
(instead of the raw output you would see for a/b/c/d).
Co-authored-by: Toon Claes <toon@iotcl.com>
Signed-off-by: Toon Claes <toon@iotcl.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
- Use `backticks` for keywords and more complex option
descriptions. The new rendering engine will apply synopsis rules to
these spans.
- In description lists, put each option on its own line, to make them more
searchable and enable automatic translation of the options.
Signed-off-by: Jean-Noël Avila <jn.avila@free.fr>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We presently use the ".txt" extension for our AsciiDoc files. While not
wrong, most editors do not associate this extension with AsciiDoc,
meaning that contributors don't get automatic editor functionality that
could be useful, such as syntax highlighting and prose linting.
It is much more common to use the ".adoc" extension for AsciiDoc files,
since this helps editors automatically detect files and also allows
various forges to provide rich (HTML-like) rendering. Let's do that
here, renaming all of the files and updating the includes where
relevant. Adjust the various build scripts and makefiles to use the new
extension as well.
Note that this should not result in any user-visible changes to the
documentation.
Signed-off-by: brian m. carlson <sandals@crustytoothpaste.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
