git-blame(1) can ignore a list of commits specified via
--ignore-revs-file or the blame.ignoreRevsFile configuration variable.
This is useful for skipping uninteresting revisions such as tree-wide
formatting changes, large-scale refactors, and code modernizations that
would otherwise obscure genuine historical authorship.
When revision-ignoring was introduced in commit ae3f36dea1 ("blame: add
blame.ignoreRevsFile config option", 2019-10-18), it intentionally
avoided adopting a default ignore file. At the time, the capability was
new and unproven, so avoiding unrequested filesystem I/O or unexpected
attribution shifts took priority over a project-wide default.
Requiring explicit opt-in per clone was therefore the prudent design.
Since then, maintaining a .git-blame-ignore-revs file at the top level
of the repository has become the standard convention across the Git
ecosystem, adopted by many open source projects (such as Chromium and
LLVM) as well as hosting platforms (GitHub, GitLab, Gerrit). However,
git-blame(1) and git-annotate(1) still ignore this file unless each
user remembers to manually configure blame.ignoreRevsFile in every
local clone.
Teach git-blame(1) and git-annotate(1) to automatically add
.git-blame-ignore-revs, if it exists, as the initial element in the
list of ignore-revs files. Similar to how read_mailmap() handles
.mailmap, consult .git-blame-ignore-revs at the root of the working
tree when a working tree is available, and otherwise consult the
HEAD:.git-blame-ignore-revs blob in bare repositories. Reading only
from the working tree in non-bare repositories makes it ergonomic to
prepare a new .git-blame-ignore-revs file, edit an existing one, or
remove a committed one and immediately test the effect with
git-blame(1) before committing, while consulting
HEAD:.git-blame-ignore-revs in bare repositories allows them to
respect the committed ignore list as well.
To ensure consistent precedence, override semantics, and safety when
reading upstream-controlled content:
- The default .git-blame-ignore-revs entry is added before reading
configuration and CLI options, preserving user and repository config
overrides.
- In git_blame_config(), blame.ignoreRevsFile entries are appended via
string_list_append() rather than inserted in sorted order via
string_list_insert() so that configuration entries preserve their
order relative to the initial default entry.
- When a working tree is present, the working-tree
.git-blame-ignore-revs path is checked with lstat() and S_ISREG(),
opened with open_nofollow(), and verified with fstat() and S_ISREG()
so symbolic links and non-regular files are silently skipped. In bare
repositories, HEAD:.git-blame-ignore-revs is resolved quietly via
get_oid_with_context() and its tree entry mode is checked with
S_ISREG() before checking the object.
- In both modes, the file or blob size is capped at
IGNORE_REVS_MAX_FILE_SIZE (100MB, matching ATTR_MAX_FILE_SIZE from
3c50032ff5 ("attr: ignore overly large gitattributes files",
2022-12-01)) before reading its contents, skipping an oversized file
or blob with a warning without loading it into memory. The contents
are parsed via a new oidset_parse_buffer_carefully() helper in
oidset.c that shares line parsing with oidset_parse_file_carefully().
- In build_ignorelist(), ignore-revs entries are processed starting
after the last empty string entry. This ensures setting
blame.ignoreRevsFile to "" or passing --ignore-revs-file "" or
--no-ignore-revs-file cleanly discards the default entry without
attempting to read or parse it, allowing users to bypass a malformed
default ignore-revs file.
Update documentation in blame-options.adoc and config/blame.adoc, and
add test coverage in t8013 for the default working-tree lookup,
uncommitted and untracked .git-blame-ignore-revs files, subdirectory
invocations, bare repositories, CLI and config overrides, symbolic
links, overly large files and blobs, and comments and whitespace
handling.
Based-on-patch-by: Abhijeetsingh Meena <abhijeet040403@gmail.com>
Helped-by: Kristoffer Haugsbakk <code@khaugsbakk.name>
Helped-by: Phillip Wood <phillip.wood@dunelm.org.uk>
Helped-by: Eric Sunshine <sunshine@sunshineco.com>
Signed-off-by: Ravi Mistry <rmistry@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Currently, blame.ignoreRevsFile and --ignore-revs-file only read paths
explicitly configured by the user, which cannot point to an
attacker-controlled file under Git's threat model. An upcoming commit
will teach git-blame(1) and git-annotate(1) to automatically read
.git-blame-ignore-revs by default if it exists, exposing the
ignore-revs parser (oidset_parse_file_carefully() in oidset.c) and its
tag-peeling callback (peel_to_commit_oid() in builtin/blame.c) to
upstream-controlled content at a well-known path.
Harden both code paths before enabling the default ignore-revs file:
- In oidset_parse_file_carefully(), strbuf_getline() reads up to the
next newline and records the full line length in sb.len, including
any embedded NUL bytes. However, strchr(sb.buf, '#') and
parse_oid_hex_algop(sb.buf, &oid, &p, algop) treat sb.buf as a
NUL-terminated string. If a line contains an embedded NUL byte after
a valid object name (such as "<oid>\0garbage" or "<oid>\0# comment"),
*p is '\0' and trailing bytes on the line are silently ignored.
Reject any line containing an embedded NUL byte via memchr() before
stripping comments and whitespace.
- In peel_to_commit_oid(), odb_read_object_info() is called without
OBJECT_INFO_SKIP_FETCH_OBJECT or OBJECT_INFO_QUICK, and deref_tag()
calls parse_object() on tag targets without checking whether the
target object exists locally first. In a partial clone, any missing
commit OID or tag target listed in the ignore-revs file would trigger
lazy promisor fetches and pack directory rescans during git-blame(1).
Use odb_read_object_info_extended() with OBJECT_INFO_LOOKUP_REPLACE |
OBJECT_INFO_SKIP_FETCH_OBJECT | OBJECT_INFO_QUICK and peel OBJ_TAG
objects one layer per iteration, verifying that each target object
exists locally and matches the tag's declared type before parsing it.
Signed-off-by: Ravi Mistry <rmistry@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Introduce a new function that allows the caller to verify whether two
oidsets contain the exact same object IDs.
Note that this change requires us to change `oidset_iter_init()` to
accept a `const struct oidset`.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The `oidset_parse_file_carefully()` function implicitly depends on
`the_repository` when parsing object IDs. Fix this by having callers
pass in the hash algorithm to use.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In a following commit, we will need to add all the oids from a set into
another set. In "list-objects-filter.c", there is already a static
function called add_all() to do that.
Let's rename this function oidset_insert_from_set() and move it into
oidset.{c,h} to make it generally available.
While at it, let's remove a useless `!= NULL`.
Signed-off-by: Christian Couder <chriscool@tuxfamily.org>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
hex.c contains code for hex-related functions, but for some reason these
functions were declared in the catch-all cache.h. Move the function
declarations into a hex.h header instead.
This also allows us to remove includes of cache.h from a few C files.
For now, we make cache.h include hex.h, so that it is easier to review
the direct changes being made by this patch. In the next patch, we will
remove that, and add the necessary direct '#include "hex.h"' in the
hundreds of C files that need it.
Note that reviewing the header changes in this commit might be
simplified via
git log --no-walk -p --color-moved $COMMIT -- '*.h'`
In particular, it highlights the simple movement of code in .h files
rather nicely.
Signed-off-by: Elijah Newren <newren@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
oidset_size() just reads a single word from memory and returns it.
Avoid the function call overhead for this trivial operation by turning
it into an inline function.
While we're at it, declare its parameter const to allow it to be used
on read-only oidsets.
Suggested-by: Jeff King <peff@peff.net>
Signed-off-by: René Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Since 610e2b9240 (blame: validate and peel the object names on the
ignore list, 2020-09-24) git blame reports checks if objects specified
with --ignore-rev and in files loaded with --ignore-revs-file and config
option blame.ignoreRevsFile are actual objects and dies if they aren't.
The intent is to report typos to the user.
This also breaks the ability to use a single ignore file for multiple
repositories. Typos are presumably less likely in files than on the
command line, so alerting is less useful here. Restore that feature by
skipping non-commits without dying.
Reported-by: Jean-Yves Avenard <jyavenard@mozilla.com>
Signed-off-by: René Scharfe <l.s.r@web.de>
Reviewed-by: Barret Rhoden <brho@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The command reads list of object names to place on the ignore list
either from the command line or from a file, but they are not
checked with their object type (those read from the file are not
even checked for object existence).
Extend the oidset_parse_file() API and allow it to take a callback
that can be used to die (e.g. when an inappropriate input is read)
or modify the object name read (e.g. when a tag pointing at a commit
is read, and the caller wants a commit object name), and use it in
the code that handles ignore list.
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Occasionally, it may be useful for callers to know the number of object
IDs in an oidset. Right now, the only way to compute this is to call
'kh_size' on the internal 'kh_set_oid_t'.
Similar to how we wrap other 'kh_*' functions over the 'oidset' type,
let's allow callers to compute this value by introducing 'oidset_size'.
We will add its first caller in the subsequent commit.
Signed-off-by: Taylor Blau <me@ttaylorr.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
"git blame" learned to "ignore" commits in the history, whose
effects (as well as their presence) get ignored.
* br/blame-ignore:
t8014: remove unnecessary braces
blame: drop some unused function parameters
blame: add a test to cover blame_coalesce()
blame: use the fingerprint heuristic to match ignored lines
blame: add a fingerprint heuristic to match ignored lines
blame: optionally track line fingerprints during fill_blame_origin()
blame: add config options for the output of ignored or unblamable lines
blame: add the ability to ignore commits and their changes
blame: use a helper function in blame_chunk()
Move oidset_parse_file() to oidset.c
fsck: rename and touch up init_skiplist()
khash lets us define a hash as either a map or a set (i.e., with no
"value" type). For the oid maps we define, "oid" is the set and
"oid_map" is the map. As the bug in the previous commit shows, it's easy
to pick the wrong one.
So let's make the names more distinct: "oid_set" and "oid_map".
An alternative naming scheme would be to actually name the type after
the key/value types. So e.g., "oid" _would_ be the set, since it has no
value type. And "oid_map" would become "oid_void" or similar (and
"oid_pos" becomes "oid_int"). That's better in some ways: it's more
regular, and a given map type can be more reasily reused in multiple
contexts (e.g., something storing an "int" that isn't a "pos"). But it's
also slightly less descriptive.
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
There is no need to inline oidset_init(), as it's typically only called
twice in the lifetime of an oidset (once at the beginning and at the end
by oidset_clear()) and kh_resize_* is quite big, so move its definition
to oidset.c. Document it while we're at it.
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Reimplement oidset using khash.h in order to reduce its memory footprint
and make it faster.
Performance of a command that mainly checks for duplicate objects using
an oidset, with master and Clang 6.0.1:
$ cmd="./git-cat-file --batch-all-objects --unordered --buffer --batch-check='%(objectname)'"
$ /usr/bin/time $cmd >/dev/null
0.22user 0.03system 0:00.25elapsed 99%CPU (0avgtext+0avgdata 48484maxresident)k
0inputs+0outputs (0major+11204minor)pagefaults 0swaps
$ hyperfine "$cmd"
Benchmark #1: ./git-cat-file --batch-all-objects --unordered --buffer --batch-check='%(objectname)'
Time (mean ± σ): 250.0 ms ± 6.0 ms [User: 225.9 ms, System: 23.6 ms]
Range (min … max): 242.0 ms … 261.1 ms
And with this patch:
$ /usr/bin/time $cmd >/dev/null
0.14user 0.00system 0:00.15elapsed 100%CPU (0avgtext+0avgdata 41396maxresident)k
0inputs+0outputs (0major+8318minor)pagefaults 0swaps
$ hyperfine "$cmd"
Benchmark #1: ./git-cat-file --batch-all-objects --unordered --buffer --batch-check='%(objectname)'
Time (mean ± σ): 151.9 ms ± 4.9 ms [User: 130.5 ms, System: 21.2 ms]
Range (min … max): 148.2 ms … 170.4 ms
Initial-patch-by: Jeff King <peff@peff.net>
Signed-off-by: Rene Scharfe <l.s.r@web.de>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This is similar to using the hashmap in hashmap.c, but with an
easier-to-use API. In particular, custom entry comparisons no longer
need to be written, and lookups can be done without constructing a
temporary entry structure.
This is implemented as a thin wrapper over the hashmap API. In
particular, this means that there is an additional 4-byte overhead due
to the fact that the first 4 bytes of the hash is redundantly stored.
For now, I'm taking the simpler approach, but if need be, we can
reimplement oidmap without affecting the callers significantly.
oidset has been updated to use oidmap.
Signed-off-by: Jonathan Tan <jonathantanmy@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When using the hashmap a common need is to have access to caller provided
data in the compare function. A couple of times we abuse the keydata field
to pass in the data needed. This happens for example in patch-ids.c.
This patch changes the function signature of the compare function
to have one more void pointer available. The pointer given for each
invocation of the compare function must be defined in the init function
of the hashmap and is just passed through.
Documentation of this new feature is deferred to a later patch.
This is a rather mechanical conversion, just adding the new pass-through
parameter. However while at it improve the naming of the fields of all
compare functions used by hashmaps by ensuring unused parameters are
prefixed with 'unused_' and naming the parameters what they are (instead
of 'unused' make it 'unused_keydata').
Signed-off-by: Stefan Beller <sbeller@google.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
This is similar to many of our uses of sha1-array, but it
overcomes one limitation of a sha1-array: when you are
de-duplicating a large input with relatively few unique
entries, sha1-array uses 20 bytes per non-unique entry.
Whereas this set will use memory linear in the number of
unique entries (albeit a few more than 20 bytes due to
hashmap overhead).
Signed-off-by: Jeff King <peff@peff.net>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Ravi Mistry