The code in 'git pack-objects' used to truncate object sizes on
platforms where 'long' is 32-bit but 'size_t' is larger (like 64-bit
Windows) when handling large objects exceeding 4GiB, which has been
corrected by using 'size_t' and 'off_t' appropriately.
* ep/pack-objects-streamed-large-blobs:
pack-objects: use size_t for streamed large blobs
Various cleanups in the 'commit-graph' builtin command, removing
global state and updating to not rely on the implicit
'the_repository' variable, in preparation for making the
commit-graph subsystem pluggable.
* ps/commit-graph-cleanups:
builtin/commit-graph: de-globalize options handling
builtin/commit-graph: de-globalize write options handling
builtin/commit-graph: simplify how we wire up common options
builtin/commit-graph: drop global "--object-dir=" switch
builtin/commit-graph: get rid of `the_repository`
builtin/commit-graph: drop declaration for `read_replace_refs`
builtin/commit-graph: rename subcommand functions
The prune_packed_objects() API and its callbacks have been updated to
take a 'struct repository' pointer instead of relying on the global
'the_repository' variable, moving the subsystem closer to being fully
libified.
* sm/prune-packed-libification:
prune-packed: remove USE_THE_REPOSITORY_VARIABLE
The output from 'git init' has been corrected to display the
absolute path of the created repository, addressing a recent
regression.
* mh/init-show-abspath:
init: show absolute path of (re)initialized repository
In builtin/pack-objects.c, write_no_reuse_object() handles blobs
exceeding core.bigFileThreshold by writing them in a streaming path. It
takes the size from st->size (size_t) and assigns it to an "unsigned
long" variable, which is then written to the pack object header, and
write_large_blob_data() adds up the compressed length in an "unsigned
long" as well.
On platforms where "long" is 32 bits but size_t is 64 bits, like 64-bit
Windows, handling a blob of 4GiB or more truncates the size in the pack
header while the data decompresses to the full size, corrupting the
object. If the compressed data also exceeds 4GiB, the length
accumulation overflows "unsigned long", misaligning the offsets of all
subsequent objects in the index.
When running repack or gc on a loose large blob, the original loose copy
is pruned, leaving only the corrupted packed copy and resulting in data
loss. Truncating st->size to 16 bits, with a 100KiB loose blob reachable
from a commit and "git -c core.bigFileThreshold=1k repack -a -d", leaves
a corrupt packed object and no loose copy.
Switch the streamed path to use size_t for the object size, and off_t
for the length of the compressed data, which write_large_blob_data()
adds up while it writes the stream piece by piece, and for
write_no_reuse_object()'s return value, like write_reuse_object()
already does. Other truncations in pack-objects are not reachable
because core.bigFileThreshold is itself an "unsigned long".
Triggering this requires processing an object of 4GiB or more on a
platform where "long" is 32 bits but size_t is 64 bits, so no test is
added.
Helped-by: Simon Richter <Simon.Richter@hogyros.de>
Signed-off-by: Elia Pinto <gitter.spiros@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The object storage format has been made configurable via the
'extensions.objectStorage' configuration variable, allowing users
to specify the storage backend and an optional payload, mirroring
the design of 'extensions.refStorage'.
* ps/odb-object-storage-extension:
builtin/submodule: propagate "--object-storage-format="
builtin/clone: introduce "--object-storage-format="
builtin/init: introduce handling of object storage format
builtin: expose the object storage format
setup: allow overriding object storage format via environment
setup: introduce object storage extension
loose: drop unused `repo_write_loose_object_map()`
The tag backfilling negotiation in 'git fetch' has been updated to
explicitly include the in-flight fetched tips as 'haves', preventing
the server from resending objects when batched reference updates are
used.
* kn/fetch-tag-backfill-batch-fix:
fetch: advertise fetched refs when backfilling tags
transport: add `transport_options_release()`
t5510: cover upstream info update on fetch failure
The output of 'git repo info' has been updated to use
'references.storageFormat' instead of 'references.format' for
consistency with recent ref-storage-format terminology updates.
* ps/repo-ref-storage-format-fix:
builtin/repo: rename "references.format" to "references.storageFormat"
The 'git status' command has been optimized to acquire the index
lock only when writing the updated index, rather than holding it
during the entire working tree traversal, avoiding lock
contention. Index validation has also been improved to use stat
data to detect concurrent modifications when 'index.skipHash' is
enabled.
* mc/status-index-lock:
status: lock the index only to write it
read-cache: verify skipHash index by stat data
The ref storage format passed to git-clone(1) propagates to recursively
cloned submodules, and git-submodule(1)'s "add" and "update" subcommands
accept `--ref-storage-format=` so that newly cloned submodules can be
set up with a specific format.
Teach git-submodule(1) to also propagate the object storage format.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Wire up the new object storage format in git-clone(1) via a new
`--object-storage-format=` option, same as we have introduced in the
preceding commit for git-init(1).
Note that "--local" clones will not work if either of the object
databases is not using the "files" format. This is because we hardlink
loose objects and packfiles into the new repository, and that is
something that simply won't work with alternative backends.
Consequently, we either downgrade to a non-local clone with a warning,
or we die in case the user has explicitly asked for a local clone.
We _could_ in theory handle the case where the "files" backend has been
configured with an alternative object directory path (that is, with
"files:///path/to/dir"). But that's left for a future patch series to
implement for the sake of simplicity.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
While the object storage format extension can already be configured
manually via git-config(1), this isn't exactly an ergonomic way of doing
so. For the reference storage format extension, users have a host of
options available to them.
Introduce the equivalent options for the object storage format.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The ref storage format of a repository can be inspected via both
git-rev-parse(1)'s "--show-ref-storage-format" and git-repo(1)'s
"references.storageFormat" field. Introduce the equivalent options for
the object storage format, as well.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The --dry-run option of git reflog expire has been corrected to
report "would prune" instead of "prune" when refs are named on the
command line, bringing it in line with the behavior of the --all
option.
* lb/reflog-expire-dry-run-fix:
builtin/reflog: say "would prune" for named refs with --dry-run
The command-line completion for 'git stash' has been updated to
dynamically discover subcommands and to correct a Zsh-specific
formatting issue where stash references were improperly unquoted.
Additionally, the deprecated 'save' subcommand has been removed from
completions and synopsis documentation.
* dk/stash-completion-updates:
completion: drop "stash save" completion altogether
doc: drop usage mentions of "git stash save"
completion: fix Zsh handling of stash names
Completion: extract common stash listing code
completion: simplify stash subcommand completion
In 0e358de64a (fetch: use batched reference updates, 2025-05-19), the
fetch code was modified to use batched updates to provide a good
performance improvement. Batched updates are used both for the fetched
references and to backfill tags.
When using batched updates, the references aren't yet committed to disk
when we start backfilling tags. For shallow fetches, the negotiation
phase while backfilling tags does not advertise the newly fetched refs.
This can cause the server to send objects the client already has.
The issue always existed when the '--atomic' flag was used, as both the
regular ref updates and the tags need to be part of the same
transaction.
Fix this by advertising the already fetched refs as 'haves' during the
negotiation. Do this by simply adding the tips to the transport's
`negotiation_include_tips` field. This also fixes the issue for
`--atomic` fetches since it follows the same route.
This does mean that all references fetched before the backfill will be
used for the negotiation. While duplicate OIDs are trimmed before
negotiation, we could still end up with a lot of distinct OIDs used in
the 'haves' section. However, this is still better than the server
transferring already existing objects to the client, which can take a
lot more time.
Add two tests, which validate that the 'haves' are filled when
backfilling tags for atomic and non-atomic flows.
Reported-by: Mitja Bezenšek <mitja.bezensek@login5.org>
Signed-off-by: Karthik Nayak <karthik.188@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Same as in the preceding commit, de-globalize handling of options that
are common to both the "verify" and "write" subcommands. With this
change we don't have any global state in "builtin/commit-graph.c"
anymore.
Same as in the preceding commit, this requires us to drop the `static`
modifier for the options array as it references a local variable now.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The commit graph write options are stored as a global variable even
though they're really only required by `graph_write()` in the first
place. The only other function that needs access to it is our config
parser, but we can trivially pass it through to it via the callback
data.
Refactor the code accordingly. Note that as part of this refactoring we
introduce a struct dedicated to parsing the configuration, and that
struct only contains the write options. It's thus a bit heavy handed
right now, but we're about to add a second member in the next commit.
Note that this requires us to drop the `static` modifier for our options
array as it now references a local variable.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
All of the git-commit-graph(1) subcommands accept the "--object-dir="
option. To avoid duplication we thus have a helper function that adds
this common option to our array of command-specific options. This works
alright, but it feels rather awkward as this forces us to allocate and
free a secondary array of options.
Refactor the code to instead have common options as a preprocessor
macro that we can add. This avoids the extra allocation and is overall a
bit less verbose.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When invoking git-commit-graph(1) it is possible for the user to pass
the "--object-dir=" switch either before or after the subcommand: both
"git commit-graph --object-dir= write" and "git commit-graph write
--object-dir=" do the same thing. This calling convention is very
atypical in Git, we don't document this and none of our tests even use
the former syntax.
Furthermore, this requires us to keep track of the object directory via
global state as we don't have an easy way to pass the parsed option down
to the individual subcommands. We _could_ of course adapt the logic and
prepend the object directory to the `argv` we end up passing down. But
that feels rather pointless.
Adapt the code to only allow the "--object-dir=" option for subcommands.
This brings the command in line with how Git typically handles options,
and it avoids adding complexity in subsequent commits where we're about
to remove all global state in "builtin/commit-graph.c".
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
There are a couple of references to `the_repository` in
git-commit-graph(1). As that command cannot run outside of a repository
these references can be trivially replaced by the repository that's
passed in as parameter to `cmd_commit_graph()`.
We don't have to worry about the repository being `NULL` when called
with "-h", either: we do pass the repository to `repo_config()` before
calling `parse_options()`, but that function can handle a `NULL`
repository alright.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
We have an `extern int read_replace_refs`, but that variable is not used
anymore since d24eda4e03 (repository: create disable_replace_refs(),
2023-06-06). In fact, that variable has been converted into a file-local
variable, and users are expected to call `disable_replace_refs()` now.
Remove the declaration.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Subcommands that we wire up via `OPT_SUBCOMMAND()` are typically called
`cmd_<command>_<subcommand>()`, but we don't follow that pattern in
"builtin/commit-graph.c". Rename the functions.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The '--dry-run' option of 'git repack --drop-filtered' used to
actually rebuild packs and delete redundant ones, which has been
corrected to only list the candidates without making changes.
* ss/repack-drop-filtered-dry-run:
repack: do not rebuild packs on --dry-run
Instead of failing to record conflicts to be resolved immediately,
wait while "rerere gc" is ongoing.
* tb/rerere-wait-for-merge-rr-lock:
rerere: add "gc --skip-locked" for auto maintenance
rerere: wait for MERGE_RR.lock before giving up
Since 20e38cba39 (setup: split up concerns of `init_db()`,
2026-09-10), git-init produces the message:
Initialized empty Git repository in .git/
instead of the previous
Initialized empty Git repository in /full/path/to/repo/.git/
Prior to 20e38cba39, the code producing this message was in function
init_db (renamed at the time to create_repository). In that context,
the git_dir variable was updated by 'git_dir = repo_get_git_dir(repo);'
earlier in the function, to set the value used in printf.
None of the discussion [1] around the original patch indicates that this
change was intentional, so address the regression by similarly using
repo_get_git_dir in this code's new location.
1: https://lore.kernel.org/git/20260825-pks-odb-write-alternates-at-creation-time-v1-0-911513ba95c3@pks.im/
Signed-off-by: Matt Hunter <m@lfurio.us>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The prune_packed_objects() function uses the global
the_repository variable. To help with the libification effort,
change this function so it takes a 'struct repository' pointer
as an argument instead.
The function for_each_loose_file_in_source() uses a callback
that only takes a single 'void *data' pointer. This means we
cannot pass both the repository pointer and the 'opts' integer
at the same time. To fix this, create a new 'struct prune_data'
to hold both of them together, and update the prune_subdir() and
prune_object() callbacks to read from this new struct.
Update the builtins that call this function to pass their local
`repo` parameters. As all of those builtins are declared with
`RUN_SETUP` we know that they won't execute outside a repository
and thus the repository parameter will be set. Unless the command
was invoked with "-h" to print help, but all new uses of the `repo`
variable are after a call to `parse_options()` which knows to exit
early in case it sees "-h"
Finally, drop the USE_THE_REPOSITORY_VARIABLE macro from
prune-packed.c.
Signed-off-by: Shashank S Mogaveera <shashu8660@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The 'rerere' subsystem has been libified to remove its dependency on
the global 'the_repository' state and global tracking variables,
allowing it to operate correctly in multi-repository contexts.
* ps/rerere-remove-global-state:
rerere: get rid of global rerere cache state
rerere: bundle global state into a struct
rerere: stop using `the_repository`
rerere: cache result of `is_rerere_enabled()`
rerere: remove redundant calls to `repo_config()`
rerere: stop using `the_repository` in internal helpers
rerere: stop using `the_repository` in `rerere_path()`
The hook.<event>.enabled configuration variable has been extended to
also disable traditional hookdir hooks in addition to configured
hooks. In addition, the hook configuration parser has been updated
to treat missing values for boolean variables like enabled and
parallel as true.
* tb/hook-enabled-covers-traditional-hooks:
hook: accept .enabled and .parallel given without a value
hook: make hook.<event>.enabled cover the hookdir hook
The 'git switch' and 'git checkout' commands have been updated to
complete the branch switch instead of aborting when the tracking
configuration fails to write, matching the behavior of 'git push -u'.
* ti/checkout-track-failure:
checkout: finish switching when the tracking config cannot be written
branch: return an error from setup_tracking()
The messages 'git checkout' prints to describe orphaned commits and
multiple references have been updated to use the Q_() macro, allowing
translators to use the correct plural forms for numbers greater than
one.
* tj/checkout-orphan-more-plural:
checkout: use Q_() for messages with counts
The verbose "Checking connectivity" message in 'git fsck' has been
updated to use the pluralizing translation macro.
* ag/fsck-connectivity-msg-i18n:
fsck: use Q_() for the verbose connectivity message
The connectivity check in 'git fetch' and 'git receive-pack' has
been optimized with an alternative incremental mode, available via
'transfer.connectivityCheck=incremental'. Instead of walking the
full object closure of the boundary commits, it compares the trees
of incoming commits against their trusted parent trees, skipping
unchanged subtrees.
* kk/connectivity-check-incremental:
connected: add "auto" connectivity check mode
connected: add incremental connectivity check via rev-list
connected: add transfer.connectivityCheck config
connected: drop dependency on `the_repository`
p5413: add perf tests for connectivity check
Documentation: describe connectivity checking
The conditional configuration inclusion mechanism via 'includeIf'
has been extended to support matching the machine's hostname, and
the system's hostname as seen by Git can now be queried using 'git
var GIT_HOSTNAME'.
* ic/includeif-hostname:
config: learn the "hostname:" includeIf condition
t6500: get hostname from "git var"
var: add GIT_HOSTNAME
The 'from' command in 'git fast-import' has been updated to
correctly resolve references to newly created commits and tags by
checking the internal object table, preventing a crash when the
objects are not yet available in the object database.
* mb/fast-import-reference-pending:
fast-import: allow 'from' to reference new commits and tags by oid
The repository initialization sequence has been refactored to treat
the repository object passed to create_repository() purely as an
out-parameter.
* ps/setup-enforce-repo-passed-to-create-repository-has-no-state:
setup: enforce that passed-in repo does not carry relevant state
repository: adapt `repo_clear()` to fully reset the repository
builtin/clone: don't apply "core.sharedRepository" to leading dirs
builtin/init: move handling of "core.sharedRepository" into "setup.c"
builtin/init: refactor messy creation of leading directories
path: introduce `safe_create_leading_directories_no_share_const()`
path: drop useless `safe_create_leading_directories_1()`
'git blame' and 'git annotate' have been updated to automatically
use the '.git-blame-ignore-revs' file from the working tree (or
'HEAD:.git-blame-ignore-revs' in bare repositories) as the default
ignore list if it exists, and the parser has been hardened.
* rm/blame-default-ignore-revs:
blame: ignore revs in .git-blame-ignore-revs by default
blame: harden ignore-revs parser and tag peeling
With --dry-run --verbose, "git reflog expire --all" reports the entries
it would remove as "would prune", but when refs are named on the command
line it reports them as "prune", which looks as if they were removed:
$ git reflog expire --dry-run --verbose --expire=now HEAD
prune commit (initial): first
Nothing is removed in either case, only the label is wrong. This was
reported at [1].
fcd2c3d9d8 (reflog + refs-backend: move "verbose" out of the backend,
2021-12-22) added dry_run to struct expire_reflog_policy_cb and set it
in the --all loop, but not in the loop for named refs. Set it there
too.
Add tests for named refs and --all. No test checked the keep/prune
wording before.
[1] https://lore.kernel.org/git/b46cbce8-d2bb-46d2-bdd3-545422813d6b@tngtech.com/
Reported-by: Sven Weiland <sven.weiland@tngtech.com>
Assisted-by: Claude Sonnet 5.5
Signed-off-by: Laveesh Bansal <laveeshb@laveeshbansal.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Editors and shell prompts run "git status" in the background. Status
takes index.lock before collecting and holds it to the end, even when
it writes nothing, so a "git add" or "git commit" run meanwhile fails
with "index.lock: File exists".
The lock does not need to cover the collection, because
repo_update_index_if_able() refuses to write an index that changed
since status read it. 226c051adb (status: enable untracked cache,
2015-03-08) moved the write after wt_status_collect() to save the
untracked cache, but left the lock where it was.
Add repo_lock_and_update_index_if_able(), which takes the lock only if
the index has something to write, and call it after collecting. When
there is nothing to write, status no longer creates index.lock; when
there is, it holds the lock only for the write. In a clean gcc checkout
with status running in a loop, 104 of 200 "git add" runs failed before
and none after.
Assisted-by: Claude Opus 5.5, Gemini 4 Argon
Signed-off-by: Mihai Cosma <mcosma@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
hook.<event>.enabled, added by dcfb5af67e (hook: add
hook.<event>.enabled switch, 2026-04-10), skips only the hooks
defined in the configuration when set to false. With
hook.pre-commit.enabled=false, "git commit" still runs
.git/hooks/pre-commit, and "git hook list" shows that hook without
the "event-disabled" mark.
That was the intent, but the documentation says "no hooks fire for
that event", which reads as covering the hookdir hook too. No other
setting turns off the hookdir hook of a single event either:
core.hooksPath=/dev/null turns off the hookdir hooks of all events.
Make the switch cover the hookdir hook as well, and say so in the
documentation. This changes behavior for anyone who set the switch
and relied on the hookdir hook still running.
The test for hook.<event>.enabled=false looks only at the standard
output, but "git hook run" sends hook output to the standard error,
so the test passes even when a disabled hook runs. Add a hookdir hook
to it and check the standard error as well.
Assisted-by: Claude Fable 5.1
Signed-off-by: Thomas Bachem <mail@thomasbachem.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The rerere cache state is tracked via a global variable. On the one hand
this ties it to one specific repository. On the other hand it has the
consequence that the cache may be leaking state across multiple calls
into the rerere subsystem.
Refactor the subsystem to instead pass around the rerere cache state
explicitly and make callers responsible for setting it up and then
clearing it again.
For now, all callers are updated to do so ad-hoc. This of course means
that state that we could carry over across multiple calls would have to
be set up multiple times. But that doesn't really matter much right now
anyway, as the number of times we call those functions is bounded.
Note that `rerere_gc_needed()` is the only caller where we explicitly
don't set up the rerere cache state. This is intentional and matches the
previous status quo, where we didn't honor "rerere.enabled", either. The
function only needs access to `struct rerere_cache::dirs` anyway though,
so it's fine to partially initialize it, only.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The function `rerere_path()` computes paths of rerere cache entries
contained in the "rr-cache" directory of `the_repository`. This makes us
implicitly depend on global state.
Add a repository parameter so that callers can specify which repository
to compute the path for. Adjust the internal helpers that call it to
pass down the repository they already have available in their contexts.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The incremental connectivity check pays a per-commit cost. For
every incoming commit it reads the new trees along the changed
paths, and the trees at the same paths in its parents to compare
against. A parent tree is often read again even though an earlier
commit already read it: on git.git, about a third of the tree reads
for 2000 incoming commits are such re-reads. The full check
instead pays a mostly fixed cost for walking the boundary trees,
so with many incoming commits the incremental check becomes the
slower one (see the previous commit). Users who mostly
receive small pushes but occasionally a large one should not have
to choose between the two.
Add an "auto" value for transfer.connectivityCheck that picks the
mode based on the incoming history. When the revision walk is
limited, which is the case whenever the receiving repository has
references, the incoming commit list is already computed by
limit_list(), so counting it is cheap. "auto" then uses the
incremental check for up to 1000 incoming commits, and the full
check above that.
The cutoff is a fixed constant based on the p5413 fixture in the
previous commit, where breakeven is between 500 and 3000 commits.
It is conservative for repositories with large trees, where the
fixed cost of the full check is higher. Measured on real
repositories (rev-list against a single boundary commit, mean
of at least 10 runs):
repository files commits full incr. full/incr
internal A 556K 1000 1.71s 0.61s 2.8x
internal A 556K 4000 2.78s 2.83s 1.0x
internal B 288K 1000 1.78s 0.88s 2.0x
linux 95K 598 0.33s 0.12s 2.7x
linux 95K 1728 0.56s 0.35s 1.6x
git 4.9K 984 0.22s 0.27s 0.8x
With small trees incremental can be slower within the cutoff,
but both modes are fast enough there (0.22s vs 0.27s for git
above) that the difference does not matter in practice.
No single cutoff is optimal for every repository, so it can be set
with "auto=<cutoff>"; plain "auto" uses 1000. A malformed cutoff
is treated like any other unknown value: it falls back to "full"
with a warning.
Suggested-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Kristofer Karlsson <krka@spotify.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The full connectivity check uses rev-list to find commits
reachable from the incoming tips but not from the
already-connected side, then walks their object closure. Commit
traversal stops at the connectivity boundary, but trees and blobs
reachable from that boundary still need to be walked so they can
be marked uninteresting, allocating a struct object for each one.
On repositories where the boundary commits have large trees, the
connectivity check for small incoming changes visits and tracks
more objects than needed.
Add an alternative connectivity check that verifies incoming
commits incrementally against their parents.
The verifier processes incoming commits with ancestors first and
compares each new tree against its trusted parent trees. Entries
already seen on the trusted side are skipped by OID, so unchanged
subtrees need not remain at the same path to be recognized.
Changed subtrees are recursively compared against same-path parent
subtrees, and new subtrees without a comparison base are verified
from scratch.
See Documentation/technical/connectivity-check.adoc for the trust
invariants and detailed algorithm.
The incremental check runs inside rev-list, selected by the
internal --connectivity-check option. After the normal
revision walk identifies the incoming commits, the verifier
consumes them before the usual object traversal; any pending
non-commit tips are still handled by the existing traversal.
Partial-clone semantics are preserved: objects promised by a
promisor remote are accepted as connected, and on-demand fetching
is prevented by excluding promisor objects from traversal.
A new transfer.connectivityCheck value "incremental" selects the
new mode, with "full" remaining the default. Deepening fetches
fall back to the full check because they do not have the normal
existing-reference boundary, and traversal instead follows the
effective shallow boundary.
Clones and pushes into an empty repository also use the full
check. Without negative revisions there is no trusted boundary,
and the incremental check would have to verify all of history
from scratch like the full check does, with extra per-commit
overhead on top. On git.git, checking 70K commits without a
boundary takes 5.3s with the full check and 9.2s with the
incremental one.
p5413 results (best of 3), scaling one dimension at a time.
Each fixture is a repository with a flat tree of many
directories containing 100 files each. The incoming set is a
chain of commits on top of the existing history, with each
commit changing files in different directories round-robin.
The three axes vary the total repository tree size, the number
of incoming commits, and the number of files changed per
incoming commit.
Scaling tree size (10 incoming commits, 10 files/commit):
files full incr. full/incr
5K 0.01s 0.01s 1.0x
50K 0.04s 0.01s 4.0x
200K 0.13s 0.01s 13.0x
800K 0.54s 0.03s 18.0x
The full mode must walk the trees of boundary commits, which
grows with overall tree size. Incremental still scans the
root trees, but avoids descending into unchanged subtrees,
so it grows much more slowly with repository size.
Scaling incoming commit count (200K files, 10 files/commit):
commits full incr. full/incr
1 0.13s 0.01s 13.0x
10 0.13s 0.01s 13.0x
100 0.16s 0.06s 2.7x
500 0.29s 0.25s 1.2x
3000 1.14s 1.44s 0.8x
5000 1.87s 2.39s 0.8x
10000 3.45s 5.13s 0.7x
With many commits the per-commit overhead of scanning both
the new and parent root trees accumulates and incremental
becomes slower. Breakeven is between 500 and 3000 commits
and the ratio stabilizes near 0.7x for this fixture.
Scaling files per incoming commit (200K files, 10 commits):
files/commit full incr. full/incr
1 0.13s 0.01s 13.0x
10 0.13s 0.01s 13.0x
100 0.15s 0.03s 5.0x
500 0.23s 0.13s 1.8x
1000 0.34s 0.27s 1.3x
2000 0.65s 0.56s 1.2x
The advantage shrinks as more files change per commit, but
incremental stays slightly faster even at 2000 files/commit,
where every directory is touched.
Selected peak RSS measurements from the same fixtures:
case full incr. ratio
200K files, 10 commits 31 MB 12 MB 0.4x
800K files, 10 commits 110 MB 26 MB 0.2x
200K files, 5000 commits 155 MB 151 MB 1.0x
Incremental uses less memory when it can prune enough of the
boundary tree walk. In the long-history case the two modes
visit similar object sets and memory converges.
The regression cases in the CPU benchmarks are in wall-clock
time rather than memory, primarily from scanning some trees
more than once: a commit's tree is read as a candidate when the
commit is verified, and again as a trusted parent tree when its
children are verified.
Signed-off-by: Kristofer Karlsson <krka@spotify.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add the transfer.connectivityCheck configuration option to choose
the algorithm used for the connectivity check. For now the only
accepted value is "full", which is the existing behavior and the
default. Unknown values fall back to "full" with a warning, so
that configuration written for a newer version of Git does not
break fetches and pushes; "full" is always a correct check.
check_connected() now always tells its rev-list subprocess which
algorithm to use with an internal --connectivity-check=<mode>
option, which so far only accepts "full". Passing the mode
explicitly means rev-list knows it is running a connectivity
check, so later optimizations can be limited to that case.
This prepares for an incremental check added in the next commit.
Suggested-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Kristofer Karlsson <krka@spotify.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The next commits will need to use the repository to read configuration
in check_connected(). Rather than accessing `the_repository`, refactor
the code to instead require the caller to pass in a repository so that
we can remove `USE_THE_REPOSITORY_VARIABLE`.
Suggested-by: Junio C Hamano <gitster@pobox.com>
Signed-off-by: Kristofer Karlsson <krka@spotify.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Even though we previously deprecated it in fd2ebf14db (stash: mark "git
stash save" deprecated in the man page, 2017-10-22), the save subcommand
clings to life. For example, 951ec747d4 (doc txt & -h consistency: make
"stash" consistent, 2022-10-13) brought back "save" to the git-stash(1)
synopsis, and 41a28eb6c1 (stash: implement '--staged' option for 'push'
and 'save', 2021-10-18) gave it new options. But we really discourage
its use in modern Git, so let's drop mention of it from -h usage and the
synopsis.
It's still described in the manual, since some descriptions talk about
the "save" command. As we haven't gone ahead with a full deprecation and
removal, we can leave these pieces for folks whose fingers still type
"save" and need a reference for its behavior.
Signed-off-by: D. Ben Knoble <ben.knoble@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When git switch -c <new> <remote>/<branch> (or checkout -b) cannot write
the tracking config, for example because .git/config.lock exists, the
worktree, the index and the new branch ref have already been updated.
Git then exits with status 1 while HEAD still points to the original
branch. The changes of the switch remain staged, and the post-checkout
hook does not run.
create_branch() now returns an error instead of exiting. Make checkout
ignore that error so that the switch completes and the command exits
with status 0. The error and the hint are still printed.
Match push -u, clone and fetch, which also print the error and continue.
The upstream can be set later with the command shown in the hint.
Keep git branch <new> <start> failing when the tracking config cannot be
written, so its behavior does not change; t3200 covers this. Making the
switch fully atomic, undoing the worktree, the index, the ref and the
config when the tracking config fails, is a separate topic.
Signed-off-by: tsukasa inoue <inoue.tsukasa.4@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
setup_tracking() calls exit(1) when it cannot write the tracking
configuration. This exit was introduced in 27852b2c53 (branch: report
errors in tracking branch setup, 2016-02-22). Because of it, callers
cannot continue after a tracking failure, even though the ref has
already been created.
Change setup_tracking() to return an error instead of exiting, and make
create_branch() and dwim_and_setup_tracking() return that value too.
This does not change behavior. Callers still exit with status 1 on
failure, as before. Changing checkout
Signed-off-by: tsukasa inoue <inoue.tsukasa.4@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Junio C Hamano