The object storage format has been made configurable via the
'extensions.objectStorage' configuration variable, allowing users
to specify the storage backend and an optional payload, mirroring
the design of 'extensions.refStorage'.
* ps/odb-object-storage-extension:
builtin/submodule: propagate "--object-storage-format="
builtin/clone: introduce "--object-storage-format="
builtin/init: introduce handling of object storage format
builtin: expose the object storage format
setup: allow overriding object storage format via environment
setup: introduce object storage extension
loose: drop unused `repo_write_loose_object_map()`
While the object storage format extension can already be configured
manually via git-config(1), this isn't exactly an ergonomic way of doing
so. For the reference storage format extension, users have a host of
options available to them.
Introduce the equivalent options for the object storage format.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The ref storage format configured via "extensions.refStorage" can be
overridden with the GIT_REF_STORAGE_FORMAT environment variable.
Introduce the GIT_OBJECT_STORAGE_FORMAT environment variable that
overrides "extensions.objectStorage" in the same way.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Introduce a new "extensions.objectStorage" extension that allows the
user to configure the primary object database source of a repository.
The value is a URI in the format "<schema>[://<payload>]", mirroring
the format used by "extensions.refStorage":
- The schema identifies the backend that shall be used. For now the
only supported schema is "files", which is the existing object
database backed by loose objects and packfiles.
- The payload can be passed to configure the backend, and its
interpretation is backend-specific. For the "files" backend, it
allows you to configure a different location for the ".git/objects"
directory.
Making the location of the "files" backend configurable is not the most
thrilling feature of this new extension. But it gives us a couple of
advantages:
- It allows us to start exercising the payload logic in our tests so
that we know it's working properly for backends that will need it.
- It allows us to easily detect tests that are broken with the new
extension because they for example assume that an object is a file
in ".git/objects/". Testing is not part of this series yet, but will
be done in subsequent patch series.
- It aligns with how we handle the ref storage extension, where we can
also change its location to an arbitrary path.
- It allows for some very esoteric and edge-case-y gimmicks. For
example, you can initialize a single repository "skeleton" and then
multiplex therein by setting both GIT_REF_STORAGE_FORMAT and
GIT_OBJECT_STORAGE_FORMAT to point to different databases. These
edge cases won't matter to many people out there, but we will use
this for example as a building block for migrations.
Eventually, this mechanism will be used to support alternative backends.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The repository initialization sequence has been refactored to treat
the repository object passed to create_repository() purely as an
out-parameter.
* ps/setup-enforce-repo-passed-to-create-repository-has-no-state:
setup: enforce that passed-in repo does not carry relevant state
repository: adapt `repo_clear()` to fully reset the repository
builtin/clone: don't apply "core.sharedRepository" to leading dirs
builtin/init: move handling of "core.sharedRepository" into "setup.c"
builtin/init: refactor messy creation of leading directories
path: introduce `safe_create_leading_directories_no_share_const()`
path: drop useless `safe_create_leading_directories_1()`
A new 'uploadpack.lazyFetchTrusted' configuration variable has been
introduced to allow 'upload-pack' to lazily fetch missing objects from
configured promisor remotes when serving trusted repositories.
* cc/lazy-fetch-trusted-bit:
builtin/upload-pack: don't disable lazy fetching on trusted repo
promisor-remote: prevent infinite recursion when lazy fetching
upload-pack: read uploadpack.lazyFetchTrusted
setup: extract path_allowlist_apply()
promisor-remote: factor out lazy_fetch_objects()
* ps/setup-enforce-repo-passed-to-create-repository-has-no-state:
setup: enforce that passed-in repo does not carry relevant state
repository: adapt `repo_clear()` to fully reset the repository
builtin/clone: don't apply "core.sharedRepository" to leading dirs
builtin/init: move handling of "core.sharedRepository" into "setup.c"
builtin/init: refactor messy creation of leading directories
path: introduce `safe_create_leading_directories_no_share_const()`
path: drop useless `safe_create_leading_directories_1()`
Originally, when designing pluggable object databases the goal was that
the object database can have multiple sources, and every source attached
to it could use a different backend. This would have allowed for quite a
lot of flexibility, as you could trivially mix and match different kinds
of object storages in whatever way you like.
But while well-intentioned, this design led to a bunch of conceptual
problems:
- We're now trying to read objects in source order, whereas we
previously tried to read objects via packfiles before trying to read
them via loose objects. This led to a performance regression when
using alternates or when using a quarantine directory.
- Some data structures are supposed to only ever exist once, like for
example bitmaps and commit graphs. At the same time, those data
structures also span across the union of all objects, so they may
cross sources.
- It is unclear how we can extend GIT_OBJECT_DIRECTORY or
GIT_ALTERNATE_OBJECT_DIRECTORIES to become backend-agnostic in a
backwards-compatible way. In general, introducing an object storage
extension into the current status quo where alternates may have to
be extended to become generic was proving to be painful.
- Some mechanisms of alternates assume way too much about how exactly
their backends work. Alternate refs for example assume that the
alternate is backed by a filesystem path, and that this filesystem
path may also allow us to read references. This is not a given
though, as backends may not even have local data at all.
In short, there are a bunch of conceptual mismatches when we have
alternates and pluggable object databases coexist. So while the original
idea was nice, it does not result in a system that is easy to reason
about.
Correct course by moving alternates into the "files" source itself so
that it becomes an implementation detail thereof so that we can avoid
all of these shortcomings. While it's unfortunate that we cannot easily
mix and match sources now, that ability doesn't go away. It's still very
much feasible to introduce a new backend that allows for exactly that
use case, and such a backend may also be a lot more flexible as we can
now add new logic to determine which objects should be stored where. So
the original motivation for having per-source backends can still be
realized with the new architecture.
Note that as part of this move, we also handle the GIT_OBJECT_DIRECTORY
and GIT_ALTERNATE_OBJECT_DIRECTORIES environment variables in the
"files" backend. This may be surprising at first, but object directories
are very much a concept of that backend, too. So these variables would
have bad interactions with other backends, and they create a bit of a
mismatch with the eventual object storage extension that we plan to
introduce.
Note that this commit is way larger than I'd like it to be. I'm sorry, I
couldn't find a way to split it up further. That being said, most of the
changes are straight-forward conversions that go from iterating over
sources to iterating over object directories. The more involved changes
are moving the infrastructure to track individual sources from "odb.c"
and moving them into "odb/source-files.c".
This also serves to show a bit of a who's-who of commands that don't
work properly with pluggable object databases. Almost all of these
commands are related to housekeeping though -- some of them will be
converted eventually, like for example commit graphs or MIDXs. But many
of them will stay incompatible going forward as they are simply too
specific to the "files" backend.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In a following commit we are going to check whether a repository is
part of an allowlist specified in a config variable.
To prepare for that let's extract existing code from
safe_directory_cb() into a new path_allowlist_apply() helper that will
help with such checks.
While at it let's make the helper's code simpler and more generic, by
passing it a `bool (*allow_path)(const char *path, void *cbdata)`
function that decides if a path is acceptable by the caller, and let's
add a NEEDSWORK comment about it silently ignoring missing, possibly
misspelled, paths.
To further simplify how to reuse that new helper, and avoid duplicating
the config-value handling in a future commit, let's also introduce a
path_allowlist_config_apply() helper.
As the new path_allowlist_apply() function reports its result through
a `bool *matches` argument, let's also change the `int is_safe` member
of `struct safe_directory_data` to a `bool`, so that its address can
be passed as that argument.
Signed-off-by: Christian Couder <christian.couder@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The terminology regarding reference storage formats has been unified
across command-line options, environment variables, configuration
variables, and source code, standardizing on the phrase "ref storage
format" (e.g., `--ref-storage-format`, `'GIT_REF_STORAGE_FORMAT'`).
Additionally, the `--ref-storage-format` option has been updated to
accept payloads in the form `<format>://<payload>`.
* ps/ref-storage-format:
setup: allow "--ref-storage-format=" to specify a payload
setup: rename "init.defaultRefFormat" to "init.defaultRefStorageFormat"
t: rename GIT_TEST_DEFAULT_REF_FORMAT
setup: rename ref storage format environment variables
setup: refactor how we configure the ref storage format
refs: expose function to parse reference URIs
help: rename "default-ref-format" to "default-ref-storage-format"
builtin/rev-parse: rename "--show-ref-format" to "--show-ref-storage-format"
builtin/submodule: rename "--ref-format=" to "--ref-storage-format="
builtin/refs: rename "--ref-format=" to "--ref-storage-format="
builtin/clone: rename "--ref-format=" to "--ref-storage-format="
builtin/init: rename "--ref-format=" to "--ref-storage-format="
parse-options: allow for hidden aliases
The setup of alternates has been deferred to object database
creation time during clone, which drops the unused ad-hoc alternate
writing API, simplifying the object database backend interface.
* ps/odb-alternates-at-creation:
odb/source: remove the ability to write alternates
builtin/clone: write alternates via `odb_create_on_disk()`
odb/source: support writing alternates when creating the database
builtin/clone: move setup of alternates for non-shared local clones
builtin/clone: move setup of alternates for shared local clones
builtin/clone: refactor handling of "--reference{,-if-able}"
builtin/clone: move around `setup_reference()`
builtin/clone: defer setup of the object database
setup: split up concerns of `init_db()`
In the preceding patches we have refactored `create_repository()` so
that the passed-in repository is not used anymore to propagate any kind
of state. This was done so that the parameter doesn't act like an in-out
parameter, but only as an out parameter that we initialize with the
state of the newly created repository.
We don't enforce though that the repository _cannot_ be used to
propagate state anymore, which makes it quite easy for state to sneak in
at a later point again.
Ideally, we'd do that by having the function create a newly allocated
repository instead of taking a repository as input. But unfortunately,
that does not work because we end up calling `repo_config_values()` when
we create the "files" ref database, and that function requires that the
passed-in repository is `the_repository`.
Instead, call `repo_clear()` at the beginning of the function, which
gives us a clean slate.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When initializing a new repository via git-init(1) we know to honor
"core.sharedRepository" and adjust permissions of newly created files
accordingly. The way we propagate that setting is quite awkward though,
as we have to set it on the repository that we pass into
`create_repository()` and pass it as a parameter. This is because there
are two different scopes in play here:
- We need to apply it to the repository so that creating the
repository's directory uses the correct permissions.
- We need to reapply it to the repository after we have created
default files so that we know to override any configuration that we
have read from the new repository's configuration.
The effect of this though is that the repository works as an in-out
parameter, which is quite awkward.
Refactor the code so that the caller only needs to pass the value.
Starting with this change, the passed-in repository can essentially be
completely blank as it doesn't carry any state anymore that we'd care
about in `create_repository()`.
Note that this change in theory also impacts the other caller of
`create_repository()` that exists in git-clone(1). But that caller
already passes `-1` as a value for this parameter, and neither does that
caller modify the repository it passes. So there shouldn't be any change
in behaviour here.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* ps/odb-alternates-at-creation:
odb/source: remove the ability to write alternates
builtin/clone: write alternates via `odb_create_on_disk()`
odb/source: support writing alternates when creating the database
builtin/clone: move setup of alternates for non-shared local clones
builtin/clone: move setup of alternates for shared local clones
builtin/clone: refactor handling of "--reference{,-if-able}"
builtin/clone: move around `setup_reference()`
builtin/clone: defer setup of the object database
setup: split up concerns of `init_db()`
When creating a repository with alternates we first initialize the
object database and then write alternates to it in a separate step. This
is unfortunate due to a couple of reasons:
- It requires us to have a `write_alternates()` callback, which is
unfortunate as we never even write alternates to an object database
after it has been created.
- We're about to make alternates an implementation detail of the
object database's backend in a future patch series, so having this
callback is suboptimal there.
- The backend has more flexibility with how exactly alternates are
configured when it itself is in full control over their setup at the
time where it creates the object database itself.
We have thus introduced the ability to write alternates right at
creation time in the preceding commits, and we have unified setup of
alternates into a single location. All that's left to do for us now is
to wire up alternates as an option for the database creation.
Do so.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Add the ability to write alternates when creating the object database.
This change allows us to remove the `write_alternates()` callback in a
subsequent patch.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The function `init_db()` is responsible for creating the on-disk
directory structure required for a Git repository. It is used by both
git-init(1) and git-clone(1), and because their expected behaviour is
different we support a couple of flags:
- The `QUIET` flag controls whether the command is quiet or not. For
git-init(1) this is user-controllable, whereas for git-clone(1)
we're always quiet.
- The `EXIST_OK` flag controls whether a preexisting repository is
okay or not. For git-init(1) it is, for git-clone(1) it's not.
- The `SKIP_REFDB` flag controls whether the reference database should
already be created or not. For git-init(1) we do, but for
git-clone(1) we don't because it does not yet know about the default
branch and about the remote object hash.
Furthermore, we're about to add another divergence in behaviour, where
we have to also skip creation of the object database in git-clone(1).
This is becoming quite cumbersome though.
Instead of introducing another flag, start to split up concerns of the
function so that we never create the reference or object database. This
becomes the responsibility of the caller, which is thus free to defer
their creation to a later point in time. This lets us get rid of most of
the divergent behaviour:
- We don't need the `SKIP_REFDB` and a potential `SKIP_ODB` flags
anymore.
- We don't need the `QUIET` flag anymore, as nothing prints output
except for the final status message that tells the user that the
repository has been (re)initialized. But as this message is specific
to git-init(1), we can easily move it there.
The only piece of information we still have to convey is whether or not
reinitialization of a preexisting repository is okay. This is handled by
a new `reinit_ok` pointer that, if non-`NULL`, indicates that it is okay
to reinitialize the repository. Furthermore, the pointer will be written
to to indicate whether the repository was reinitialized or not, which we
need in git-init(1) to print the correct initialization message.
With these refactorings, `init_db()` is named quite misleadingly though,
as we don't create any of the reference or object databases anymore.
Rename it to `create_repository()`.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Reference storage backends can be configured with a payload via the
"extensions.refStorage" config key and the "GIT_REF_STORAGE_FORMAT"
environment variable, both of which accept a URI in the format
"<format>://<payload>". The payload may contain backend-specific
information, for example an alternate refs directory or which database
references should be stored in.
The `--ref-storage-format=` option of git-init(1) and git-clone(1) does
not know about payloads though: its value is parsed as a plain format
name, so backends that require a payload cannot be conveniently set up
at initialization time via the command line.
Teach the option to accept the same URI syntax. Also, document the
optional payloads for both the "files" and "reftable" backends.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
With the same reasoning as for git-init(1), rename the
"init.defaultRefFormat" config option to "init.defaultRefStorageFormat"
and keep the old name as an alias.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
With the same reasoning as for git-init(1), rename the environment
variables GIT_REFERENCE_BACKEND and GIT_DEFAULT_REF_FORMAT to
GIT_REF_STORAGE_FORMAT and GIT_DEFAULT_REF_STORAGE_FORMAT, respectively.
The old names are kept as an alias to retain compatibility.
While at it, fix indentation for `GIT_REF_STORAGE_FORMAT` docs to use
tabs instead of spaces.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When (re)initializing a repository we need to figure out the ref storage
format that the repository ought to use. This logic is surprisingly
complex, as we have grown a lot of different mechanisms over time to
configure the format. Unfortunately, as a result of this organic growth,
the logic that configures the storage format has grown very complex.
The biggest culprit here is that we're mixing the logic that determines
the desired storage format with the logic that validates whether the end
result is sane. This leads to some repetitive code, and makes it very
easy to forget validation for some of the branches.
In fact, the way we handle GIT_REFERENCE_BACKEND shows exactly one such
edge case where we don't properly validate. When initializing a
repository with one storage format and then reinitializing it with the
environment variable set to a different format then we'd corrupt the
repository because we silently change the format:
$ git init repo
$ git -C repo commit --allow-empty -m message
$ GIT_REFERENCE_BACKEND=reftable git -C repo init
fatal: could not open '.../refs/heads' for writing: Is a directory
$ git -C repo log
fatal: your current branch appears to be broken
Refactor the code so that we clearly distinguish between these two
different concerns. This lets us clearly spell out the precedence order
and makes the whole logic significantly easier to extend going forward.
Note that the new logic intentionally changes the precedence order so
that "GIT_REFERENCE_BACKEND" is now overridden by the
"--ref-storage-format=" command line option. This matches our usual
precedence order, where explicit command line arguments override
environment variables.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In the next commit we're about to add more sites that want to parse a
reference backends URI into a format and payload. Expose a new function
`ref_storage_format_by_uri()` that enables this.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The git worktree repair command failed to rewrite the .git file of
a working tree from a relative path to an absolute path when the
command was run in the working tree itself. The
read_gitfile_gently() function was modified to also return whether
the path originally recorded in the file was absolute, and this new
capability is used to correctly detect such mismatches.
* yn/worktree-repair-relative:
worktree repair: detect relative path in .git file correctly
The object database layer has been simplified by eagerly loading
alternate object directories upon initialization, instead of
deferring it to the first object lookup. This eliminates the need
for scattered lazy-loading calls throughout the codebase and paves
the way for integrating alternates with the pluggable backends.
* ps/odb-eagerly-load-alternates:
odb: drop `alternates_db` field
odb: drop `loaded_alternates` field
odb: eagerly initialize alternates
odb: decouple source path comparisons from `the_repository`
setup: create ref and object databases after config is written
Given a state in which the cross-references between the worktree and
the repository (specifically worktree/id/gitdir in the main repository
and the .git file in the worktree) are recorded using absolute paths,
setting 'worktree.useRelativePaths=true' and running 'git worktree
repair' within the main worktree converts them to relative paths.
Conversely, given a state in which the cross-references are recorded
using relative paths, one would expect that setting
'worktree.useRelativePaths=false' and running 'git worktree repair'
would convert them to absolute paths. However, they remain as relative
paths.
This is because we incorrectly use read_gitfile_gently(), which always
returns an absolute path. To fix this, introduce read_gitfile_raw(),
which reads the path from the .git file without resolving it to an
absolute path.
Because read_gitfile_raw() does not validate the path with
is_git_directory(), repair_gitfile() performs this validation to
preserve the existing behavior.
Signed-off-by: Yoichi NAKAYAMA <yoichi.nakayama@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The unused name parameter in 'struct chdir_notify_entry' has been
removed from chdir_notify_register(), chdir_notify_unregister(), and
related callback signatures across several subsystems, simplifying the
API now that trace output no longer uses it.
* ch/chdir-notify-drop-name:
chdir-notify.h: Removed unused param 'name'
The creation of the on-disk data structures for the object database
has been made pluggable, allowing future backends to customize their
setup. As part of this, the initialization of the object database
has been deferred, and the loading of the loose-object map has been
detangled from repository initialization.
* ps/odb-make-creation-pluggable:
odb: make creation of on-disk structures pluggable
odb/source: introduce function to map source type to name
setup: defer object database creation
setup: handle ODB-related environment variables in `odb_new()`
setup: detangle loading of loose object maps
loose: load loose object map for the correct source
When creating a new repository we create both the reference and object
databases after we have finalized the repository. This ensures that
those subsystems find a fully-configured repository at the time where
they are asked to create their own on-disk data structures.
There is one exception though: while we have already fully configured
the repository at this point, we haven't yet written both
"core.sharedRepository" and "receive.denyNonFastforwards". The latter
configuration doesn't really matter to us, but the first one does as the
"files" object database source reads it.
This doesn't cause any problems right now, but it will in a subsequent
patch where we will start to read "core.ignoreCase" when creating the
object database. Move the initialization of both of these data
structures towards the end of `init_db()`. The only thing that now comes
after is status reporting, but that's it.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The `name` parameter in `chdir_notify_entry` was only ever used by
chdir_notify_reparent() to produce trace output. That function was
removed in 5bf546755c (chdir-notify: drop unused
`chdir_notify_reparent()`, 2026-06-25), which left `name` with no
remaining consumers.
Prior to that removal, most callers had already stopped passing a
meaningful name, switching to NULL in 1f43ff2c7e (refs: unregister
reference stores from "chdir_notify", 2026-06-25) and 0de2467e6c
(odb/source-packed: start converting to a proper `struct odb_source`,
2026-06-17).
Since no caller has populated `name` with real data for some time,
and its last consumer is gone, drop it from chdir_notify_register(),
chdir_notify_unregister(), and the callback signature to simplify
the API.
Signed-off-by: Colin Hinton <colinlewishinton@gmail.com>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
* ps/odb-make-creation-pluggable:
odb: make creation of on-disk structures pluggable
odb/source: introduce function to map source type to name
setup: defer object database creation
setup: handle ODB-related environment variables in `odb_new()`
setup: detangle loading of loose object maps
loose: load loose object map for the correct source
When creating a new "files" object database source we have to create a
couple of directories. These directories are of course specific to this
particular backend, and a different backend may require a setup that is
completely different.
Make the creation of on-disk structures pluggable to accommodate for
this.
Note that there is one exception though: the "objects" directory must
exist in a repository regardless of which backend is in use. If it
doesn't exist then the repository is not treated as a Git repository at
all. Consequently, we create this directory regardless of the backend.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In a subsequent commit we'll make the creation of the on-disk data
structures of an object database pluggable. This will lead to an
in-between state where we have already configured the repository's
object database, but it's not usable yet until we eventually call
`create_object_directory()`.
Lift the call to `odb_new()` out of `apply_repository_format()` so that
callers have more wiggle room with when exactly they call it, and adapt
them accordingly. The only exception is `init_db()`, where we now defer
creating the object database until we call `create_object_database()`.
With this change, initializing and creating the object database on disk
is now neatly encapsulated in a single function, which will make it
easier for a subsequent commit to move creation of the on-disk data
structures into the `struct odb_source` backends.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When initializing a repository's object database we have to respect the
GIT_OBJECT_DIRECTORY and GIT_ALTERNATE_OBJECT_DIRECTORIES environment
variables, which can be set by the user to override the default location
of where we write objects to and read objects from.
This is handled in `apply_repository_format()`, which is fine. But in a
subsequent commit we'll have to defer constructing the object database
to a later point in some cases, and that will require a second site
where we call `odb_new()`. And of course, that second site would have to
handle those environment variables, as well.
It would be somewhat awkward to duplicate the logic though. But there's
a better alternative: instead of handling this logic in "setup.c", we
can easily handle environment variables in `odb_new()` itself. This
ensures that object database creation is neatly self-contained, and we
don't have to duplicate any of the logic.
Another benefit is that in a future patch series we plan to move
handling of alternates into the backends themselves [1], and that will
require us to also handle those environment variables in the "files"
backend itself. So moving the logic into the ODB level already gets us
one step closer to that goal.
Refactor the logic accordingly.
[1]: https://lore.kernel.org/git/amLgMqkqxR8mKIbT@pks.im/
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When a repository is configured to use a compatibility hash function
then we load the loose object map when we initialize the repository.
This object map provides the mappings between the canonical object hash
and the compatibility object hash.
Loading the object map happens in `repo_set_compat_hash_algo()`, which
calls `repo_read_loose_object_map()` in case the compatibility object
hash is non-zero. This setup sequence has two major downsides:
- We assume that the primary object database is the "files" object
database and unconditionally downcast it. This will cause us to BUG
in case a different object database type was used together with a
compat hash algorithm.
- We require the object database to already have been initialized when
configuring the object database. This means that we must intermix
configuration of the repository and initialization of its
sub-structures in a weird way.
Refactor the logic so that we instead load the loose object map via the
"loose" backend, which fixes both of the above issues.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The copy_file() and copy_file_with_time() functions have been
refactored to take a repository parameter, allowing the removal of the
implicit dependency on the global 'the_repository' variable in
'copy.c'.
* ps/copy-wo-the-repository:
copy: drop dependency on `the_repository`
The ref subsystem and the worktree API have been refactored to pass a
repository pointer down the call chain, allowing them to drop
references to the global 'the_repository' variable. As part of this,
the handling of the 'core.packedRefsTimeout' configuration has been
moved into the per-repository ref store structure.
* ps/refs-wo-the-repository:
refs: remove remaining uses of `the_repository`
worktree: pass repository to public functions
worktree: pass repository to file-local functions
worktree: refactor code to use available repositories
refs/files: drop `USE_THE_REPOSITORY_VARIABLE`
refs/packed: de-globalize handling of "core.packedRefsTimeout"
The repository discovery and repository configuration phases, which
were previously intertwined in 'setup.c', have been split. Repository
discovery has been updated to populate a 'struct repo_discovery'
without modifying the repository state, which is then taken by
repository configuration to initialize the repository, paving the way
for clean unification of repository configuration.
* ps/setup-split-discovery-and-setup:
setup: mark `set_git_work_tree()` as file-local
setup: pass worktree to `init_db()`
setup: drop redundant configuration of `startup_info->have_repository`
setup: make repository discovery self-contained
setup: propagate prefix via repository discovery
setup: drop static `cwd` variable
setup: move prefix into repository
setup: embed repository format in discovery
setup: introduce explicit repository discovery
setup: split up concerns of `setup_git_env_internal()`
setup: unify setup of shallow file
setup: mark bogus worktree in `apply_repository_format()`
setup: rename `check_repository_format_gently()`
When copying a file we need to potentially adapt permissions of the new
file based on whether or not "core.shared" is enabled. Parsing this
configuration makes us implicitly depend on `the_repository`.
Refactor the code to instead require the caller to pass in a repository
so that we can remove `USE_THE_REPOSITORY_VARIABLE`.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Refactor remaining public functions that still depend on
`the_repository` to instead receive a repository as parameter. This
allows us to get rid of `USE_THE_REPOSITORY_VARIABLE`.
Adapt callers accordingly.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In the preceding commit we have removed the last callers of
`set_git_work_tree()` that is located outside of "setup.c". Remove its
declaration and mark the function as file-local.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In the preceding commits we have refactored how we discover and set up
repositories so that we cannot end up with partially-configured repos.
Instead, we apply the gitdir, worktree and repository format in a single
location, only.
Initializing a new repository has the same antipattern though: while
most of the information for the new repository is passed via parameters,
the work tree is instead propagated by configuring the repository's work
tree.
Refactor the code so that we also pass the work tree as an explicit
parameter. Like this, configuration fo the repository happens in a
single spot, too, just as with repository discovery.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In `init_db()` we set `startup_info->have_repository` twice: once before
reading and applying the repository format and once after. This is
redundant though, as configuring the repository format does not rely on
this variable at all.
Remove the first such site. While at it, fix up formatting a bit.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In the preceding commits we have introduced a separate repository
discovery phase and refactored the logic so that we have two clear
phases:
1. Repository discovery, which doesn't modify the repository itself at
all.
2. Repository configuration, which takes the information we have
discovered to set up the repository.
Extract the first phase into a new function `repo_discover()` to further
stress these two different phases.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
In the preceding commits we have started to propagate all information
required for the configuration of the repository via a new `struct
repo_discovery`. The only exception is the repository's prefix, which we
still return via the return parameter.
This is conceptually fine, but somewhat inconsistent. Refactor this to
instead propagate the prefix via the repository discovery, too.
While at it, drop a static variable in `repo_discover_bare_gitdir()`.
We apply its value to the repository discovery anyway, so we don't have
to keep it around afterwards anymore.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The current working directory is stored as part of a static strbuf
variable. This variable had to have a lifetime longer than its
containing function because the value we return typically points into
that buffer.
In the preceding commit we have moved the prefix into the repository
though. Consequently, we can now return the repository's prefix instead
of the local one and thus properly manage the lifecycle of this local
variable.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The repository prefix is currently stored in the startup info. This
feels somewhat awkward though, as it is inherently a property of a given
repository.
Move the prefix into the repository accordingly.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
All functions related to repository discovery receive both a `struct
repository_discovery` and `struct repository_format` as input, and the
expectation is that both will be populated. Refactor this so that the
repository format is part of the discovery result.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
When setting up the global repository we intermix repository discovery
and repository configuration: we repeatedly call `set_git_work_tree()`
and `apply_and_export_relative_gitdir()` until we're happy with the
result. The result of this is then a partially-configured repository
that we use for further setup.
This process is quite hard to follow, as it's never quite clear which
parts of the repository have been configured already and which haven't.
Furthermore, it means that the repository configuration is distributed
across many different places instead of having it neatly contained in a
single location. Ultimately, this is the reason that we cannot use a
central function like `repo_init()`.
Refactor the logic so that we stop partially-configuring a repository
and instead populate a new `struct repo_discovery`. This allow us to
essentially split repository setup into two phases:
- The first phase only figures out parameters required to configure
the repository.
- The second phase then takes these parameters and applies them to the
repository.
Like this, we'll never end up with a partially-configured repository and
can eventually extend `repo_init()` to handle the full initialization
for us.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
The function `setup_git_env_internal()` does two completely unrelated
things:
- It configures the repository's gitdir and propagates environment
variables into it.
- It configures a couple of global parameters via environment
variables.
The function is called when we initialize the repository's path, but
it's also called via `chdir_notify_register()` whenever we change the
current working directory. While we indeed have to reconfigure the
gitdir in case it's a relative path, it doesn't make sense to reapply
the global environment variables.
Split up concerns of this function along the above delineation. Handling
of the global environment variables is moved into `init_git()`, as they
can be considered part of our setup procedure.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
It is possible to configure an arbitrary "shallow" file via two
mechanisms, and the respective logic to handle these is split across two
locations:
- Via the "GIT_SHALLOW_FILE" environment variable, which is handled in
`setup_git_env_internal()`.
- Via the global "--shallow-file=" command line option, which is
handled in `handle_options()`.
We can rather easily unify this logic by not configuring the shallow
file in `handle_options()`, but instead overwriting the environment
variable. The environment variable itself is then handled inside of
`apply_repository_format()`, which is responsible for configuring a
discovered Git directory.
This new logic is similar in nature to how we handle the other global
options already, all of which end up setting an environment variable.
So for one this gives us more consistency. But more importantly, this
change means that `the_repository` will not contain any relevant state
anymore before we hit `apply_repository_format()` once we're at the end
of this patch series. Consequently, it will become possible for us to
completely discard `the_repository` and populate it anew.
Note that on first sight, this change looks like it might change the
precedence order. Before this change, we used to configure the shallow
file in the arguments handler first, and then it looks like we override
it via the environment variable. What's important to note though is the
last parameter to `set_alternate_shallow_file()`, which tells us whether
we want to overwrite a preexisting value, and when applying the value
from the environment we tell it not to overwrite preexisting values. So
in effect, the command line has precedence over the environment. After
this change, we now overwrite preexisting environment variables when we
see the argument, and consequently we keep the precedence order in tact.
With this change though we don't need the final parameter anymore that
tells `set_alternate_shallow_file()` whether or not to overwrite. We
only have a single callsite for this function now, and that function is
itself only ever called exactly once. Remove that parameter.
Signed-off-by: Patrick Steinhardt <ps@pks.im>
Signed-off-by: Junio C Hamano <gitster@pobox.com>
Junio C Hamano