Commit Graph

441 Commits

Author SHA1 Message Date
Lennart Poettering
90f5f3552c import: use truncate_nl() where appropriate 2023-03-24 18:33:59 +01:00
Lennart Poettering
50ed5cbf5f tree-wide: port various places to find_line_startswith() 2023-03-24 18:30:20 +01:00
Daan De Meyer
7c2f5495e2 copy: Move chattr arguments to full function signatures
These are almost never used, so let's move them to the _full()
functions signatures.
2023-03-21 20:53:09 +01:00
Jan Janssen
9214828313 meson: Introduce userspace dep
This will help in a later commit to separate userspace from EFI builds.
2023-03-10 11:40:14 +01:00
Lennart Poettering
82fa9f28f9 importd: hook up memory pressure, SIGRTMIN-18 (and in fact SIGINT + SIGTERM) 2023-03-01 09:43:23 +01:00
Yu Watanabe
4788f635e3 Merge pull request #26203 from medhefgo/meson
meson: Use dicts for test/fuzzer definitions
2023-02-22 10:27:16 +09:00
Jan Janssen
822cd3ff25 meson: Use dicts for test definitions
Although this slightly more verbose it makes it much easier to reason
about. The code that produces the tests heavily benefits from this.

Test lists are also now sorted by test name.
2023-02-21 15:10:26 +01:00
Yu Watanabe
0c2aedb451 tree-wide: use FORK_REARRANGE_STDIO and FORK_CLOSE_ALL_FDS 2023-02-21 07:39:18 +09:00
Daan De Meyer
c17cfe6ef7 copy: Make copy_xattr() more generic
Let's make copy_xattr() a little more generic in preparation for
copying symlink xattrs.
2023-02-14 12:51:40 +01:00
Yu Watanabe
f3f2d02e97 tree-wide: set FORK_RLIMIT_NOFILE_SAFE flag
No functional changes, just refactoring.
2023-02-07 14:39:49 +09:00
Arsen Arsenović
181eea677d importd: Always specify file unpacked by tar
Despite popular belief, the default file extracted by GNU tar is not stdin.  It
is the value of the TAPE environment variable, falling back on a compile-time
constant.  On my system, the default value is /dev/full, which causes tar to
just spin forever due to --ignore-zeros.  Always specifying this flag is the
safe thing to do.

  ~$ tar --show-defaults
  --format=gnu -f/dev/full -b20 --quoting-style=escape
  --rmt-command=/usr/sbin/grmt

See also: ``(tar)defaults'', available via Info viewers, and in HTML form at:
https://www.gnu.org/s/tar/manual/html_node/defaults.html
2023-01-31 12:21:44 +01:00
Jan Janssen
4a7ee0a521 meson: Do not include headers in source lists
Meson+ninja+compiler do this for us and are better at it.

https://mesonbuild.com/FAQ.html#do-i-need-to-add-my-headers-to-the-sources-list-like-in-autotools
2023-01-24 22:04:03 +01:00
Frantisek Sumsal
e61a4c0b7c import: use CURLOPT_PROTOCOLS_STR with libcurl >= 7.85.0
CURLOPT_PROTOCOLS [0] was deprecated in libcurl 7.85.0 with
CURLOPT_PROTOCOLS_STR [1] as a replacement, causing build warnings/errors:

../build/src/import/curl-util.c: In function ‘curl_glue_make’:
../build/src/import/curl-util.c:255:9: error: ‘CURLOPT_PROTOCOLS’ is deprecated: since 7.85.0. Use CURLOPT_PROTOCOLS_STR [-Werror=deprecated-declarations]
  255 |         if (curl_easy_setopt(c, CURLOPT_PROTOCOLS, CURLPROTO_HTTP|CURLPROTO_HTTPS|CURLPROTO_FILE) != CURLE_OK)
      |         ^~
In file included from ../build/src/import/curl-util.h:4,
                 from ../build/src/import/curl-util.c:6:
/usr/include/curl/curl.h:1749:3: note: declared here
 1749 |   CURLOPTDEPRECATED(CURLOPT_PROTOCOLS, CURLOPTTYPE_LONG, 181,
      |   ^~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors

Since there's no grace period between the two symbols, let's resort
to a light if-def-ery to resolve this.

[0] https://curl.se/libcurl/c/CURLOPT_PROTOCOLS.html
[1] https://curl.se/libcurl/c/CURLOPT_PROTOCOLS_STR.html
2023-01-09 13:36:42 +01:00
Frantisek Sumsal
2285c462eb import: use CURLINFO_SCHEME instead of CURLINFO_PROTOCOL
CURLINFO_PROTOCOL has been deprecated in curl 7.85.0 causing compilation
warnings/errors:

../build/src/import/pull-job.c: In function ‘pull_job_curl_on_finished’:
../build/src/import/pull-job.c:142:9: error: ‘CURLINFO_PROTOCOL’ is deprecated: since 7.85.0. Use CURLINFO_SCHEME [-Werror=deprecated-declarations]
  142 |         code = curl_easy_getinfo(curl, CURLINFO_PROTOCOL, &protocol);
      |         ^~~~
In file included from ../build/src/import/curl-util.h:4,
                 from ../build/src/import/pull-job.h:6,
                 from ../build/src/import/pull-common.h:7,
                 from ../build/src/import/pull-job.c:16:
/usr/include/curl/curl.h:2896:3: note: declared here
 2896 |   CURLINFO_PROTOCOL    CURL_DEPRECATED(7.85.0, "Use CURLINFO_SCHEME")
      |   ^~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors

Since both CURLINFO_SCHEME and CURLINFO_PROTOCOL were introduced in
the same curl version (7.52.0 [0][1]) we don't have to worry about
backwards compatibility.

[0] https://curl.se/libcurl/c/CURLINFO_SCHEME.html
[1] https://curl.se/libcurl/c/CURLINFO_PROTOCOL.html
2023-01-09 12:43:37 +01:00
Yu Watanabe
5bb1d7fbab tree-wide: use -EBADF more 2022-12-21 01:50:33 +09:00
Yu Watanabe
19ee48a6c2 tree-wide: introduce PIPE_EBADF macro 2022-12-20 11:12:58 +09:00
Zbigniew Jędrzejewski-Szmek
3401477982 tree-wide: use -EBADF also in pipe initializers
In some places, initialization is dropped when unnecesary.
2022-12-19 15:00:59 +01:00
Zbigniew Jędrzejewski-Szmek
254d1313ae tree-wide: use -EBADF for fd initialization
-1 was used everywhere, but -EBADF or -EBADFD started being used in various
places. Let's make things consistent in the new style.

Note that there are two candidates:
EBADF 9 Bad file descriptor
EBADFD 77 File descriptor in bad state

Since we're initializating the fd, we're just assigning a value that means
"no fd yet", so it's just a bad file descriptor, and the first errno fits
better. If instead we had a valid file descriptor that became invalid because
of some operation or state change, the other errno would fit better.

In some places, initialization is dropped if unnecessary.
2022-12-19 15:00:57 +01:00
Ivan Shapovalov
c7779a61ac import: wire up SYSTEMD_IMPORT_BTRFS_{SUBVOL,QUOTA} to importd
Btrfs quotas are actually being enabled in systemd-importd via
setup_machine_directory(), not in systemd-{import,pull} where those
environment variables are checked. Therefore, also check them in
systemd-importd and avoid enabling quotas if requested by the user.

Fixes: #18421
Fixes: #15903
Fixes: #24387
2022-12-03 20:31:48 +04:00
Lennart Poettering
7d1e61cab6 tree-wide: make constant ratelimit compound actually const
The compiler should recognize that these are constant expressions, but
let's better make this explicit, so that the linker can safely share the
initializations all over the place.
2022-11-22 01:30:55 +01:00
Daan De Meyer
a424958aa6 copy: Support passing a deny list of files/directories to not copy 2022-11-10 16:40:31 +01:00
Zbigniew Jędrzejewski-Szmek
28db6fbff1 Rename def.h to constants.h
The name "def.h" originates from before the rule of "no needless abbreviations"
was established. Let's rename the file to clarify that it contains a collection
of various semi-related constants.
2022-11-08 18:21:10 +01:00
Zbigniew Jędrzejewski-Szmek
3ae6b3bf72 basic: rename util.h to logarithm.h
util.h is now about logarithms only, so we can rename it. Many files included
util.h for no apparent reason… Those includes are dropped.
2022-11-08 18:21:10 +01:00
Zbigniew Jędrzejewski-Szmek
d6b4d1c7c4 basic: move version() to build.h+c 2022-11-08 13:41:14 +01:00
Yu Watanabe
ee3455cf11 fd-util: rename CLOSE_AND_REPLACE() -> close_and_replace()
We have free_and_replace() and friends, they are all named with lower
letters, even they are macros, not functions.
For consistency, let's rename CLOSE_AND_REPLACE() with lower letters.

This also mekes the macro used more places.
2022-09-17 09:54:41 +02:00
David Tardon
995340074e tree-wide: use ASSERT_PTR more 2022-09-13 08:13:27 +02:00
Lennart Poettering
45519d13a4 tree-wide: port things dirname_malloc() → path_extract_directory() 2022-08-23 15:10:15 +02:00
Ludwig Nussel
5243331fb8 pull: fix PullFlags numbering 2022-08-04 17:27:03 +02:00
Michael Biebl
41d6f3bf4d Use https for freedesktop.org
grep -l -r http:// | xargs sed -E -i s'#http://(.*).freedesktop.org#https://\1.freedesktop.org#'
2022-06-28 13:10:05 +02:00
Lennart Poettering
ba4e0427e9 time-util: assume CLOCK_BOOTTIME always exists
Let's raise our supported baseline a bit: CLOCK_BOOTTIME started to work
with timerfd in kernel 3.15 (i.e. back in 2014), let's require support
for it now.

This will raise our baseline only modestly from 3.13 → 3.15.
2022-03-28 16:55:41 +02:00
Lennart Poettering
69811f4763 import: improve error message
As suggested: https://github.com/systemd/systemd/pull/20156#discussion_r810941489
2022-03-15 22:43:48 +00:00
Zbigniew Jędrzejewski-Szmek
ffb8c82715 Merge pull request #22092 from keszybz/docs-links
Add more doc pages, adjust links, add explanatory headers to examples and relax license to CC-0
2022-01-12 16:06:39 +01:00
Zbigniew Jędrzejewski-Szmek
d6e2c1ab71 policy files: adjust landing page link 2022-01-12 16:05:59 +01:00
Jan Janssen
e1eeebbb11 meson: Use files() for tests
Not having to provide the full path in the source tree is much
nicer and the produced lists can also be used anywhere in the source
tree.
2022-01-11 14:15:54 +01:00
Zbigniew Jędrzejewski-Szmek
03f7ddf0bf meson: stop building out convenience libraries by default
The meson default for static_library() are:
build_by_default=true, install=false. We never interact with the
static libraries, and we only care about them as a stepping-stone towards
the installable executables or libraries. Thus let's only build them if
they are a dependency of something else we are building.

While at it, let's drop install:false, since this appears to be the default.

This change would have fixed the issue with lib_import_common failing
to build too: we wouldn't attempt to build it.

In practice this changes very little, because we generally only declare static
libraries where there's something in the default target that will make use of
them. But it seems to be a better pattern to set build_by_default to false.
2021-12-16 11:01:08 +01:00
Zbigniew Jędrzejewski-Szmek
b93f018f57 meson: don't compile import sources four times
Use a 'convenience library' to do the compilation once and then link the
objects into all the files that need it. Those files are small, so this probably
doesn't matter too much for speed, but has the advantage that we don't get the
same error four times if something goes wrong.

The library is conditionalized in the same way importd itself, because we
cannot build it without the deps.
2021-12-16 10:54:46 +01:00
Zbigniew Jędrzejewski-Szmek
c7f0d9e5ac tree-wide: make FOREACH_DIRENT_ALL define the iterator variable
The variable is not useful outside of the loop (it'll always be null
after the loop is finished), so we can declare it inline in the loop.
This saves one variable declaration and reduces the chances that somebody
tries to use the variable outside of the loop.

For consistency, 'de' is used everywhere for the var name.
2021-12-15 16:19:13 +01:00
Zbigniew Jędrzejewski-Szmek
939387bdc6 Merge pull request #21170 from keszybz/delibgcryptify
Allow systemd-resolved and systemd-importd to use libgcrypt or libopenssl
2021-12-03 13:44:53 +01:00
Zbigniew Jędrzejewski-Szmek
6214d42bd2 import: port importd from libgcrypt to openssl^gcrypt
This is heavily based on Kevin Kuehler's work, but the logic is also
significantly changed: instead of a straighforward port to openssl, both
versions of the code are kept, and at compile time we pick one or the other.

The code is purposefully kept "dumb" — the idea is that the libgcrypt codepaths
are only temporary and will be removed after everybody upgrades to openssl 3.
Thus, a separate abstraction layer is not introduced. Instead, very simple
ifdefs are used to select one or the other. If we added an abstraction layer,
we'd have to remove it again afterwards, and it don't think it makes sense to
do that for a temporary solution.

Co-authored-by: Zbigniew Jędrzejewski-Szmek <zbyszek@in.waw.pl>

# Conflicts:
#	meson.build
2021-11-30 23:00:21 +01:00
Yu Watanabe
8add30a03c tree-wide: use ERRNO_IS_TRANSIENT() 2021-11-30 23:06:43 +09:00
Lennart Poettering
35cd0ba516 shared: clean up mkdir.h/label.h situation
Previously the mkdir_label() family of calls was implemented in
src/shared/mkdir-label.c but its functions partly declared ins
src/shared/label.h and partly in src/basic/mkdir.h (!!). That's weird
(and wrong).

Let's clean this up, and add a proper mkdir-label.h matching the .c
file.
2021-11-16 17:03:28 +01:00
Lennart Poettering
7c248223eb tree-wide: use new RET_NERRNO() helper at various places 2021-11-16 08:04:09 +01:00
Lennart Poettering
aedec452b9 tree-wide: always use TAKE_FD() when calling rearrange_stdio()
rearrange_stdio() invalidates specified fds even on failure, which means
we should always invalidate the fds we pass in no matter what. Let's
make this explicit by using TAKE_FD() for that everywhere.

Note that in many places we such invalidation doesnt get us much
behaviour-wise, since we don't use the variables anymore later. But
TAKE_FD() in a way is also documentation, it encodes explicitly that the
fds are invalidated here, so I think it's a good thing to always make
this explicit here.
2021-11-03 23:05:26 +00:00
Lennart Poettering
7950211df3 tree-wide: port more code to sigkill_wait() 2021-11-03 16:36:19 +01:00
Lennart Poettering
8f03de5323 tree-wide: port various places to use TAKE_PID() 2021-11-03 16:36:09 +01:00
Lennart Poettering
bf819d3ac2 basic: split out sync() family of calls from fs-util.[ch] into new c/h file
No changes in code, just some splitting out.
2021-10-05 15:47:32 +02:00
Lennart Poettering
983b69c3f5 import-fs: create paren dirs in --direct mode too
There's no reason not to create these in advance if they are missing.
2021-08-20 21:57:08 +02:00
Lennart Poettering
50b9c30cea import: don't attempt full-file clones if we only are supposed to write a part of the file
Otherwise we'll copy more than we were told to.
2021-08-20 21:57:04 +02:00
Lennart Poettering
bf284aee23 import: enable sparse file writing logic only for files we create
Only if we create a file we know for sure that it is empty and hence our
sparse file logic of skipping over NUL bytes can can work. If we hwoever
are called to write data to some existing file/block device, we must do
regular writes to override everything that might be in place before.

Hence, conditionalize sparse file writing on the write offset not being
configured (which is how we internally distinguish write to existing
file and write to new file)
2021-08-20 21:56:59 +02:00
Lennart Poettering
53e03c9692 import: when completed, say how many bytes we wrote/acquired
This is just too useful.
2021-08-20 21:56:55 +02:00