* ci: run e2e registry jobs from one matrix
Replace the per-registry dockerhub and gitlab jobs with a single matrix
job. Each registry is configured by the same E2E_<ID>_* settings, and
the registry host moves to a repository variable, so adding a registry
is a matrix entry plus its settings.
Signed-off-by: Terry Howe <terrylhowe@gmail.com>
* ci: drop kind cluster from e2e registry jobs
The Kubernetes install test exercises the same pull path as the
registry tests and already runs per-PR against kind in e2e.yml.
Signed-off-by: Terry Howe <terrylhowe@gmail.com>
* ci: read e2e registry usernames from secrets
Signed-off-by: Terry Howe <terrylhowe@gmail.com>
---------
Signed-off-by: Terry Howe <terrylhowe@gmail.com>
Add a gitlab job to the e2e-registries workflow, configured from
vars.E2E_GITLAB_USERNAME, vars.E2E_GITLAB_NAMESPACE,
vars.E2E_GITLAB_REPOSITORY and secrets.E2E_GITLAB_PASSWORD. GitLab nests
repository paths under the project, so the suite runs in its default
layout. Its token endpoint only implements the distribution GET flow,
which covers the OAuth2 POST fallback that Docker Hub does not exercise.
Document running the suite against GitLab with a deploy token.
Signed-off-by: Terry Howe <terrylhowe@gmail.com>
* feature: automated OCI end-to-end tests
Signed-off-by: Terry Howe <terrylhowe@gmail.com>
* test: address review feedback on OCI e2e tests
Move the OCI end-to-end tests out of package cmd into a dedicated
test/e2e package behind an 'e2e' build tag, and drive a real helm binary
rather than executeActionCommand.
- Parameterize the registry, namespace, and credentials so the tests run
against any OCI registry, not a hard-coded personal GHCR namespace.
- Isolate each run's repositories and namespace with a random run ID.
- Fail rather than skip when required configuration is missing, since the
build tag is already an explicit opt-in.
- Pass the registry password over stdin so credentials never reach argv
or test output.
- Assert an authentication-specific failure in the invalid credential
test instead of accepting any error.
- Install through a real helm binary against the ambient kubecontext, so
the Kubernetes test no longer runs against the fake kube client.
- Stop dereferencing the os.Stat result after a stat error.
- Add a 'make test-e2e' target and test/e2e/README.md.
Signed-off-by: Terry Howe <thowe@nvidia.com>
* test: address second round of review feedback on OCI e2e tests
- Verify pulled chart contents against the source fixture instead of only
checking that a path of the expected name exists, so a registry serving a
different artifact under the same tag fails the test.
- Skip the invalid-credential test against a registry that serves anonymous
access, where there is no authentication to exercise.
- Delete the namespace the Kubernetes test creates, and wait for it to go,
so runs do not accumulate namespaces. A caller-supplied
HELM_E2E_NAMESPACE is left in place.
Signed-off-by: Terry Howe <thowe@nvidia.com>
* test: run OCI e2e tests across registry implementations
The suite has no registry-specific behavior, so covering a new registry
is a matter of pointing it at one. Add the CI wiring to do that, and make
the repository paths suitable for shared registries.
- Repository paths are now stable rather than per-run. The fixtures are
immutable, so re-running overwrites each tag with identical content.
This bounds the repositories a shared registry accumulates and lets
registries that require repositories to exist before a push (ECR) have
them created ahead of time. HELM_E2E_ISOLATE restores per-run paths.
- Add HELM_E2E_REPO to set the repository path under the registry.
- Add e2e.yml: local authenticated registry plus kind, no secrets, so it
runs on pull requests from forks.
- Add e2e-registries.yml: nightly matrix over GHCR, Quay, and ECR with
fail-fast disabled. A registry with no secrets configured is skipped,
so they can be wired up one at a time.
Signed-off-by: Terry Howe <thowe@nvidia.com>
* test: address review feedback on e2e harness and workflow
- Build the helm binary by relative package path instead of the v4 module
path, so the harness also works on branches where the module carries a
different major version suffix.
- Scope Kubernetes release names to the run. A caller-supplied
HELM_E2E_NAMESPACE may already hold releases, and a fixed name could
collide with one that cleanup would then uninstall.
- Fail a partially configured registry leg in the nightly workflow
instead of reporting it as configured and failing later in the harness
or the AWS CLI. A leg with no registry set is still skipped.
Signed-off-by: Terry Howe <thowe@nvidia.com>
* test: honor HELM_KUBECONTEXT when deleting the e2e namespace
Helm subprocesses inherit HELM_KUBECONTEXT from the environment, but the
cleanup client loaded the kubeconfig's current-context. When the two
differ, releases are installed into one cluster while the namespace
delete is sent to another, where it returns NotFound and is treated as
success, leaking the namespace it was meant to remove.
Apply the same context override when building the cleanup client.
Signed-off-by: Terry Howe <thowe@nvidia.com>
* test: build e2e cleanup client from helm's own settings
Overriding only HELM_KUBECONTEXT still left the cleanup client able to
diverge from the helm subprocesses, which also honor HELM_KUBEAPISERVER,
HELM_KUBETOKEN, HELM_KUBECAFILE, HELM_KUBETLS_SERVER_NAME and
HELM_KUBEINSECURE_SKIP_TLS_VERIFY. Any of those could send the namespace
delete to a different endpoint, where NotFound reads as success and the
namespace leaks.
Resolve the client through cli.New().RESTClientGetter() instead, so it is
built exactly the way the subprocesses build theirs and stays in step
without mirroring each variable by hand.
Signed-off-by: Terry Howe <thowe@nvidia.com>
* test: run hosted e2e against docker hub in single-repository mode
Replace the GHCR, Quay and ECR legs of e2e-registries with a Docker Hub
job using the E2E_DOCKER_* secrets. Docker Hub does not nest repository
paths and a scoped token can only write to one repository, so add
HELM_E2E_SINGLE_REPO: each fixture is repackaged under that repository's
name with its own prerelease version, e.g. 0.1.0-unicode-chart.
Signed-off-by: Terry Howe <terrylhowe@gmail.com>
* ci: read non-secret docker hub e2e settings from repository variables
Signed-off-by: Terry Howe <terrylhowe@gmail.com>
* ci: run e2e workflow on main only
Signed-off-by: Terry Howe <terrylhowe@gmail.com>
* ci: update kind, registry, httpd, checkout and setup-go in e2e workflows
Signed-off-by: Terry Howe <terrylhowe@gmail.com>
* ci: serialize e2e-registries runs that share docker hub tags
Signed-off-by: Terry Howe <terrylhowe@gmail.com>
---------
Signed-off-by: Terry Howe <terrylhowe@gmail.com>
Signed-off-by: Terry Howe <thowe@nvidia.com>
Terry Howe