Implement HelmReleaseTest operator by twallac10 · Pull Request #1 · networkengineer-cloud/testrun-operator

twallac10 · 2026-03-07T13:39:18Z

Summary

Scaffold kubebuilder project with HelmReleaseTest CRD (group testing.testing.platform.io/v1alpha1); spec references a HelmRelease, Kustomization, and suspended CronJob by name
Flux webhook handler (internal/webhook) on :8080/trigger — HMAC-SHA256 validation, upgrade-succeeded event filtering, 5-minute dedup window, Job creation by copying the CronJob's jobTemplate
Job watcher controller (internal/controller) — watches batch/v1 Jobs filtered by app=helm-release-test, lazily resolves commit SHA from Kustomization status, patches HelmReleaseTest status conditions, posts GitHub commit status
Helper packages: internal/kustomization (unstructured Kustomization fetch + SHA parsing), internal/github (POST to /repos/{owner}/{repo}/statuses/{sha})
cmd/main.go wired with webhook goroutine, probes on :8082, metrics on :8081, leader election ID helm-release-test-operator.testing.platform.io
RBAC ClusterRole covers all required verbs for batch jobs/cronjobs, Flux Kustomization/HelmRelease CRDs, and helmreleasetests
Sample manifests: HelmReleaseTest CR, suspended CronJob, Flux Provider + Alert

Test Plan

make generate — deepcopy functions regenerate without errors
make manifests — CRD and RBAC YAML regenerate correctly
make build — compiles cleanly
make test — unit tests pass (HMAC validation, SHA parsing, dedup logic, GitHub status posting)
Manual: apply CRD to cluster, create suspended CronJob + HelmReleaseTest CR, send mock webhook POST, verify Job created from CronJob template

🤖 Generated with Claude Code

- Scaffold kubebuilder project with HelmReleaseTest CRD - CRD types: HelmReleaseTestSpec (helmReleaseRef, kustomizationRef, cronJobRef), full status fields, kubebuilder print columns - Webhook handler (internal/webhook): HMAC-SHA256 validation, Flux event parsing, HelmReleaseTest matching, 5-min dedup window, Job creation from CronJob template - Job watcher controller: watches batch/v1 Jobs by label, lazy SHA resolution, updates HelmReleaseTest status, posts GitHub commit status - Kustomization SHA resolver (internal/kustomization): unstructured Kustomization fetch - GitHub status reporter (internal/github): POST /statuses/{sha} with env-based config - cmd/main.go: webhook goroutine on :8080, probes on :8082, metrics on :8081, leader election ID helm-release-test-operator.testing.platform.io - RBAC markers: batch jobs/cronjobs, kustomize+helm flux CRDs, helmreleasetests - Sample manifests: HelmReleaseTest CR, suspended CronJob, Flux Provider+Alert - Unit tests: HMAC validation, SHA parsing, dedup logic, GitHub status posting - All verification steps pass: make generate, manifests, build, test Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Scope push trigger to main only so feature branch pushes don't fire a second run alongside the pull_request trigger. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

- errcheck: wrap resp.Body.Close in func literal, remove duplicate os.Setenv calls - lll: break long flag.StringVar line in main.go - staticcheck QF1008: drop embedded .Time from CreationTimestamp.After call Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Multi-platform build (linux/amd64 + linux/arm64) covers Ubuntu k8s nodes and Mac OS kind clusters (Apple Silicon). On PRs, builds both platforms without pushing to verify the Dockerfile. On push to main, pushes ghcr.io/<owner>/testrun-operator:latest and :sha-<sha> tags. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Critical: - Controller: add LastRunJob guard to prevent re-processing completed Jobs, eliminating redundant status patches and duplicate GitHub API calls Important: - Webhook: add 1 MiB body size limit via http.MaxBytesReader - Webhook: remove unused testing.platform.io/helmrelease annotation write - GitHub: convert PostCommitStatus to Poster struct with injectable HTTPClient, read Token/Repo once at startup in main.go instead of inside the leaf function - main.go: implement fluxWebhookRunnable (manager.Runnable) for graceful shutdown with 5s drain timeout instead of bare goroutine - main.go: inject github.Poster into reconciler - manager.yaml: fix health probe arg and port from :8081 to :8082 - api types: remove LastTestedRevision field that was declared but never populated Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

…FORM Without --platform=\$BUILDPLATFORM, Docker pulls the arm64 golang image and runs the entire Go compile under QEMU on the amd64 runner, which takes many minutes. With this flag the builder always runs natively; GOARCH=\$TARGETARCH handles cross-compilation. Only the final distroless layer needs QEMU. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

Copilot

Pull request overview

This PR scaffolds and implements a new Kubebuilder-based Kubernetes operator for the HelmReleaseTest CRD, integrating a Flux notification webhook to trigger Jobs and a controller to watch those Jobs and report results (including GitHub commit statuses).

Changes:

Add Flux webhook HTTP handler (/trigger) with HMAC validation, upgrade-succeeded filtering, deduping, and Job creation from a suspended CronJob template.
Add Job-watcher controller that updates HelmReleaseTest status and posts GitHub commit statuses using a SHA resolved from Flux Kustomization status.
Add operator scaffolding (CRD/RBAC/manifests), CI workflows, and test/e2e harness.

Reviewed changes

Copilot reviewed 62 out of 64 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
`cmd/main.go`	Wires controller + Flux webhook server into the manager, adds metrics/probes/leader election config.
`internal/webhook/handler.go`	Implements Flux event ingestion, HMAC validation, deduping, and Job creation from CronJob templates.
`internal/webhook/handler_test.go`	Adds unit tests for signature validation, deduping, and event filtering.
`internal/controller/helmreleasetest_controller.go`	Watches Jobs and patches `HelmReleaseTest` status; posts GitHub commit statuses.
`internal/controller/helmreleasetest_controller_test.go`	Scaffolds controller reconcile test.
`internal/controller/suite_test.go`	Envtest suite setup for controller package.
`internal/kustomization/revision.go`	Fetches Flux Kustomization as unstructured and parses commit SHA from status revision.
`internal/kustomization/revision_test.go`	Tests SHA parsing behavior.
`internal/github/status.go`	Implements GitHub commit status posting client.
`internal/github/status_test.go`	Tests GitHub status posting behavior via httptest server.
`api/v1alpha1/helmreleasetest_types.go`	Defines the `HelmReleaseTest` API types and status fields/printcolumns.
`api/v1alpha1/groupversion_info.go`	Registers the API group/version with the scheme.
`api/v1alpha1/zz_generated.deepcopy.go`	Generated deepcopy implementations for API types.
`config/crd/bases/testing.testing.platform.io_helmreleasetests.yaml`	Generated CRD manifest for `HelmReleaseTest`.
`config/crd/kustomization.yaml`	CRD kustomization wiring for default install.
`config/crd/kustomizeconfig.yaml`	Kustomize config for name/namespace substitutions in CRDs (webhook-related).
`config/rbac/role.yaml`	ClusterRole rules for Jobs/CronJobs, Flux CRDs, and `helmreleasetests` resources.
`config/rbac/role_binding.yaml`	ClusterRoleBinding for the manager role.
`config/rbac/service_account.yaml`	ServiceAccount for the controller manager.
`config/rbac/leader_election_role.yaml`	Namespaced Role for leader election resources.
`config/rbac/leader_election_role_binding.yaml`	Binds leader election Role to manager ServiceAccount.
`config/rbac/metrics_reader_role.yaml`	ClusterRole allowing GET access to `/metrics`.
`config/rbac/metrics_auth_role.yaml`	ClusterRole for TokenReview/SAR used by secure metrics authn/z.
`config/rbac/metrics_auth_role_binding.yaml`	Binds metrics auth role to the manager ServiceAccount.
`config/rbac/kustomization.yaml`	RBAC kustomization bundle for default install.
`config/rbac/helmreleasetest_admin_role.yaml`	Scaffolded admin ClusterRole for the CRD.
`config/rbac/helmreleasetest_editor_role.yaml`	Scaffolded editor ClusterRole for the CRD.
`config/rbac/helmreleasetest_viewer_role.yaml`	Scaffolded viewer ClusterRole for the CRD.
`config/manager/manager.yaml`	Manager Deployment + Namespace scaffolding.
`config/manager/kustomization.yaml`	Manager kustomization bundle.
`config/default/kustomization.yaml`	Default install bundle combining CRDs/RBAC/manager + metrics service + patches.
`config/default/metrics_service.yaml`	Service exposing controller metrics.
`config/default/manager_metrics_patch.yaml`	Patch enabling metrics bind address arg.
`config/default/cert_metrics_manager_patch.yaml`	Patch for mounting cert-manager secrets for secure metrics (optional).
`config/prometheus/monitor.yaml`	ServiceMonitor scaffold for Prometheus scraping of metrics.
`config/prometheus/monitor_tls_patch.yaml`	Patch for TLS-secured ServiceMonitor configuration.
`config/prometheus/kustomization.yaml`	Prometheus kustomization bundle.
`config/network-policy/allow-metrics-traffic.yaml`	NetworkPolicy scaffold to restrict metrics ingress.
`config/network-policy/kustomization.yaml`	Network policy kustomization bundle.
`config/samples/testing_v1alpha1_helmreleasetest.yaml`	Sample `HelmReleaseTest` CR.
`config/samples/suspended_cronjob.yaml`	Sample suspended CronJob used as a Job template source.
`config/samples/flux_provider_alert.yaml`	Sample Flux Provider/Alert to call the webhook endpoint.
`config/samples/kustomization.yaml`	Sample kustomization bundle referencing sample manifests.
`test/utils/utils.go`	E2E helper utilities (kubectl/kind helpers, cert-manager install/uninstall, etc.).
`test/e2e/e2e_suite_test.go`	E2E suite setup: build/load image, optional cert-manager install, teardown.
`test/e2e/e2e_test.go`	E2E tests for manager running and metrics endpoint availability.
`Makefile`	Kubebuilder build/test/deploy targets + CI-friendly envtest/e2e targets.
`Dockerfile`	Multi-stage build for the manager binary into distroless image.
`go.mod`	Module definition and dependencies.
`go.sum`	Dependency checksums.
`hack/boilerplate.go.txt`	License header template for generated code.
`.golangci.yml`	Linter configuration (including custom plugin).
`.custom-gcl.yml`	Config for building custom golangci-lint with plugins.
`.github/workflows/test.yml`	CI workflow running unit tests.
`.github/workflows/test-e2e.yml`	CI workflow running e2e tests on Kind.
`.github/workflows/lint.yml`	CI workflow running lint checks.
`.github/workflows/docker.yml`	CI workflow building/pushing multi-arch container images to GHCR.
`.dockerignore`	Restricts Docker build context to Go sources + module files.
`.gitignore`	Ignores build artifacts (bin/, cover.out).
`README.md`	Scaffolded project README (still contains TODO placeholders).
`PROJECT`	Kubebuilder project metadata.
`AGENTS.md`	Agent/maintenance guide and kubebuilder workflow notes.
`.devcontainer/devcontainer.json`	Devcontainer definition (Go + docker-in-docker + tooling).
`.devcontainer/post-install.sh`	Installs kind/kubebuilder/kubectl and configures shell completions.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Copilot · 2026-03-07T14:42:28Z

+	dir, _ := GetProjectDir()
+	cmd.Dir = dir
+
+	if err := os.Chdir(cmd.Dir); err != nil {
+		_, _ = fmt.Fprintf(GinkgoWriter, "chdir dir: %q\n", err)
+	}


The test utility changes the process-wide working directory via os.Chdir(). This is a global side effect that can break parallel tests and any code that relies on the original cwd; cmd.Dir is already set and is sufficient for running the command. Consider removing the os.Chdir call (or restoring the previous cwd with defer) and rely solely on cmd.Dir.

Copilot · 2026-03-07T14:42:28Z

+	secretName := fmt.Sprintf("%s-token-request", serviceAccountName)
+	tokenRequestFile := filepath.Join("/tmp", secretName)
+	err := os.WriteFile(tokenRequestFile, []byte(tokenRequestRawString), os.FileMode(0o644))
+	if err != nil {
+		return "", err
+	}


This helper writes a TokenRequest JSON file to a deterministic path under /tmp and never removes it. That can cause collisions when tests run concurrently (or rerun after a failure) and leaves artifacts behind. Prefer os.CreateTemp (or t.TempDir) and defer os.Remove to ensure uniqueness and cleanup.

Copilot · 2026-03-07T14:42:29Z

+func (h *Handler) validateSignature(header string, body []byte) bool {
+	if len(h.HMACSecret) == 0 {
+		return true
+	}


validateSignature() currently returns true (accepts the request) when no HMAC secret is configured. That means the /trigger endpoint becomes unauthenticated by default, allowing anyone who can reach it to create Jobs. Consider failing closed by default (reject when secret is empty) and/or making an explicit "insecure/disable-auth" flag for local development.

Copilot · 2026-03-07T14:42:29Z

+	// Skip Jobs that haven't completed yet.
+	if job.Status.CompletionTime == nil {
+		return ctrl.Result{}, nil
+	}


The controller skips any Job where status.completionTime is nil. For Kubernetes Jobs, completionTime is only set when the Job completes successfully; failed Jobs may never set it. This would prevent reporting failed test runs and leave HelmReleaseTest status stale. Use Job status conditions (Complete/Failed) or check Succeeded/Failed counts to detect terminal Jobs.

Copilot · 2026-03-07T14:42:29Z

+	ghCtx := fmt.Sprintf("helm-release-tests/%s", hrt.Name)
+	if err := r.GitHubPoster.PostCommitStatus(ctx, sha, ghCtx, ghState, condMsg); err != nil {
+		logger.Error(err, "Failed to post GitHub commit status")


Reconcile() unconditionally dereferences r.GitHubPoster. If the reconciler is ever constructed without a poster (e.g., tests or alternative wiring), this will panic. Consider guarding with a nil check (and/or making GitHubPoster a non-pointer value) before attempting to post a status.

Copilot · 2026-03-07T14:42:29Z

+  # logcheck validates structured logging calls and parameters (e.g., balanced key-value pairs)
+  - module: "sigs.k8s.io/logtools"
+    import: "sigs.k8s.io/logtools/logcheck/gclplugin"
+    version: latest


The golangci-lint plugin is pinned to version "latest", which makes lint results and even the ability to build the custom linter non-deterministic over time. Pin this to a specific, known-good version (and bump intentionally) to keep CI reproducible.

Suggested change

version: latest

version: v0.7.0

Copilot · 2026-03-07T14:42:30Z

+# atomic-munching-donut
+// TODO(user): Add simple overview of use/purpose
+
+## Description
+// TODO(user): An in-depth paragraph about your project and overview of use
+


The README still contains scaffold TODOs and does not describe the HelmReleaseTest operator behavior, required env vars (HMAC_SECRET/GITHUB_TOKEN/GITHUB_REPO), or how to configure Flux Provider/Alert. This makes the top-level documentation misleading for consumers of this repo; please replace the TODO sections with operator-specific setup and usage info.

Copilot · 2026-03-07T14:42:30Z

+        # TODO(user): The option insecureSkipVerify: true is not recommended for production since it disables
+        # certificate verification, exposing the system to potential man-in-the-middle attacks.
+        # For production environments, it is recommended to use cert-manager for automatic TLS certificate management.
+        # To apply this configuration, enable cert-manager and use the patch located at config/prometheus/servicemonitor_tls_patch.yaml,
+        # which securely references the certificate from the 'metrics-server-cert' secret.


This comment references a patch file named "config/prometheus/servicemonitor_tls_patch.yaml", but the repository adds "config/prometheus/monitor_tls_patch.yaml". Please fix the filename in the comment (or rename the patch) to avoid confusion when enabling TLS.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

twallac10 and others added 6 commits March 7, 2026 08:33

Fix duplicate workflow runs on PR creation

a8cb5cf

Scope push trigger to main only so feature branch pushes don't fire a second run alongside the pull_request trigger. Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

twallac10 force-pushed the feature/atomic-munching-donut branch from dd5ea6f to 529b34c Compare March 7, 2026 14:33

twallac10 requested a review from Copilot March 7, 2026 14:34

Copilot started reviewing on behalf of twallac10 March 7, 2026 14:34 View session

Copilot AI reviewed Mar 7, 2026

View reviewed changes

twallac10 and others added 8 commits March 7, 2026 22:10

docs: add Helm chart design doc

e580818

feat: add helm chart scaffold

49834d2

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

feat: add CRD to helm chart

27dc9eb

feat: add RBAC templates to helm chart

ad6027e

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

feat: add deployment template to helm chart

b6f05c5

fix: add imagePullSecrets support for private registry

2a1e10b

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

feat: add service template to helm chart

5a9f18a

feat: add helm chart package and push to CI

62dbca5

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

twallac10 merged commit 41ea5c1 into main Mar 8, 2026
5 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Implement HelmReleaseTest operator - #1

Implement HelmReleaseTest operator#1
twallac10 merged 14 commits into
mainfrom
feature/atomic-munching-donut

twallac10 commented Mar 7, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Mar 7, 2026

Uh oh!

Copilot AI Mar 7, 2026

Uh oh!

Copilot AI Mar 7, 2026

Uh oh!

Copilot AI Mar 7, 2026

Uh oh!

Copilot AI Mar 7, 2026

Uh oh!

Copilot AI Mar 7, 2026

Uh oh!

Copilot AI Mar 7, 2026

Uh oh!

Copilot AI Mar 7, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Uh oh!

Conversation

twallac10 commented Mar 7, 2026

Summary

Test Plan

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Mar 7, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 7, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 7, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 7, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 7, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 7, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 7, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 7, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants