Skip to content

feat(plugins): add playground plugin for filter/rule simulation#2346

Merged
JocLRojas merged 4 commits into
release/v12.0.0from
feature/plugin-playground
Jul 14, 2026
Merged

feat(plugins): add playground plugin for filter/rule simulation#2346
JocLRojas merged 4 commits into
release/v12.0.0from
feature/plugin-playground

Conversation

@JocLRojas

Copy link
Copy Markdown
Contributor

Changes

Adds a new event-processor plugin com.utmstack.playground that exposes an HTTP REST API for testing filters and rules against sample logs without touching production. Runs colocated with the event-processor manager, gated by MODE=manager, and supervises the EP playground -watch daemon as a child process.

What ships

New plugin — plugins/playground/

  • HTTP server on :8091, internal to the swarm network (not published to the host).
  • Auth via X-Internal-Key header, matched against the shared com.utmstack SDK config (same pattern as soc-ai and inputs).
  • At startup, runs playground -init once to scaffold /playground-workdir, then supervises playground -watch as a child process with automatic restart on exit.
  • Endpoints (all X-Internal-Key gated except /health):
    • POST /playground/filters — write a filter YAML
    • POST /playground/rules — write a rule YAML
    • POST /playground/filters/copy — clear-then-copy production filters
    • POST /playground/rules/copy — clear-then-copy production rules
    • DELETE /playground/filters — clear playground filter files
    • DELETE /playground/rules — clear playground rule files
    • POST /playground/run — submit a log, wait for parsed event + optional alert
    • GET /playground/status — idle | running (with current uuid)
    • GET /health — unauthenticated healthcheck

Installer (installer/docker/compose.go)

  • PLAYGROUND_BASE_URL=http://event-processor-manager:8091 added to the backend service (same pattern as SOC_AI_BASE_URL).
  • /playground-workdir host volume mount added to event-processor-manager so the plugin's shared filesystem survives container restarts.

CI (.github/workflows/v12-deployment-pipeline.yml)

  • playground added to the build_plugins matrix. No Dockerfile change is needed — event_processor.Dockerfile already copies the entire ./plugin-binaries/ directory, and the new artifact is picked up automatically via the plugin-bin-* merge pattern.

Why

Testing new filters or correlation rules currently requires modifying production configuration and observing live traffic, which is risky. The playground provides a safe, scripted way to submit a log and observe how it flows through the parsing and correlation pipelines — enabling both manual debugging (via curl/Postman) and future analyst-facing UI features (the backend consumes PLAYGROUND_BASE_URL).

Key design decisions

  • Interaction with the EP playground CLI is filesystem-based, not process-per-request. The plugin supervises playground -watch as a long-running child so the pipeline stays warm between calls. POST /run writes input/<uuid>.json, then polls the JSONL outputs the daemon appends.
  • Alert matching uses Alert.events[].id, not Alert.id. cel/generateAlert in the EP assigns a fresh UUID to Alert.id that is unrelated to the triggering event. The original event id only survives inside Alert.events[], so that's what we match on. Documented in runner/scan.go.
  • Log input is serialized with protojson.Marshal, not encoding/json. The daemon reads inputs with utils.StringToProtoMessage (protojson-backed). The two serializations diverge on fields whose protobuf JSON name differs from the Go json tag — notably Timestamp, which protojson emits as @timestamp and encoding/json would emit as timestamp (silently dropped by the daemon).
  • Single global mutex serializes CRUD + Run. Prevents races between filter/rule writes and an in-flight simulation. Worst-case wait ~17s for a simulation whose log never matches any filter; typical simulations against a warm daemon resolve in ~200ms.
  • TimedOut is a normal outcome (HTTP 200 with timedOut: true), not a 504. A log that doesn't match any filter is a valid debugging result, not a server error.
  • UUID auto-generation. If the request body omits log.id, the Runner generates one via uuid.NewString() and returns it in the response so clients can submit lightweight requests without pre-generating IDs.
  • OOM guard. POST /run uses http.MaxBytesReader (1 MiB) on the request body.

Reference

No tracking issue — internal feature request.

@github-actions

github-actions Bot commented Jul 14, 2026

Copy link
Copy Markdown

🛑 AI review — Sensitive area, extra care recommended

This PR touches critical paths or introduces changes the model cannot judge with sufficient confidence. Review carefully before merging.

🛑 architecture (gemini-3-flash-lite) — high/critical — please review

Summary: Introduction of a new plugin service with significant infrastructure changes and potential security implications.

  • high installer/docker/compose.go:217 — Modifying the installer to add a new service dependency (PLAYGROUND_BASE_URL) and volume mapping requires a coordinated deployment strategy for existing production environments.
  • high plugins/playground/internal/api/middleware.go:16 — Custom authentication implementation using X-Internal-Key. Ensure this aligns with the centralized auth/secret handling architecture rather than introducing a new, isolated secret mechanism.
  • medium plugins/playground/internal/runner/fs.go:80 — Hardcoded directory structure in EnsureDirs. This logic should ideally be managed by the installer or a shared configuration schema to avoid drift between the plugin and the deployment environment.
  • medium plugins/playground/internal/api/handlers.go:113 — The run handler uses a 1 MiB limit, but the logic performs file system operations (write/read) based on user-provided input. Ensure this is properly sandboxed to prevent path traversal or resource exhaustion.

🛑 bugs (gemini-3-flash-lite) — high/critical — please review

Summary: Potential resource leak in HTTP server and race condition in file handling

  • high plugins/playground/internal/runner/fs.go:30 — Potential resource leak: srcFile is opened but not closed if the subsequent os.Create or io.Copy fails. Use defer srcFile.Close() immediately after checking the error from os.Open.
  • medium plugins/playground/internal/api/handlers.go:48 — Insecure file path construction: destPath is joined using user-provided body.Name. While nameAllowlist provides some protection, it is safer to use filepath.Base(body.Name) to prevent directory traversal.
  • low plugins/playground/internal/runner/runner.go:158 — The pollUntil function uses time.NewTicker but does not handle the case where the context is cancelled before the first tick, potentially delaying shutdown by up to pollInterval.

🛑 security (gemini-3-flash-lite) — high/critical — please review

Summary: Introduction of a new playground plugin with potential path traversal and insecure file handling in API handlers.

  • high plugins/playground/internal/api/handlers.go:46 — Potential path traversal vulnerability in makeWriteHandler. While the filename is validated against a regex, the destDir is constructed using filepath.Join with user-controlled input. Ensure that the resulting path is strictly contained within the intended directory.
  • medium plugins/playground/internal/runner/fs.go:13 — Insecure file handling in CopyDir. The function walks a source directory and copies files to a destination without validating that the source paths are safe or that the destination paths remain within the target directory, potentially allowing arbitrary file reads/writes if the source is manipulated.
  • medium plugins/playground/internal/api/handlers.go:74 — Insecure file operations in makeCopyHandler. The function clears and copies entire directories from a production path to a playground work directory. If the production path is misconfigured or accessible, this could lead to unauthorized data exposure or corruption.

🟢 go-deps — up to date

No pending Go dependency updates.

@JocLRojas JocLRojas merged commit a002609 into release/v12.0.0 Jul 14, 2026
1 check passed
@JocLRojas JocLRojas deleted the feature/plugin-playground branch July 14, 2026 19:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant