Skip to content

feat(tools): add architecture violation scanner with CLI/REPL GitHub issue workflow (#3235)#3625

Open
Devesh36 wants to merge 25 commits into
Tracer-Cloud:mainfrom
Devesh36:issue/3235
Open

feat(tools): add architecture violation scanner with CLI/REPL GitHub issue workflow (#3235)#3625
Devesh36 wants to merge 25 commits into
Tracer-Cloud:mainfrom
Devesh36:issue/3235

Conversation

@Devesh36

@Devesh36 Devesh36 commented Jul 2, 2026

Copy link
Copy Markdown
Collaborator

This pull request introduces the new Architecture Issue Tool, which enables maintainers to scan the repository for architectural violations, generate refactor task proposals, and optionally file GitHub issues directly from the CLI or REPL. The implementation includes a new CLI command, comprehensive documentation, and enhancements to GitHub integration token resolution. The changes also update import boundaries and improve integration with the interactive shell.

Major features and improvements:

1. Architecture Issue Tool introduction and CLI integration

  • Added a new architecture-issue-tool documentation page describing the tool's purpose, usage, checks performed, parameters, and API examples (docs/architecture-issue-tool.mdx).
  • Implemented the opensre architecture-scan CLI command group with subcommands for scanning, proposing, and filing GitHub issues, including options for repo root, file size limits, and baseline inclusion (surfaces/cli/commands/architecture_scan.py, surfaces/cli/commands/__init__.py) [1] [2] [3].
  • Registered the new documentation in the docs navigation (docs/docs.json).

2. GitHub integration enhancements

  • Improved GitHub token resolution logic to support tokens from the integrations store, environment variables, and explicit input, with better error handling and logging (integrations/github/client.py) [1] [2] [3] [4] [5].
  • Updated error messages to guide users to configure GitHub tokens or run the setup command if missing (integrations/github/client.py) [1] [2].

3. Import boundary and dependency updates

  • Updated .importlinter.strict to allow necessary imports for the architecture issue tool and new integrations, including GitHub client and Slack delivery (.importlinter.strict) [1] [2] [3].

4. Interactive shell enhancements

  • Improved imports in the interactive shell command registry to support new UI components and highlighting for better CLI/REPL parity (surfaces/interactive_shell/command_registry/cli_parity.py).

References:

/fix #3235

demo shots

Screenshot 2026-07-04 at 5 02 48 PM Screenshot 2026-07-04 at 5 02 55 PM Screenshot 2026-07-04 at 5 03 03 PM Screenshot 2026-07-04 at 5 03 08 PM

@github-actions

github-actions Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Greptile code review

This repo uses Greptile for automated review. Before merge, aim for Confidence Score: 5/5 with zero unresolved review threads — see CONTRIBUTING.md.

Run a review — add a PR comment with:

@greptile review

Give it ~5-10 minutes (sometimes longer) for results, then fix feedback and re-trigger until you reach Confidence Score: 5/5.

Optional: automate with the greploop skill.

@greptile-apps

greptile-apps Bot commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

Greptile Summary

This PR introduces the architecture-issue-tool — a read-only static scanner that detects layering violations, oversized files, compatibility shims, and misplaced modules, and optionally proposes or files GitHub issues for each finding. It wires the tool into both the CLI (opensre architecture-scan) and the REPL (/architecture-scan) with an interactive propose/file-issues menu, and extends GitHub token resolution to also check the integrations store.

  • New tool + CLI surface: tools/architecture_issue_tool/ consolidates all four scanners into scanners.py with lazy CI bridge loading, correct BaseTool base-class detection, and UnicodeDecodeError/SyntaxError guards; scan.py orchestrates the scan-propose-file pipeline; surfaces/cli/commands/architecture_scan.py exposes a Click group with propose and file-issues subcommands that correctly merge group-level flags into subcommand scans via _merged_scan_kwargs.
  • GitHub token resolution: integrations/github/client.py gains resolve_github_token_from_integration_store and expands resolve_github_token to check GITHUB_TOKEN, GH_TOKEN, and GITHUB_MCP_AUTH_TOKEN, though GITHUB_TOKEN_SETUP_HINT (shown in CLI errors) still only names the first two.
  • REPL integration: cli_parity.py adds _cmd_architecture_scan with an interactive picker gated on exclusive_stdin_active; input_policy.py grants exclusive stdin only for plain scans, correctly preventing follow-up menus from racing with prompt_async.

Confidence Score: 4/5

Safe to merge with minor follow-up — the scan and CLI paths are well-tested; the main remaining nits are a stale hint string and an unguarded OS-level read error.

The PR addresses most of the pre-review feedback (lazy CI bridge, UnicodeDecodeError guard, SyntaxError guard, BaseTool base-class detection, clean-repo propose exit, group-flag propagation, explicit GITHUB_REPO forwarding). The outstanding items are: GITHUB_TOKEN_SETUP_HINT omitting GITHUB_MCP_AUTH_TOKEN (users with that env var will see a misleading error message), read_source_safe not catching OS-level read errors (a mid-scan file deletion or permission change would surface as an unhandled traceback), and an import of the private _parse_git_remote_url symbol that is not covered by the type checker.

tools/architecture_issue_tool/scan.py (hint string + private import) and tools/architecture_issue_tool/scanners.py (OSError guard in read_source_safe).

Important Files Changed

Filename Overview
tools/architecture_issue_tool/scan.py Core orchestration for scan, propose, and file-issue workflows; addresses previously flagged empty-proposals and token-check guard issues; new import of private _parse_git_remote_url and an incomplete env-var hint string.
tools/architecture_issue_tool/scanners.py Consolidated scanner replacing the previous per-type file structure; correctly implements lazy CI bridge loading, read_source_safe (UnicodeDecodeError), parse_module (SyntaxError), and inherits_from base-class check; OSError in read_source_safe is still unguarded.
surfaces/cli/commands/architecture_scan.py New Click group implementing architecture-scan, propose, and file-issues subcommands; _merged_scan_kwargs correctly propagates group-level flags to subcommands via ParameterSource checks.
surfaces/cli/commands/architecture_scan_parsing.py Parsing helpers for GITHUB_REPO argument extraction and follow-up CLI argument forwarding; resolve_architecture_scan_repo_scope_from_args correctly handles explicit positional repo argument in follow-up commands.
surfaces/interactive_shell/command_registry/cli_parity.py Adds REPL /architecture-scan handler with interactive propose/file-issues menus; follow-up menu correctly gated on exclusive_stdin_active; run_cli_command result is still unchecked in _run_architecture_scan_github_subcommand before calling _architecture_scan_offer_issue_numbers_follow_up.
integrations/github/client.py Adds resolve_github_token_from_integration_store and extends resolve_github_token to check three env vars; error messages in GitHubRestClient and scan.py only mention two of the three vars.
surfaces/interactive_shell/runtime/utils/input_policy.py Adds _architecture_scan_needs_exclusive_stdin that correctly grants exclusive stdin only for plain scans (no subcommand), preventing concurrent prompt_async from racing with the interactive follow-up menus.
tools/architecture_issue_tool/tool.py Registers find_architecture_violations and propose_github_issues_from_architecture_tasks as read-only tools with correct side_effect_level, cost_tier, and input schemas.

Sequence Diagram

%%{init: {'theme': 'neutral'}}%%
sequenceDiagram
    participant User
    participant REPL as REPL / CLI
    participant ScanOrch as scan.py orchestration
    participant Scanners as scanners.py
    participant GHClient as integrations/github/client.py

    User->>REPL: /architecture-scan (or opensre architecture-scan)
    REPL->>REPL: turn_needs_exclusive_stdin grants exclusive stdin
    REPL->>REPL: _architecture_scan_initial_choice() menu
    User->>REPL: selects propose or file-issues
    REPL->>ScanOrch: run_architecture_scan(repo_root, max_file_lines, include_baselines)
    ScanOrch->>Scanners: scan_dependency_violations()
    ScanOrch->>Scanners: scan_oversized_files()
    ScanOrch->>Scanners: scan_compatibility_shims()
    ScanOrch->>Scanners: scan_misplaced_modules()
    Scanners-->>ScanOrch: list[ArchitectureViolation]
    ScanOrch->>ScanOrch: build_refactor_tasks(violations)
    ScanOrch-->>REPL: scan result dict

    alt propose
        REPL->>ScanOrch: run_architecture_scan_and_propose_github_issues()
        ScanOrch-->>REPL: proposals (read-only)
        REPL->>User: display proposals
    else file-issues
        REPL->>GHClient: resolve_github_token() store then env vars
        REPL->>ScanOrch: run_architecture_scan_and_file_github_issues()
        ScanOrch->>GHClient: execute_github_issue_mutation per proposal
        GHClient-->>ScanOrch: issue results
        ScanOrch-->>REPL: issue results
        REPL->>User: display created issues
    end

    REPL->>User: offer --issue-numbers follow-up menu
Loading
%%{init: {'theme': 'base', 'themeVariables': {"darkMode": true, "background": "#0d1117", "primaryColor": "#21262d", "primaryTextColor": "#e6edf3", "primaryBorderColor": "#8b949e", "lineColor": "#8b949e", "textColor": "#e6edf3", "edgeLabelBackground": "#161b22", "actorBkg": "#21262d", "actorBorder": "#8b949e", "actorTextColor": "#e6edf3", "actorLineColor": "#8b949e", "signalColor": "#8b949e", "signalTextColor": "#e6edf3", "noteBkgColor": "#373320", "noteBorderColor": "#d4a72c", "noteTextColor": "#f0e6c0", "labelBoxBkgColor": "#21262d", "labelBoxBorderColor": "#8b949e", "labelTextColor": "#e6edf3", "loopTextColor": "#e6edf3", "activationBkgColor": "#30363d", "activationBorderColor": "#8b949e"}}}%%
sequenceDiagram
    participant User
    participant REPL as REPL / CLI
    participant ScanOrch as scan.py orchestration
    participant Scanners as scanners.py
    participant GHClient as integrations/github/client.py

    User->>REPL: /architecture-scan (or opensre architecture-scan)
    REPL->>REPL: turn_needs_exclusive_stdin grants exclusive stdin
    REPL->>REPL: _architecture_scan_initial_choice() menu
    User->>REPL: selects propose or file-issues
    REPL->>ScanOrch: run_architecture_scan(repo_root, max_file_lines, include_baselines)
    ScanOrch->>Scanners: scan_dependency_violations()
    ScanOrch->>Scanners: scan_oversized_files()
    ScanOrch->>Scanners: scan_compatibility_shims()
    ScanOrch->>Scanners: scan_misplaced_modules()
    Scanners-->>ScanOrch: list[ArchitectureViolation]
    ScanOrch->>ScanOrch: build_refactor_tasks(violations)
    ScanOrch-->>REPL: scan result dict

    alt propose
        REPL->>ScanOrch: run_architecture_scan_and_propose_github_issues()
        ScanOrch-->>REPL: proposals (read-only)
        REPL->>User: display proposals
    else file-issues
        REPL->>GHClient: resolve_github_token() store then env vars
        REPL->>ScanOrch: run_architecture_scan_and_file_github_issues()
        ScanOrch->>GHClient: execute_github_issue_mutation per proposal
        GHClient-->>ScanOrch: issue results
        ScanOrch-->>REPL: issue results
        REPL->>User: display created issues
    end

    REPL->>User: offer --issue-numbers follow-up menu
Loading

Reviews (20): Last reviewed commit: "style: fix ruff format in architecture-s..." | Re-trigger Greptile

…ibility shim detection and misplaced module handling
@Devesh36

Devesh36 commented Jul 2, 2026

Copy link
Copy Markdown
Collaborator Author

@greptile review

violations: list[ArchitectureViolation] = []
for py_file in iter_python_files(repo_root, roots):
rel_path = str(py_file.relative_to(repo_root))
source = py_file.read_text(encoding="utf-8")

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 UnicodeDecodeError can crash the entire scan

py_file.read_text(encoding="utf-8") raises UnicodeDecodeError for any Python file that contains non-UTF-8 bytes — common in legacy source files with Latin-1–encoded comments or string literals. When that happens the exception propagates all the way out of run_architecture_scan, returning nothing to the caller instead of a partial result. The same unguarded pattern exists in _scan_core_prefix_violations (dependencies.py line 125), scan_misplaced_modules (misplaced.py line 62), and scan_compatibility_shims (shims.py line 72). parse_module in ast_utils.py already handles the analogous SyntaxError case — a matching read_source_safe helper there that catches UnicodeDecodeError and returns None, with callers skipping None results, would close this gap consistently across all four scanners.

…chitecture violation reporting

Added a new command `opensre architecture-scan` for CLI and `/architecture-scan` for interactive shell, enabling users to scan for architecture violations with a deterministic report. The report includes detailed summaries of violation types and integrates with the existing architecture issue tool, enhancing usability and consistency across interfaces. Updated documentation to reflect these changes and ensure clarity on usage.
@Devesh36

Devesh36 commented Jul 2, 2026

Copy link
Copy Markdown
Collaborator Author

@greptile review

…ture issue tool

Updated variable names for clarity and introduced a new Protocol for type hinting in the scanners module. This enhances code readability and maintainability, ensuring better alignment with type expectations in the architecture issue tool.
@Devesh36

Devesh36 commented Jul 2, 2026

Copy link
Copy Markdown
Collaborator Author

@greptile review

…issue tool

Introduced the `propose_github_issues_from_architecture_tasks` function to build read-only GitHub issue proposals from architecture scan tasks. Updated documentation to clarify the process of filing issues and modified existing code to integrate this new functionality. Enhanced tests to ensure proper behavior of the new proposal generation feature.
@Devesh36

Devesh36 commented Jul 2, 2026

Copy link
Copy Markdown
Collaborator Author

@greptile review

… capabilities

Expanded the architecture issue tool to support filing GitHub issues directly from scan results. Introduced new commands for proposing and creating issues via the CLI and interactive shell. Updated documentation to clarify usage and added guidance for GitHub issue creation. Enhanced tests to validate the new functionality and ensure proper integration with existing features.
@Devesh36 Devesh36 changed the title feat(tools): add find_architecture_violations scanner for layering and refactor debt feat(tools): add architecture violation scanner with CLI/REPL GitHub issue workflow (#3235) Jul 2, 2026
Comment thread integrations/github/client.py Fixed
@Devesh36

Devesh36 commented Jul 2, 2026

Copy link
Copy Markdown
Collaborator Author

@greptile-apps review again

Comment thread tools/architecture_issue_tool/scan.py Outdated
Added logging to the GitHub token resolution process to capture warnings for invalid credentials and environment configurations. Introduced a new helper function to parse task indices, improving error handling for invalid inputs. Updated CLI commands to utilize the new parsing function and added tests to ensure proper behavior for invalid task indices.
@Devesh36

Devesh36 commented Jul 2, 2026

Copy link
Copy Markdown
Collaborator Author

@greptile review

…ands with flags

Merge group-level architecture-scan options into subcommand context instead of
overwriting defaults, and parse propose/file-issues even when scan flags precede
the subcommand in CLI and REPL wiring.

Co-authored-by: Cursor <cursoragent@cursor.com>
@Devesh36

Devesh36 commented Jul 2, 2026

Copy link
Copy Markdown
Collaborator Author

@greptile review

Comment thread tools/architecture_issue_tool/scan.py
Short-circuit the integrated CLI workflow when a scan finds no refactor
tasks instead of surfacing the LLM-tool empty-input error as a fatal exit.

Co-authored-by: Cursor <cursoragent@cursor.com>
@Devesh36

Devesh36 commented Jul 2, 2026

Copy link
Copy Markdown
Collaborator Author

@greptile review

Defer importing .github/ci modules until dependency scanning runs so the
tool package loads in pip-installed deployments while still using CI graph
checks from a source checkout.

Co-authored-by: Cursor <cursoragent@cursor.com>
@Devesh36

Devesh36 commented Jul 2, 2026

Copy link
Copy Markdown
Collaborator Author

@greptile review

global _CI_BRIDGE, _CI_BRIDGE_LOAD_ATTEMPTED
if _CI_BRIDGE_LOAD_ATTEMPTED:
return _CI_BRIDGE
_CI_BRIDGE_LOAD_ATTEMPTED = True
Broaden the lazy CI bridge loader to treat renamed or incomplete CI scripts
like an unavailable bridge instead of crashing architecture scans.

Co-authored-by: Cursor <cursoragent@cursor.com>
@Devesh36

Devesh36 commented Jul 2, 2026

Copy link
Copy Markdown
Collaborator Author

@greptile review

Comment thread tools/architecture_issue_tool/scan.py
…st imports

Resolve leftover post-merge lint failures from the upstream ReplSession->Session
rename in cli_parity.py and test_commands.py, and fix import ordering in the
gateway agent lifecycle test.

Co-authored-by: Cursor <cursoragent@cursor.com>
@Davidson3556

Copy link
Copy Markdown
Collaborator

looks good to me, it needs more review from the team

@muddlebee

Copy link
Copy Markdown
Collaborator
/architecture-scan <github-url>

instead of Tracer-Cloud opensre

and other features like file-issues etc should be sub commands and not in the same line

@Devesh36

@Devesh36

Devesh36 commented Jul 3, 2026

Copy link
Copy Markdown
Collaborator Author
/architecture-scan <github-url>

instead of Tracer-Cloud opensre

and other features like file-issues etc should be sub commands and not in the same line

@Devesh36

okay working on it

- Refactor architecture-scan commands to accept GitHub repository URLs in the format `https://github.com/OWNER/REPO` or `OWNER/REPO`.
- Update documentation and examples to reflect the new argument format.
- Enhance error handling for invalid GitHub repository inputs.
- Modify tests to ensure compatibility with the new URL format.

This change improves usability by allowing users to specify repositories more flexibly and clearly.
@Devesh36

Devesh36 commented Jul 4, 2026

Copy link
Copy Markdown
Collaborator Author

@greptile-apps review again

Comment thread surfaces/interactive_shell/command_registry/cli_parity.py Outdated
Merge upstream/main so CI picks up the github_mcp -> integrations.github.mcp
move, update token-resolution imports and tests, preserve scan flags in REPL
follow-up subcommands, and reformat architecture-scan CLI tests.

Co-authored-by: Cursor <cursoragent@cursor.com>
@Devesh36

Devesh36 commented Jul 4, 2026

Copy link
Copy Markdown
Collaborator Author

@greptile-apps review again

Comment thread surfaces/cli/commands/architecture_scan.py
Merge group-level --repo-root, --max-file-lines, and --include-baselines
into propose/file-issues when set before the subcommand, with subcommand
flags taking precedence when explicitly provided.

Co-authored-by: Cursor <cursoragent@cursor.com>
@Devesh36

Devesh36 commented Jul 4, 2026

Copy link
Copy Markdown
Collaborator Author

@greptile-apps review again

@Devesh36

Devesh36 commented Jul 4, 2026

Copy link
Copy Markdown
Collaborator Author
/architecture-scan <github-url>

instead of Tracer-Cloud opensre

and other features like file-issues etc should be sub commands and not in the same line

@Devesh36

Screenshot 2026-07-04 at 12 54 59 PM Screenshot 2026-07-04 at 12 55 17 PM

@muddlebee

Copy link
Copy Markdown
Collaborator

@Devesh36 should be subcommands no?

Default GitHub repo to Tracer-Cloud/opensre, show propose/file-issues menu
up front, offer --issue-numbers follow-up after filing, and rename the
task-indices flag across CLI, REPL, docs, and tests.

Co-authored-by: Cursor <cursoragent@cursor.com>
Comment on lines +86 to +108
def architecture_scan_github_subcommand(args: list[str]) -> str | None:
"""Return ``propose``/``file-issues`` when present, skipping scan-only flags."""
i = 0
while i < len(args):
token = args[i]
if token in _SCAN_OPTION_FLAGS:
i += 2
continue
if token in _SCAN_FLAG_ONLY:
i += 1
continue
if any(token.startswith(f"{flag}=") for flag in _SCAN_OPTION_FLAGS):
i += 1
continue
if token.startswith("-"):
i += 1
continue
lowered = token.lower()
if lowered in ARCHITECTURE_SCAN_GITHUB_SUBCOMMANDS:
return lowered
return None
return None

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

move this and the other parsing logic to a parsing util.

@Davidson3556

Copy link
Copy Markdown
Collaborator

kindly fix the merge conflict @Devesh36

Devesh36 and others added 2 commits July 7, 2026 15:02
Resolve the .importlinter.strict conflict, extract architecture-scan argument
parsing into a shared util per review feedback, trim duplicated CLI decorators,
and add parsing/failure regression tests for the GitHub issue workflow.

Co-authored-by: Cursor <cursoragent@cursor.com>
Resolve .importlinter.strict by adopting the renamed azure_openai path and
dropping stale config.gateway_output_sink ignores after the sink moved to gateway/.

Co-authored-by: Cursor <cursoragent@cursor.com>
@Devesh36

Devesh36 commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator Author

@greptile review

Comment thread surfaces/interactive_shell/command_registry/cli_parity.py Outdated
Parse the positional GITHUB_REPO from scan args when resolving repo scope and
building follow-up CLI commands so --issue-numbers reruns target the user's repo
instead of the git-inferred default.

Co-authored-by: Cursor <cursoragent@cursor.com>
@Devesh36

Devesh36 commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator Author

@greptile review

Co-authored-by: Cursor <cursoragent@cursor.com>
@Devesh36

Devesh36 commented Jul 7, 2026

Copy link
Copy Markdown
Collaborator Author

@greptile review

@VaibhavUpreti

Copy link
Copy Markdown
Member

tagging @w3joe

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Find architecture violations and create refactor tasks

6 participants