Skip to content

Add AI adoption tracking and update related documentation - #25

Merged
gundersen-lumn merged 7 commits into
mainfrom
ai/eric-cop
Jul 20, 2026
Merged

Add AI adoption tracking and update related documentation#25
gundersen-lumn merged 7 commits into
mainfrom
ai/eric-cop

Conversation

@gundersen-lumn

Copy link
Copy Markdown
Collaborator

This pull request introduces a comprehensive, zero-friction AI adoption tracking system using git hooks, and updates documentation and templates to support robust, automated detection and reporting of AI-assisted commits. The main changes include adding an auto-tagging git hook, enhancing issue templates and documentation to guide teams through setup, and updating analysis requirements to leverage these new signals—including handling of squashed commits and merge strategies for accurate adoption metrics.

AI Adoption Tracking Automation

  • Added .githooks/prepare-commit-msg, a git hook that auto-detects and tags AI-assisted commits (e.g., those using Copilot, GPT, Claude, etc.) with [AI-ASSISTED] using multiple signals (explicit keywords, metadata, implicit patterns), ensuring idempotency and zero manual developer action.

  • Updated the README.md to document the AI adoption tracking system, explaining the hook’s operation, setup for Node.js and non-Node projects, and rationale for using a committed .githooks/ path.

Template and Documentation Enhancements

  • Enhanced .github/ISSUE_TEMPLATE/create-copilot-instructions.yml to provide step-by-step instructions for setting up the AI adoption git hook, including scripts for automatic configuration in Node.js projects and a setup script for other languages. The template ensures all developers have the hook with minimal friction.

  • Updated .github/ISSUE_TEMPLATE/comprehensive-codebase-review.yml to:

    • Require detection and documentation of the AI adoption hook.
    • Expand AI signal detection to include hook-based, metadata, and implicit pattern signals.
    • Add requirements for squashed commit analysis, merge strategy documentation, and reporting of detection methods and their coverage. [1] [2]

AI Adoption Analysis Improvements

  • Extended the AI adoption CSV and markdown report specifications to include new columns for squash_merge_indicator and signal_type, and to summarize squashed commit recovery and merge strategies in adoption metrics. [1] [2]

These changes together enable robust, automated, and low-friction tracking of AI-assisted contributions across repositories, improving both the accuracy and transparency of adoption metrics.

…entation

- Introduced a new git hook at `.githooks/prepare-commit-msg` to auto-tag AI-assisted commits.
- Enhanced `create-copilot-instructions.yml` to include setup instructions for AI adoption tracking.
- Updated `README.md` to document the AI adoption tracking system and its integration with existing templates.

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR adds a repo-committed git hook for tagging AI-assisted commits and updates the README and issue templates to document and require AI adoption signal collection (including intended handling of squash-merge workflows) for downstream reporting.

Changes:

  • Added .githooks/prepare-commit-msg to auto-prefix commit subjects with [AI-ASSISTED] based on simple message signals.
  • Expanded documentation and issue templates to guide hook setup and to extend AI adoption reporting requirements (signal type + squash-related fields).
  • Updated the comprehensive review template to require multi-signal detection and include merge/squash analysis requirements.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 7 comments.

File Description
README.md Documents AI adoption tracking via committed git hooks and setup approaches.
.github/ISSUE_TEMPLATE/create-copilot-instructions.yml Adds step-by-step instructions to generate/install the hook and related setup guidance.
.github/ISSUE_TEMPLATE/comprehensive-codebase-review.yml Expands AI adoption analysis requirements (signals, squash handling, output schema).
.githooks/prepare-commit-msg Implements the commit-message auto-tagging hook.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread README.md
Comment thread README.md
Comment thread .github/ISSUE_TEMPLATE/create-copilot-instructions.yml
Comment thread .github/ISSUE_TEMPLATE/comprehensive-codebase-review.yml Outdated
Comment thread .github/ISSUE_TEMPLATE/comprehensive-codebase-review.yml Outdated
Comment thread .githooks/prepare-commit-msg Outdated
Comment thread .githooks/prepare-commit-msg Outdated
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Copilot AI review requested due to automatic review settings July 20, 2026 16:08
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
gundersen-lumn and others added 2 commits July 20, 2026 10:09
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Copilot Autofix powered by AI <175728472+Copilot@users.noreply.github.com>

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (5)

README.md:123

  • The Non-Node setup instructions reference ./setup-dev-env.sh, but the issue template later instructs creating the script under .githooks/setup-dev-env.sh. As written, developers following the README will likely run a non-existent script. Consider pointing to the .githooks/ path (or explicitly stating the script is created at repo root).
Run the setup script once after cloning:

```bash
./setup-dev-env.sh
**README.md:180**
* This section says “No Setup Scripts Needed”, but earlier the README describes a non-Node setup script and Node npm scripts for configuring `core.hooksPath`. The current heading/bullets are contradictory and can confuse adopters about whether any setup automation exists/should be used.

No Setup Scripts Needed

The system is designed for simplicity:

  • No separate setup-ai-hooks.sh script required
  • The issue template creates files directly
**.github/ISSUE_TEMPLATE/create-copilot-instructions.yml:311**
* The template instructs creating the setup script at `.githooks/setup-dev-env.sh` (line 279), but the later example tells developers to run `./setup-dev-env.sh` from the repo root. This mismatch will cause copy/paste setup failures.
    ./setup-dev-env.sh
**.github/ISSUE_TEMPLATE/comprehensive-codebase-review.yml:201**
* The proposed “squashed commit recovery” steps are not technically achievable from git history after a squash-and-merge. In particular, `git log <merge-commit>^..<merge-commit>` will only show the squash commit itself (the pre-squash commits are not in the target branch history), and `git diff <commit>^..<commit>` can’t be run for commits that no longer exist in the branch graph. This guidance will lead to incorrect/irreproducible adoption reporting unless it explicitly relies on PR metadata (GitHub API).
          - If PR metadata is not available locally, document that individual pre-squash commits cannot be reliably recovered from git history alone.
          - Attribute detected AI signals to the original PR commit authors (not the squash-merge author) when PR commit data is available.
          - Document the observed merge strategy (squash vs. merge vs. rebase) and any recovery limitations in the Caveats section.
          - For squashed commits with AI signals, preserve the original author/date and create a logical "unmerged" entry in ai-adoption.csv with the source commits listed in a supplementary column.
          - If PR metadata is available (GitHub API), cross-reference the PR number from the merge commit message to retrieve the full commit history before squashing.
**README.md:149**
* This bullet mixes repo-root (`./setup-dev-env.sh`) with “in the `.githooks` folder”. If the script lives under `.githooks/`, the path should be `.githooks/setup-dev-env.sh` (and the run command should reflect that), otherwise it should say it’s created at repo root.
  • Create ./setup-dev-env.sh in the .githooks folder
</details>


Comment thread .github/ISSUE_TEMPLATE/create-copilot-instructions.yml
Comment thread .github/ISSUE_TEMPLATE/comprehensive-codebase-review.yml Outdated
Comment thread .githooks/prepare-commit-msg
Copilot AI review requested due to automatic review settings July 20, 2026 16:14

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

Comments suppressed due to low confidence (4)

README.md:180

  • This section says “No Setup Scripts Needed”, but earlier the README requires a setup script for non-Node projects. This contradiction can cause teams to skip necessary setup and end up with hooks not configured.
### No Setup Scripts Needed

The system is designed for simplicity:
- No separate `setup-ai-hooks.sh` script required
- The issue template creates files directly

.github/ISSUE_TEMPLATE/create-copilot-instructions.yml:312

  • This template says to create .githooks/setup-dev-env.sh, but the instructions later tell developers to run ./setup-dev-env.sh (repo root). That mismatch will cause the documented command to fail for non-Node repos.
        Before starting development, run the setup script once:

        ```bash
        ./setup-dev-env.sh
        ```

.githooks/prepare-commit-msg:43

  • Using echo to write the remainder of the commit message can be lossy/unsafe (some shells treat leading -n or backslash escapes specially). Using printf avoids option/escape interpretation and preserves the message content more reliably.
    echo "[AI-ASSISTED] $FIRST_LINE" > "$COMMIT_MSG_FILE"
  else
    echo "[AI-ASSISTED] $FIRST_LINE" > "$COMMIT_MSG_FILE"
    echo "$REST" >> "$COMMIT_MSG_FILE"
  fi

README.md:123

  • The README instructs non-Node projects to run ./setup-dev-env.sh, but the issue template guidance creates the script under .githooks/ (and elsewhere in this README it says the script lives in the .githooks folder). Align the run command with the actual script location to avoid broken setup steps.
**For Non-Node Projects (Java, Python, etc.):**

Run the setup script once after cloning:

```bash
./setup-dev-env.sh
</details>


Comment thread .githooks/prepare-commit-msg
Comment thread .githooks/prepare-commit-msg
@gundersen-lumn
gundersen-lumn merged commit 2e9e340 into main Jul 20, 2026
1 check passed
@gundersen-lumn
gundersen-lumn deleted the ai/eric-cop branch July 20, 2026 18:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants