GitHub Copilot: Expanded Metrics Guide

GitHub Copilot offers insights into user activity through its Copilot Metrics API, which aggregates telemetry data from the IDE. These metrics are provided by GitHub, and Hivel’s platform visualizes them. Below is a guide to understanding these metrics and their key definitions.

---

🔍 Metric Categories

These categories break down how Copilot metrics are measured:

Area	Metrics Covered
Adoption	Assigned seats, Active users
Engagement	Suggestions shown, Suggestions accepted, Active days per user
Effectiveness	Acceptance rate (lines accepted vs suggested), Total lines suggested/accepted
Usage Depth	IDE vs Chat completions, average usage per day

---

🧮 Key Metric Definitions

✅ Total Lines Suggested

• Definition: The total number of lines suggested by GitHub Copilot, including inline, block, or multi-line completions.

• Tracked Per: User, per day

✅ Total Lines Accepted

• Definition: The lines from Copilot suggestions that the user accepts explicitly via IDE interactions (e.g., pressing Tab or Enter).

• Tracked Per: User, per day

• Notes: Even if the accepted suggestion is later deleted, it is still logged.

✅ Acceptance Rate

• Formula:

  
• Definition: The percentage of suggested lines that were accepted. This metric is a measure of adoption, not quality.

• Implications: A higher acceptance rate may indicate better alignment with developer intent or greater usability.

---

👥 User Classification

✅ Active Users

• Definition: A user who has received at least one suggestion or has interacted with Copilot Chat on that day.

⚠️ No formal “Passive” User Metric

• How to Identify: You can estimate passive behavior by checking for:

  • A seat is assigned but no suggestions received or accepted in the last X days.

  • No threshold (e.g., 10 lines/day) is explicitly set for activity.

---

🧱 Technical Foundations

• Telemetry: GitHub tracks usage metrics using telemetry from the user’s IDE (with opt-in).

• Finalization: Metrics are finalized at the end of each UTC day and have a 1-day lag.

• Rolling Window: Data is available for up to 28 days of history.

⚠️ Known Limitations

Scenario	Counted in Metrics?	Notes
Full suggestion accepted via Tab/Enter	✅ Yes	Counts as accepted lines.
Partially accepted suggestions (some lines only)	✅ Yes (partial)	Only the accepted lines are logged.
Copy-pasting suggestion content (manual)	❌ No	Only interactions via IDE UI are counted.
Rewriting code based on AI inspiration	❌ No	GitHub does not capture intent through telemetry.
Accepting a suggestion, then deleting it soon after	✅ Yes	The deletion may not be fully filtered out, possibly inflating adoption slightly.
Using Copilot in offline mode	❌ No	No telemetry is captured, so no metrics are logged.
Chat suggestion used but edited before insertion	❌/⚠️ Ambiguous	Depends on how the chat event was logged in the system.

Summary: Copilot metrics may not capture the full extent of AI influence, as partial usage, copy-pasting, and inspiration-driven code revisions are not tracked.

---

📦 Sample Copilot API Payload

Here’s an example of a typical API payload with metrics:

jsonCopy{
  "timestamp": "2025-07-29",
  "active_users": 21,
  "total_code_lines_suggested": 5800,
  "total_code_lines_accepted": 1640,
  "acceptance_rate": 28.27
}

In this example:

• 21 active users interacted with Copilot.

• 5,800 lines were suggested in total.

• 1,640 lines were accepted.

• The acceptance rate is 28.27%.

---

This guide provides an overview of the key metrics available via GitHub Copilot. By understanding these metrics, you can better track adoption, engagement, effectiveness, and usage depth of Copilot in your development environment.