# How to measure impact of Copilots using Hivel?

While AI Copilots like GitHub Copilot don't provide direct data on their usage (e.g., suggestions and acceptances), you can evaluate their impact through Hivel by focusing on key productivity metrics. 

Here’s a step-by-step guide:

***

### **1. Establish a dedicated Copilot team**

* Identify a team of developers who will adopt Copilot for their daily workflows.
* Ensure the team is diverse in roles and experience levels to gain balanced insights.
* [<mark style="color:purple;">Create the team</mark>](https://docs.hivel.ai/setup/teams/how-to-create-teams) in Hivel.

***

### **2. Define key metrics to measure impact**

Consider certain metrics around delivery speed, throughput, and quality whose change will indicate the effectivenes of the Copilot on developer's productivity.

These are the metrics that we recommend to be tracked:

**Core Metrics**

1. **Cycle Time**: The time it takes from first commit of a PR till the PR is opened.
2. **Deployment Frequency**: How often code is merged to release branch.
3. **Active Days**: The number of days developers actively contribute to the codebase.
4. **Commits**: The number of commits per developer or team.
5. **Open PRs**: The number of PRs opened.
6. **Completed Story Points**: Total story points completed within the time period.

**Optional Metrics**

* **Review Time**: Measure whether Copilot affects the time it takes to review and approve PRs.
* **Rework and Change Failure Rate**: Track any correlation between Copilot usage and code quality.

***

### **3.** Create a dashboard to track the metrics:

In the custom dashboards, create a private or public dashboard with the metrics and save a filter from when Copilot was introduced.

<figure><img src="/files/R1hN03XdWrjifPyZRiza" alt=""><figcaption></figcaption></figure>

***

### **4. Analyze Metrics After Copilot Adoption**

* Monitor the Copilot team’s metrics **post-adoption** to see if the metrics have improved or not.
* Subsequently, check the same for other teams to understand if the impact is generally across the org or only for the Copilot team.
* Utilize the tabular view to get a quick snapshot of the same.<br>

  <figure><img src="/files/Abp7w3k6FKsFCoOvmHl3" alt=""><figcaption></figcaption></figure>

### **Key Questions to Ask:**

* Has cycle time improved?
* Is deployment frequency higher?
* Are there changes in the number of active days or contributions (e.g., commits, PRs)?
* Are story points completed more consistently or at a faster rate?

***

### **5. Interpret the Results**

Look for trends or significant changes in the metrics:

* **Positive Indicators**: Faster cycle times, higher deployment frequency, and increased completed story points suggest that Copilot is helping developers ship faster and more effectively.
* A meaningful performance gap favoring the Copilot team over the control group strengthens the case for its impact.

***

By following these steps, you can derive actionable insights into how Copilot contributes to developer productivity, even without direct usage data. For assistance with setting up metrics dashboards, reach out to us.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://docs.hivel.ai/faqs/reporting-and-goal-tracking-faqs/how-to-measure-impact-of-copilots-using-hivel.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.