Why Did GitHub Copilot’s Growth Stall According to JetBrains?

GitHub Copilot emerged as a revolutionary AI-powered pair programmer that promised to redefine developer workflows. As an AI product lead with extensive experience shipping AI-assisted B2B SaaS workflows, I always emphasize that the model is not the product. The true question is: how does it integrate into workplace usage and influence tool adoption within the developer workflow? Recent observations from JetBrains, a leader in developer tools, indicate that Copilot’s initial explosive growth has plateaued, raising questions about what really drives or stalls AI assistant adoption at scale.

In this analysis, we will unpack the core factors behind GitHub Copilot’s growth stalling, drawing on vehicle frameworks like the PM Toolkit tools, incorporating learnings from enterprise clients like PayPal, AI firms such as H2O.ai, and patterns I’ve seen embed into spreadsheets and lightweight eval suites prior to fully automated pipelines.

Understanding the Growth Stall: Beyond the Model

At first glance, it’s tempting to attribute Copilot’s slowed growth to technical shortcomings or model limitations. Yet, JetBrains highlights a crucial insight: Task-dependent model behavior and failure modes profoundly impact user retention and adoption.

GitHub Copilot’s underlying language models excel in many contexts, but the variance in performance across different programming tasks creates friction. For example:

  • Simple autocomplete tasks — it shines.
  • Complex algorithm design — results can be unpredictable or erroneous.
  • Context switching during multi-file projects — occasionally struggles to keep relevant context.

Because Copilot is embedded into the developer workflow but is not perfectly reliable across all tasks, developers often default back to familiar tooling when the AI’s suggestions impede, rather than help.

The Model Is Not the Product

This point cannot be overstated. The performance of the AI model itself is just one component of the product experience. From my experience at H2O.ai and PayPal, you must consider:

  • Integration quality: How seamlessly does the tool fit into existing workflows?
  • Usability and onboarding: How quickly can users internalize the tool’s value?
  • Fail safes and kill switches: Can users easily revert or dismiss unwanted suggestions?

Copilot delivers good model outputs, but the absence of a well-defined user evaluation process before release sometimes results in misaligned expectations and early churn.

Why Defining an Eval Before Release Matters

JetBrains’ analysis aligns well with a principle I codified in my personal checklist called the ship/no-ship thresholds. Before integrating AI-powered tools like Copilot into the developer ecosystem, teams must define:

  1. The specific workflow tasks the model will augment or automate.
  2. Failure modes that would cause an immediate rollback.
  3. Success criteria measured in impact on velocity, cycle time, activation rates, and retention.

Here’s where PM Toolkit’s suite of calculators is invaluable. Using calculators for:

  • RICE, ICE, and Kano scoring frameworks to prioritize feature improvements focused on real developer pain points.
  • Business case, ROI, and pricing calculators to justify ongoing investment in the AI tool based on measured usage.
  • A/B test and sample size calculators to rigorously verify uplift claims.

For instance, a team might evaluate Copilot’s value by comparing its conversion rate to active usage over time (DAU/MAU), or monitor activation rates post-onboarding. Without this empirical foundation, expectations become hand-wavy “accuracy improved” claims — a pitfall I see often with new AI tools shipped in rushed cycles.

Automation vs Augmentation vs Innovation Patterns

Copilot’s stall can also be explained through the classic lens of workflow impact patterns:

Pattern Description Example in Developer Tools Copilot’s Fit Automation Replacing manual tasks end-to-end CI/CD pipelines automating deployments Limited - Copilot assists but doesn't fully automate coding Augmentation Helping users do tasks faster or better Code autocompletion and syntax highlighting Primary mode - Copilot generates suggestions to speed coding Innovation Enabling tasks not previously possible or broadening scope AI-driven code refactoring or automatic bug detection (future) Emerging - potential to evolve with stronger context understanding

Copilot shines in augmentation but does not yet automate entire workflows or reliably introduce disruptive innovation. This limits how deeply it penetrates the daily habits of developers, especially at enterprise scale.

What Do Developers Do Today? Understanding Existing Workflows

One of my “quirks” from the trenches: Always ask “What do users do today?” before proposing AI-driven workflow changes. For GitHub Copilot, this means acknowledging that developers:

  • Rely heavily on trusted IDEs and environments like JetBrains’ suite or VS Code
  • Use numerous plugins and linters to catch issues early
  • Often prefer manual control over code to avoid introducing ambiguous errors
  • Collaborate via code review workflows that mandate clarity and code style consistency

Copilot attempts to insert itself into these well-oiled workflows but sometimes disrupts developer mental models, especially when suggestions introduce subtle bugs or require verification overhead.

Bridging the Gap with Evaluation Spreadsheets

Before organizations at PayPal, H2O.ai, or elsewhere commit to adopting Copilot-like solutions, they create evaluation spreadsheets mimicking real user stories:

  • Define expected input and output for typical dev tasks (e.g., boilerplate code, API consumption, refactoring)
  • Record model outputs alongside failure instances flagged by devs or code reviewers
  • Quantify impact on time saved, errors reduced, and overall developer satisfaction

This hands-on evaluation exposes the nuanced failure modes and task-dependent performance variability that JetBrains highlights. It also helps build the dataset to drive fine-tuning, interface improvements, or new feature prioritization.

Key Takeaways and The Road Ahead

GitHub Copilot’s slowed growth as observed by JetBrains is less about the language model capabilities and more about:

  1. Task-dependent model performance gaps creating inconsistency in real-world usage.
  2. Lack of rigorous, workflow-oriented evaluation defining clear acceptance thresholds prior to release.
  3. Product teams underestimating the importance of seamless integration and fail-safe mechanisms.
  4. The challenge of moving from AI augmentation to genuine automation or innovation within dev workflows.

Companies like JetBrains, PayPal, and H2O.ai have shown that successful AI tool adoption hinges not on “model wars” but on a clear focus on user workflows, robust evaluation using tools like PM Toolkit calculators, and a commitment to iterate on product experience informed by data.

For AI product managers and teams eyeing workplace AI tools, the core lessons are:

  • Define your eval early — what exactly will success look like in your users’ hands?
  • Measure impact on relevant metrics — activation rate, DAU/MAU, retention, NPS, velocity, cycle time.
  • Design for diverse task failure modes — and build kill switches or fallback modes to maintain trust.
  • Focus on augmentation patterns first, but plan for automation and innovation horizons.

GitHub Copilot’s journey illustrates how building an AI model is the starting line, not the finish line. The product’s ultimate success or stall depends on the nuanced interplay between model, workflow, and human factors — a truth echoed across the computer use agents examples AI tools landscape.