How does Uplevel classify Issues?

Overview

Uplevel’s ML Issue Classification model provides key visibility into engineering effort and time spent on New Feature Development vs. Defects vs. Sustenance without dependencies on how teams are tagging work.

How it works

The Uplevel Machine Learning (ML) Issue Classifier uses issue metadata from standard fields like Issue Type, Summary, Description, and Assignee, as well as similar Epic metadata if the issue is linked. From there, an issue is classified as one of three categories:

• Defect - This category contains issues for work related to bugs and defects.
• New value creation - This category contains issues for work related to new features and enhancements.
• Sustenance - This category contains issues for work related to operational efficiency and improvements to maintain and improve reliability and safety. Some types of work included are KTLO, maintenance, and tech debt.

Details

• Uplevel first classifies issues as defect or not defect. Uplevel then classifies the non-defect issues as either new value creation or sustenance.
• Uplevel utilizes a default confidence percentage of 60% to categorize the issues. If below 60% confidence for a specific issue, Uplevel will aggregate the work under “Not linked to a ML Issue Classification”. This confidence percentage is configurable.

****💡Example: A task titled "BE: verify issue handling and NPE logic" has been flagged as Sustenance, but with low confidence it is only slightly favored to be sustenance compared to a defect. In this case, Uplevel would put it into "Uncertain Classification" by default.

Accuracy / Results

Uplevel has seen great success in the accuracy of our ML Issue Classification model with current customers to-date. We’ve directly compared our ML Issue Classification models to custom aggregations and manually tagged categories already leveraged by Uplevel customers today, which has provided the following results:

1. The current model for defect vs not defect:
   • Overall ~93% (87%-96%) of issues have the predicted classification matching the manual classification. Looking at allocation time it is ~93% (86%-99%) matching.
2. The current model for new value creation vs sustenance:
   • Overall ~70% (64%-77%) of issues have the predicted classification matching the manual classification. Looking at allocation time it is ~71% (69%-76%) matching.

FAQ

• Is this an LLM?
  • No, it is not an LLM. It is an ML (machine learning) model.
• **Does my data get sent to a third party?**No, the data stays within Uplevel using our own proprietary ML model and does not use a third party.