Accessibility metrics and JIRA

Many companies use JIRA for bug-tracking and JIRA’s query language can help you pull out some great metrics for managing your company’s product accessibility.

Needle in the Haystack

The first key to JIRA tracking is to understand how to find accessibility issues. Most companies will use the term accessibility to reference data access, servers offline, setting up employee security access, etc. You cannot depend on a text search for accessibility. Further, your company may have a accessibility option within the worktype selector. This also will be full of unusable tickets.

With this in mind, it’s best to look at alternative methods for defining which tickets are for accessibility. At Intuit, we add “accessibility” to the labels field on appropriate issues. This has worked well, but unfortunately is dependent on people adding the issue. I will also do regular searches of JIRA for “aria-“, “keyboard accessibility”, “VoiceOver”, “JAWS”, and similar queries to find issues that lack the accessibility label. You can now begin using the below JQL queries to surface your issues once you have this label established.

JIRA Query Language (JQL) Queries

I am using the following queries to build an accessibility dashboard within JIRA. This has become a tremendous asset for understanding the activity and breadth of changes.

All bugs with accessibility label

labels=accessibility

This is your base query. This will pull up all issues that have the accessibility label. The data is raw and we’ll begin fine tuning it below

Find issues that need the accessibility label

text ~ “aria-” and labels != accessibility

I’m using “aria-” in this text search and hiding tickets that already have the accessibility label. This will allow you to quickly go through your tickets and start adding the label. Switch up the query to include other accessibility terms

All open accessibility bugs

labels=accessibility and status=Open

We will use the status field often. Status options may vary if your company has customized the set. Common options are:

• Open
• In Progress
• Blocked
• Integration
• Reopened
• Closed

Blocked and Won’t Fix

labels=accessibility and status = blocked

labels=accessibility and resolution = “Won’t Fix”

These are important issues and I would suggest filtering them within the last year or similar time period to make sure the new issues stand out and not legacy issues. These are the tickets that need your attention!

All closed accessibility bugs

labels=accessibility and status=Closed

accessibility bugs closed within the last 4 weeks

labels=accessibility and status=Closed and resolved > “-4w”

Now we are starting to look at the performance of our engineers. This query is a nice one to see the current rate of issue completion. This is saying the resolved date is less than four weeks ago. I like to add this to a dashboard as a piechart by project.

accessibility bugs closed since beginning of FY17

labels=accessibility and status=Closed and resolved > “2016/08/01”

Intuit’s fiscal year starts on August 1. This query allows you to see how many issues have been closed since a particular date. You may change this to January 1, 2016 or similar date.

accessibility bugs that are open and have been updated in FY17

labels=accessibility and status =Open and updated > “2016/08/01”

This will let you know which bugs have been updated since a particular date. Updated may mean the issue was reassigned, a new comment was added, it was added to a sprint, or other touches. This is good to see which issues are still alive.

accessibility bugs that are not open or closed

labels=accessibility and status in (“In Progress”, New, Verify) ORDER BY status DESC

The middle zone between Open and Closed can be very interesting. This query will also sort the results by status and start with Verify (descending alphabeticallly).

This post will be updated as I create new queries and please share your favorite JIRA queries in the comments below.

Creating Dashboards

I’ll do a post later on making dashboards, but I wanted to share a lesson I learned the hard way. JIRA allows you to create custom dashboards and you can add these filters to that dashboard to watch progress with nice charts and tables. You can also share that dashboard with other people. However, you MUST save the query as a filter and then share that filter before it can be viewed by other people on your shared dashboard. It’s a bit backwards and confusing, but the takeaway is to not save a search query to your dashboard without first saving it as a filter.