Learn how your company can collect and share data metrics for accessible product development, employee resources, and customer outreach. This presentation reflects lessons learned from the Intuit Accessibility team and other technology companies. Continue Reading Accessibility Data Metrics and Reporting – Industry Best Practices
The key to good decision making is evaluating the available information – the data – and combining it with your own estimates of pluses and minuses. As an economist, I do this every day.
–Emily Oster, Brown University
Continue Reading Accessibility Data Metrics and Reporting – Industry Best Practices
Many companies use JIRA for bug-tracking and JIRA’s query language can help you pull out some great metrics for managing your company’s product accessibility.
Needle in the Haystack
The first key to JIRA tracking is to understand how to find accessibility issues. Most companies will use the term accessibility to reference data access, servers offline, setting up employee security access, etc. You cannot depend on a text search for accessibility. Further, your company may have a accessibility option within the worktype selector. This also will be full of unusable tickets.
With this in mind, it’s best to look at alternative methods for defining which tickets are for accessibility. At Intuit, we add “accessibility” to the labels field on appropriate issues. This has worked well, but unfortunately is dependent on people adding the issue. I will also do regular searches of JIRA for “aria-“, “keyboard accessibility”, “VoiceOver”, “JAWS”, and similar queries to find issues that lack the accessibility label. You can now begin using the below JQL queries to surface your issues once you have this label established.
JIRA Query Language (JQL) Queries
I am using the following queries to build an accessibility dashboard within JIRA. This has become a tremendous asset for understanding the activity and breadth of changes.
- All bugs with accessibility label
- This is your base query. This will pull up all issues that have the accessibility label. The data is raw and we’ll begin fine tuning it below
- Find issues that need the accessibility label
- text ~ “aria-” and labels != accessibility
- I’m using “aria-” in this text search and hiding tickets that already have the accessibility label. This will allow you to quickly go through your tickets and start adding the label. Switch up the query to include other accessibility terms
- All open accessibility bugs
- labels=accessibility and status=Open
- We will use the status field often. Status options may vary if your company has customized the set. Common options are:
- In Progress
- Blocked and Won’t Fix
- labels=accessibility and status = blocked
- labels=accessibility and resolution = “Won’t Fix”
- These are important issues and I would suggest filtering them within the last year or similar time period to make sure the new issues stand out and not legacy issues. These are the tickets that need your attention!
- All closed accessibility bugs
- labels=accessibility and status=Closed
- accessibility bugs closed within the last 4 weeks
- labels=accessibility and status=Closed and resolved > “-4w”
- Now we are starting to look at the performance of our engineers. This query is a nice one to see the current rate of issue completion. This is saying the resolved date is less than four weeks ago. I like to add this to a dashboard as a piechart by project.
- accessibility bugs closed since beginning of FY17
- labels=accessibility and status=Closed and resolved > “2016/08/01”
- Intuit’s fiscal year starts on August 1. This query allows you to see how many issues have been closed since a particular date. You may change this to January 1, 2016 or similar date.
- accessibility bugs that are open and have been updated in FY17
- labels=accessibility and status =Open and updated > “2016/08/01”
- This will let you know which bugs have been updated since a particular date. Updated may mean the issue was reassigned, a new comment was added, it was added to a sprint, or other touches. This is good to see which issues are still alive.
- accessibility bugs that are not open or closed
- labels=accessibility and status in (“In Progress”, New, Verify) ORDER BY status DESC
- The middle zone between Open and Closed can be very interesting. This query will also sort the results by status and start with Verify (descending alphabeticallly).
This post will be updated as I create new queries and please share your favorite JIRA queries in the comments below.
I’ll do a post later on making dashboards, but I wanted to share a lesson I learned the hard way. JIRA allows you to create custom dashboards and you can add these filters to that dashboard to watch progress with nice charts and tables. You can also share that dashboard with other people. However, you MUST save the query as a filter and then share that filter before it can be viewed by other people on your shared dashboard. It’s a bit backwards and confusing, but the takeaway is to not save a search query to your dashboard without first saving it as a filter.