Accessibility Data Metrics Survey Results

Survey PointIn preparation for an upcoming talk at CSUN 2017 on data metrics for accessibility, I created a survey for fellow accessibility managers to share what they are doing to quantify the accessibility of their products and services. The following is the raw survey results and I will continue working with the data and responses as we prepare for the presentation: Accessibility Data Metrics and Reporting – Industry Best Practices

Some Initial Observations:

  • Some of the questions were confusing. This is especially true when asking about percentages, as we may know the number of events, but could never compare them to an unknown larger set.
  • WCAG conformance is the starting point for many teams. This includes closed captioning support and issues by category.
  • Most people track the priority of issues, but how is this determined? Is it the priority set in bug tracking software or determined by automated tools?
  • Compliance is a key metric for product managers (VPAT, AODA, and CVAA)
  • Employee and customer participation with open source, conferences, training, and beta testing is a key opportunity for metrics.
  • Few companies are using Net Promoter Score for customer satisfaction regarding accessibility.

You can begin understanding your company’s progress by harnessing the data within your bug tracking software. This article has more information: Accessibility Metrics and JIRA.

Question 1:  Which of the following product quality metrics are you currently tracking? (select all that apply)

Ranked by popularity

  1. Issues by Priority – 77%
  2. Issues by WCAG category – 54%
  3. last time product had a manual evaluation – 46%
  4. Closed Captioning support – 38%
  5. Number of accessibility bugs opened per month/year – 31%
  6. Test coverage for a product – 23%
  7. Number of accessibility bugs marked as blocked – 23%
  8. Automated tests pass/fail – 23%
  9. Number of accessibility bugs closed per month/year – 23%
  10. Percentage of UI test cases written within a time period/release that test for explicit accessibility requirements (e.g. keyboard, ARIA,  markup…) – 15%
  11. Average time to close new bugs – 8%
  12. Most common errors detected via automated testing – 8%
  13. Error density per page (# of issues/size of page) – 0
  14. Percentage of UI tickets closed within a time period/release that were tagged with accessibility (had accessibility requirements) – 0

Question 2: Which of the following metrics do you track for your company’s accessibility Management (select all that apply)

Ranked by popularity

  1. AODA compliance – 55%
  2. Number of media inquiries about product/company accessibility – 45%
  3. Conference talks by employees on accessibility – 45%
  4. Product VPAT Coverage – 45%
  5. Compliance towards 21st Century Communications Act (CVAA) – 36%
  6. Status of PDF Accessibility – 36%
  7. Training completion per team/engineer – 27%
  8. Percentage of company-paid conference attendees attending accessibility conferences – 27%
  9. Customer empathy training completion – 18%
  10. Percentage of employees that have self-disclosed as having a disability (section 503) – 18%
  11. Number of times product developers used a screen reader or other AT 9%
  12. Open source contributions related to accessibility – 0

Question 3: Customer-focused data metrics

  1. Percentage of usability participants who have a disability – 63%
  2. Quantity of feedback via customer support channels – 50%
  3. Percentage of usability activities including user personas with disabilities – 38%
  4. Customer support calls by product/technology 38%
  5. Number of tickets filed by customer support – 38%
  6. Percentage of beta testers with a disability – 25%
  7. Customers who use our products with AT – 13%
  8. Net Promoter Score for customers with a disability – 13%


Accessibility metrics and JIRA

Many companies use JIRA for bug-tracking and JIRA’s query language can help you pull out some great metrics for managing your company’s product accessibility.

Needle in the Haystack

The first key to JIRA tracking is to understand how to find accessibility issues. Most companies will use the term accessibility to reference data access, servers offline, setting up employee security access, etc. You cannot depend on a text search for accessibility. Further, your company may have a accessibility option within the worktype selector. This also will be full of unusable tickets.

With this in mind, it’s best to look at alternative methods for defining which tickets are for accessibility. At Intuit, we add “accessibility” to the labels field on appropriate issues. This has worked well, but unfortunately is dependent on people adding the issue. I will also do regular searches of JIRA for “aria-“, “keyboard accessibility”, “VoiceOver”, “JAWS”, and similar queries to find issues that lack the accessibility label. You can now begin using the below JQL queries to surface your issues once you have this label established.

JIRA Query Language (JQL) Queries

I am using the following queries to build an accessibility dashboard within JIRA. This has become a tremendous asset for understanding the activity and breadth of changes.

All bugs with accessibility label
This is your base query. This will pull up all issues that have the accessibility label. The data is raw and we’ll begin fine tuning it below
Find issues that need the accessibility label
text ~ “aria-” and labels != accessibility
I’m using “aria-” in this text search and hiding tickets that already have the accessibility label. This will allow you to quickly go through your tickets and start adding the label. Switch up the query to include other accessibility terms
All open accessibility bugs
labels=accessibility and status=Open
We will use the status field often. Status options may vary if your company has customized the set. Common options are:

  • Open
  • In Progress
  • Blocked
  • Integration
  • Reopened
  • Closed
Blocked and Won’t Fix
labels=accessibility and status = blocked
labels=accessibility and resolution = “Won’t Fix”
These are important issues and I would suggest filtering them within the last year or similar time period to make sure the new issues stand out and not legacy issues. These are the tickets that need your attention!
All closed accessibility bugs
labels=accessibility and status=Closed
accessibility bugs closed within the last 4 weeks
labels=accessibility and status=Closed and resolved > “-4w”
Now we are starting to look at the performance of our engineers. This query is a nice one to see the current rate of issue completion. This is saying the resolved date is less than four weeks ago. I like to add this to a dashboard as a piechart by project.
accessibility bugs closed since beginning of FY17
labels=accessibility and status=Closed and resolved > “2016/08/01”
Intuit’s fiscal year starts on August 1. This query allows you to see how many issues have been closed since a particular date. You may change this to January 1, 2016 or similar date.
accessibility bugs that are open and have been updated in FY17
labels=accessibility and status =Open and updated > “2016/08/01”
This will let you know which bugs have been updated since a particular date. Updated may mean the issue was reassigned, a new comment was added, it was added to a sprint, or other touches. This is good to see which issues are still alive.
accessibility bugs that are not open or closed
labels=accessibility and status in (“In Progress”, New, Verify) ORDER BY status DESC
The middle zone between Open and Closed can be very interesting. This query will also sort the results by status and start with Verify (descending alphabeticallly).

This post will be updated as I create new queries and please share your favorite JIRA queries in the comments below.

Creating Dashboards

I’ll do a post later on making dashboards, but I wanted to share a lesson I learned the hard way. JIRA allows you to create custom dashboards and you can add these filters to that dashboard to watch progress with nice charts and tables. You can also share that dashboard with other people. However, you MUST save the query as a filter and then share that filter before it can be viewed by other people on your shared dashboard. It’s a bit backwards and confusing, but the takeaway is to not save a search query to your dashboard without first saving it as a filter.

Ubiquitous Transactions – Financial Future and Accessibility

This short presentation was created for the Wearable Computing Solutions panel at the M-Enabling Summit 2016. It introduces some new and upcoming standards that could simplify financial transactions and thus making them more accessible.

Continue Reading Ubiquitous Transactions – Financial Future and Accessibility

Notes from G3ict International Briefing: Inclusive Financial Services for Seniors and Persons with Disabilities

I was invited to represent Intuit at a meeting in Paris to discuss how International financial institutions could provide accessible experiences for their customers and employees. The event was organized by G3ICT and built upon the work done for the Convention on the Rights of Persons with Disabilities. The following are notes I’ve extracted from the presentations listed below. There’s a wealth of information contained within these presentations and each is worth opening and studying.

Ted Drake Intuit and Robin Sargent, Business Development Executive, IBM Accessibility

Speakers and links

  • Matt Ater, Vice President, Services, Freedom Scientific • Presentation and Presentation
  • Dominique Burger, President, BrailleNet, European eAccessibility Forum
  • Alireza Darvishy, Head, Accessibility Center, Crédit Suisse • Presentation
  • Ted Drake, Accessibility Principal Engineer, Intuit • Presentation
  • Gita Esmieu, Director, Financial Services Accessibility Program, G3ict
  • Serge Leblal, Editorial Director, CIO and Le Monde Informatique
  • Axel Leblois, President and Executive Director, G3ict • Presentation
  • Dr. Monique Mai, Accessibility Group Department, Director “Communication, Remote Sales & Public Affairs,” Orange Group • Presentation
  • Sara Mansell, Design Lead, IBM • Presentation
  • Jean-Michel Mépuis, Director, Sustainable Development and CSR, Société Générale
  • Nicola Palmarini, Global Digital Creative & Technology Advocate, IBM Accessibility / IBM Research • Presentation
  • Inmaculada Placencia-Porrero, Deputy Head of Unit for Rights of Persons with Disabilities, Directorate General for Justice, European Commission • Presentation
  • Jean Royné, Director General, IT News Info
  • Robin Sargent, Business Development Executive, IBM Accessibility • Presentation
  • Paul Smyth, Head of IT Accessibility, Barclays • Presentation
  • James Thurston, Vice President, Global Strategy and Development, G3ict • Presentation
  • Yves Veulliet, Global Disability & Inclusion Program Manager – Global Diversity HR, IBM • Presentation
  • Frances West, Chief Accessibility Officer, IBM

Alireza Darvishy, Head, Accessibility Center, Crédit Suisse

For our clients, we provide the following accessible products and services

  • Bank statements in Braille
  • Bank statements in larger fonts
  • Talking ATMs
  • Sign language interpreter for deaf clients
  • Induction system for hearing impaired clients

Axel Leblois, President and Executive Director, G3ict

Three Major Global Drivers for Accessibility
Compliance in Financial Services

  1. Convention on the Rights of Persons with Disabilities
  2. United States jurisprudence and regulatory activity
  3. European Accessibility Act

ICT Accessibility: An Extensive Set of Dispositions in the CRPD

  • Preamble – Defines accessibility as an enabler for Persons with Disabilities to exercise their rights
  • Article 3 (f) – Identifies Accessibility as one of its 8 general principles
  • Article 9 – Elevates ICT Accessibility obligations on par with those for the built environment and transportation

“Access/Accessible/Accessibility” – 17 uses throughout the CRPD

CRPD includes avenues for documenting failures and steps to compliance.

UN CRPD Committee Decision Sets Clear Precedent (May 16, 2013)

Case against Hungarian banks not providing accessible ATMs was submitted to CRPD Committee

Committee referred to article 9.2 (b) of the Convention

Concluded that:

  • Hungary had failed its CRPD State Party obligations
  • Must remedy the situation of inaccessible ATMs

UN CRPD Committee also Mandated Hungary to:

  • Establish minimum standards for the accessibility of banking services provided by private financial institutions
    for persons with visual and other types of impairments
  • Create a legislative framework with concrete, enforceable and time – bound benchmarks for monitoring and assessing the gradual modification and adjustment by private financial institutions of previously inaccessible banking services provided by them into accessible ones
  • Ensure that all newly procured ATMs and other banking services are fully accessible for persons with disabilities

Dr. Monique Mai, Accessibility Group Department, Director “Communication, Remote Sales & Public Affairs,” Orange Group

3 categories of service: Mobile Money, NFC Payment, and Mobile Banking
Orange Accessibility’s missions

  • integrate accessibility from conception to delivery
  • create adapted product and service range for the seniors and disabled people (Fr, Spain…)
  • develop distribution networks
  • drive web accessibility policy
  • communicate and set up partnerships
  • public affairs (CSR, regulation, standardization)

Sara Mansell, Design Lead, IBM

IBM Design Thinking
Sample design specification
Accessible products begin with clearly defined keyboard navigation specifications

Universal design

The design of products and environments to be usable by all people, to the greatest extent possible, without the need for adaptation or specialised design (Ron Mace, 1988)

Inclusive Design

Products, services and environments that include the needs of the widest number of consumers (UK Government 2000)

Design for All

Everything that is designed and made by people to be used by people – must be accessible, convenient for everyone in society to use and responsive to evolving human diversity (European Institute for Design and Disability (EIDD) Stockholm Declaration in 2004)

Nicola Palmarini, Global Digital Creative & Technology Advocate, IBM Accessibility / IBM Research

New clients: the aging

  • Market for aging population is a >$20 Trillion World Wide opportunity.
  • US – Estimated assets for this demographic $8.4 to $11.6 Trillion
  • Rising Eldercare costs will disrupt economies 6% of US GDP will account for social service costs for the Elder. Double the current percentage.

The challenge is converting a world built by and for the young into a world that supports and engages population that live 100 years and beyond.

Other demographic factors affect the business

Seniors cannot be treated as a homogeneous population, but comprise different subgroups.

The third age ( young old ) describes older adults’ healthy and active life phase, which is characterized by the continuation of their former lifestyle after retirement (approximately from 65 to 80).

The fourth age ( oldest old ) beginning roughly at 80, is associated with fading health and independence.

Elderly people can also be very active

  • 30% to 80% of seniors over 65 still travel.
  • On average, seniors spend 5 of every 7 days outside their homes.
  • 68% of seniors have their own cars.
  • Two-thirds of seniors have a partner.
  • 75% of seniors are grandparents.
  • Approximately 70% of seniors with children see the m several times per month or week.
  • 45% of seniors engage in volunteering activities .
  • 42% of seniors feel healthy or very health

Some challenges

Need for story telling and simplification to understand and buy new business models
Compliance & Access
Bare access to digital services is still an issue/ Compliance is driven by business performance
Usability and Accessibility
Huge Usability and accessibility gap: in the United Stat es, older boomers are over 95% underserviced and senior s manage to complete only 55.3% of tasks online.
Biometric rise the issue of privacy
Easy of access rise the risk of fraud: in US only elder fra ud accounted for $36 Billion in losses in 2014

Inmaculada Placencia-Porrero, Deputy Head of Unit for Rights of Persons with Disabilities, Directorate General for Justice, European Commission

Why an European Accessibility Act?

Economic reasons

  • Free circulation of accessible products & services
  • more accessible and cheaper products/services for 80M of EU citizens • Divergence of national legislations
  • fragmentation of the EU Market
  • counterproductive for enterprises
  • Opening markets for being ready for global competitiveness

Legal Obligations

  • The UN Convention on the Rights of Persons with Disabilities (UNCRPD) entered into force for the EU in 2011
  • Its obligations increase the risk of divergent accessibility legislations in MS
  • The EAA helps to implement the obligations of article 9 on Accessibility

Products & services in the scope of the EAA

  • Computers and operating systems
  • ATMs, ticketing and check – in machines
  • Telephones and smartphones
  • TV equipment related to digital television services
  • Telephony services and related equipment
  • Audio – visual media services (AVMS) and related equipment
  • Air, bus, rail and waterborne passenger transport services
  • Banking services
  • E – books
  • E – commerce

Robin Sargent, Business Development Executive, IBM Accessibility

Apple, IBM, and Japan Post Group are reimagining elder care in Japan

The Aging Population is growing at rapid rate

Seniors=25 % of Japan’s population. Projected to grow to 40 percent by 2055

  • Fear of being alone
  • Afraid of being Institutionalized
  • Concerned about someone else taking control or
  • Losing control

So much of what happens within your social sphere affects your healthcare …… And depression is big issue with elderly people

IBM and Apple collaboratively designed a suite of apps that provide structure and transparency to eder care at home. There are three components

  1. Elder Support – handled by Postal/Commercial Elder Support Services or Assistance Worker
  2. Elder at Home – The Senior at home alone
  3. Elder Advocate – Friend, Family, Carer

Elder Support

1 of 3 interdependent apps in the Elder at Home Suite, Elder Support is for enterprise workers assigned to monitor and assist sponsored or subscribed elders living alone and using the Elder at Home app and their families using the Elder Advocate app.

Elder at Home

Central to Elder at Home Suite, the app is specially designed for use by elders providing reminders for meds, real – time interactions with family and support service provider through associated apps, and ability to incorporate 3rd party services in ecosystem established by a client organization. Such services may include shopping, community activities, and social service requests

Elder Advocate

The app for families of subscribed or sponsored seniors, it supports and interacts with senior activity on the Elder at Home app as well as Elder Support worker app for monitoring and remote support of senior for collaborative, interactive, assistance, and communication with far – away loved ones

Paul Smyth, Head of IT Accessibility, Barclays

Accessibility resources from Barclays

Ted Drake, Accessibility Principal Engineer, Intuit

Read the full presentation: Future Accessibility for Financial Services for Seniors and Persons with Disabilities

Reducing the effort for a customer will bring better usability and accessibility. The evolution from paper based financial tracking to electronic has already given users greater control over their financial records

How do we balance the need for security while also minimizing the cognitive load for our customers? Near Field Communication, bio-metrics, and multi-factor authentication have already improved the experience, what is next?

Convention on the Rights of Persons with Disabilities

The CRPD is an international resolution to increase the rights and opportunities for people with disabilities. This builds upon the Universal Declaration of Human Rights, but adds support for those that need it the most.

The rights enumerated in the Universal Declaration of Human Rights, in a perfect world, would be enough to protect everyone. But in practice certain groups, such as women, children and refugees have fared far worse than other groups and international conventions are in place to protect and promote the human rights of these groups. Similarly, the 650 million people in the world living with disabilities—about 10 per cent of the world’s population—lack the opportunities of the mainstream population. They encounter a myriad of physical and social obstacles that:

  • Prevent them from receiving an education;
  • Prevent them from getting jobs, even when they are well qualified;
  • Prevent them from accessing information;
  • Prevent them from obtaining proper health care;
  • Prevent them from getting around;
  • Prevent them from “fitting in” and being accepted.

Why a Convention? – Convention on the Rights of Persons with Disabilities – United Nations

Continue Reading Convention on the Rights of Persons with Disabilities