Google IO 2017 – Accessibility Notes

Building an accessibility team

The project champion needs to make sure training is available to all people on the team.

  • Create a set of primary and secondary key work paths, i.e. put an item in a shopping cart (first) change avatar image (secondary)
  • Add a checklist and try user testing

Resources and Ideas:

What’s new in Android Accessibility

Accessibility | Android Developers

Accessibility is an important part of any app. Whether you’re developing a new app or improving an existing one, ensure that components are accessible to everyone.

Why develop for accessibility

  • 1 in 5 people will have a disability in their life. – 2010 census
  • Designing for accessibility benefits blind, low vision, and eyes occupied (driving)

Android includes 4 types of assistive technology:

  • TalkBack: Screen reader
  • BrailleBack: Braille output for refreshable braille devices
  • Switch Access: switch control of device
  • Voice Access: control device by voice activation: “scroll up”

Android O’s major focus: increase productivity for users

  • new api additions for accessibility
  • print disabilities (reading disabilities)

New to TalkBack

accessibilityVolume: adjust audio volumen for accessibiity independently from media. So you can watch youtube and control that separately than talkback. This is available when talkback is on.

Volume from youtube is quieted while talkback is being used. it then fades back into the foreground. There’s a new accessibility volume slider

New gestures for talkback.

If there’s a fingerprint sensor on the back of the device, it can be used by talkback users. The sensor has its own set of customizable gestures. For instance,  swipe up on the fingerprint. These can be assigned, such as longpress action

Quickly enable/disable TalkBack

long press the volume keys to quickly turn on/off talkback.

this works on any screen, this makes it easier to test apps and turn off talkback to type information. Press both keys at the same time for a long press and eventually it will turn on/off talkback. the accessibility shortcut can be assigned to switch control, zoom, or other service.

New Text to speech engine can handle multiple languages. Use LocaleSpan to trigger language switching.

2 new APIs

Continuous Gesture API: enable motor impaired users who use head tracker to perform drag and drop, zoom, etc.

Accessibility Button:

A new accessibility button is located in the navigation bar. This allow users to quickly invoke context dependent accessibility features. This sits in the row with back and home buttons

Print disabilities

People with dyslexia, low vision, learning new language… They can now use select to speak. part of talkback 5.2. Select element on screen and talkback will read it. It has a floating action button to enable.

In android o. read whole page, and advanced controls, word level highlighting, set up wizard.

Testing

manual testing: try your app with TalkBack and SwitchAccess.

  • if it is ok in talkback, it should be good for brailleback and select to speak
  • if it works with switch access, it should also work with voice access.

Accessibility Scanner, free app to download on play store. Android's Accessibility Scanner

This analyzes the current screen and provides an audit that can be shared.

The accessibility test framework is still requiring espresso and/or robolectric

Android Accessibility Documentation

Android has a new developer hub for understanding accessibility. There’s a page for Android testing.

Google IO 2017 – Android 0, Studio, and Things Notes

Android Things

Use Android Things with AI to create smart devices: Do it Yourself AI kit

IOT Security with Android Things

Android Things – System on Module architecture. Developer console

Security features are built in: on by default, easy to maintain, infrastructure by Google.

Feature features: OS Hardening, updates verified boot, hardware backed keys, attestation

The main problem in IOT secuirty is economics. the cost of building in security vs. cost of risking security. smaller, lower cost devices may not build in security. Exploits have become their own market

How much is the cost to an engineer to create an attack?

  • Time
  • Money
  • Skill
  • Grit

attack ROI:

How valuable is it?

Pervasive

Can this attack scale enough to be valuable? WiFi injection may be effective, but it doesn’t scale if you have to be near the router. A default credential, i.e. security cams, can be attacked by the thousands.

Privileged

Does the attack give the person privilege to the hardware, accounts?

Persistant

Does the attack give the engineer significant persistance? Can it survive a device reboot?

Security Cost: not every company has the resources to build and maintain security features and infrastructure.

Android Things goal: raise attack costs, reduce ROI, and reduce security costs.

OS Hardening: All of Android’s hardening is enabled in Android Things. Permissions, app sandbox, mandatory access control (Selinux), kernel syscall filtering, full ASLR, FORTIFY, stack-protector-strong…

Developer Action: Declare permissions only as needed, split out privileged code.

All Android Things devices will get infrastructure updates directly from Google. This reduces attack persistance and drives down the attack ROI. Updates can be controlled during critical operations, like a drone is flying. Developers can also test updates and request an update to be stopped

Android Things Keystore

Attestation: Authentication for devices. Attests to what? Authnetic Android Thing Device, product info, device identity, device state (verified boot state), key attributes (allowed modes,…)

Building for Billions

Building for Billions is Androids best practices for low cost/bandwidth devices. Best practices on how to optimize Android apps for low- and no-bandwidth and low-cost devices.

Kotlin

  • Kotlin is now a first-class language for Android.
  • Kotlin Programming Language
  • Kotlin is a statically typed programming language for the JVM, Android and the browser, 100% interoperable with Java.
  • There’s no change in support for Java or C++.
  • Comparable to Swift and Objective C.

Watch Google IO talks  for more info on using Kotlin

Android Studio 3.0 canary is available today.

Watch for unified network profiler. It is interactive, you can go into functions and dive into specific code.

Android Architecture Components

A new collection of libraries that help you design robust, testable, and maintainable apps. Start with classes for managing your UI component lifecycle and handling data persistence.

Android is going to have a device catalog, find out the specifics of different phones and devices.

Android Instant Apps

Android Instant Apps available to everyone

They allow Android users to run your apps instantly, without installation. Android users experience what they love about apps—fast and beautiful user interfaces, high performance, and great capabilities—with just a tap.

Android Instant Apps is now open to all developers, so anyone can build and publish an instant app today.

Use the same code to generate the instant app, use feature modules to define what can be included/removed for instant apps. Look for modularize feature in android studio. They also will have optimization tools to make these features faster.

What’s new in Android O keynote…

Picture in picture is coming to Android O. similar to the way YouTube will let the video reduce to a thumbnail while you scroll other videos in a search. Only now the video can be a thumbnail while you open evernote and write notes

Android O color management will allow colors to be more consistent across devices.

Multi-display. user can choose to send information to multiple displays.

Look for ActivityOptions

$adb shell dumpsys display
getMetrics() on mediaPlayer, mediaRecorder…

You can now use an arbitrary number of audio/video tracks.

WebView allows Google Safe Browsing API.

<meta-data
    android:name=“andorid.webkit.webview.enableSafeBrowsing”
    android:value=“true” />

Web view safe browsing goes back to lollipop

Seek within videos

animatorSet setCurrentPlayTiem (long); - seek within

Reverse

you can now reverse: reverse();

AutoFill

You can now get user name/password and other from Chrome. https://youtu.be/1N9KveJ-FU8?t=9m17s

This will make logging into apps much easier.

Autofill will be included in basic form inputs:  text view, edit text.. no extra work required

You can use hints to the auto-fill api to define data types. Autofill apis for custom veiws and opaque hierarchies

Fonts

Text Stuff: font files can be added to font directory. Downloadable fonts: declare font to be downloaded and cached.  Font provider in Google play services v1.  There is a beta version available. Access to all 800 google fonts

Auto-sizing Text Views

Fonts with auto-resize will change font size as you resize their container. In the past, the container may grow, but the fonts stay the same size. Now they can grow with the container.

AccessibilityService Utilities

Language detection, accessibility button, separate volume controls, finger print

FindViewById update

Current:

View.java: public View findViewById (int id);
TextView tv = (TextView) findViewbyId(R.id.mytextview);

Android O

View.java: public <T extends view> findViewById(int id)
TextView tv = findViewById (R.id.mytextview);

Adaptive Icons.

We will need to worry about Adaptive Icons with future releases. Developers provide background + foreground * mask for multiple devices.

Pin request.

Ask people to pin our app to desktop. ShortcutManager and appWidgetManager. requestPinAppWidget…

Notifications

Notifications are getting more power in O. The user should always be in control. Users and developers want ability to tweak notifications from an app. Notification channels give developers and users fine grain control. Apps can define channels, assign notifications to channels, post notifications

Note: Once you target O. You must use channels or notifications will be dropped!

Strict Mode

Thread policy. unbuffered i/o, VM policy

Media file access

Seekable file descriptors from custom document provider. useful for large remote sources.  Cached data. statey below the quota to avoid aggressive deletion. Use storage manager

getCacheQuoteBytes()
allocateBytes()
setCacheBehaviorTombstone()

Security

Privacy ANDROID_ID is now different for every app,user

net.hostname is empty

Google Play Protect, scans apps on user’s phone. now they are making it obvious by giving the user a notification when it scans and lets them know the state.

Android Studio 3.0

Kotlin available today.

  • You can copy java and paste within kotlin file and it will automatically convert to kotlin code.
  • Java: new packages for java.time, java.nio.file, java.lang.invoke
  • Runtime: Concurrent -copying collector.
  • Smaller heap, faster allocations, faster collections
  • v26 has emojiCompat – bundled or updatable.

Animation

Physics animation: velocity, springs, force. This simplifies the process. It is more natural, interactive, interruptible.

Look at ChainedSpringDemo

Architecture components:

easier android development. lifecycle

Background apps:

  • location is only going to get coarse grained location info
  • wakelocks for cached apps – wakelock is released
  • background execution limits

Alert windows

TYPE_APPLICATION_OVERLAY must be used

Mystery Meat 2.0 – Making hidden mobile interactions accessible

Mystery Meat 2.0

  • Ted Drake, Intuit Accessibility
  • Poonam Tathavadkar, TurboTax
  • CSUN 2017
  • Slides: slideshare.net/7mary4

This presentation was created for the CSUN 2017 conference. It introduces several hidden interactions available within Android and iOS. Learn how these work and how to make them accessible.
Blue Bell Ice Cream Blue Bell Ice Cream web site with mystery meat navigationis a classic example still live on the web.

The user must hover over the different images to see what they represent. It uses an image map and lacks alt text.

Android Touch and Hold

A.K.A.: Android’s Right Click or Android Long Press to Add context-specific menus

  • Default: Touch and hold
  • With TalkBack:
    Double tap and hold to long press

Mint Transactions

This short video shows how you can use the touch and hold/long press to quickly change category or merchant name within the Mint application

Developers

  • onLongClick: Called when a view has been clicked
    and held
  • Define and Show your menu
  • Not for vital interactions.
  • This is a short
    cut.

It is possible to modify the default notification to the user

iOS 3D Touch

iOS 3D Touch was introduced on the iPhone 6S. It detects the pressure a person applies to the screen with their finger. I light touch is seen as a tap. A medium touch will
trigger a peek view. A continued firm touch will launch the peek’s content into a full screen.

This also allows a person to trigger a shortcut menu on app icons.

  • Peek:
    Quick glance at relevant information and
    actions
  • Pop:
    Open full content previewed in the Peek
  • Quick Actions:
    Custom task list from app icon

User Experience: A light press opens a hovering window so you can “Peek” at the content. When you press just a little bit harder, you will “Pop” into the actual content you’d just been
previewing in a Peek.

Developer info

Quick Actions

This short video shows how 3d touch is also available via the app’s icon for quick tasks.

Pressing and holding the ItsDeductible icon will trigger a menu with customized tasks. App Icon Developer resources

Developers

Swipe Revealed Actions

Alternative actions allow users to quickly make changes without having to open a detail screen. For instance, they can delete a transaction or change an email’s status. The standard interface is to display the options when a user swipes a row to the left. For voiceOver users, the options are announced as alternate actions

It’s Deductible Actions

This short video shows how the alternative actions menu is used in standard mode and how VoiceOver announces the options.

In iOS, editActionsForRowAtIndexPath defines the actions to display in response to swiping the specified row

  • Accessible by default
  • Define:
    • Target Action and Response
    • Visible Text
    • Background color

Swipe Based Navigation

TurboTax uses a custom swipe based navigation between views. It lacks button or suggestions to move back and forth. User testing has showed it to be effective for
sighted users, but required some extra work for accessibility.

Default Experience With VoiceOver

The default experience on Turbo Tax uses a custom swipe gesture that lacks navigation buttons.

TurboTax detects a user’s Screen Reader/Switch Control status to show navigation buttons on Android and iOS

This video shows the default and VoiceOver/SwitchControl experience.

Notice in the standard experience how the screen tracks the user’s finger movement. This is not a standard swipe gesture, so it will not work with VoiceOver enabled.

We detect VoiceOver and SwitchControl is running to display alternate back and continue buttons

Swipe Navigation

  • Animated transition between views
  • Next and Back flow with every screen
  • Eliminates navigation buttons
  • No buttons? Accessibility?
  • Have I reached the end of the screen?

Instead of a basic swipe gesture, this interface tracks the movement of the finger across the screen. This allows the animation to match the user’s finger speed for a more
natural transition.

However, the finger touch is intercepted by VoiceOver, so the custom navigation does not work when VoiceOver is enabled.

Detect Accessibility ON

UIAccessibility kit provides two methods

True == Show Navigation Buttons

These booleans always return true or false. We use this to insert navigation buttons into the screen.

State Change Notification

This does not solve for the more complex of when the user decides to turn it on/off in the middle of the flow of the application for changes to take place dynamically.

For that as well, iOS has great support. Fires an accessibilityChanged event that helps detect changes even when the user is in the middle of the flow and chooses to
turn voice over on/off.

User enables VoiceOver while on a screen

Detect the status change

TurboTax Helper Function

  • How can we refactor code to detect any
    accessibility related settings and address them
    together?
  • Helper function to the rescue!
  • NSNotificationCenter adds observers to track any
    settings that may require us to show buttons.
  • This is an OR logic. Example – if voice over OR
    switch control status changed, display buttons.

Code specifics

  • Boolean is assigned a value – true if buttons need to be shown.
  • Consider a React Native project, all this happens in the native code side (Objective C). This boolean is then handed over to the JAVASCRIPT side since it is not feasible
    for Javascript to get information directly from the device.

Android Accessibility – The Missing Manual

This presentation was created for the Bangalore Accessibility Week at Intuit, October 2015. It’s a collection of hard to discover information on making an Android application accessible.

Testing

DevicesTesting is done both on actual devices and with automation, such as Calabases and Android Lint. Intuit’s product development also relies heavily on user testing, including users with disabilities.

Intuit also has a mobile device library that allows anyone within Intuit to check out a mobile device for testing. This has significantly lowered equipment cost and makes it much easier to test applications on an assortment of phones. This can be important as phone manufacturers may break accessibility, such as the Samsung keyboard.

Android Lint

Android lint allows you to run accessibility tests within your development environment. It’s easy to find and fix the issues.

AccessibilityChecks

This session, Improve your Android App’s Accessibility, from Google IO 2015 introduces AccessibilityChecks and teases the upcoming testing app.

TalkBack

TalkBack is the built in screen reader for Android devices. ChromeVox is the screen reader used in Chrome and on ChromeOS

Turn on TalkBack

This video shows how to enable TalkBack and how to use it as a developer. It also shows how to use the context menu for reading the entire screen and turning of TalkBack temporarily.

Two Fingered Gestures

It is easy to use custom gestures with Android. If the app depends on a single finger gesture, TalkBack will pass the same gesture with two fingers. This is very helpful with signatures and swipe navigation.

Android Accessibility Shortcut

Android provides a convenient shortcut for enabling TalkBack on a device. This short video shows how to use it.

Programming

accessibilityLiveRegion

Place this declaration on the container that includes data that changes dynamically. The new content will be announced when it appears.

android:accessibilityliveregion="polite"

This short video shows how a live-region’s content is announced whenever it is changed by deselecting a row within the table. This is an HTML page with aria-live=“assertive”

AccessibilityAction

  • Swipes and other hard to discover actions
  • Actions are activated from the Local Context Menu
  • While this could be used to provide hints for actions, I haven’t found the documentation/examples on how this is accomplished.
Create AccessibilityAction
/**
 * @param actionId The id for this action. This should either be one of
 * the standard actions or a specific action for your app. In that case it
 * is required to use a resource identifier.
 */
 public AccessibilityAction(int id, CharSequence label)
 new AccessibilityAction(R.id.dismiss, getString(R.string.dismiss));
 new AccessibilityAction(ACTION_CLICK, getString(R.string.play_song));
// Constants for all the standard actions with default label:
 AccessibilityAction.ACTION_CLICK
Handling a Custom Action
eventView.setAccessibilityDelegate(new AccessibilityDelegate {
 @Override
 public onInitializeAccessibilityNodeInfo(
        View host, AccessibilityNodeInfo info) {
   super.onInitializeAccessibilityNodeInfo(host, info);
   info.addAction(new AccessibilityAction(R.id.dismiss,
 }
 @Override
   getString(R.string.dismiss)));
     public boolean performAccessibilityAction(
            View host, int action, Bundle args) {
      if (action == R.id.dismiss) {} // Logic for action
     }
 });

android:importantForAccessibility

diagram of UI layers within Android and setting noHideDescendants on unused layers. Use Auto on current layer

If you are using a stacked interface, where the hidden layers are still receiving focus, you can use importantForAccessibility to define the layers that are not important. Hamburger menus would be the most common use for this technique.

Set the visible layer to “auto”, which lets Android manage it’s functionality. Set the “hidden” layers to “noHideDesendants”, which should remove the layer and its children from the Accessibility API. Switch these values when you hide/show layers.

android:importantForAccessibility = "auto"
android:importantForAccessibility = "yes"
android:importantForAccessibility = "no"
android:importantForAccessibility = "noHideDescendants"

ListPopupWindow

diagram of a modal layer within Android. Use <a href=setModal(true) to make sure it acts like a modal view”>

For popups, such as a set of options A better option, however, may be to use a ListPopupWindow or PopupWindow if you’re attempting to display information in a modal context. Just be sure to setModal(true) to ensure the contents of that window, and only the contents of that window are focusable with TalkBack.

Forms

Forms are critical for a great user experience and we need to make sure our users understand what each input represents.

Form Labels

Which is correct for your app?

  • android:hint
  • android:labelFor
  • android:contentDescription

android:hint

The hint is like the HTML “placeholder” attribute. It is a visible label within the input and is surfaced to the Accessibility API when the input first receives focus. However, the hint is ignored when the input has a value. SSB Bart published a great article on this topic:
Android Accessibility Properties and TalkBack

Android form using android:hint, but no visual label

  • This create a placeholder text string within an input
  • This was the preferred method but is a hack
  • The hint is removed when a user adds a value to the input
  • Still a valid method of adding a label to an input

labelFor

The strongest method for adding a label to an input is the “labelFor” method. If your app has a visual text label, add “labelFor” to this text view and point it to the form input. The user will always know what the form input represents.
Android app using visual labels

<TextView
 android:layout_height="match_parent"
 android:labelFor="@+id/edit_item_name"
 android:text="Invoice amount"/>
<EditText
 android:id="@+id/edit_item_name"
 android:layout_height="wrap_content"
 android:hint="Invoice Amount"/>

contentDescription

ContentDescription is much like HTML’s “aria-label”. It’s an invisible text string that is surfaced directly to the Accessibility API. Android documentation specifically warns against using the contentDescription directly on the input.

Note: For EditText fields, provide an android:hint attribute instead of a content description, to help users understand what content is expected when the text field is empty. When the field is filled, TalkBack reads the entered content to the user, instead of the hint text.
Making Applications Accessible – Android Developer Portal

  • Invisible description for TalkBack
  • Should not be used directly on an input
  • You can use it on an input’s container and combine with labelFor

textinputlayout

It is possible to use “contentDescription” and “labelFor” to include a hidden label for your application. For instance, this pattern works with the recently introduced “textinputLayout” for Material Design layouts. This same pattern should work with a basic container around a form input.
Material design for form inputs include the android:hint as a visual label.

<textinputlayout
 android:labelfor="@id/signupemail"
 android:contentdescription="Email"
 android:accessibilityliveregion="polite">
  <edittext
   android:id="@id/signupemail"
   android:hint="Email"
   android:seterror="Create a valid email address"
    …/>
</textinputlayout> 

This is the pattern suggested for Material Design in Marshmallow. It has some bugs with Android support, but these should be solved soon. More information: Accessible Android Inputs with Material Design

Checking for TalkBack

AccessibilityManager am = (AccessibilityManager)
         getSystemService(ACCESSIBILITY_SERVICE);
boolean isAccessibilityEnabled = am.isEnabled();
boolean isExploreByTouchEnabled = am.isTouchExplorationEnabled();

You can check to see if the user has talkBack enabled and then make modifications to your application. For instance, this could be used to add continue and back buttons to a swipe-based navigation interface.

More Android Documentation

Accessible Android Inputs with Material Design

Update – August 2016

I first wrote this post shortly after textinputlayout was introduced and there was a distinct lack of documentation. My original code example for setting the error was incorrect. Per Victor Tsaran:

There is no such attribute as android:setError. There is a method called setError in the View class that can set the error message dynamically. That method actually works with TalkBack.
Victor Tsaran

I will be updating the code examples soon. For now, do not include the android:seterror line shown below. I’m leaving it right now for archival purposes.


Google has done an admirable job defining the Material Design style guide. They’ve also begun rolling out new APIs that make it much easier to implement the interaction designs within Android and HTML. However, there are still some gaps. This article looks at the popular Text Input for Android interaction. Please note: the code in this article is not fully documented and the best practice may change as the Google Accessibility team updates their documentation. Consider this a beta pattern and I will gladly update it as we learn better practices.
Continue Reading Accessible Android Inputs with Material Design