Google IO 2017 – Android 0, Studio, and Things Notes

Android Things

Use Android Things with AI to create smart devices: Do it Yourself AI kit

IOT Security with Android Things

Android Things – System on Module architecture. Developer console

Security features are built in: on by default, easy to maintain, infrastructure by Google.

Feature features: OS Hardening, updates verified boot, hardware backed keys, attestation

The main problem in IOT secuirty is economics. the cost of building in security vs. cost of risking security. smaller, lower cost devices may not build in security. Exploits have become their own market

How much is the cost to an engineer to create an attack?

  • Time
  • Money
  • Skill
  • Grit

attack ROI:

How valuable is it?


Can this attack scale enough to be valuable? WiFi injection may be effective, but it doesn’t scale if you have to be near the router. A default credential, i.e. security cams, can be attacked by the thousands.


Does the attack give the person privilege to the hardware, accounts?


Does the attack give the engineer significant persistance? Can it survive a device reboot?

Security Cost: not every company has the resources to build and maintain security features and infrastructure.

Android Things goal: raise attack costs, reduce ROI, and reduce security costs.

OS Hardening: All of Android’s hardening is enabled in Android Things. Permissions, app sandbox, mandatory access control (Selinux), kernel syscall filtering, full ASLR, FORTIFY, stack-protector-strong…

Developer Action: Declare permissions only as needed, split out privileged code.

All Android Things devices will get infrastructure updates directly from Google. This reduces attack persistance and drives down the attack ROI. Updates can be controlled during critical operations, like a drone is flying. Developers can also test updates and request an update to be stopped

Android Things Keystore

Attestation: Authentication for devices. Attests to what? Authnetic Android Thing Device, product info, device identity, device state (verified boot state), key attributes (allowed modes,…)

Building for Billions

Building for Billions is Androids best practices for low cost/bandwidth devices. Best practices on how to optimize Android apps for low- and no-bandwidth and low-cost devices.


  • Kotlin is now a first-class language for Android.
  • Kotlin Programming Language
  • Kotlin is a statically typed programming language for the JVM, Android and the browser, 100% interoperable with Java.
  • There’s no change in support for Java or C++.
  • Comparable to Swift and Objective C.

Watch Google IO talks  for more info on using Kotlin

Android Studio 3.0 canary is available today.

Watch for unified network profiler. It is interactive, you can go into functions and dive into specific code.

Android Architecture Components

A new collection of libraries that help you design robust, testable, and maintainable apps. Start with classes for managing your UI component lifecycle and handling data persistence.

Android is going to have a device catalog, find out the specifics of different phones and devices.

Android Instant Apps

Android Instant Apps available to everyone

They allow Android users to run your apps instantly, without installation. Android users experience what they love about apps—fast and beautiful user interfaces, high performance, and great capabilities—with just a tap.

Android Instant Apps is now open to all developers, so anyone can build and publish an instant app today.

Use the same code to generate the instant app, use feature modules to define what can be included/removed for instant apps. Look for modularize feature in android studio. They also will have optimization tools to make these features faster.

What’s new in Android O keynote…

Picture in picture is coming to Android O. similar to the way YouTube will let the video reduce to a thumbnail while you scroll other videos in a search. Only now the video can be a thumbnail while you open evernote and write notes

Android O color management will allow colors to be more consistent across devices.

Multi-display. user can choose to send information to multiple displays.

Look for ActivityOptions

$adb shell dumpsys display
getMetrics() on mediaPlayer, mediaRecorder…

You can now use an arbitrary number of audio/video tracks.

WebView allows Google Safe Browsing API.

    android:value=“true” />

Web view safe browsing goes back to lollipop

Seek within videos

animatorSet setCurrentPlayTiem (long); - seek within


you can now reverse: reverse();


You can now get user name/password and other from Chrome.

This will make logging into apps much easier.

Autofill will be included in basic form inputs:  text view, edit text.. no extra work required

You can use hints to the auto-fill api to define data types. Autofill apis for custom veiws and opaque hierarchies


Text Stuff: font files can be added to font directory. Downloadable fonts: declare font to be downloaded and cached.  Font provider in Google play services v1.  There is a beta version available. Access to all 800 google fonts

Auto-sizing Text Views

Fonts with auto-resize will change font size as you resize their container. In the past, the container may grow, but the fonts stay the same size. Now they can grow with the container.

AccessibilityService Utilities

Language detection, accessibility button, separate volume controls, finger print

FindViewById update

Current: public View findViewById (int id);
TextView tv = (TextView) findViewbyId(;

Android O public <T extends view> findViewById(int id)
TextView tv = findViewById (;

Adaptive Icons.

We will need to worry about Adaptive Icons with future releases. Developers provide background + foreground * mask for multiple devices.

Pin request.

Ask people to pin our app to desktop. ShortcutManager and appWidgetManager. requestPinAppWidget…


Notifications are getting more power in O. The user should always be in control. Users and developers want ability to tweak notifications from an app. Notification channels give developers and users fine grain control. Apps can define channels, assign notifications to channels, post notifications

Note: Once you target O. You must use channels or notifications will be dropped!

Strict Mode

Thread policy. unbuffered i/o, VM policy

Media file access

Seekable file descriptors from custom document provider. useful for large remote sources.  Cached data. statey below the quota to avoid aggressive deletion. Use storage manager



Privacy ANDROID_ID is now different for every app,user

net.hostname is empty

Google Play Protect, scans apps on user’s phone. now they are making it obvious by giving the user a notification when it scans and lets them know the state.

Android Studio 3.0

Kotlin available today.

  • You can copy java and paste within kotlin file and it will automatically convert to kotlin code.
  • Java: new packages for java.time, java.nio.file, java.lang.invoke
  • Runtime: Concurrent -copying collector.
  • Smaller heap, faster allocations, faster collections
  • v26 has emojiCompat – bundled or updatable.


Physics animation: velocity, springs, force. This simplifies the process. It is more natural, interactive, interruptible.

Look at ChainedSpringDemo

Architecture components:

easier android development. lifecycle

Background apps:

  • location is only going to get coarse grained location info
  • wakelocks for cached apps – wakelock is released
  • background execution limits

Alert windows


Google IO 17 – Home, Assistant, and Actions Notes

Actions built for Google Home are now available for Google Assistant on phone

New interfaces for actions:

  • Tap
  • Voice
  • Typing

Purchases are also built in via google wallet. Google facilitated Payments.

New app directory for google actions.  Shortcuts will allow you to more easily market the action.

Actions console on Google

Google Home

Google Home can now do phone calls to any number in US and Canada. The number is generic, but you can use your personal #.

Actions on Google development platform.

Now available on Android and iOS. Includes transactions.  is the new hub for artificial intelligence at Google. IO17

Designing for Voice Interactions: value of voice interactions: speed, simplicity, ubiquity

Design strategies

  • Keep people comfortable
  • Ask questions that are easy to answer
  • Structure information in a way that supports easy recall
  • Capabilities: recognize what users say. understand what they mean.

People have higher expectations for voice accuracy. spend extra time planning for exceptions. make it really easy to get back on track. leverage techniques used in everyday conversations.

Conversation UI and why it matters.

Google IO17 Notes – VR, AR, and AI – smart realities

Google wants to democratize Artificial Intelligence with Tensor Flow  and API.AI

Conversational UX Platform for products and services – API.AI

Create conversational experiences across platforms: chatbots, smart home automation, connected cars, mobile and web apps, robots, wearables, Enterprise, etc. (6kB)

Sign up to be one of the first to write AI on Google’s new Tensor Processing Units:

These are part of the google computing engine cloud service and are extremely fast for AI-based computing

Visual Positioning Service, Google Lens, and other Visual/AI Combinations

Google’s Visual Positioning Service uses Augmented Reality to find key visual points for indoor navigation withiut GPS or beacons.

Google Lens is also able to pull objects out of photos and create a data presentation. For instance, a photo of the Chicago cityscape could be analyzed and then provide information about the individual buildings.

Google IO 2017 – Web Development Notes

Lighthouse is being integrated directly into dev tools

Firebase now gives a real time analytics. Cloud functions for Firebase sounds like Akamai for small node functions, like resizing images. But being able to handle these tasks in a global, cached environment.

Google IO: state of the mobile web

Chrome’s mission: move the web platform forward

Real world javascript language performance: key focus for this past year. How does V8 perform in real world scenarios, not just perfect testing spaces.  35% increase in performance since last year on Android

scroll anchoring: page jumps after content loads after page renders, such as a banner ad. This locks the screen, even when content loads at the top. Scroll anchoring can reduce 3 page jumps per load on average.

AMP: Accelerated Mobile Pages. improve mobile experience.

  • On average, AMP pages load in less than a second and use 10x less data
  • LinkedIN found people were 10% more likely to read an article when it is AMP .
  • amp-bind: merchants can build e-commerce experiences
  • Progressive web apps: app focused experiences, reliable, fast, engaging.

Mobile Twitter just relaunched as a PWA

  • Fast loading on slow networks, less data, works well on smart phones.
  • Data saver mode can reduce data downloads by 70%
  • Introducing Twitter Lite | Twitter Blogs

Twitter Lite is a more accessible, faster and more affordable way for people to use Twitter when they are on slow mobile networks, have expensive data plans and with limited storage on their mobile device.

PWAs can be added to the home screen. Developers can soon control the home page button prompt. They can be displayed in app launcher, android settings, android intents, notifications, and launch as a full-screen immersive view

Mobile payments

  • 123B in US alone
  • paymentRequest – simple web payments within Chrome
  • paymentRequest can now use more forms of payment. paypal, alipay, samsung pay… could this integrate quickbooks payments?


  • 70% of internet traffic is video.
  • Biograf: This is a sample video-rich PWA app
  • Notice this has an airplane mode and allows video download
  • Workbox is a tool to better manage service workers

Polymer 2.0 launches today. 10% faster and 80% smaller

Ola Cabs

India’s largest ride sharing app. They are using PWA. over 1m daily rides. 110 cities and 600k drivers. They needed to work with customers that use low cost phones, minimal data plans, and bad connections to reach the entire customer base.

Off line and caching provides faster performance with low data loads. They used polymer for fast web components, Shadow DOM, and HTML import.

  • They rely heavily on cache, but also have an initial load of 1.3 seconds.
  • They strategically load components to only get critical elements first.
  • They are using workbox, they cache these elements for repeated use.
  • Now, they are only requesting new data, such as transaction information.
  • They have a 100 score in lighthouse
  • 20% of their PWA bookings come from users that previously uninstalled their Android app


<amp-install-serviceworker>. This takes an AMP page and allows it to install a service worker so a user shifts to a PWA when they click. This gives the fast initial page load of an AMP page and prepares the browser to make the second page load just as fast.

Let’s say someone shares a PWA page with a friend. We’d want the friend to have a fast page load, but if they go to a PWA page as the first experience, they won’t get the acceleration via the service worker. So we could do some detection at the page load to see if the service worker is available, if not, move to the AMP experience of that page.

Shadow DOM and iFrames

Use shadowDOM to update an AMP page with new content to create a fast, progressive web app

This is good for a site that has a lot of static content, such as our marketing site. it wouldn’t be logical for an application, such as QBO

Mobile Vaani, get feedback on your product design via this network for rural India.

Web4All Conference Notes – Day 3

Crowd sourcing accessibility evaluations

2013-6: 350 government web sites and 2,000 non-government sites have been evaluated for accessibly in China

conformance testing included

  • Automatic Evaluation
  • Manual Assessment

Crowd sourcing can integrate the power of crowds to solve the manual assessment bottleneck

It was proposed in 2006 and has been used in reCaptcha, spoken wikipedia, labeling.

the current crowd sourcing is not suitable for web accessibility because the assessment tasks require a high level of expertise and experience.

There was an assignment of tasks. The results were compared to

  • total work
  • time out
  • give up
  • errors detected.

An algorithm was developed to compare these values to determine a cost model. This allows them to look at historical data to find that a person is more efficient at one of the rulesets. For instance a completely blind person may be great at form labels but not at color contrast.

Assessment of semantic taxonomies for blind indoor navigation based on a shopping center use case

Location-based services (LBS)

  • many LBS are available  thanks to smart phones
  • provide turn by turn navigation support using vocal instructions
  • we know little about what environmental elements an features are useful, such as tactile paving or braille buttons

The did a survey of taxonomies

Looking at these data sets, they created a simplified taxonomy based on their similarities

  • Pathways
  • doorways
  • elevators
  • venues
  • obstacles (not included in the previous taxonomies)

These elements defined by their fixed positions within floor map. Vocal instructions use this information to generate vocal instructions. Locate tactile paving:

  • “proceed 9 meters on braille blocks, and turn right”
  • “proceed 20 meters, there are obstacles on both sides”

Announcements of obstacles and tactile paving was confusing and unnecessary for one guide dog user.

Do web users with autism experience barriers when searching for information within web pages

The study looked at eye gazing to see if there was a difference between two groups: with and without autism.

With a series of search tasks, the group with autism had less success than the control group for completing the tasks.

tracking the eye gaze. Five elements: a, b, c, d, e. Their eye map could be a-b-c-e-d

Check the variance between the two groups.


The #DysMusic study is creating a language independent test for detecting #dyslexia in children. #w4a2017 @luzrello

Most dyslexia detection tools are still linguistics based, which isn’t appropriate until the child is already 7-12 years old. This study tries to find a detection method that is non-language based, this would allow detection at a much younger age.

There is a memory game with music elements.


  • Find the matching sounds
  • distinguish between sounds
  • short time interval perception

Raw sound is modified via frequency, length, rise time, rhythm.  Only one property is modified at a time. People with dyslexia tend to have trouble detecting rise time changes.

Accessibility Challenge

Producing Accessible Statistics Diagrams in R

Data visualization is increasingly important. R is an existing language for statistics. Jonathan (co-writer) had been using R to output printed diagrams of statistics. They worked together to convert R into an accessible SVG format

Histograms and Boxplots were discrete data presentations  targeted layout for the initial project. Time series and scatter plots are continuous data graphs

Extract the important data points, convert it to an xml document, and attach this to the SVG. The final experience provide easy navigation (arrow keys), supports screen readers via aria live regions.


GazeTheWeb is a simplified browser designed for eye tracking navigation. #w4a2017 #a11y

Math Melodies

Math Melodies makes math easier to learn for children that are blind or low-vision. Math exercises as puzzles, audio icon maps, different exercises. It was funded via crowdfunding and has been downloaded 1400 times


NavCog is a navigation project from CMU for blind individuals. It uses low energy blue tooth beacons.

Installation of the beacons is not scalable across large areas. To crowd source the task, they created a set of instructions to walk through the process of configuring and installing the beacons.


LuzDeploy is a Facebook messenger bot: easy to use.


VizLens is a crowd sourced interpretation of interfaces, such as microwave oven. Multiple volunteers are recruited to generate labels for the interface. the app then uses augmented reality to virtually overlay the labels.

Chatty Books

Chatty Books is an html5 + Daisy reader that creates an audio version of documents. It can now convert from pdf to multimedia Daisy.

  1. PDF – NiftyReader (text)
  2. export to multimedia daisy or epub3
  3. drag and drop to chatty books, the daisy player and library
  4. upload daisy content to chatty books service (cloud) and use chatty books app on iPad

Able to read my mail

Simplified email program for people with learning and intellectual disabilities. Gmail plugin that converts to simplified text or pictograms.

Closed ASL Interpreting for online videos

Created a framework for incorporating an interpreter. Closed Interpreting, instead of Closed captioning.

the interpreter window needs to be flexible to allow the user to move it around and change size to reduce distractions. IT’s closed, so i can be turned on/off

Moving the eyes back and forth for long periods of time can be exhausting. so the window can be moved to be closer to the screen’s content.

eye-gaze tracking to pause the video when looking away from the video.

Closed Interpreting [CI]

Provide a video interface that allows closed interpreting, like closed captioning. The interface provides a second screen that includes an ASL interpreter

The users appreciated the ability to customize the interpreter’s location. They also liked the ability to pause the interpreter as the gaze moved from content to the interpreter