06/04/2021 Assistant Readiness Tests (ART) Guide | Assistant Partners

Assistant Readiness Tests (ART) Guide 

Impact of Covid-19

Due to the Covid-19 situation, we want to provide you with the latest information on the status
of our ART accredited third party labs. This information will be updated whenever there is a
change in status with our ART certi ed labs. See the table below.

For the most updated information, check directly with the third party lab you wish to use, prior
to sending your device for ART testing/certi cation.

Third Party ART Lab Status as of 07/07/2020

UL (Fremont, CA) Lab is open for business.

Sound Certi cate (Vienna, Austria) Lab is open for business.

PAL (Taipei, Taiwan) Lab is open for business.

Pegatron (Taipei, Taiwan) Lab is open for business.

Overview

ART is a suite of tests designed to certify Assistant functionality for devices that integrate the
Google Assistant. To be certi ed as an Assistant Ready device, your device must pass the
speech testing (/assistant-partners/resources/art/speech) requirements outlined within this guide.

The ART test software and dataset is available for download here
 (https://drive.google.com/drive/folders/10QZwQNjUQZGca-XSs9EzJbALPbO6vX5R?usp=sharing).

Prerequisites for pe orming Assistant testing

https://developers.google.com/assistant-partners/resources/art 1/3
06/04/2021 Assistant Readiness Tests (ART) Guide | Assistant Partners

Refer to Device Requirements (/assistant-partners/resources/art/dut-reqs).

Scope

ART is a speech recognition quality test suite comprised of the following:

Automatic Speech Recognition (ASR) and Hotword Acceptance - WER tests

AEC - Eraser tests

Hotword False Accept - False Accept tests

Tests may be executed in any order but are generally executed in order of complexity starting
with ASR and WER tests with no added noise.

Assistant speech validation tests utilize a series of queries to validate hotword acceptance
and ASR in quiet conditions, noisy conditions, and with media playback from the device.
Recordings of speech and media which do not contain hotwords are used to validate hotword
false accept performance.

OEMs, SOC vendors, and other interested parties may acquire a copy of the certi cation tools
and dataset by contacting their Google Technical Account Manager (TAM).

For further details, refer to Speech testing (/assistant-partners/resources/art/speech).

Out of Scope

While other certi cations and testing are required for a device (e.g. GVA Cast) to be Google
certi ed, this guide has limited scope. It does NOT incorporate details of the following tests:

FDR (Factory Device Reset)

Multiroom Certi cation

Speci cation Compliance

Cast Readiness Tests

Feedback Testing

https://developers.google.com/assistant-partners/resources/art 2/3
06/04/2021 Assistant Readiness Tests (ART) Guide | Assistant Partners

Cast Device Setup Testing

Validation of UX States

Validation of Device Actions

OoBE

Refer to the complete certi cation process here (/assistant-partners/cast-assistant/certi cation).

Baseline/success parameters for testing

Success parameters of testing may change in scope, at any point in time, at the discretion of
Google. Success parameters for the current version of testing is de ned in simple pass or fail
terms. A device either meets or doesn't meet the requirements of the tests.

Device performance is compared against a reference device tested in the same environment.
See lab accreditation policy (/assistant-partners/resources/art/policy#lab-accreditation) for details.

All rights reserved. Java is a registered trademark of Oracle and/or its a liates.

Last updated 2020-07-08 UTC.

https://developers.google.com/assistant-partners/resources/art 3/3