1 Duolingo English Test

The Duolingo English Test is a measure of English language proficiency. The test has been

designed for maximum accessibility; it is delivered via the internet,without a testing center,

and is available 24 hours a day,365 days a year. It has been designed to be efficient. It

takes less than one hour to complete the entire process of taking the test (i.e.,onboarding,test

administration,uploading). It is a computer-adaptive test (CAT),and it uses item types that

provide maximal information about English language proficiency. It is designed to be user

friendly; the onboarding,user interface,and item formats are easy to interact with.

This document provides an overview of the design the Duolingo English Test. It contains a

discussion of:

? the test’s accessibility,delivery,proctoring and security processes;

? the demographic information of the test taking population;

? the test’s items,how they were created,and how they are are delivered and scored;

? and the statistical characteristics of the Duolingo English Test.

The test scores are intended to be interpreted as reflecting test takers’ English language ability

and used in a variety of settings,including for university admissions decisions.

2 Accessibility

Broad accessibility is one of the central motivations for the Duolingo English Test. Tests

administered at test centers consume resources which limit accessibility: they require

appointments at a physical testing center within certain hours on specific dates (and travel

to the test center),and carry considerable registration fees. The maps in Figure 1 show the

concentration of test centers in the world (left panel) compared to internet penetration in the

world (right panel). The figure shows a stark difference in how much more easily an internet

based test can be accessed than a test center* . While the ratio of internet access to test center

access is a somewhat limited metric — not every internet user has access to a device that can

run the Duolingo English Test,and physical test centers can usually handle dozens of test-takers

at once — it is still clear that the potential audience for the Duolingo English Test is orders of

magnitude larger than those who could be served currently by more traditional test centers. By

delivering assessments on-demand,24 hours a day,to an estimated 2 billion internet-connected

devices anywhere in the world for US$49,we argue that the Duolingo English Test holds the

potential to be the most accessible valid and secure language assessment platform in the world.

?Central Africa is underserved by both models.

? 2019 Duolingo,Inc4

Duolingo Research Report

Figure 1. Global heatmaps comparing the number of physical test centers (left) per 1 million inhabitants

(TOEFL?,IELTS?,and pTE? combined) vs. the percentage of internet users (right).

3 Test Administration and Security

The Duolingo English Test is administered online,via the internet to test takers. The security of

Duolingo English Test scores is ensured through a robust and secure onboarding process,rules

that test takers must adhere to during the test administration,and a strict proctoring process.

All of these procedures are evaluated after the test has been administered and prior to score

reporting.

3.1 Test Administration

Test takers are required to take the test alone in a quiet environment. The Duolingo English Test

can be taken in the Chrome and Opera browsers worldwide. In China,the test can be taken on

the the 360 and QQ browsers as well. An internet connection with at least 2 Mbps download

speed and 1 Mbps upload speed is recommended for test sessions.

3.2 Onboarding

Before the test is administered,test takers complete an onboarding process. This process

checks that the computer’s microphone and speaker work. It is also at this time that the test

takers’ identification information is collected and that test takers are informed of the test’s

administration rules.

3.3 Administration Rules

The list behaviors that are prohibited during an administration of the Duolingo English Test are

listed below. In addition to these behavioral rules,there are rules for the test takers’ internet

browsers. The browsers are locked down after onboarding,which means that any navigation

away from the browser invalidates the test session. Additionally,all browser plugins must be

disabled.

? Interacting with anyone

? Allowing other people in the room

? Using headphones or earbuds

? Disabling the microphone or camera

? 2019 Duolingo,IncDuolingo English Test: Technical Manual

5

? Looking off screen

? Moving out of frame of the camera

? Accessing any outside material or devices

? Leaving the web browser

3.4 proctoring & Reporting

After the test has been completed and uploaded,it undergoes a thorough proctoring review using

human proctors with TESOL/applied linguistics expertise,which is supplemented by artificial

intelligence to call proctors’ attention to suspicious behavior. This process take no more than

48 hours after the test has been uploaded. After the process has been completed,score reports

are sent electronically to the test taker and any institutions they elect to share their scores with.

Test takers can share their scores with an unlimited number of institutions.

4 Test Taker Demographics

In this section,test taker demographics are reported. During the onboarding process of each test

administration,test takers are asked to report their first language (L1),date of birth,and their

gender identity. Their country of residence is logged when they show their proof of identification

during the onboarding process. There were 23,460 people who took certified Duolingo English

Tests between August 1,2017 and June 30,2019.

The most frequent L1s of Duolingo English Test test takers include Mandarin,Spanish,

Arabic,and portuguese (see Table 1). Duolingo English Test test takers represent 115 unique

L1s and 168 unique countries. The full tables of all test taker L1s and countries of origin can be

found in the Appendix.