Smart doorbells may be fun, but we don’t know who

is using your face

This changes everything | Training facial recognition algorithms requires masses of data, and big
tech companies can go to extreme lengths to get that, says Annalee Newitz

HUMANS | COMMENT 11 December 2019

Nest Hello Smart Doorbell

Ellie Reed/Alamy Stock Photo

I AM one of millions of people around the globe who have a smart doorbell with a hidden camera, controlled
through an app. Now I can’t stop watching videos of my porch and the street beyond. I have a Google Nest smart
home system, and it sends alerts to my phone that say things like: “Your doorbell camera thinks it has spotted a
person.” Who wouldn’t be curious after receiving that? Is the person on my porch a suspicious lurker? Someone
delivering a package? An old friend? In my case, the answer has, so far, been none of these. Usually, the people my
camera has spotted are my neighbours’ children racing out the door to school, or a person walking by with their
dog.

It doesn’t feel like surveillance, exactly. There is something almost fun about peering out of your front door
without anyone knowing. Yet hundreds of police departments in the US are using footage from Amazon’s Ring
doorbells to snoop on streets. They ask residents to install Ring to create a “neighborhood watch” of street cameras
that law enforcement can access. Amazon even offers discounts and freebies to people buying Ring through the
police.

Officers need user consent to access footage through Ring’s social app, Neighbors, but they don’t need a warrant.
That means I could spy on my neighbours and passersby for the police without any court oversight. My Google
Nest system doesn’t currently have an app like Neighbors, nor partnerships with police, but it is still easy to share
footage. Though some cities like San Francisco have banned police from using facial recognition, many places and
organisations are embracing it.

There is a big difference between smart doorbells and traditional CCTV. For one thing, they are privately owned.
In the US, that means police can get all kinds of surveillance data just by asking nicely through apps like
Neighbors.

I am constantly uploading data about people’s faces and whereabouts to the cloud. Each time a person rings my
bell, my app takes a snapshot and asks whether I know them. I can choose to label friends’ faces with their names,
or simply say that I recognise them so Nest can alert me when a “known person” arrives. My camera also captures
images of cars driving on my street and the junction nearby, which means my doorbell could be logging number
plates too.

Where is all that data going? We can be sure that at least some of it is used to train algorithms to recognise people.
Companies are desperate for this kind of face data. A Google contractor was recently busted for paying homeless
people if they agreed to pose for selfies in a “game”. That game turned out to be facial recognition software for the
new Google Pixel phone. And there are other sneaky ways to harvest faces. Security at a Taylor Swift concert in
2018 set up a kiosk that secretly scanned the faces of her fans to check for her known stalkers. Several concert
venues and airports have similar technologies in place.

“Facial recognition algorithms are notoriously unreliable – especially for people of colour”
What’s more, facial recognition algorithms are notoriously unreliable – especially when it comes to people of
colour. Due to biased data sets and other technical problems, they misidentify people of colour far more often than
white people. This means police often nab the wrong person. But tech companies are working to close the face
divide. One reason why Google hired that contractor to take pictures of the homeless was reportedly to get more
people of colour into its database.

Chinese companies who make surveillance software are taking things to the next level. In 2018, China-based
company CloudWalk Technology cut a deal with the government of Zimbabwe to do mass surveillance of its
population to improve algorithmic recognition of the faces of Zimbabweans. Recently, Chinese authorities have
taken genetic samples from Uighurs, a primarily Muslim minority group, in the hopes that they will learn how to
recreate Uighur facial features from DNA. The Chinese government has put roughly a million Uighurs
into “reeducation camps”. It hopes to simplify the process by using DNA to identify Uighurs in surveillance
footage.

I just had a conversation with my partner through the doorbell. He was fixing the lock on our door, and my phone
alerted me to a “familiar face”. When I saw him on camera, I turned on its speaker and yelled “EXTERMINATE!
EXTERMINATE!” Somehow, he guessed it was me. It’s harmless footage, but who knows what else was captured
behind him? I have no idea what will happen to my Dalek impression video if Google sells Nest to a surveillance
company, and I can’t control who might be targeted inadvertently by my doorbell’s constant vigilance. Even when
I tell my app to delete footage weekly, I’ll never feel completely sure.

Read more: https://www.newscientist.com/article/mg24432603-800-smart-doorbells-may-be-fun-but-we-dont-
know-who-is-using-your-face/#ixzz70eCTaSE3