CITY OF NEW YORK

COMMISSION ON HUMAN RIGHTS

In the Matter of the Complaint of:

NEW YORK CITY COMMISSION ON HUMANRIGHTS,

Complainant,

- against - Investigation No.: M-P-P-19-54310

TUMBLR, INC.,

Respondent.
STIPULATION AND ORDER
U

WHEREAS, the New York City Commission on Human Rights (“Commission”) haslaunched an investigation into certain algorithmic decision-making processes of Tumblr,
Inc.(“Tumblr” or “Respondent”) as used on its Tumblr.com website to classify or flag as “adult”images posted to the Tumblr.com website, which are not permitted and are
removed from Tumblr.com (the “Adult Content Ban” or “Adult Content Ban Process”); and

WHEREAS, Respondent Tumblr has informed the Commission that since its acquisition by Automattic Inc., it has made changes, including replacing its algorithmic
decision-making classifier (the “Original Classifier”) for “adult” content to more accurately classify images and data and revising its appeal process for images flagged as
“adult” under its Adult Content Ban sothat human reviewers (“Reviewers”) now have final say over the Adult Content Ban Process; and

WHEREAS, the Commission and Respondent (collectively, “Parties”) now desire toamicably, fully, and finally settle the issues arising out of the investigation without
further proceedings;

NOW, THEREFORE, PURSUANT TO THIS STIPULATION AND ORDER (“AGREEMENT”), IT IS HEREBY STIPULATED AND AGREED, by and among the
Parties as follows:

Stipulation and Order

New York City Commission on Human Rights v. Tumblr, Inc. (M-P-P-19-54310)Page 1 of 11
• Submissions called for by this Agreement shall be made to Geoffrey L. Wertime at: NewYork City Commission on Human Rights, Attn: Geoffrey L. Wertime, Esq., 22 Reade Street,
3rd Floor, New York, NY 10007, (347) 988-5787 (cell), (212) 416-0104 (office), (646) 500-7427 (fax), gwertime@cchr.nyc.gov.

• Trainings.

• NYCHRL Training. No later than one hundred and eighty (180) calendar days after the execution of this Agreement, Respondent will provide training regarding
Respondent’s obligations under the New York City Human Rights Law (the “NYCHRL Training”) to all of Respondent’s employees and subcontractors, both within the
United States and internationally, who reviewimages flagged as “adult” under the Adult Content Ban, with a focus on the NYCHRL’s protections for members of the public
based on actual or perceived sexual orientation and gender identity (“SOGI”). As much of Respondent’s workforce is distributed around the world, the training will be
virtual.

• The Commission’s Community Relations Bureau shall provide training fortwenty-five (25) employees identified by Respondent.

• Respondent shall provide training for its remaining subcontractors(approximately 125), as follows:

• The NYCHRL trainer shall possess substantial knowledge of andhave substantial experience advising companies like Tumblr on anti-discrimination law.

• Within ninety (90) calendar days of the execution of this Agreement, Respondent shall provide the following information, tothe individual identified in
Paragraph 1 of this Agreement, for finalapproval by the Commission:

Stipulation and Order

New York City Commission on Human Rights v. Tumblr, Inc. (M-P-P-19-54310)Page 2 of 11

• The name and qualifications of the individual(s) or entitywhich will conduct the NYCHRL Training; and

• Draft training materials.

• The individuals who attend the Commission’s NYCHRL Training andRespondent’s NYCHRL Training shall sign in and out of an electronicattendance sheet, and
Respondent will keep the attendance sheet for Respondent’s NYCHRL Training for a period of 90 days.

• Within ten (10) calendar days of completion by all trainees of Respondent’s NYCHRL Training, Respondent must provide proof of completion of Respondent’s
NYCHRL Training by returning an affidavit,modeled on Appendix A, to the Commission. Respondent may omit the names of attendees and replace them with unique
identifiers, such as employee identification numbers and job titles, for the purposes of compliance with relevant data protection rules, including the European Union’s
General Data Protection Regulation (“GDPR”) as well as for protection of Respondent’s employees and subcontractors.

• Unconscious bias training. No later than one hundred and eighty (180) calendar days after the execution of this Agreement, an entity with substantialexperience advising
 companies like Respondent Tumblr on issues of diversityand inclusion shall provide training to all of Respondent’s employees and subcontractors in the United States and
internationally (“SOGI Trainees”) who review images under the Adult Content Ban. The training will cover unconscious bias and SOGI (the “SOGI Training”).

• The SOGI trainer must possess substantial knowledge of and have substantial experience advising companies like Tumblr on issues of diversity and inclusion and
SOGI issues. The training shall include a discussion and illustrative examples of individual, interpersonal, institutional, and structural homophobia, heterosexism,
transphobia, and

Stipulation and Order

New York City Commission on Human Rights v. Tumblr, Inc. (M-P-P-19-54310)Page 3 of 11

cissexism in the United States. The SOGI Training must also include a discussion and illustrative examples of homophobia and transphobia in the
United States as well as implicit and unconscious bias. The training must be at least two (2) hours long and shall be interactive. As much of
Respondent’s workforce is distributed around the world, the training will be virtual.

• Within ninety (90) calendar days of the execution of this Agreement, Respondent shall provide the following information, for final approval bythe Commission:

• The name and qualifications of the individual(s) or entity whichwill conduct the SOGI Training; and

• Draft training materials.

• SOGI Trainees shall sign in and out of an electronic attendance sheet, andRespondent will keep the attendance sheet for a period of 90 days.

• Within ten (10) calendar days of the SOGI Training by all SOGI Trainees,Respondent must provide proof of completion of the SOGI Training by returning an
affidavit modeled on Appendix A. Respondent may omit the names of attendees and replace them with unique identifiers for the purposes of protection and
compliance with relevant data protection rules,including the GDPR.

• Appeal Process.

• Respondent has advised the Commission that it has improved its appeal process under the Adult Content Ban to have Reviewers review images flagged as adult before
removing them from Tumblr.com, rather than havingReviewers only review post-removal appeals.

Stipulation and Order

New York City Commission on Human Rights v. Tumblr, Inc. (M-P-P-19-54310)Page 4 of 11

• Respondent will begin tracking reviews of content flagged by its algorithmicdecision-making classifier (the “Classifier”) as “adult” under the Adult Content Ban, as follows:
 • Within 90 (ninety) calendar days of the execution of this Agreement, Respondent shall revise the appeal form for its Adult Content Ban (“AdultAppeal Form”) to
include the text and text entry boxes in Appendix B.

• Within 60 (sixty) calendar days of the published changes to the Adult Appeal Form (the “Tracking Start Date”), Respondent shall search all filedappeals with a list of
SOGI search terms on a monthly basis, to be approved by the Commission, to identify the number of reviews alleging SOGI bias. Respondent hereby agrees to
incorporate new search terms based on Respondent’s own research and the Commission’s instructions.

• Respondent shall also examine five hundred (500) appeals from each of six (6) separate days (for a total of 3,000 images), randomly selected withone (1) day from
each of the past six (6) months, and filed within the six
(6) months prior to the execution of this Agreement, to identify possible SOGI bias in its determinations, and shall use five hundred (500) or 75%of
the incorrectly identified images, whichever is smaller, to retrain the Classifier.

• Respondent shall track the number of SOGI-related appeals that result in reversal of the decision to flag the content as “adult.” Starting on the first business day of the
second month following the Tracking Start Date, and continuing for a period of eighteen (18) months, Respondent shall, on a monthly basis, report to the individual
identified in Paragraph 1 of this Agreement the total number of appeals filed the previous month, the total number of SOGI-related appeals, and the percentage of the
whole that the SOGI appeals represent in that month. Reports will be considered timely ifreceived within ten (10) business days of the first business day of the month.
Stipulation and Order
New York City Commission on Human Rights v. Tumblr, Inc. (M-P-P-19-54310)Page 5 of 11

• Starting by the Tracking Start Date, Respondent shall, on a monthly basis,use a random sample of five hundred (500) or 75%, whichever number is smaller, of the
content from successful SOGI appeals to further train Respondent’s Reviewers and the Classifier.

• Classifier Review. Respondent shall take the following additional steps to review the Classifier for SOGI bias and restructure it as needed to avoid any such bias, as follows:

• Classifier Expert. Respondent shall hire an individual or entity with substantial knowledge of SOGI issues and with substantial experience advising companies like
Respondent on issues of diversity and inclusion, including SOGI, as they relate to image classifiers (the “Classifier Expert”) to advise Respondent on reviewing its Classifier
to address SOGI discrimination.

• Within sixty (60) days of the Tracking Start Date, Respondent shall submitto the individual identified in Paragraph 1 of this Agreement, for the Commission’s
approval, the following information:

• The name and qualifications of the individual(s) or entity whichwill act as the Classifier Expert.

• Classifier Review. The Classifier Expert shall undertake a holistic review ofthe Classifier and any potential SOGI discrimination. Respondent shall grantthe Classifier
Expert access to the Classifier, its appeals data, its SOGI data, and anything else the Classifier Expert deems necessary.

• Classifier Training. No later than one hundred twenty (120) calendar days after the Tracking Start Date, the Classifier Expert will provide training (the “Classifier Training”)
 on how to avoid and eliminate SOGI bias in algorithmsfor all of Respondent’s employees, agents, interns, and independent contractors who work on the Classifier (the
“Classifier Group”). As much of Respondent’s workforce is distributed around the world, the training will be virtual.

Stipulation and Order

New York City Commission on Human Rights v. Tumblr, Inc. (M-P-P-19-54310)Page 6 of 11

• Within ninety (90) calendar days after the Tracking Start Date, Respondent shall provide the following information, to the individualidentified in Paragraph 1 of
this Agreement, for final approval by theCommission:

• Draft training materials.

• The individuals who attend the Classifier Training shall sign in and out ofan electronic attendance sheet, and Respondent will keep the attendance sheet for a period
of 90 days.

• Within ten (10) calendar days of completion by all trainees of the Classifier Training, Respondent must provide proof of completion of the Classifier Training by
returning an affidavit, modeled on Appendix A, to the Commission. Respondent may omit the names of attendees and replacethem with unique identifiers, such as
employee identification numbers andjob titles, for the purposes of protection and compliance with relevant dataprotection rules, including the GDPR.

• Consultation. After the Classifier Training, Respondent shall consult with theClassifier Expert and take all reasonably necessary steps, as determined by theClassifier Expert,
to test and retrain the Classifier to address any SOGI discrimination.

• If Respondent believes that any particular step that the Classifier Expert determines is reasonably necessary is infeasible, Respondent may requestthat the
Commission find that that step is an undue burden.

Stipulation and Order

•
In the event that Respondent makes such a request, it shall submit a request to the individual identified in Paragraph 1 of this Agreement and include detailed
 information about (a) the step the Classifier Expert has requested and (b) the reason(s) that it would constitute an undue burden. Ifthose reasons are financial,
Respondent shall include financial informationsufficient to show that it is beyond Respondent’s means.

New York City Commission on Human Rights v. Tumblr, Inc. (M-P-P-19-54310)Page 7 of 11

• Review Report. Within one hundred eighty (180) calendar days of the Tracking Start Date, Respondent shall submit to the individual identified in Paragraph 1 of this
Agreement a report by the Classifier Expert (the “ReviewReport”) that contains the following information:

• The Classifier Expert’s initial assessment of the Classifier and any relatedSOGI issues;

• The steps Respondent and the Classifier Expert took to address anypotential SOGI bias; and

• The Classifier Expert’s ultimate evaluation of the Classifier and SOGIdiscrimination after the Review.

• The Review Report shall be written in plain English to the extent possible.

• Respondent shall also contact the individual identified in Paragraph 1 toarrange a meeting with the Commission to discuss the Review Report.

• Postings.

• No later than thirty (30) calendar days after the execution of this Agreement,Respondent shall display postings outlining its obligations under the NYCHRL in the employee
notice area on Respondent’s internal employee communications system.

• Respondent shall prominently display the Commission’s “Notice of Rights”poster, in both English and Spanish, attached as Appendix C, and “LGBTQRights” poster,
attached as Appendix D, both in color, on Respondent’s internal employee communications system.

• Within thirty (30) calendar days after the execution of this Agreement, Respondent shall submit proof of its compliance with this Paragraph to theindividual identified in
Paragraph 1 of this Agreement.

Stipulation and Order

New York City Commission on Human Rights v. Tumblr, Inc. (M-P-P-19-54310)Page 8 of 11

• Proof shall be submitted in the form of photographic proof of postings ofthe Notice on the internal employee communications system.

• Tumblr Team.

• To augment the team involved in the Adult Content Ban Process, the Appeal Process as described in Paragraph 3 of this Agreement, and the Classifier Group (collectively, the
 “Team”), Respondent will assign at least nine (9) Team employees to assist with the processes and reviews described in this Agreement as needed for a period of at least six
(6) months from the executiondate of this Agreement, and then assign as many employees as needed to maintain those processes and reviews thereafter.

• Respondent may request extensions of time to complete any item required by the Agreement by requesting in writing to the individual identified in Paragraph 1 of this
Agreement.

• By signing this Agreement, Respondent acknowledges that failure to timely comply with any of the provisions in paragraphs 1 through 6 of this Agreement shall constitute
noncompliance with a Commission Order pursuant to § 8-107(8) of the Administrative Code of the City of New York. In the event the Commission believes that Respondent has
violated any provision of this Stipulation and Order, the Commission will provide Respondent with notice of such violation, and Respondent shall then respond to such notice and/or
cure such non-compliance within thirty (30) days.

• Electronic, facsimile, and scanned signatures are to be treated as original, and thisAgreement may be executed in counterparts.

• This Stipulation and Order of Settlement embodies the entire agreement between theParties in this matter.

• Upon completion of the requirements and/or conditions contained in Paragraphs 2 through 6 of this Agreement, Investigation Number M-P-P-19-54310 against Respondentshall be
closed as a successful intervention.
Stipulation and Order
New York City Commission on Human Rights v. Tumblr, Inc. (M-P-P-19-54310)Page 9 of 11

IN WITNESS WHEREOF, the Parties hereto have set their hand as of the date and place sostated.

Dated:

ACKNOWLEDGMENT BY CORPORATION

STATE OF e>.., ,, )
COUNTY OF ,'i. )

On the ._ day of I!,._. in the year 20before me, personally came

\ «..; to
me known, who, being by me duly sworn, did depose and say that he/she/they reside(s) in
 2..2i::;;,,. 4-s........a., Q-. .t:s. . c=:Jb...,,..;;;:,
that he/she/they is/are the --' :::..::EL==s:Ei-=- ::..:: c..=.....;
(president or other officer or direct r attorney in fact duly appointed) of
\.cp- \ L • (name of corporation), the corporation describedin and which executed the above instrument; that he/she/they know(s) the seal of said corporation; that
the seal affixed to said instrument is such corporate seal; that it was so affixed by authority of the board of directors of said corporation, and the he/she/they signed his/her/
their name(s) theretoby like authority.

NOTARYPUBLIC
,1.ICT'HONY MONTERO .
Motary
C , "'
SanFrancisco ou n , ,
Public. California
,..
Commi,sion " 2297 4S8 3
MYcomm.
Expire, t,.ug 13' 101

Stipulation and Order

New York City Commission on Human Rights v. Tumblr, Inc. (M-P-P-19-54310)Page 10of 11

ACKNOWLEDGMENT
State of California
County of SAN
FRANCISCO
On JANUARY 18, 2022
beforeme, ANTHONY MONTERO, NOTARY PUBLIC
(insert name and title of the officer)
personally appeared _P_A_U_L_E_S_I_E_M_I_N_S_K_I_*_**_*_*_*_*_*_*_*_*_*_*_*_*_*_* _who proved to me on the basis of satisfactory evidence to be the person(s) whose name(s) is/are
subscribed to the within instrument and acknowledged to me that he/she/they executed the same in his/her/their authorized capacity(ies), and that by his/her/their signature(s) on the instrument the person(s), or the entity
upon behalf of which the person(s) acted, executed the instrument.

I certify under PENALTY OF PERJURY under the laws of the State of California that the foregoing
paragraph is true and correct.

.........f
WITNESS my hand and official seal.
ANTHONYMONTERO
Notary Public - Cdlifornia San Franci1co County Commi11ion # 2297458 -
y Comm. Expires Aug 13, 2023
(Seal)
A notary public or other officer completing this certificate verifies only the identity of the individual who signed the document to which this certificate isattached, and not the truthfulness, accuracy, or
validitv of that document.

Law Enforcement Bureau

New York C' Commission on Hu

By:

Sapna V.
Deputy C mm1ss10ner Law Enforcement Bureau

Commissioner/Chair
New York City Commission on Human Rights
 Prepared by: Geoffrey L. Wertime
Agency Attorney

Dated: b \• L 4 ? 0'22

Stipulation and Order

New York City Commission on Human Rights v. Tumblr, Inc. (M-P-P-19-54310)Page 11 of 11