Benjamin Wickham 

Code.org Evaluation 
February 18, 2020 

Who is Code.org? 
“Code.org® is a nonprofit dedicated to expanding access to computer science in schools and 
increasing participation by women and underrepresented minorities.” (About Us, 2020). In this 
evaluation we will be looking at our nonprofit organization, Code.org. Code.org is an 
organization dedicated to spreading computer science education to every school and student 
who has the desire to learn it, no matter their life situation. We provide K-12 curriculum for 
computer science to the largest school districts in the United States, and organize the campaign 
known as the “Hour of Code”, designed to introduce students to coding concepts.   

 
  

A saddening fact is that to the majority of students, opportunities to learn about computer 
science are both terrifying, and few and far in-between. Code.org’s mission is to help students 
get more access to these great opportunities, and simplify them in a way that will demystify and 
even excite students about learning to code. 

How do we measure our success? 

We would like to develop certain criteria with which to judge our success in reaching this goal. 
In order for us to say we have brought access to more students, I would like to judge: first, if the 
program benefits students who have less access to computer science education, second, if the 
education helps them learn computer science principles, and third, whether or not the students 
enjoy it. 

Step I: Reach Everyone 

The first important criteria is whether our programs are accessible to all students. This is 
important because if we only serve the students who already have access to CS education, we 
won’t be able to expand CS education much more than it already is. Groups that have less 

 
  

access to CS education are minorities, women, and poorer school districts. Code.org is happy to 
report that most of our students are women or minorities. The graphic below shows the 
distribution of students that are female, underrepresented minorities, and in high needs schools. 
(Image accessed from Code.org’s website at h
​ ttps://code.org/about​)

 
Code.org aims to combat this lack in diversity by focusing at the base, K-12 schooling. As stated 
on our website “​Our focus is primarily on K-12 learning, which is where the diversity problem begins:
Among women, those who try AP Computer Science in high school are 10 times more likely to major
in computer science.” (Diversity in K-12 Computer Science, 2020). The key is to help the younger
generation of underrepresented groups learn, so that they can represent these populations in
universities and the workforce in the future.

Step II: Teach Well 

The next criteria we’d like to focus on is teaching well. We would like to look at how much students
actually learn from the puzzles and challenges we give them. Will they be better prepared to learn
actual programming languages by working on our puzzles and activities? It doesn’t do good to reach
lots of students if we don’t teach them as well.

While there isn’t a lot of data on long-term success of students who have used Code.org
specifically--Code.org started in 2013 (About Us, 2020)--we have done some studies of students
using data from Code.org and have some information on their proficiency within the program.

 
  

We first must define proficiency. To do this we have taken key elements of coding--such as
sequencing and looping--and determined what a puzzle with a difficulty level 1-5 would entail for that
element. From there, we measured how long it took for a student to become proficient in that
element. Our website states:

“‘Proficiency’ within a concept is defined as the successful completion (using no hints and with
the optimal number of blocks) of 3 puzzles of difficulty level 3 within that concept. Reaching
‘Basic Coding Proficiency’ is defined as reaching proficiency in three different concepts.”
(Basic Coding Proficiency, 2017)

Essentially, by completing three or more puzzles ranked

difficulty three or higher without hints or mistakes in three
different concepts, a student is determined proficient in coding.
A graphic here (accessed from Code.org’s website at
https://code.org/about/evaluation/proficiency2016​) shows how
long it takes for students to get there:

We can see that over 90% of students who do over 200 puzzles are able to reach coding proficiency.

Obviously there are some drawbacks to this approach to measurement, primarily that this system is
contained within our own programs. We have yet to gather substantial data from outside, but these
coding concepts are well accepted as necessary coding knowledge by the CS community, and we
have supporting evidence that students will learn them by working in our program.

Step III: Have Fun 

One of the biggest hurdles to learning code is how boring it
can look (and sometimes feel). Screens full of words and
symbols with very specific syntax aren’t easy for children to
understand and pay attention to. For those that are new, it
can also be terrifying. That’s why spreading access to CS
education isn’t just in the content, but the presentation. If
students aren’t engaged, they’re not learning anything.

 
  

We are happy to report that as part of our program we use block-based coding (as opposed to
text-based) to help avoid the confusing (and sometimes frustrating) syntax issues. We also use
programs that involve well-known games and graphics to keep the children interested. This way we
can avoid boring the children and focus on teaching them.

But don’t just take my word for it. The Code.org site includes quotes from excited educators talking
about their students (About Us, 2020):

"Every single day yielded the same results— 100% engagement."  

- Nina Nichols Peery, Teacher 
 
"I have never, ever seen my students so excited about learning."  
- Frank Martinez, Teacher 
 
The students love it, teachers love it, we at Code.org love what we’ve created, too. It’s fun,
challenging, and keeps kids focused.

A Great Force for Change 

At Code.org we feel it is very important to address the lack of access to computer science knowledge
and bridge the gap for K-12 students. We can see that many underrepresented groups are now able
to learn using our programs. We can see that students learn as they go and slowly achieve
proficiency along the way, and we can also see that they have fun doing it. While we may not have
much long-term data about the impact we’ve made, going at only a few years of life, we expect to see
a lot of positive differences in Computer Science programs and in the workforce in years to come.

References
About Us. (n.d.). Retrieved February 21, 2020, from ​https://code.org/about

Code.org's plan for diversity in K-12 computer science. (n.d.). Retrieved February 21, 2020, from
https://code.org/diversity

Towards Measuring Basic Coding Proficiency. (2017, March). Retrieved from

https://code.org/about/evaluation/proficiency2016