CS349 User Interfaces

Syllabus

Jeff Avery & Ed Lank

Pop Quiz
§ Why is Waterloo so well-known in Canada for Computer Science?
§ Answer: Because 40 years ago, this is what a computer looked like …

Red Room
October 1974

2
Another Way to Think About This
§ This is what a user-interface looked like before the early 1980s:

3 CS 349 - Syllabus
4
5
6 CS 349 - Syllabus
Today?

7 CS 349 - Syllabus
 2015
e s 2010 iPad
rf ac
e
Int 2005
r
U se iPhone
al 2000
hic
ap iPod
Gr 1995
WWW
1990
c es
fa
er 1985
Int
l
ona Macintosh
a ti 1980
rs IBM PC
ve
Co
n 1975 Apple II
1970 Intel 4004 Microprocessor
s ARPANET
fa ce 1965
r
te
In 1960
h
tc
Ba
1955
1950

1945 8 CS 349 - Syllabus

ENIAC
How has the role of the UI programmer changed?

40 years ago Today

Who used a computer Specialist Everyone

What they knew about the Lots Little

computer

How they learned to use a Reading operators Tinkering

computer manuals

Where they found the applications Built them App store, www
that ran on the computer

How many different computers Maybe one Many !

they would have interacted with
during the course of their day
9
What is a User Interface?
§ Maybe, “the human’s view of the computer”

§ Maybe, “the place where humans and computers meet”

10 CS 349 - Syllabus
Definition: User Interface
§ A user interface is the place where a person expresses intention to an
artifact, and the artifact presents feedback to the person.

express intention

presents feedback

person artifact

11 CS 349 - Basic Concepts

User Interfaces
§ Does a microwave have an interface? A user interface is the
place where a person
§ A refrigerator? expresses intention to
an artifact, and the
§ A door bell?
artifact presents
§ A hammer? feedback to the person.

§ A jet?

12 CS 349 - Basic Concepts

13 CS 349 - Syllabus
CS349 Objectives
The goal of CS349 is to teach you:
1) how to design, implement, and evaluate user interfaces
- provide the foundational knowledge for building highly interactive, usable
desktop, web and mobile applications
- illustrate the underlying architecture of modern GUI toolkits
- teach strategies applicable across a range of interface problems
- teach essential design tools, techniques, and processes
2) ways to understand users
- physical and cognitive abilities of users
- visual design principles

User-centered design is covered more extensively in CS449.

14 CS 349 - Syllabus
15
List of Topics
§ Syllabus (introduction, motivation), Basic
Concepts (user interfaces, interaction)
§ Multiple windows (BWS, Windows Mgr, X
examples)
§ Java: advanced (interfaces, listeners, canvas,
drawing)
§ Events (event capture, event loop, dispatch) w.
Java dispatch examples
§ 2D graphics (shape models, mouse interaction,
selection)
§ MVC (rationale, demos, widgets)
§ Widgets (physical vs. logical input, consistency,
toolkits)
§ Layout (Dynamic/Responsive, Layout
Managers, Composite/Strategy DP)
§ Visual perception (temporal/spatial resolution,
color, displays)
§ Users: visual design (gestalt principles,
structure)
§ Responsiveness (perception, UI feedback, Java
UI threads, Web async calls)
§ Undo/Redo; Cut/Paste
16
§ Design principles (useful vs. usable, Norman, gulf
evaluation/execution, metaphor, feedback)
§ Design process (UCD, prototyping)
§ History (interaction up to WIMP interfaces)
§ Touch interfaces (Input, Interaction problems, DM
on WIMP/touch, gestural, mobile)
§ Touchless interfaces (gestural/speech design and
problems)
§ Android: advanced (intents, activities, mvc,
graphics)
§ Users: perception and cognition (cognetics,
memory, bias)
§ Accessibility (impairments, curb-cut, legal
obligations)
§ Input (types of input, keyboards, positional input,
absolute/relative, direct/indirect)
§ Input performance (KLM, Fitt’s Law, Steering
Law)
§ HCI research, methodology
CS349 People
§ Instructors:
- Jeff Avery & Ed Lank [Office hours by appointment]
§ Instructional Assistant:
- Jingjie (Vincent) Zheng
§ Teaching Assistants:
- Reza Saputra
- Terence Dickson
- Qi Feng (Edmund) Liu
- Shaishav Siddhpuria
- Alex Williams

§ Instructional Support Coordinator:

- Caroline Kierstead

17 CS 349 - Syllabus
Website, Schedule, Textbook
§ Web Site: www.student.cs.uwaterloo.ca/~cs349/
redirects to http://student.cs.uwaterloo.ca/~cs349/w16/about.html

§ Schedule on website
- lecture topics & slides, assignment & midterm dates, tutorials, sample code
§ Textbooks - useful but not required!

- Building Interactive Systems,

Dan R. Olsen Jr.

- The Design of Everyday Things,

Donald A. Norman

- Designing with the Mind in Mind, Jeff Johnson

18 CS 349 - Syllabus
Lectures
§ Mon, Wed 11:30-12:50 MC 2054 Ed Lank
§ Tues, Thurs 08:30-09:50 PHY 150 Jeff Avery
§ Tues, Thurs 01:00-02:20 MC 4063 Jeff Avery
§ Tues, Thurs 02:30-03:50 MC 4041 Jeff Avery

During lectures:
§ Participate by asking and answering questions
§ Try and limit computer use for anything other than taking notes.

We’ll attempt to keep all lecture sections in-sync; if you miss your
scheduled lecture, you have three other chances to catch it!

19 CS 349 - Syllabus
Getting Help Outside of Lectures
§ Office Hours
- Jeff & Ed: by appointment
- TAs: to be determined
§ Piazza for Q&A
- Please enroll with a meaningful name
- Search before you post
- Use a meaningful title for every post
- Answer questions, but don’t be too explicit
- Build one collaborative answer rather than many follow-ups

20 CS 349 - Syllabus
Assignments
Assignments meant to provide meaningful, engaging experiences in
constructing interfaces… while giving you the opportunity to create
applications you will want to share with others.
§ You will build both desktop and mobile applications.
- We use Java 8 and Android for assignments.
- We’ll post material online to help you learn SVN and Java. You are
expected to cover this material on your own time!
§ There’s lots of room for creativity in assignments
- Typically have a component for going above and beyond the spec
- Marking is inherently subjective

§ Assignments require significant time coding

- Do not underestimate the time it takes to code interactive applications that
are intuitive and easy-to-use. Start early!

21 CS 349 - Syllabus
Assignments

§ A0: Java and Subversion Check (Bonus!)

§ A1: Breakout (2D graphics)
§ A2: Photo Browser (MVC)
§ A3: TBD (Custom widgets)
§ A4: TBD (Android)

22 CS 349 - Syllabus
Assignment Policies
§ Due dates: 5:00pm on the date in schedule
- Late assignments are NOT accepted!
§ Submission is via Subversion (SVN) source code repository
- Enrolled students will receive an email with username/password

§ Assignments are your individual work

§ Feel free to use:
- Examples provided in class and on the course website
- Examples/tutorials linked from Resources Page

§ You should NOT be doing general Internet searches for more specific
solutions.

23 CS 349 - Syllabus
Grading
§ Assignments: 40%
- 10%, 10%, 10%, 10%
- A0 is a 1% bonus if you complete it on time
§ Midterm: 20%
- Friday Feb 12 @ 6:30 PM – 8:00 PM (location TBD)
§ Final: 40%
- Scheduled by the Registrar’s Office

§ To pass the course, you must pass the weighted exam average and the
weighted assignment average
- In other words, if you fail the assignments, you fail the course, regardless
of how well you do on the exams.

24 CS 349 - Syllabus
 NOTE: Midterm exam is
on Friday, February 12,
6:30 – 8:00pm!!!

CS 349 - Syllabus
25
26
Next Steps
§ Explore the web site
§ Get signed up for Piazza
§ Start A0
- Install Java 8 JDK
- Install SVN client
- Get the world’s simplest Java program running
- Check it into SVN (once you receive your account details!)

27 CS 349 - Syllabus
Questions?

28 CS 349 - Syllabus
Definition: User Interface
§ A user interface is the place where a person expresses intention to an
artifact, and the artifact presents feedback to the person.

express intention

presents feedback

person artifact

29 CS 349 - Basic Concepts

Interactive System Architecture

30 CS 349 - Basic Concepts

Interactive System Architecture

User Interactive System

mental
model system
model

present feedback

express intention

31 CS 349 - Basic Concepts

Interactive System Architecture

User Interactive System

mental
model system
model

perceive present

express translate

32 CS 349 - Basic Concepts

Model-View-Controller (MVC)

present
View
perceive notify

mental system
model Model model

express
change
Controller
translate

33 CS 349 - Basic Concepts

 Graphical Temperature Control

30

20 draw widget

10 present
View value changed
perceive notify

temperature max = 30°

is colder than temp = 18°
Model
usual min = 10°

express
change
mouse movement
Controller change temp to 20°
translate
mouse
Event: "An observable occurrence,
events Event: "A message to notify
phenomenon, or an extraordinary
occurrence" an application that something
happened"
34 CS 349 - Basic Concepts
 Speech Temperature Control

text to
speech
“Temperature is present
18 degrees.” View
perceive notify
value changed

temp max = 30°

temp = 20°
is 18° Model
min = 10°

express
change
“Set temperature to
20° degrees.” Controller change temp to 20°
translate
speech
recognition

35 CS 349 - Basic Concepts

Interface vs. Interaction
§ What is the difference between an interface and interaction?
§ Interface refers to the external presentation to the user
- Controls (what you can manipulate to communicate intent)
- Visual, physical, auditory presentation (what the program uses to
communicate its response)
§ Interaction is used to connote behavior: The actions the user must
invoke to perform a task and the corresponding responses
- Interaction is action and dialog
- Unfolds over time

36 CS 349 - Basic Concepts

Interface and Interaction Design
§ What is the interface?
§ What is the interaction?

§ Why is interaction design so hard?

37 CS 349 - Basic Concepts

Interaction Design
§ Challenging because of variability in users and tasks
- Varying levels of expertise among users
- Often a range of tasks will performed with the same tool (i.e. tools can be
generic) – can you anticipate all uses and scenarios?
§ No one “right way” to design an interface; interfaces can always be
improved

38 CS 349 - Basic Concepts

Why Study Interaction? Empowering People
§ Well designed interfaces empower people
to do things they couldn’t otherwise do
- Desktop publishing, grassroots journalism
(blogs), movie production, music
production, image editing, assistive
technologies…
§ A well designed tool can literally change
the world
- The web browser, Linux, original Napster,
the spreadsheet, email, instant messaging…

39 CS 349 - Basic Concepts

GUI Basics
A case study of a graphical user-interface architecture.
Evolution of GUI Programming
§ On early computers, everything was basically rolled by hand
- Re-inventing the wheel with every iteration
§ Interaction was very, very limited.
§ No graphical input or output
- Ivan Sutherland Sketchpad system

§ Leading up to the 1980’s, began to develop GUI architectures and

systems to simplify programming
§ One early and highly influential GUI architecture was the XWindows
System, built to run on top of various Unix’s.

41 CS 349 - Syllabus
 2015
e s 2010 iPad
rfac
e
Int 2005
r
U se iPhone
c al 2000
phi iPod
a
Gr 1995
1990 Linux 1.0
c es
fa
ter 1985
l In
na Macintosh
it o 1980
AT&T UNIX System V
s a
er BSD 4.2 + X Windows
onv 1975
C Apple II
1970 Intel 4004 Microprocessor
Early Unix (PDP)
s
ac
e 1965
rf
te
In 1960
tch
B a
1955
1950

1945 42 CS 349 - Syllabus

ENIAC
X (X11) Window System
§ A base window system first developed in 1984
§ A standard for low-level graphical output and user input
- A protocol to create windows, handle input, draw graphics, …
- Not a window manager (more on that later)
- Does not specify style of user interface

§ Free and cross-platform

(os-, processor-, form
factor-agnostic)
§ Separate from operating
system

43 CS 349 - X Windows Basics

X Windows Design Criteria (~1986)
1. implementable on a variety of displays
2. applications must be device independent
3. must be network transparent
4. support multiple, concurrent application displays
5. support many different applications
6. support output to overlapping windows
(… even when partially obscured)
7. support a hierarchy of resizable windows
(… an application can use many windows at once)
8. high-performance, high-quality text, 2-D graphics, imaging
9. system should be extensible

(from Scheifler & Gettys, 1986)

44 CS 349 - X Windows Basics
Displays, Screens, Windows
§ In X, a display may have multiple screens
§ A display may have multiple windows
§ A window may
cross multiple
screens

45 CS 349 - X Windows Basics

X Client-Server Architecture
§ Separate user interface and application:
- the X Client handles all application logic
- the X Server handles all display output and user input
§ A server handles requests from multiple clients, processes data as
requested, and returns the results to the clients
§ X inverts conventional www
server and client relationship
(in www, web browser is the “client”,
web site is the “server”)

46 CS 349 - X Windows Basics

Why Client-Server?
§ Goal was flexibility and
economy
- Many clients (perhaps on
multiple machines), one
display

47 CS 349 - X Windows Basics

X Windows as MVC Architecture
View

present X Server
Output notify
perceive

huh…?
X Client Model

express
X Server change
translate Input

Controller

48 CS 349 - X Windows Basics

Structure of a Typical X Program
1. perform client initialization
2. connect to the X server
3. perform X related initialization
4. event loop:
get next event from the X server
handle the event:
if the event was a quit message, exit the loop
do any client-initiated work
send drawing requests to the X server
5. close down the connection to the X server
6. perform client cleanup

49 CS 349 - X Windows Basics

Xlib
§ library to wrap low level X Window protocol
- to avoid implementing message passing for every new program

§ uses buffered input and output queues

- need to flush them: XSync, XFlush

§ Xlib functions:
- connection operations: e.g. XOpenDisplay, XCloseDisplay, …
- connection operation requests: e.g. XCreateWindow, XCreateGC,…
- connection information requests: e.g. XGetWindowProperty, …
- local event queue operations: e.g. XNextEvent, XPeekEvent, …
- local data operations: e.g. XLookupKeysym, XParseGeometry, XSetRegion,
XCreateImage, XSaveContext, …
§ Xlib data types:
- e.g. Display, Window, GC, XSizeHints, XWhitePixel, XBlackPixel, etc.

§ Xlib is not a window manager

§ Xlib does not specify style of user interface or provide “widgets”

50 CS 349 - X Windows Basics

Display a Window (openwindow.min.cpp)
Display* display;
Window window; // save the window id

int main( int argc, char* argv[] ) {

display = XOpenDisplay(""); // open display
if (!display) exit (-1); // couldn't open, so bail
int screen = XDefaultScreen(display);// info about the display
window = XCreateSimpleWindow(display,
XDefaultRootWindow(display), // window's parent
10, 10, 400, 300, 2, // location: x, y, width, height
XBlackPixel(display, screen), // foreground colour
XWhitePixel(display, screen)); // background colour
XMapRaised(display, window); // put window on screen
XFlush(display); // flush the output buffer
std::cout << "ENTER2exit"; std::cin.get(); // wait for input
XCloseDisplay(display);
}

51 CS 349 - X Windows Basics

Opening a Window in Java

52 CS 349 - Syllabus
Evolution of GUI Programming
§ Much of the XWindows architecture was influenced by its time period
- Larger server machines and low-cost client displays because computation was
expensive
§ As computation got cheaper, certain aspects of program behavior could be
assumed:
- Software runs on the client computer
- OS handles display rendering and has device drivers to coordinate with specific
hardware.
§ Over time, in a multi-OS world, even this restriction became a burden
- Services could be abstracted even farther from the underlying architectures,
and programs could run on virtual machines (e.g. JVM, .NET VM, etc.)
- Write once for a generic ‘virtual’ architecture, run the program on the virtual
machine.
- Any platform for which the ‘VM’ is implemented could execute the same
program.

53 CS 349 - Syllabus
Next Day: Windowing Systems and Event Loops

54 CS 349 - Syllabus