Professional Documents
Culture Documents
Ofrssizlasruokuattwuds - Atq1Ti Two: Uh Uh
Ofrssizlasruokuattwuds - Atq1Ti Two: Uh Uh
So uh the big discussion came in January uh in general uh thered been some talk
guess
back and forth about things like this over number of months and all of sudden word
came down that was thinking of doing this and they wanted to
management really seriously
know is there any reason why we couldnt do it Uh and they also had suggestion that
they wanted to release it in the first quarter which is by 3-31 uh hence the name of this
Uh it
was uh-uh the big question was what-wotthbtop-us what
ongoing project
was the reason why we couldnt do this And they gave us think almost somewhere
between three days and week to try and decide if there was something that we were not
about and uh what they could do you know how we could do it how we
thinking
about getting involved and finally said definitely were going to do this mean
against us and uh this-the idea is to try and roll the dice little bit andmake this
guess
more interesting for everyone Uh theres some old quotes about uh terrain being the
most critical in combat situation and think were smaller company able to move faster
and the hope is that we can bring this to bear so that Microsoft cant compete as well Fact
Ive really seen positive things come out of this where uh weve been rated as better
browser becaus now were cost competitive But anyway so it-it came out in late January
uh there Iliad meeting with Bob Lisbon the Senior VP of all of client division
wer
and theyve gtT20 other people saying Are there any show stoppers Are there
any
show stoppers No one could really think of uh a-a dead problem and then uh guess
Mark Mark Andreeson went public with this that we were definitely going to do it
The decision was made mid to late January so this gave us little over two months and the
tmth is if
you back off the schedule and you say well if
you want to be done on 3-31 you
really
need to be aim for 3-15 so that youre dead youre dead done and then you have
months is an incredibly short piojt cycle In some sense its not product In another
sense it
really is Were gonna show the code to people Uh and so there-theres the first
that people were nervous about is all sorts of profanity that might sit in the code Anything Kiu
uh-uh people didnt like certain variant of UNIX and uh there sort of abusive terms
relating to the operating system and-and truth be known lot of us consider this non
professional mean uh theres there are examples where companies have had things
appear accidentally on their disk profanity which is even outside of the code and gotten all
77 r--
sorts of flaks-flak from different groups you know its not part of thcir able code so
were very concerned about that There was concern that were going to be infringing on-
business to know whats patented Patent things things are moving along quickly
very
uh theres certain chance that were using something and dont even realize it People
from the outside cant-cant see what theyre doing what were doing but once you look
at the code you go oh theyre using this algo-rhythm They are using this-this
technique But for engineer it would very difficult to see what source code it would
very straight forward to see it not that we stole it but simply independent
ub we started looking at that The big thing that hit us that we didnt really forsee in that
first week is the complete consideration of third pirty cride It turns out the numbers in
are
that we consider communicator And now the realization was gee we cant this
give away
third party code without their agreement So now there are-there are several possibilities..
one is to get them to sign up to the actual uh they can call it NPL Netscape public lisense
and try and convince them to for it Uh but you know in truth of these people_are
go alot
making money by selling this code mean there are-there are versions of code where you
Page2
-1
Netscape Documentary
Tape 10 Jim Roskind Interview
code and ship the binary Thats common scenario Uh as an example Microsoft has
routine of sorts uh Quick Sort And be it there are no unstandard algorhythms but
theyll actually give you copy of their source But the intent is not that you will
redistribute that source and make your own compiler using their system but rather youll
.JesV A-
different One of here has been
players my jobs historically the job the job-of security
architect obia littl bit the liason through J-Java for source 3-31 one as we call it Um
and so had to consider the issue of Java and approach Java Soft Am running on too
long
So uh so laison to Java had to raise the elements of what can we do about Java Java
represents large amount of source which is bundled into the communicator and is clearly
restricted in source form by Java Soft The question is can we ship it What-Whats the
actually distributes their source under fairly uh easy to get lisense The one restriction is
commercial use you have to you pay you know dollars and you-you get certain rights to
use it But noncommercial use people get to see it So the approached that another large
distribute that
patch is list of changes from the freely distributable source Now the
idea is linex gives this and you get from Java Soft the
you actual
underlying source
Then there are tools that just merge them together and now you have really home-brewed
Soft and get it Well just shif-ship patch But everyone and their brother knowsthe
Page
Netscape Documentary
Tape 10 Jim Roskind Interview
virtual machineJava Softs source code that it was hard to use tool to do this So the
approach that took is that we went to Java Soft and we said hey could we ship your
source with the changeweve made under your highly restrictive lisense Now it doesnt
virtual machine They can plug maybe Cava theres whole bunch of different
Obi
variations Also other vendors can think about integrating it All VM vendors that are
concerned about doing integration already have lisenses can say oh this is how we plug
it in and theyll have much easier time integrating So it would nice for everybody
although it wouldnt be the perfect scenario it would be pretty good Uh the problem is
listen youve got to agree to this weve got to get it in writing if were gonna move
and thats exactly what happened We passed 2-24 February 24th that was the official
deadline and at that point uh we can no longer wait for agreement from Java Softwace
We really were gonna be put in position where we couldnt out of- get out of this
get
trouble uh so at that goint we began cutting So weve been cutting now roughly 2-24 2-
you want to gracefully remove it You want move Java but still leave navigator
That sounds
functioning like saying okay we decided we dont want to side window
Uh and maybe you have to now remove it but you dont air out so you
really blowing
you cant look out the window You cant do everything you could do with the window
c4
but it still functions like so its an interesting thing removing pieces and trying to leave
functional underneath
body SIn some cases weve had harder troubles with this uh theres
database that we were using its part of mail system and in that
case not only couldnt
3S
/0
Page
Netscape Documentary
Tape 10 Jim Roskind Interview
we ship it and couldnt really get all the stuff straight but this was fairly pivotal to all of
our mail side of this product and so the bottom line is we ended up think the current plan
is still lithe bit in the air is fear that this will be viewed
up is
...eres
little as
you do
the
misrepresentation to net but my answer is the best
you can you tell the net
what youre doing you try to be honest an open and thats think what were going for
right now Youll see that number of different pieces theyll be funny sections uh
security is another example of major piece of code Now this is probably the most
interesting one because the third party we unintelligble all the code but theres this
ery
restriction by the Federal Government on how we can distribute this code its getThe
effectively is the third party and ship it Um and this is even stickier because now were
question we put to the government is how much do we have to cut and their answer is
pretty simple they said you have to make it look like there never was any security in the
system you have to make sure theres no sign of crypto or photography encription if
you
hidden its tied deeply into the system its almost like with car saying you can ship it
dont need lighting the cars only driven by day thats right but the wires you cant
leave the wires in because it shows how you would wire the car and so theres
very
large effort think this was Jeff Wienstein and hes put in quite an effort to try and
more comformtable communicating across this but this uses cryptography uh and the
Pages
iY/
Netscape Documentary
Tape 10 Jim Roskind Interview
bottom line is we couldnt leave that in at all People started to wonder well gee heard
weak
you can export cryptography but not strong cryptography why wont they allow you
to export source for weak cryptography the answer is typically the difference between
ti
weak and strong is whether 40 or 128 oneoint which visible So
you type at is
very
theres no real
way to narrow it down This is an example of the same idea of cutting
things out but the goal is to make it useable so now comes question what happens
when you go to site that to use SSL how does the and
expects you system respond is it
exceptable And so the cutting has to be very delicate Im sure you can imagine if you
had no electricity all of sudden youdf have to think know how do we fire the
you
chambers well OK its diesel engine thats right we use diesel engine no electronic
ignition no headlights uh we cant have warning lights on it so all sorts of things have to
going away uh in the end some people will come and say hey why did you say you were
shipping this and were hoping that the Net will be understanding of some of this uh we
cant
it
its
interesting
unclear
crypto
where
laws we cant
the
crypto
even
goes
leave holes
in There are
for plugging
laws that
in
youre
crypto
not
ii
allowed to whats of plug and play crypto oh
you cant make
ship called sort even it so
has to be completely devoid of connection so thats its very difficult this is all theres
case you have too months to go you cant go about any slow
this in
way you have to be
Page
Netscape Documentary
10 Jim Roskind Interview
Tape
and
There is some code creation in the analogy said you remove window you have to
and down you have to tie that in you have to in and make delicate adjustment the bad
go
party uh software you have to replace that crank with uh uh an electric power
window just to get by no more cranking up and down just locking so theyre chaning
for the
the lock youre changing the window and when you go to connect you each look
wow its gone the guy putting in the window looks for the crank and goes theres nothing
to connect with this issue of when you have many people working on the project at the
same time and really rushing you cant just say OK you replaced the window OK
youre done and now that youre done OK you replace the crank People are trying to
this large code base think its in the of million lines of code and
modify neighborhood
theyre running around we have Im not sure what the count of engineers is certainly Idl
around with different tasks and then they have whats called merging problem basically
with youre of the car which is the code the code is really
youve go off snapshot design
you know design and you come back with the changes that youve made with your
notice right where it checks in that the picture has been changed and now you have to
merge you have to carefully look at their changes and your changes find way to work
them together Now when youre working on veryjiow schedule or if its just one
person everything is nice and sequential everyone knows whats going on Uh when
ii
Page
iC
Netscape Documentary
of security
and were Well java it turns out makes alot
andUbert Shaw tearing out java
in and cut out all the security so they actually went into
security group is assigned to go
lets just adjust this of code so that it doesnt make the call
to call so ptI piece
anymore
technical
that its about to break if it
gets put in Now if you break the build thats the
thats
you now have to crank the car the
the electricity
you took away
all
the answer is if
new way and the oh didnt even realize yeu had to crank the car and jill
guy says
the problem
the goal is to stop changing anything uh the hard part in debugging or finding
sure theres only been couple of changes and we know to look at exactly those changes
what broke the So typical1 once the system breaks we just stop dead
really
is system
and we to then do little and look back so we can see whats chang and
try autopsy
iS
Page
Netscape Documentary
view
have people spinning their wheels saying well want to check in my new side
this stuff checked in But now what youve done is youve instigated inefficiency
get
have
the system Again two months its desperate schedule you cant any ineffeciency7
and go check
and finally say ah now Im done and then you turn around face the system
call whats called the car pool and they say listen youre in merge hell we can get you out
and then release the tree will be the phrase So then youll see
drawing change in re-open
its to land
the car car pool really easy
changing theyll allow pool
call
merge they
small change if all you want to do is crss off the DOT sticker you know on the window
in and do that and be done And then you go to another window and do
you can go
Page
Netscape Documentary
Tape 10 Jim Roskind Interview
another DOT If you have to remove all the glass in the car at once and it turns out that
the glass is related to other things electricity thats an easier example you cant jsut say
well Ill remove the battery so then the car stops working You have both remove the
battery remove the starter insert crank you have to remove the ignition switch you
have to insert all these things have to be done at once you cant get away with doing one
without the other And so thats why you have this large change Its always nicer some
electricity has to go Ill take out the lights first OK thats out Now can remove this
other But at some point you get down to the starter motor and the ignition system
piece
CoCV r4pC
and it all has to be changed out at once you have to switch
fro ignition system to
diesel and you cant do meal and theres lot of other people doing skip checks
it
piece
Yesterday
Yesterday was Monday the start of the week and people were getting really nervous
cause now its 3-10 and theyre hoping to be done really by 3-15 basically by Friday
and there are two major car pools that hadnt landed One is the security car pool and the
second is the java car pool Now there was sort of race between the two of us to see
who can get out there fat faster or be ready faster there was maybe almost there was
another thing we talk about here called schedule chicken uh schedule chicken is where uh
one group realizes maybe my group could realize we couldnt be done by Thursday but
we tell them we could be done by Wednesday knowing that the other group wont be done
by Friday and all the attention will be focused on them and then we can be done by
Thursday and no one ever knew that we were really ever going to be late So its game of
whos CCn In
chicken the question is going to tac cnt7
Page 10
Netscape Documentary
Tape 10 Jim Roskind Interview
check uh and was going to in the afternoon the tree was being closed so that
it it
happen
call now the back burner They told us at the time at the meeting
certainly a14 p001/ put on
worse case could be done by Wednesday which is thought because again Friday
it
scary
cGr
is the goal we have major eØl pool coming in were all so we dont want to be
scaredk
on the hot seat we dont want to be the one to cause the prob the program to slip uh so
car
yesterday there was alot of discussion about the landing the security call pool and what
what is...exactly is uh the uh the Java lIpool gonna do about this the first thing we
mentioned we were then now were talking about recovery plans mentioned we took
snapshots of where things were before the security Si pool started landing in their change
caused too much bustage again another clever technical term uh the-the chanebes we
could always roll them back out and let us go first and let them think about what they were
e4 Iccrd
going to do little bit more clearly Uh Ynfle mentionedfl Jeff Weinstein or his
people coming from his group his goal he was just going to stay all night and be sure the
call pool cleared everytime there was an error he was going to turn it back around The
tree wasnt going to be moving so the ground wasnt going to be shaking under his feet
sure if it was or 11 oclock at night he actually was completely done So then we got
call was here Rahman and Yube and uh Hey their call pool cleared how are you
ri
scary going in late at night when whole bunch of changes have landed because you dont
have the full support of the other engineering staff if there was problem outside your urea
e4 expertise its
pretty problematic Youll see today in meetings theyll be talking about
involved Last night we did merge the tip which is trying to adjust our design to be as
LC
11
Page
Netscape Documentary
Tape 10 Jim Roskind Interview
close as possible to the most recent most recent set of code and uh merged i3
design it
up .j
links you know major function you bring up the whoeage you see an underlined word
sort of stares you in the face and go holy cow somethings brokenl and now came
question.. did someone else break the system before we entered this new sytem status or
did we just break it when our merges conflicted with items that were sitting there So we
were trying probably til about three in the morning to figure out what the heck is going
on finally decided the answer is take break go home get some sleep come back in
today Tuesday uh have some more people with more expertise in Unix to look at the exact
Unix problem were facing and uh see if we can diagnose it So were little bit on the
hot seat We thought we were good to go were about ready to check in were doing
final final final checks making sure everything merges again We cant afford to land
unless we can assure safe landing Uh and now we suddenly had a-a problem Were
going around were circling the clock is ticking its Tuesday theyre going to wonder
are we going to make it Theyre on the outside theyre listening to me and they have to
wonder. is Jim really understanding whats going on are these guys misrepresenting
things work and then often you find another bug But each time you find one everything
up to there ran perfectly so you have this funny feeling that everythings working
wonderfully without there being problem in the design uh and so its pretty nerve-
for
wracking management Management commonly has this problem with software
that
engineering its-theres alot of stuff going on its
easy to misunderstand details uh
you know its-the-anothe/guery is that ninety percent of the work is finished in the last ten
Page 12
Netscape Documentary
hot
they dont
want to be done want to be in the
and uh it
gets very confusing People
confidence
Michael Uh hes driving this uh ovei time theres rapport and theres
what So
me clock and you have to see happen
11
But at the same time the is ticking
ii cr
slowed down Soa-eah
more and more people standing up and shouting theyre being
and constrantly have to trade off if you slow the group too much and you finally have
you
constant question Michael is wondering. are they being honest with me Can they
done in just few hours Or should let the people who are stacked up behind
get this
them go And he has to mak at judgement call And uh guess thats why they pay
Page 13
Netscape Documentary
Tape 10 Jim Roskind Interview
going why is this breaking sheet whats going on here Last night you know we are
nights and Thursday nights have been trying to go out and play raquet ball -.hfec.ea1lLL
And this is now like unless we get this check in theres no raquet ball tonight and
looking
comic that
callid pointy-head people its take off on certain strips that represent managers
every two seconds and say Hey are you done yet eally doesnt help them It
the
adds to the tension Uh the truth is my job is to try to shield the pressure so go to
meetings and Im going to sit through and take the flack about us not being in and us not
In
be nice guy as come around asking without interrupting being curteous to system
fact you know my job bring cookies bring water Uh if they have problems and they
want me Ill sit down and look at code with them as well uh It varies so its just
moving along wthei4-ache.4 and time you never know when you have the last bug
You know when the last on the main is about hope it all works And
never thing system
VS
good does time again-is immemorial here
the truth is if youre doing it
really
Thats it And it
really does work Okay that was the problem and were good to go but
you hit your head on another one and youre going Oh darn its not over you know
the marathon hasnt ended could see the-the light at the end of the tunnel but
thought
it..
Page 14
SUM4UUJ
1301 p%wwox wg