4/21/2017 ToSpreadScience,KeepItSecretLessWrong

MAIN DISCUSSION WIKI SEQUENCES ABOUT

CustomSearch
LessWrongisacommunityblogdevotedtorefiningtheartofhumanrationality.PleasevisitourAboutpageformore
information.
Register/Login

ToSpreadScience,KeepItSecret Password

Rememberme
37 Eliezer_Yudkowsky 28March200805:47AM Login Recoverpassword

Followupto:JoyinDiscovery,BindYourselftoReality,ScientificEvidence,Scarcity
SubscribetoRSSFeed
SometimesIwonderifthePythagoreanshadtherightidea.

Yes, I've written about how "science" is inherently public. I've written that "science" is distinguished from merely NEARESTMEETUPS
rationalknowledgebytheinprincipleabilitytoreproducescientificexperimentsforyourself,toknowwithoutrelyingon
Moscow:regularmeetups:01January
authority. I've said that "science" should be defined as the publicly accessible knowledge of humankind. I've even 202002:00PM
suggestedthatfuturegenerationswillregardallpapersnotpublishedinanopenaccessjournalasnonscience,i.e.,it
GamesinKochergaclub:
can'tbepartofthepublicknowledgeofhumankindifyoumakepeoplepaytoreadit.
Fallacymania,TowerofChaos,
ScientificDiscovery:26April2017
But that's only one vision of the future. In another vision, the knowledge we now call "science" is taken out of the
07:40PM
public domainthe books and journals hidden away, guarded by mystic cults of gurus wearing robes, requiring
fearsomeinitiationritualsforaccesssothatmorepeoplewillactuallystudyit.
VIRTUALSTUDYROOM
Imean,rightnow,peoplecanstudysciencebuttheydon't.
Coworkwithotherrationalistsonline.
"Scarcity",it'scalledinsocialpsychology.Whatappearstobeinlimitedsupply,ismorehighlyvalued.Andthiseffect LessWrongStudyHall

isespeciallystrongwithinformationwe'remuchmorelikelytotrytoobtaininformationthatwebelieveissecret,and
tovalueitmorewhenwedoobtainit. RECENTCOMMENTS

With science, I think, people assume that if the information is freely available, it must not be important. So instead Definitelyasidepoint,butthanks
byKennyonExcludingthe
fortheinfoanyways!
peoplejoincultsthathavethesensetokeeptheirGreatTruthssecret.TheGreatTruthmayactuallybegibberish,but
Supernatural|0points
it'smoresatisfyingthancoherentscience,becauseit'ssecret.
Thanks.:)
ScienceisthegreatPurloinedLetterofourtimes,leftoutintheopenandignored. byCrimeThinkeronProjectHufflepuff:
PlantingtheFlag|0points
Sure,scientificopennesshelpsthescientificelite.They'vealreadybeenthroughtheinitiationrituals.Butfortherest
"AsimovonChemistry"wasa
of the planet, science is kept secret a hundred times more effectively by making it freely available, than if its books byjohnlawrenceaspdenonFact
childhoodfavouriteofmine.
wereguardedinvaultsandyouhadtowalkoverhotcoalstogetaccess.(Thisbeingafearsometrialindeed,sincethe Posts:HowandWhy|1point
greatsecretsofinsulationareonlyavailabletoPhysicistInitiatesoftheThirdLevel.)
Responsetooldpost:[Appearsto
byJiroonThenoncentralfallacythe
beanurbanlegend.]
If scientific knowledge were hidden in ancient vaults (rather than hidden in inconvenient payforaccess journals), at
worstargumentintheworld?|1point
leastthenpeoplewouldtrytogetintothevaults.They'dbedesperatetolearnscience.Especiallywhentheysawthe
powerthatEighthLevelPhysicistscouldwield,andweretoldthattheyweren'tallowedtoknowtheexplanation. Masculinityisn'toffputting.
byfmgnonInstrumentalRationalityis
Andifyoutriedtostartacultaroundoh,say,Scientology,you'dgetsomedegreeofpublicinterest,atfirst.Butpeople aChimera|0points

would very quickly start asking uncomfortable questions like "Why haven't you given a public demonstration of your
Eighth Level powers, like the Physicists?" and "How come none of the Master Mathematicians seem to want to join RECENTPOSTS
your cult?" and "Why should I follow your Founder when he isn't an Eighth Level anything outside his own cult?" and
ProjectHufflepuff:Plantingthe
"WhyshouldIstudyyourcultfirst,whentheDentistsofDoomcandothingsthataresomuchmoreimpressive?"
Flag
byRaemon|35v(97c)
When you look at it from that perspective, the escape of math from the Pythagorean cult starts to look like a major
strategicblunderforhumanity. EuropeanCommunityWeekend
2017
Now, I know what you're going to say: "But science is surrounded by fearsome initiation rituals! Plus it's inherently byDreamFlasher|15v(8c)
difficult to learn! Why doesn't that count?" Because the public thinks that science is freely available, that's why. If "Flinchingawayfromtruthis
you'reallowedtolearn,itmustnotbeimportantenoughtolearn. oftenabout*protecting*the
epistemology
It'sanimageproblem,peopletakingtheircuesfromothers'attitudes.Justanyonecanwalkintothesupermarketand byAnnaSalamon|69v(51c)
buy a light bulb, and nobody looks at it with awe and reverence. The physics supposedly aren't secret (even though
FurtherdiscussionofCFARsfocus
you don't know), and there's a oneparagraph explanation in the newspaper that sounds vaguely authoritative and
onAIsafety,andthegoodthings
convincingessentially,noonetreatsthelightbulbasasacredmystery,soneitherdoyou. folkswantedfromcause
neutrality
Eventhesimplestlittlethings,completelyinertobjectslikecrucifixes,canbecomemagicalifeveryonelooks at them byAnnaSalamon|35v(40c)
likethey'remagic.Butsinceyou'retheoreticallyallowedtoknowwhythelightbulbworkswithoutclimbingthemountain
Besecretlywrong
tofindtheremoteMonasteryofElectricians,there'snoneedtoactuallybothertolearn.
byBenquo|30v(47c)

Now,becausesciencedoesinfacthaveinitiationritualsbothsocialandcognitive,scientistsarenotwhollydissatisfied CFARsnewfocus,andAISafety
withtheirscience.Theproblemisthat,inthepresentworld,veryfewpeoplebothertostudyscienceinthefirstplace. byAnnaSalamon|30v(87c)
Science cannot be the true Secret Knowledge, because just anyone is allowed to know iteven though, in fact, they FactPosts:HowandWhy
don't. bysarahconstantin|75v(32c)

IftheGreatSecretofNaturalSelection,passeddownfromDarwinWhoIsNotForgotten,wasonlyeverimpartedtoyou DoubleCruxAStrategyfor
ResolvingDisagreement
afteryoupaid$2000andwentthroughaceremonyinvolvingtorchesandrobesandmasksandsacrificinganox,then
byDuncan_Sabien|58v(101c)
when you were shown the fossils, and shown the optic cable going through the retina under a microscope, and finally

http://lesswrong.com/lw/p0/to_spread_science_keep_it_secret/ 1/9
4/21/2017 ToSpreadScience,KeepItSecretLessWrong
toldtheTruth,youwouldsay"That'sthemostbrilliantthingever!"andbesatisfied.Afterthat,ifsomeotherculttried OntheimportanceofLessWrong,
totellyouitwasactuallyabeardedmaninthesky6000yearsago,you'dlaughlikehell. oranothersingleconversational
locus
And you know, it might actually be more fun to do things that way. Especially if the initiation required you to put byAnnaSalamon|82v(357c)

together some of the evidence for yourselftogether, or with classmatesbefore you could tell your Science Sensei MIRI's2016Fundraiser
youwerereadytoadvancetothenextlevel.Itwouldn'tbeefficient,sure,butitwouldbefun. bySo8res|20v(13c)

Ifhumanityhadnevermadethemistakenevergonedownthereligiouspath,andneverlearnedtofearanythingthat
smacksofreligionthenmaybethePh.D.grantingceremonywouldinvolvelitaniesandchanting,because,hey,that's LATESTRATIONALITYQUOTE

whatpeoplelike.Whytakethefunoutofeverything? Witchesdontoftengetangry.All
byRubixonRationalityQuotesApril
thatshoutingbusinessneverreally
Maybewe'rejustdoingitwrong. June2017|3points

Andno,I'mnotseriouslyproposingthatwetrytoreversethelastfivehundredyearsofopennessandclassifyallthe
science secret. At least, not at the moment. Efficiency is important for now, especially in things like medical RECENTWIKIEDITS

research. I'm just explaining why it is that I won't tell anyone the Secret of how the ineffable difference between
bluenessandrednessarisesfrommereatomsforlessthan$100,000
RECENTONRATIONALITYBLOGS
Ahem!Imeanttosay,I'mtellingyouaboutthisvisionofanalternateEarth,sothatyougivescienceequaltreatment
withcults.Sothatyoudon'tundervaluescientifictruthwhenyoulearnit,justbecauseitdoesn'tseemtobeprotected
appropriately to its value. Imagine the robes and masks. Visualize yourself creeping into the vaults and stealing the TOPCONTRIBUTORS,30DAYS
LostKnowledgeofNewton.Anddon'tbefooledbyanyorganizationthatdoesuserobesandmasks,unlesstheyalso Raemon(490)
showyouthedata. Lumifer(205)
Viliam(123)
PeopleseemtohaveholesintheirmindsforEsotericKnowledge,DeepSecrets,theHiddenTruth.AndI'mnoteven ChristianKl(99)
criticizingthispsychology!Therearedeepsecretesoterichiddentruths,likequantummechanicsorBayesstructure. gjm(93)
Elo(79)
We've just gotten into the habit of presenting the Hidden Truth in a very unsatisfying way, wrapped up in false
Alexei(73)
mundanity.
gwern(70)
Benquo(61)
But if the holes for secret knowledge are not filled by true beliefs, they will be filled by false beliefs. There is nothing
lifelonglearner(50)
but science to learnthe emotional energy must either be invested in reality, or wasted in total nonsense, or bogus(47)
destroyed.Formyself,Ithinkitisbettertoinvesttheemotionalenergyfunshouldnotbeneedlesslycastaway. username2(44)
Dagon(40)
Right now, we've got the worst of both worlds. Science isn't really free, because the courses are expensive and the DustinWehr(36)
textbooksareexpensive.Butthepublicthinksthatanyoneisallowedtoknow,soitmustnotbeimportant. tristanm(32)

Ideally,youwouldwanttoarrangethingstheotherwayaround.
RECENTKARMAAWARDS

PartoftheJoyintheMerelyRealsubsequenceofReductionism

Nextpost:"InitiationCeremony"

Previouspost:"Scarcity"

ArticleNavigation

Comments(42) Tags: philosophy religion

Comments(42) SortBy:Old

Hopefully_Anonymous 28March200806:47:01AM 1point []

Great, interesting, thoughtprovoking post, Eliezer. As you suggest towards the end, a multiplex of strategies
regardingrelatingsciencetothepublicmaybebest.

I'mforcreatingstraussianbrandsofwidespreadand/orfastgrowingbeliefsystemstocompetewiththeirsincere
or exploitive versions. For example, rather than a cult of science made from scratch, I think a version of
evangelicalchristianitywherethetopexecutivesarefocusedonmakingthebehaviorofthemassesmoreinour
rationalselfinterestwouldbethebetterapproach(althoughitwouldbeinterestingtoseebothtried).

ReplyPermalink
Psychohistorian3 28March200806:55:56AM 2points []

Insightful, as always, but this seems like it may have the esoteric value of some knowledge the wrong way
around.Therearecertainquestions,like"Whatisthemeaningoflife?"thatsciencecannotanswerthewaypeople
want to hear (as, "that questions is incoherent and pointless" is rarely viewed as satisfactory, regardless of its
accuracy). It seems people choose religion because they are seeking answers to some such question (or,
because their parents chose it), and they end up swallowing the earth being 6000 years old almost as an
afterthought.

ReplyPermalink
Grant 28March200807:04:53AM 0points []

Ilovemostofyourposts,butIthinkyoumightbeoffonthisone.

Whysuccessfulculturesrewardpeoplewhotriedtodiscoverandunderstandknowledgethatwasalreadyknown?
Idon'tthinktheywouldIthinkthey'drewardpeoplewhogoaftersecrets,mysteries,etc.Peoplewhomakeuseof
knowledge(preexistingorotherwise)fortheirowngainareofcourserewardedanyway,butforthemostpartIdon't
thinkthatdescribesscienceverywell,doesit?

http://lesswrong.com/lw/p0/to_spread_science_keep_it_secret/ 2/9
4/21/2017 ToSpreadScience,KeepItSecretLessWrong
What would be the purpose of rewarding people (who cannot make use of the knowledge) to 'discover' things
already in the public domain? Wouldn't we rather have them striving to solve puzzles? I know its enjoyable for
many of us to 'discover' science on our own, but for your average person I think thats a waste of time. In other
words,Ithinkscarcityinknowledgeisbeneficialforsociety,thoughmaybenotforsciencenuts.

Can'tyouusuallyauditcoursesinmostuniversitiesforfree?

ReplyPermalink
Lowly_Undergrad 28March200807:37:37AM 1point []

I think I, along with Grant, may have to disagree with portions of this entry. It seems your basic point is that if
science was set up in the way the scientology and other cults are, it would obtain an increase in interest and
pursuit. The problem I see with that is that cult knowledge is easy science is hard. Learning a couple of simple
beliefs at each level is simple. However, much like many people who would love to be a CEO of a major
corporation, but abstain from attempting to achieve this goal due to the increased stress and demands of the
position,the"cultofscience"wouldfaceasimilarissue.Learningadvancedmathematicsisnotforthetimid.

ReplyPermalink
PsyKosh 28March200808:12:12AM 5points []

Well, _ideally_ one would want us to be smart enough to not automatically overvalue hidden information or
undervalueavailibleinformation.:)

Also,I'mremindedofabitfromtheDiscworldbookThiefofTime.KaossaystoSweepersomethingtotheeffect
of"...Youknowallthehiddenwisdoms.No,morethanthat.Isuspectyouareevenwiser.Youknowtheexplicit
wisdoms,theknowledgehiddeninplainsightandignoredbymostastheyseekhiddenknowledge."

ReplyPermalink
Anonymous27 commentscorebelowthreshold [+](0children)

Eliezer_Yudkowsky 28March200808:35:31AM 14points []

Anonymous,

TheSingularityInstitutehasahumanresourcesproblemandaPRproblem.I'vebeentakinghaphazardpotshots
atthisfor8yearsandithasn'tworked.SoI'mtakingthetimetowriteoutfullexplanations,andthebasics,and
willeventuallyproduceabook,andteenagerswillreadthebook,goontoreadotherbooks,andthenbehiredby
SIAI7yearslater.That,I'mafraid,ishowithasalwaysworkedaroundhere.

Also, if you're an actual SIAI donor, please identify yourself and cite amount donated via private email, if you
prefer.

ReplyPermalink
Sam_B 28March200810:00:20AM 8points []

This post assumes a very positive view of humanity. It assumes that people aren't studying science in large
enough numbers because the knowledge isn't exciting and attractive enough. The alternative assumption is that
peoplearen'tstudyingsciencebecausethey'rethick,orlazy,orboth.

InBritain,fewerandfeweryoungpeoplearechoosingtostudyscienceatAlevel(1618)anduniversity,despite
theincreasednumberofthemcontinuingeducationaftercompulsoryeducationends(16).Thisismainlyblamed
on them choosing to do new, 'soft' options, of which Media Studies is the primary scapegoat (but a Google for
'MickeyMousedegrees'oraflickthroughtheprospectusofalowerorderuniversitywillfindplentyofothers).This
can't be attributed to the nonsecret nature of science, because Media Studies is just as nonsecret. If the open
nature of science was really the problem, it would be a problem shared by every single subject from science to
mediastudiestoplumbingapprenticeships,andenrolmentwouldbefallingineverysubject,notjustscience.

Idon'tactuallydisagreewiththemainpointofthepostthatsecretknowledgeismoreattractivebutitwouldn't
solve the problem of lack of interest in science. If science went cultish, they would see a shortterm increase in
interest, but then media studies academics would hide their 'knowledge' as well, and we'd be back where we
started.EveniftheknowledgewassecretI'venodoubtthatmostyoungpeoplewouldconsiderthesecretmedia
studiesknowledgemoreattractivethanthesecretscienceknowledge."Thechantingisn'tasweird,andtherobes
arebetter,andevenunderthosehoodsyoucantellthePhysicscultisatotalsausagefest."

ReplyPermalink
jschulter 11September201012:44:22AM 4points []

Ithinkyoumighthavefailedtorealizewhatwilldeterminewhichcultpeoplewillchoose.WhentheMediacult
makestheirpresentation,they'llbereducedtoshowingamovie(orequivalent,maybealofivirtualreality)and
saying"lookatthisfancymediawecancreate,wouldn'tyouliketobeabletodothat?"ButthenthePhysics
andMathematicscult(Ireallydofailtoseehowtheycouldbeseparatedsuccessfully)presentsalightbulb,a
tesla coil, and possibly a miniature sun and gets to say "this isn't even the half of what we could do if we
wantedto.Ifyouwanttoknowhowtodoit,you'llhavetodealwithus."

Parent
ReplyPermalink

Will_Pearson 28March200811:54:54AM 7points []

"Rightnow,we'vegottheworstofbothworlds.Scienceisn'treallyfree,becausethecoursesareexpensiveand
thetextbooksareexpensive.Butthepublicthinksthatanyoneisallowedtoknow,soitmustnotbeimportant."

http://lesswrong.com/lw/p0/to_spread_science_keep_it_secret/ 3/9
4/21/2017 ToSpreadScience,KeepItSecretLessWrong
Anyoneisallowedtopickupandreadabible.Theyareevengivenawayfree!Thepublicstillseemstoratethe
teachingsinthatthough.

IfIwastryingtospreadscience,Iwouldn'tmakeitscare,Iwouldmakeitsocial.Howaboutaroleplayinggame,
Scientist: The Discovery! Scenarios are scientific problems with real world data and the players level up by
solvingthem.Orperhapsfantastical,suchasdeflectingasteroids,butusingrealequations.

Also cults are not the thing you have to get science to the level of, today it is celebrity worship/sports
following/worldofwarcraft.

Thewholechantingthingputmeoffreligion,I'dmuchpreferaritualdance.Andnobloodysacrifices.

I'm curious what exactly your HR problems are with SIAI, it doesn't seem to have any jobs or research posts
open.

ReplyPermalink
quadibloc 06May201005:41:55PM 6points []

ComputersandCyclotronsistheroleplayinggame.Scientist:theDiscoveryisthecollectablecardgame.

Parent
ReplyPermalink
jschulter 11September201012:49:31AM 0points []

Thesacrificeputmeoffabittoo.maybeabarbecueinstead?Theoxisstilldeadandexposedtofire,butwe
don'twasttheutilityoftastytastyanimalflesh.

And it definitely would be nice if all prominent scientists/mathematicians got the same responses we see for
celebrities,insteadofjusttheselectfewwhobecomehouseholdnames.

Parent
ReplyPermalink

Caledonian2 28March200812:13:57PM 9points []

Mostpeopledon'tknowhowlightbulbsworkbecauseitdoesn'tmatterhowlightbulbswork. They can't use that

knowledge in their daily lives, so it really doesn't matter how difficult it is to acquire that data as long as it's
difficultenough,it'snotworthwhiletoexpendtheeffort.

Beingaphysicistdoesn'tgiveyouanyniftypowers,andthephenomenatheyhavetheknowledgetopredictdon't
affecttheeverydaylivesofhumanbeingsonaperceptiblelevel.

Mostpeopleweren'tclamoringtolearnthePythagoreansecrets,either.

ReplyPermalink
Aziz_Poonawalla commentscorebelowthreshold [+](0children)

spindizzy2 28March200801:22:19PM 5points []

Manymorepeoplearestudyingsciencethancanactuallyhopetofindjobsinthefield.

Therealproblemisnotascarcityofpeople,butascarcityof_smart_people.Theaverageguyinthestreetwill
notimprovehisownlifeoranyoneelse'sbythestudyofscience.Postsforlabtechniciansareeasyenoughtofill,
afterall.

Conversely,thepeoplewhoreallycanmakeadifferencebyandlargedonotneedanyencouragement.

On a practical note, I would be very interested in a discussion of the best ways an individual can make a
monetary / political / social contribution to the development of an AGI. Assuming this has already been argued
out,doesanyonehavealink?

ReplyPermalink
Silas 28March200801:44:31PM 2points []

Psychohistorian:Insightful,asalways,butthisseemslikeitmayhavetheesotericvalueofsomeknowledgethe
wrongwayaround.Therearecertainquestions,like"Whatisthemeaningoflife?"thatsciencecannotanswerthe
way people want to hear (as, "that questions is incoherent and pointless" is rarely viewed as satisfactory,
regardlessofitsaccuracy).

I don't think that's the answer science gives, at least, not the complete answer. This would be an excellent
example of "wrong questions" that Eliezer Yudkowsky discussed before (note: he linked this despite it not
containing the solution to qualia suggested in the anchor text) and an excellent opportunity to right the wrong
question.

"Whatisthemeaningoflife?"rephraseasyourconfusion>"WhydoIwanttoknowthemeaningoflife?"taboo
"meaningof">"WhydoIwanttoknowwhatsignficancelifehasbeyondwhatIcanobserve?"andsoon.

Dismissingthefirstquestionshouldnotbetheendofit.

ReplyPermalink
Tom_McCabe2 28March200803:35:59PM 0points []

"So I'm taking the time to write out full explanations, and the basics, and will eventually produce a book, and
teenagerswillreadthebook,goontoreadotherbooks,andthenbehiredbySIAI7yearslater."

http://lesswrong.com/lw/p0/to_spread_science_keep_it_secret/ 4/9
4/21/2017 ToSpreadScience,KeepItSecretLessWrong
Iwouldcertainlyenjoyreadingyourbook,andI'mconfidentthatthousandsofotherpeoplewouldenjoyitaswell.
However,I'mnotconfidentthatabookonrationalityalonewilldomuchforrecruitingHofstadter'sGEBwaswell
writtenandverywidelyread,anditdidn'thelprecruitpeopleforanyspecificorganization.

"I'm just explaining why it is that I won't tell anyone the Secret of how the ineffable difference between blueness
andrednessarisesfrommereatomsforlessthan$100,000"

I'll give you $100 for a reasonably detailed explanation of this, as long as you publish it under the GFDL or a
similarlicense.

ReplyPermalink
Silas 28March200803:58:11PM 0points []

"I'mjustexplainingwhyitisthatIwon'ttellanyonetheSecretofhowtheineffabledifferencebetweenblueness
andrednessarisesfrommereatomsforlessthan$100,000"

I'll give you $100 for a reasonably detailed explanation of this, as long as you publish it under the GFDL or a
similarlicense.

IcanthinkofareasonwhyEliezerYudkowskywon'ttakeyouuponthisoffer...

Imean,aboveandbeyondnotbeingabletoholduphisend)

ReplyPermalink
kevin3 28March200804:00:17PM 0points []

Mastering science is hard work...and it requires a willingness to ask hard questions and explore them wherever
they may lead. Most secret cults survive by demanding allegiance to the faith, obedience to doctrine, and
rejectinganycriticalexaminationofthosebeliefsortheirorigins.

Science, despite its lack of pop culture acclaim, is responsible for most of the progress in this worldprogress
againstdisease,againstpoverty,progressindescribingandunderstandingtheworldandtheuniverseinwhichwe
live. The openness of science to questions and to critical examination, and its adherence to standards of
verification,makeitthebestmethodwehaveforseparatingtruthfrommyth.

Itdoesnothoweverridtheworldofstupidpeople....andthat,myfriend,istherealproblem

ReplyPermalink
John 28March200806:37:38PM 0points []

In Nick Bostrom's paper on the survival of humanity, several potential catastrophe scenarios are technological
ones.Thatmakesmethinkthatitmightactuallybeabadideatopopularizescience.

The irony here is that information about how to create a catastrophe how to make a nuke, how to construct
virusesinalaboratory,howtomakeananobotisjustabouttheonlyscientificinformationthatpeoplearehiding.
(Fortunatetly,though,theydon'tmakeabigdealaboutthefactthey'rehidingit.)

ReplyPermalink
Nato_Welch 28March200807:14:44PM 1point []

AnimalsacrificeshavebeenreplacedbyDRM.

Ihaven'tfollowedthistoanyofnumerouspossibleconclusions,butIfoundtheanalogyirresistible.Thinkaboutit.

ReplyPermalink
Rolf_Nelson2 28March200811:33:01PM 0points []

@spindizzy:

No, this hasn't been "argued out", and even if it had been in the past, the "single best answer" would differ from
person to person and from year to year. I would suggest starting a thread on SL4 or on SIAI's Singularity
Discussionlist.

ReplyPermalink
Gwern_Branwen 30January200907:21:46PM 6points []

John: quite right. This actually reminds me of one of the common threads in Michael Crichton's works. From
JurassicPark:

>'...Malcolmsaid."Akaratemasterdoesnotkillpeoplewithhisbarehands.Hedoesnotlosehistemperandkill
his wife. > The person who kills is the person who has no discipline & no restraint, and who has purchased his
powerintheformofaSaturdaynightspecial.>Andthatisthekindofpowerthatsciencefosters,andpermits.
Andthatiswhyyouthinkthattobuildaplacelikethisissimple.">"Itwassimple!",Hammondinsisted.>'Then
whydiditgowrong?"'

Or:

>'"IwilltellyouwhatIamtalkingabout,"hesaid."Mostkindsofpowerrequireasubstantialsacrificebywhoever
wants the power. There is an apprenticeship, a discipline lasting many years. Whatever kind of power you want.
President of the company. Black belt in karate. Spiritual guru. Whatever it is you seek, you have to put in the
time, the practice, the effort. You must give up a lot to get it. It has to be very important to you. And once you
have attained it, its your power. It can't be given away: it resides in you. It is literally the result of your

http://lesswrong.com/lw/p0/to_spread_science_keep_it_secret/ 5/9
4/21/2017 ToSpreadScience,KeepItSecretLessWrong
discipline.>>Nowwhatisinterestingaboutthisprocessisthat,bythetimesomeonehasacquiredtheabilityto
killwithhisbarehands,hehasalsomaturedtothepointwherehewon'tuseitunwisely.Sothatkindofpowerhas
abuiltincontrol.Thedisciplineofgettingtheyousothatyouwon'tabuseit.>Butscientificpowerislikeinherited
wealth: attained without discipline. You read what others have done, and you take the next step. You can do it
veryyoung.Youcanmakeprogressveryfast."'

ReplyPermalink
quadibloc 06May201006:37:59PM 1point []

Ifoundthisarticlehilarious.Theideathatthegenuinelyvaluabletruthabouthowtheworldreallyworkscouldbe
foundinlosttomesofarcanelorehascrossedmymindattimes.

Thehistoricalreasonwhywearenotruledbyascientificpriesthood,though,iseasytosee.Untilfairlyrecently,
science did not produce the power to perform seeming miracles. Only by operating in the open could scientists
prove they weren't dabbling in witchcraft, because that crisis took place long before the atomic bomb or the
Gatlinggun.

"WingsOvertheWorld"fromH.G.Wells'ThingstoCome,ofcourse,istheclassicliteraryexampleofhowthis
couldcometopassafteracollapseofcivilization.

ReplyPermalink
JoshuaZ 06May201007:02:19PM 4points []

Knowledge in lost tomes isn't completely unheard of. For example, the RiemannSiegel formula (an important
formulaabouttheRiemannzetafunction)wasfoundbySiegelinthe1930swhenSiegelwasgoingthroughold
papers of Riemann's from the 1850s. This sort of thing was more common in the Middle Ages where Greek
mathematical works helped jumpstart our understanding. Sometimes also today in sociology and economics,
usefuldatasourcesarefoundinunexpectedplaces(FinkandStark'sworkonearlyreligioninAmericainpart
relied on discovering that the old US census data contained a lot more about religion than anyone had
previouslyrealized).Butthat'snotreallythesamethingsinceitisjustdata,nottheory.IthinktheRSformula
isprobablythebestmodernexampleofthissortofevent.

Parent
ReplyPermalink

TheOtherDave 05November201003:15:32PM 12points []

Thethingis,eveninourworldmostreligiouspeopledon'tinfactjoinesotericorders.

In the US, a lot of religious people are mainstream Christians, and many of those don't even read the Bible, let
alone study theology. They pick up a "password"level understanding of Christianity which allows them to signal
membership,andarecontentwiththat.Thesameisbasicallytrueofotherreligions.

Inyouralternateworld,Isuspectwhatyou'dactuallyendupwithisalotofcargocults.

Anathemtakesastabatthisidea,bytheway.It'sdelightfulreading,butitdoesn'ttakeplaceinauniversevery
muchlikeourown.

ReplyPermalink
[deleted] 07February201109:20:12PM 0points []

Wellthisexplainsalot.

ReplyPermalink
handoflixue 08July201109:47:34PM 4points []

"Whyhaven'tyougivenapublicdemonstrationofyourEighthLevelpowers,likethePhysicists?"

Icontinuetofindthisquoteaninvaluableheuristic.Thankyouforthegreatpost:)

ReplyPermalink
TheStevenator 27December201106:01:43AM 2points []

Ahh...SothisiswhyHarryandDracomeetatnightwearinghoodedmasksinsteadofjuststudyinginthelibrary
behindaquietuscharm.:)

This was a fun post. While I enjoy the public access and general lack of rock climbing ablilities associated with
learning science, it is a fun thing to contemplate. Maybe there is a way to implement this sort of thing in
elementary school classrooms. Maybe kids would think science is more fun if you offered to teach them the
amazingsecretsifandonlyiftheywereabletogiveyouanonpasswordanswerforwhathypothesesandtheories
are.

ReplyPermalink
MixedNuts 16January201210:13:48PM 15points []

Wait,can'tweactuallydothat?Peoplestartcultsallthetime.Wecouldstartourown,teachingdeepsecretsthat
are actually true. We won't bring it up, but when an initiate asks "Hey, isn't this exactly what they teach in
college?", nod and say "Yes, but it's more fun our way.". Making your own light bulb might still look pretty
mundane,soitlosessomeofthecultcompetition,butit'sstillmorefunthatschool.

ReplyPermalink
thomblake 16January201210:22:08PM 1point []

http://lesswrong.com/lw/p0/to_spread_science_keep_it_secret/ 6/9
4/21/2017 ToSpreadScience,KeepItSecretLessWrong

Idon'tthinkanyone'spresentedthatangleherebefore.Theassumptionhadbeenthatthissystemonlyworksif
yousomehowmakeallofthescientificknowledgesecretinthefirstplace.

Parent
ReplyPermalink
Minds_Eye 05December201407:02:18PM 0points []

Isthisideaactualydoable,ifsoitsoundslikeitwouldbefun.Theobviousquestions(tome)are:

Howdoyouscreenformembership,andcanwedothisinreallifewithoutthegovernment(s)screwingitover?

Parent
ReplyPermalink
ike 05December201407:11:58PM 2points []

Rule#1:don'ttalkaboutstartingacultinapublicplace.

Parent
ReplyPermalink
Minds_Eye 11December201407:37:01PM* 0points []

...IfIactuallywantedtostartacultIwouldn'tpostit.Iwasthinkingsomethingofstartingsomethingakin
tothesecularsolstice.(ThoughIdoubtmostpeoplecouldtellacultfromagrouplikethis.)

Also Rule #74: When I create a multimedia presentation of my plan designed so that my fiveyearold
advisorcaneasilyunderstandthedetails,Iwillnotlabelthedisk"ProjectOverlord"andleaveitlyingon
topofmydesk.

(WeareontheEvilOverlordchecklistright?Ineverfinishedcopyingitdown.)

Parent
ReplyPermalink

Gambler_Justice 30June201210:45:10AM 4points []

LearningJapanesewasaveryfunexperienceforme.Why?BecauseitfeltIwasinfiltratingsomekindofsecret
cult.Learningtoactuallyreadallthoseincomprehensiblemoonruneswaslikelearninghowtodecodetheirsecret
messages.Itwas...exciting.

ReplyPermalink
Dahlen 25July201201:11:45PM 1point []

Youseemtobeworkingontheassumptionthatimbuingsciencewithanauraofmysteriousness,attheexpense
ofavailabilityofknowledge,wouldsomehowattractmoreenthusiaststhanitwouldrepel.Or,inotherwords,that
therearemorepeoplewhosemainreasonfornotbeingintoscienceisan(inaccurate)ideaofitasadry,boring,
completelyunmysticalpursuit,thantherearescientificallyinclinedpeoplethatwouldbecomefrustratedwiththe
cultishnessandthesecrecyandeventuallyquit,intheeventualityofascenarioliketheoneyouproposed.

Thisseemsquiteunlikelytome,andI'llexplainwhy.

For the most part it's a feeling that the scenario you came up with is pandering to the wrong audience. Consider
thepresentstateofthings.Scienceismoreorlessoutintheopen,ifyouhavetheresources(mental,financial,
time)tolearnalargemajorityofpeopleisapatheticaboutscienceevenafter(presumably)havinghadencounters
with it at various points during their education some people that have probably had about the same encounters
withsciencehavegottenenthusiasticaboutit,andobviouslydidnotneedanysciencecultstogetthemtopursue
it.Yourthoughtexperimentseemstobedesignedtoattractmoreofthefirst.

Firstthere'stheconcernaboutwhetherthat"untapped"audiencereallyhasthepotentialtobringcontributionsthat
matchtheeffortspentwithattractingthem.Thesepeoplehavebeenexposedtosciencealreadyandithadno
effectontheirinterests!Inthiscase,doyouthinkthey'dreallyjoinamathematics"cult"fortheequations,andnot
forthesocialstandingorneedtoadheretoagroup?Ifnot,thendotheyholdpromiseaspotentialthinkers?

Asforthesecondcategoryofpeople,thosewhowouldn'tneedanyofthatbullshitsocialincentivesanyway,likeI
said,theunavailabilityofknowledgemightbeadisincentive.Scienceishardenoughasitis.Ittakesyearsand
yearstogatheradecentenoughbodyofknowledgeinordertoactuallygettomakeadvancesinyourownfield
introducingsillysocialritualsalongthewaywon'tspeedupthelearningprocess.Moreover,there'sthedangerof
not getting your message across in the first place, of it not reaching all of the people that could have been
interested.So,allinall,badidea.

Ofcourse,Iknowyoumeantthismoreasspeculationthananythingthepremisejustseemedfaultytome.

ReplyPermalink
afeller08 28October201203:04:47AM* 0points []

Idon'tknowtowhatextentwecanhackourownperceptionsofscarcitybyintentionallydirectingourthoughts,but
itseemslikeit'ssomethingworthtryingtodo:

"Scientificinformationiswidelyavailable.Asaresult,peoplewillpaylessattentiontoitthantheywouldifitwas
hidden.Asaresult,it'sbetterhiddenthanifitwerekeptpartiallysecret.Thismeansthatscientificinformationis
veryscarce,andalmostnobodyknowsthatitisscarce."

Is there a way to phrase the above statement so that it carries the same psychological weight as, "Only a few
peoplerealizethisnow,butthere'sabouttobeabeefshortage"?

http://lesswrong.com/lw/p0/to_spread_science_keep_it_secret/ 7/9
4/21/2017 ToSpreadScience,KeepItSecretLessWrong
Edit:(...andisthatthewholepointofthispostalongwiththestoryafterit?)

ReplyPermalink
fractalman 19May201307:56:56AM 0points []

Ooh,goodpointwiththosetextbooks!...andapainfulonetothinkabout.

ReplyPermalink
ChristianKl 23May201301:37:23PM 1point []

I think that the assumptions that people join cults for gaining knowledge is flawed. Most people don't care about
knowledgeandtruth.

Takescientologyasexample.LukeMuehlhauserabouttakingscientology101:

ThisclassareligiousclassItookasanatheistinordertoachieveanunrelatedgoalturnedouttobeone
ofthemostimportantclassesIhaveevertakeninmylife.

Mostscienceclassesontheotherhanddon'tteachlessonsthatthestudentcanafterwardsuseinhisdailylife.
Science101isboringtoalotofstudents.

Most students of science classes don't feel like they get knowledge in the class that they has an immdieate
effect.Religiousritualsdosomethingtopeople.

ReplyPermalink
MarsColony_in10years 06March201502:20:27AM 2points []

I can't help but think of the Ritual of the Calling of the Engineer. They date back nearly a century, and their
ceremoniesareconductedinprivate.TheinitiationritualisconductedbytheCorporationoftheSevenWardens,
with only new and past initiates present, but no guests. Upon initiation though, each new member is given a
symbolic Iron Ring, which is rumored to be forged from the mangled iron scavenged from the Quebec Bridge
disasteron1907.Theringisdistinctlyangularratherthansmoothandcomfortable,andiswornonthepinkyofthe
dominant hand so that it will slide across an engineer's drafting table or paper as he or she writes. This is a
constant reminder of an engineers lofty duties and responsibilities, and of those who died during the Quebec
Bridgecollapse.

ForpeopleintheUS,there'stheOrderoftheEngineer.Beforejoining,membersmustswearanoathtoabideby
"TheObligationofanEngineer":

Iamanengineer,inmyprofessionItakedeeppride.

ToitIowesolemnobligations.

Sincethestoneage,humanprogresshasbeenspurredbytheengineeringgenius.

Engineershavemadeusablenature'svastresourcesofmaterialandenergyforhumanity'sbenefit.

Engineershavevitalizedandturnedtopracticalusetheprinciplesofscienceandthemeansoftechnology.

Wereitnotforthisheritageofaccumulatedexperience,myeffortswouldbefeeble.

Asanengineer,Ipledgetopracticeintegrityandfairdealing,tolerance,andrespect,andtoupholddevotion
tothestandardsandthedignityofmyprofession,consciousalwaysthatmyskillcarrieswithittheobligation
toservehumanitybymakingthebestuseofEarth'spreciouswealth.

Asanengineer,Ishallparticipateinnonebuthonestenterprises.

Whenneeded,myskillandknowledgeshallbegivenwithoutreservationforthepublicgood.

Intheperformanceofdutyandinfidelitytomyprofession,Ishallgivetheutmost.

ReplyPermalink
themusicgod1 17October201508:30:25PM* 0points []

(thisisthesecondcopyofthiscomment,thefirstwasregrettablylostinabrowsercrash.Usesystemsthatback
upyourcommentsautomatically)

This advice seems to fly in the face of Richard Hamming's advice to keep an open door. However perhaps the
difference is subtle: Hamming suggested to have an open door but not necessarily to share your secrets, so
perhaps there is room for a big science mystery cult to retain its own mysteries at every level of initiation.
Perhapsthereisamiddleground[1]tobefoundbetweenthisandcurrent'openscience'whereinsecretsandritual
are more emphasized, but where the public has the ability to always query deep into the bureaucracy of the
sciencetemple/university.

Morelikely,howeverthebestapproachisalloftheabove,somekindsofthinkingareenhancedbyacertainsize
ofateam,andtheremaybesomeproblemsthatrequireanopensciencesized'ingroup',andsomeproblemsthat
aremoretractablewithaningroupthesizeofamysterycult.

1(keepinmindthemiddlegroundisnotnecessarilythebestpathit'sjustapath)

ReplyPermalink
Alex_Arendar 30November201501:31:34PM* 1point []

http://lesswrong.com/lw/p0/to_spread_science_keep_it_secret/ 8/9
4/21/2017 ToSpreadScience,KeepItSecretLessWrong

Weshouldnotalsoneglectthematerialsideofthescientistsstatus.Forexample,ajuniorinafamilymaybetold
byhisfather:"lookourneighborBob,hespent30yearsofhislifeonscienceandlookwhatajunkiecarhehas
andwhatasmallhousehelivesin.Betterhaveasimplebutreliablejob,likebuyingtshirtsinChinaandselling
themhereoralike,thatwillgiveyouareasonableliving."Ihaveseenquitealotofsuchjustificationsinreallife
andthishastobetakenintoaccountbecauseitreflectsthefactualstateofthings.

Regardingmakingtheproductsofscientificknowledgehiddenandsecret,well....Eliezerhasnotmadethissite's
contentprotectedbyinitiationritualsanditispubliclyaccessible:PIfIhadthat$100,000Iwouldputonagrey
robe,gotothatElectricityMonasteryandgrantittoEliezertobecomemysenseiandteachmeaboutblueness
andrednessimpliedfromatoms:)

ReplyPermalink

ABOUTLESSWRONG REPORTISSUES PRIVACY&TERMS PoweredbyReddit

http://lesswrong.com/lw/p0/to_spread_science_keep_it_secret/ 9/9