Scribd
Upload
104 views

Separating Programming Sheep From Non-Programming Goats

This document summarizes a research paper that proposes a simple 12-question test to determine if students have an aptitude for programming before beginning computer science courses. The test focuses on students' understanding of basic assignment statements. The results of administering the test showed that students fell into three groups - 44% had a consistent mental model of assignment, 39% did not develop a consistent model, and 8% did not attempt the questions. Retesting students after 3 weeks of instruction showed almost no movement between these groups, suggesting aptitude is determined very early. The paper argues this test could help identify students likely to succeed or struggle in programming courses.

Uploaded by

wclonn
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
104 views

Separating Programming Sheep From Non-Programming Goats

This document summarizes a research paper that proposes a simple 12-question test to determine if students have an aptitude for programming before beginning computer science courses. The test focuses on students' understanding of basic assignment statements. The results of administering the test showed that students fell into three groups - 44% had a consistent mental model of assignment, 39% did not develop a consistent model, and 8% did not attempt the questions. Retesting students after 3 weeks of instruction showed almost no movement between these groups, suggesting aptitude is determined very early. The paper argues this test could help identify students likely to succeed or struggle in programming courses.

Uploaded by

wclonn
Copyright
© © All Rights Reserved
We take content rights seriously. If you suspect this is your content, claim it here.
Available Formats
Download as PDF, TXT or read online on Scribd
You are on page 1/ 3

SeparatingProgrammingSheepfromNonProgrammingGoats

blog.codinghorror.com /separatingprogrammingsheepfromnonprogramminggoats/

Abunchofpeoplehavelinkedtothisacademicpaper,whichproposesawaytoseparateprogrammingsheepfromnonprogramminggoatsincomputerscience
classeslongbeforethestudentshaveevertouchedaprogramoraprogramminglanguage:

Allteachersofprogrammingfindthattheirresultsdisplaya'doublehump'.Itisasiftherearetwopopulations:thosewhocan[program],andthose
whocannot[program],eachwithitsownindependentbellcurve.Almostallresearchintoprogrammingteachingandlearninghaveconcentratedon
teaching:changethelanguage,changetheapplicationarea,useanIDEandworkonmotivation.Noneofitworks,andthedoublehumppersists.We
haveatestwhichpicksoutthepopulationthatcanprogram,beforethecoursebegins.Wecanpickapartthedoublehump.Youprobablydon'tbelieve
this,butyouwillafteryouhearthetalk.Wedon'tknowexactlyhow/whyitworks,butwehavesomegoodtheories.

Iwasn'tawarethatthedichotomybetweenprogrammersandnonprogrammerswassopronouncedatthisearlystage.DanBricklintouchedonthistopicinhisessay,
WhyJohnnyCan'tProgram.Butevidentlyit'scommonknowledgeamongstthosewhoteachcomputerscience:

Despitetheenormouschangeswhichhavetakenplacesinceelectroniccomputingwasinventedinthe1950s,somethingsremainstubbornlythesame.
Inparticular,mostpeoplecan'tlearntoprogram:between30%and60%ofeveryuniversitycomputersciencedepartment'sintakefailthefirst
programmingcourse.Experiencedteachersarewearybutneverobliviousofthisfactbrighteyedbeginnerswhobelievethattheoldonesmusthave
beendoingitwronglearnthetruthfrombitterexperienceandsoithasbeenforalmosttwogenerations,eversincethesubjectbeganinthe1960s.

Youmaythinkthetestthey'reproposingtodetermineprogrammingaptitudeiscomplex,butit'snot.Here'squestionone,verbatim:
Read the following statements and tick the box next to the correct answer.
int a = 10;
int b = 20;
a = b;
The new values of a and b are:
[ ] a = 20 b = 0
[ ] a = 20 b = 20
[ ] a = 0 b = 10
[ ] a = 10 b = 10
[ ] a = 30 b = 20
[ ] a = 30 b = 0
[ ] a = 10 b = 30
[ ] a = 0 b = 30
[ ] a = 10 b = 20
[ ] a = 20 b = 10

Thistestseemstrivialtoprofessionalprogrammers,butremember,it'sintendedforstudentswhohaveneverlookedatalineofcodeintheirlives.Theother12
questionsareallvariationsonthesameassignmenttheme.

Theauthorsofthepaperpositthattheprimaryhurdlesincomputerscienceare..

1.assignmentandsequence
2.recursion/iteration
3.concurrency*

..inthatorder.Thus,westartbytestingtheveryfirsthurdlenoviceprogrammerswillencounter:assignment.Thetestresultsdividedthestudentscleanlyintothree
groups:

44%ofstudentsformedaconsistentmentalmodelofhowassignmentworks(evenifincorrect!)
39%studentsneverformedaconsistentmodelofhowassignmentworks.
8%ofstudentsdidn'tgiveadamnandlefttheanswersblank.

Thetestwasadministeredtwiceonceatthebeginning,beforeanyinstructionatall,andagainafterthreeweeksofclass.Thestrikingthingisthattherewasvirtuallyno
movementatallbetweenthegroupsfromthefirsttosecondtest.Eitheryouhadaconsistentmodelinyourmindimmediatelyuponfirstexposuretoassignment,thefirst
hurdleinprogrammingorelseyouneverdevelopedone!

Theauthorsfoundanextremelyhighlevelofcorrelationbetweensuccessatprogrammingandformingaconsistentmentalmodel:
Clearly,Dehnahdi'stestisnotaperfectdividerofprogrammingsheepfromnonprogramminggoats.Nevertheless,ifitwereusedasanadmissions
barrier,andonlythosewhoscoredconsistentlywereadmitted,thepass/failstatisticswouldbetransformed.Inthetotalpopulation32outof61
(52%)failedinthefirsttestconsistentgrouponly6outof27(22%).Webelievethatwecanclaimthatwehaveapredictivetestwhichcanbetaken
priortothecoursetodetermine,withaveryhighdegreeofaccuracy,whichstudentswillbesuccessful.Thisis,sofarasweareaware,thefirsttesttobe
abletoclaimanydegreeofpredictivesuccess.

Ihighlyrecommendreadingthroughthedraftpaper(pdf),whichwasremarkablyentertainingforwhatIthoughtwasgoingtobeadry,academicpaper.Instead,itreads
likeablogentry.It'sfilledwithinterestinginsightslikethisone:

Ithastakenussometimetodaretobelieveinourownresults.Itnowseemstous,althoughweareawarethatatthispointwedonothavesufficientdata,
andsoitmustremainaspeculation,thatwhatdistinguishesthethreegroupsinthefirsttestistheirdifferentattitudestomeaninglessness.

Formallogicalproofs,andthereforeprogramsformallogicalproofsthatparticularcomputationsarepossible,expressedinaformalsystemcalleda
programminglanguageareutterlymeaningless.Towriteacomputerprogramyouhavetocometotermswiththis,toacceptthatwhateveryou
mightwanttheprogramtomean,themachinewillblindlyfollowitsmeaninglessrulesandcometosomemeaninglessconclusion.Inthetest
theconsistentgroupshowedapreacceptanceofthisfact:theyarecapableofseeingmathematicalcalculationproblemsintermsofrules,andcanfollow
thoseruleswheresoevertheymaylead.Theinconsistentgroup,ontheotherhand,looksformeaningwhereitisnot.Theblankgroupknowsthatitis
lookingatmeaninglessness,andrefusestodealwithit.

Everyoneshouldknowhowtouseacomputer,butnoteveryoneneedstobeaprogrammer.Butit'sstillalittledisturbingthattheactofprogrammingseemsliterally
unteachabletoasizablesubsetofincomingcomputersciencestudents.Evidentlynoteveryoneisasfascinatedbymeaninglessrulesandmeaningless
conclusionsasweareIcan'timaginewhynot.

*whichIhopetomastersometimebetweennowandmydeath

You might also like