Seldom am I as impressed with a course offering as I am with this one at the University of Washington. Far below I have a copy of the syllabus, case studies, tools, and links to all the videos. The course is written listed as under both the Information School and the Biology Department as
At first I thought this was bullshit. No college or university would allow such a word to be uttered within the halls of academia. Then to offer such a course seemed insane. But then it occurred to me, in this era of overload of information, of fake news, of overwhelming distrust in the information we feed on, perhaps this is exactly what we need to sift through all the chaff to get to the weed. You know, calling bullshit when needed.
A close hyperacademic friend wrote:
This gets to the core of what deceivers do.
Precisely. I have not had time to review all the videos, there’s only 45 of them. But the rationale is real, the purpose seems pure, and the hypotheses and conclusions are spot on, in what I saw and read. Now the key is to get this as a required course for anybody participates in an information operation, an influence operation, strategic communications, or public diplomacy. Especially you deception operations gurus.
A bit about the course itself, all laid out in one place.
The University of Washington, UW, has something called the iCollege which allows freer thought, freer expression, and still maintains the same standards asl Hard Science <fill in the blank> 101 – 401, which I suffered through for years.
They even have a YouTube channel where all their lectures are recorded and posted, 45 in total. UW iSchool
Welcome | Information School | University of Washington The course is not currently listed, I need to discover where and when it will be held again. It was held in the Spring Quarter of 2017, so I would venture to guess this course will be offered again in the Spring of 2018.
Of the two professors for the course, Carl T. Bergstrom and Jevin West, only West is listed as faculty in the iCollege. Bergstrom is a professor of biology, so he must in the Biology Department (iBiology?). In his twitter profile he says:
Professor of Biology
@UW. I use math & evolutionary theory to study information flow in biology & society. Love corvids.
So I dug through his University of Washington profile. This, alone, is also worth wading through. The theories he shares, the research, is incredible.
We use mathematical models and computer simulations to study how information flows through the world.
- How do living organisms acquire, store, process, and utilize information?
- How and why does communication evolve?
- How does information travel through biological or social networks?
- How do the structure of scientific institutions and the norms of scientific communication influence the questions that scientists ask and the answers that they find?
Our work on these questions leads us through evolutionary biology, animal behavior, philosophy of biology, network theory, epidemiology, and even into domains of the social sciences such as economics, sociology, and bibliometrics.
Jevin D. West is also very interesting, he’s a big data guy.
I am an an Assistant Professor in the Information School at the University of Washington. I co-direct the DataLab. I study the Science of Science. My laboratory consists of millions of scholarly papers and the billions of links that connect these papers. I develop knowledge discovery tools to both study and facilitate science. In particular, I am interested in the origin of scholarly disciplines and how sociological and economic factors drive and slow the evolution of science.
Now, dive into the course itself:
- How to Call B.S. on Big Data: A Practical Guide New Yorker
- “Calling Bullshit” Is the College Course for Our Times Big Think
- These University of Washington professors are teaching a course on bullshit ReCode
- Tag: calling bullshit in the age of big data Scholastica Blog
Bottom line, I plan to see if I can remotely enroll in the University of Washington just for this course. I would also love to collaborate on a joint project with these gentlemen.
Syllabus: Calling Bullshit in the Age of Big Data
Course: INFO 198 / BIOL 106B. University of Washington
To be offered: Spring Quarter 2017
Credit: 1 credit, C/NC
Enrollment: 160 students
Instructors: Carl T. Bergstrom and Jevin West
Synopsis: Our world is saturated with bullshit. Learn to detect and defuse it.
The course will be offered as a 1-credit seminar this spring through the Information School at the University of Washington. We aim to expand it to a 3 or 4 credit course for 2017-2018. For those who cannot attend in person, we aim to videotape the lectures this spring and make video clips freely available on the web.
Our learning objectives are straightforward. After taking the course, you should be able to:
- Remain vigilant for bullshit contaminating your information diet.
- Recognize said bullshit whenever and wherever you encounter it.
- Figure out for yourself precisely why a particular bit of bullshit is bullshit.
- Provide a statistician or fellow scientist with a technical explanation of why a claim is bullshit.
- Provide your crystals-and-homeopathy aunt or casually racist uncle with an accessible and persuasive explanation of why a claim is bullshit.
We will be astonished if these skills do not turn out to be among the most useful and most broadly applicable of those that you acquire during the course of your college education.
Schedule and readings
Each of the lectures will explore one specific facet of bullshit. For each week, a set of required readings are assigned. For some weeks, supplementary readings are also provided for those who wish to delve deeper.
- Introduction to bullshit
- Spotting bullshit
- The natural ecology of bullshit
- Statistical traps
- Big data
- Publication bias
- Predatory publishing and scientific misconduct
- The ethics of calling bullshit.
- Fake news
- Refuting bullshit
Week 1. Introduction to bullshit. What is bullshit? Concepts and categories of bullshit. The art, science, and moral imperative of calling bullshit. Brandolini’s Bullshit Asymmetry Principle.
- Harry Frankfurt (1986) On Bullshit. Raritan Quarterly Review 6(2)
- G. A. Cohen (2002) Deeper into Bullshit. Buss and Overton, eds., Contours of Agency: Themes from the Philosophy of Harry Frankfurt Cambridge, Massachusetts: MIT Press.
- Philip Eubanks and John D. Schaeffer (2008) A kind word for bullshit: The problem of academic writing. College Composition and Communication 59(3): 372-388
- J. L. Austin Performative Utterance, in Austin, Urmson, and Warnock (1979) Philosophical Papers. Clarendon.
Week 2. Spotting bullshit. Truth, like liberty, requires eternal vigilance. How do you spot bullshit in the wild? Effect sizes, dimensions, Fermi estimation, and checks on plausibility. Claims and the interests of those who make them. Forensic data analysis: GRIM test, Newcomb–Benford law.
- Carl Sagan 1996 The Fine Art of Baloney Detection. Chapter 12 in Sagan (1996) The Demon-Haunted World
- Case studies: Food stamp fraud, 99% caffeine-free
Week 3. The natural ecology of bullshit. Where do we find bullshit? Why news media provide bullshit. TED talks and the marketplace for upscale bullshit. Why social media provide ideal conditions for the growth and spread of bullshit.
- Gordon Pennycook et al. (2015) On the reception and detection of pseudo-profound bullshit. Judgement and Decision Making 10:549-563
- Adrien Friggeri et al. (2014). Rumor Cascades. Proceedings of the Eighth International AAAI Conference on Weblogs and Social Media
Week 4. Causality One common source of bullshit data analysis arises when people ignore, deliberately or otherwise, the fact that correlation is not causation. The consequences can be hilarious, but this confusion can also be used to mislead. Confusing causality with necessity or sufficiency. Regression to the mean pitched as treatment effect. Milton Friedman’s thermostat. Selection masked as transformation.
- Robert Matthews (2000) Storks deliver babies (p=0.008). Teaching Statistics22:36-38
- Case study: Traffic improvements
- Karl Pearson (1897) On a Form of Spurious Correlation which may arise when Indices are used in the Measurement of Organs. Proceedings of the Royal Society of London 60: 489–498. For context see also Aldrich (1995).
Week 5. Statistical traps and trickery. Bayes rule and conditional probabilities. Base-rate fallacy / prosecutor’s fallacy. Simpson’s paradox. Data censoring. Will Rogers effect, lead-time bias, and length time bias. Means versus medians. Importance of higher moments.
- Simpson’s paradox: an interactive data visualization from VUDlab at UC Berkeley.
- Alvan Feinstein et al. (1985) The Will Rogers Phenomenon — Stage Migration and New Diagnostic Techniques as a Source of Misleading Statistics for Survival in Cancer. New England Journal of Medicine 312:1604-1608.
- Case studies: Musicians and mortality, Track records
Week 6. Data visualization. Data graphics can be powerful tools for understanding information, but they can also be powerful tools for misleading audiences. We explore the many ways that data graphics can steer viewers toward misleading conclusions.
- Edward Tufte (1983) The Visual Display of Quantitative Information Chapters 2 (Graphical integrity) and 5 (Chartjunk: vibrations, grids, and ducks).
- Tools and tricks: Misleading axes
- Tools and tricks: Proportional Ink
Week 7. Big data. When does any old algorithm work given enough data, and when is it garbage in, garbage out? Use and abuse of machine learning. Misleading metrics. Goodhart’s law.
- danah boyd and Kate Crawford (2011) Six Provocations for Big Data. A Decade in Internet Time: Symposium on the Dynamics of the Internet and Society.
- David Lazer et al. (2014) The Parable of Google Flu: Traps in Big Data Analysis. Science 343:1203-1205
- Alyin Caliskan et al. (2017) Semantics derived automatically from language corpora contain human-like biases Science 356:183-186
- Jevin West (2014) How to improve the use of metrics: learn from game theory. Nature 465:871-872
- Cathy O’Neil (2016) Weapons of Math Destruction Crown Press.
- Peter Lawrence (2014) The mismeasurement of science. Current Biology17:R583-585
Week 8. Publication bias. Even a community of competent scientists all acting in good faith can generate a misleading scholarly record when — as is the case in the current publishing environment — journals prefer to publish positive results over negative ones. In a provocative and hugely influential 2005 paper, epidemiologist John Ioannides went so far as to argue that this publication biashas created a situation in which most published scientific results are probably false. As a result, it’s not clear that one can safely rely on the results of some random study reported in the scientific literature, let alone on Buzzfeed. Once corporate funders with private agendas become involved, matters become all the more complicated.
- John Ioannidis (2005) Why most published scientific results are false. PLOS Medicine 2:e124.
- David Michaels and Celeste Monforton (2005) Manufacturing Uncertainty: Contested Science and the Protection of the Public’s Health and Environment. American Journal of Public Health 95:S39-S48
Week 9. Predatory publishing and scientific misconduct. Predatory publishing. Beall’s list and his anti-Open Access agenda. Publishing economics. Pathologiesof publish-or-perish culture. Pursuit of PR instead of progress.
- Fake academe looking much like the real thing.
New York Times
- Dec. 29, 2016.
- Adam Marcus and Ivan Oransky (2016) Why fake data when you can fake a scientist? Nautilus November 24.
- Tools and tricks: How can you know if a paper is legit?
Week 10. The ethics of calling bullshit. Where is the line between deserved criticism and targeted harassment? Is it, as one prominent scholar argued, “methodological terrorism” to call bullshit on a colleague’s analysis? What if you use social media instead of a peer-reviewed journal to do so? How about calling bullshit on a whole field that you know almost nothing about? Pubpeer. Principles for the ethical calling of bullshit. The Dunning-Kruger effect. Differences between being a hard-minded skeptic and being a domineering jerk.
- Alan Sokal (1996) A physicist experiments with cultural studies. Lingua Franca 6:62-64.
- Jennifer Ruark (2017) Anatomy of a hoax. Chronicle of Higher Education
- Robert Service (2014) Nano-Imaging Feud Sets Online Sites Sizzling. Science343:358.
- Susan Fiske (2016) Mob Rule or Wisdom of Crowds? APS Observerpreliminary draft. Also read commentaries  and .
- Michael Blatt (2016) Vigilante Science. Plant Physiology 169:907-909.
Week 11. Fake news.. Fifteen years ago, nascent social media platforms offered the promise of a more democratic press through decentralized broadcasting and a decoupling of publishing from advertising revenue. Instead, we get sectarian echo chambers and, lately, a serious assault on the very notion of fact. Not only did fake news play a substantive role in the November 2016 US elections, but recently a fake news story actually provoked nuclear threats issued by twitter.
New York Times
- Nov. 25, 2016
- Donath, Judith (2016) Why fake news stories thrive online. CNN Opinion.
- Brian Feldman (2017) Google’s dangerous identity crisis. New York Magazine
Week 12. Refuting bullshit. Refuting bullshit requires different approaches for different audiences. What works for a quantitatively-skilled professional scientist won’t always convince your casually racist uncle on facebook, and vice versa.
- John Cook and Stephan Lewandowsky (2012) The Debunking Handbook.
- Craig Bennett et al. (2009) Neural correlates of interspecies perspective taking in the post-mortem Atlantic Salmon: An argument for multiple comparisons correction
- Case study: Gender gap in 100 m times
Exercise 1: A bullshit inventory. How much bullshit are you dealing with, anyway? Keep track of your encounters with bullshit over the course of a week, and come up with a way to visualize your results.