Information operations · Information Warfare

Discovery: Calling Bullshit in the Age of Big Data


Seldom am I as impressed with a course offering as I am with this one at the University of Washington.  Far below I have a copy of the syllabus, case studies, tools, and links to all the videos. The course is written listed as under both the Information School and the Biology Department as INFO 198 / BIOL 106B but it does not appear to be currently offered.

At first I thought this was bullshit. No college or university would allow such a word to be uttered within the halls of academia.  Then to offer such a course seemed insane. But then it occurred to me, in this era of overload of information, of fake news, of overwhelming distrust in the information we feed on, perhaps this is exactly what we need to sift through all the chaff to get to the weed. You know, calling bullshit when needed.

A close hyperacademic friend wrote:

This gets to the core of what deceivers do.

Precisely.  I have not had time to review all the videos, there’s only 45 of them. But the rationale is real, the purpose seems pure, and the hypotheses and conclusions  are spot on, in what I saw and read.  Now the key is to get this as a required course for anybody participates in an information operation, an influence operation, strategic communications, or public diplomacy. Especially you deception operations gurus.

A bit about the course itself, all laid out in one place.

The University of Washington, UW, has something called the iCollege which allows freer thought, freer expression, and still maintains the same standards asl Hard Science <fill in the blank> 101 – 401, which I suffered through for years.

They even have a YouTube channel where all their lectures are recorded and posted, 45 in total. UW iSchool

Welcome | Information School | University of Washington The course is not currently listed, I need to discover where and when it will be held again. It was held in the Spring Quarter of 2017, so I would venture to guess this course will be offered again in the Spring of 2018.

Of the two professors for the course, Carl T. Bergstrom and Jevin West, only West is listed as faculty in the iCollege. Bergstrom is a professor of biology, so he must in the Biology Department (iBiology?).  In his twitter profile he says:

Professor of Biology . I use math & evolutionary theory to study information flow in biology & society. Love corvids.

So I dug through his University of Washington profile.  This, alone, is also worth wading through. The theories he shares, the research, is incredible.

Overview

We use mathematical models and computer simulations to study how information flows through the world.

  • How do living organisms acquire, store, process, and utilize information?
  • How and why does communication evolve?
  • How does information travel through biological or social networks?
  • How do the structure of scientific institutions and the norms of scientific communication influence the questions that scientists ask and the answers that they find?

Our work on these questions leads us through evolutionary biology, animal behavior, philosophy of biology, network theory, epidemiology, and even into domains of the social sciences such as economics, sociology, and bibliometrics.

Jevin D. West is also very interesting, he’s a big data guy.

I am an an Assistant Professor in the Information School at the University of Washington. I co-direct the DataLab. I study the Science of Science. My laboratory consists of millions of scholarly papers and the billions of links that connect these papers. I develop knowledge discovery tools to both study and facilitate science. In particular, I am interested in the origin of scholarly disciplines and how sociological and economic factors drive and slow the evolution of science.

Now, dive into the course itself:

This has been written about before in the media.

Bottom line, I plan to see if I can remotely enroll in the University of Washington just for this course. I would also love to collaborate on a joint project with these gentlemen.



Syllabus: Calling Bullshit in the Age of Big Data

Logistics

Course: INFO 198 / BIOL 106B. University of Washington
To be offered: Spring Quarter 2017
Credit: 1 credit, C/NC
Enrollment: 160 students
InstructorsCarl T. Bergstrom and Jevin West
Synopsis: Our world is saturated with bullshit. Learn to detect and defuse it.

The course will be offered as a 1-credit seminar this spring through the Information School at the University of Washington. We aim to expand it to a 3 or 4 credit course for 2017-2018. For those who cannot attend in person, we aim to videotape the lectures this spring and make video clips freely available on the web.


Learning Objectives

Our learning objectives are straightforward. After taking the course, you should be able to:

  • Remain vigilant for bullshit contaminating your information diet.
  • Recognize said bullshit whenever and wherever you encounter it.
  • Figure out for yourself precisely why a particular bit of bullshit is bullshit.
  • Provide a statistician or fellow scientist with a technical explanation of why a claim is bullshit.
  • Provide your crystals-and-homeopathy aunt or casually racist uncle with an accessible and persuasive explanation of why a claim is bullshit.

We will be astonished if these skills do not turn out to be among the most useful and most broadly applicable of those that you acquire during the course of your college education.


Schedule and readings

Each of the lectures will explore one specific facet of bullshit. For each week, a set of required readings are assigned. For some weeks, supplementary readings are also provided for those who wish to delve deeper.

Lectures

  1. Introduction to bullshit
  2. Spotting bullshit
  3. The natural ecology of bullshit
  4. Causality
  5. Statistical traps
  6. Visualization
  7. Big data
  8. Publication bias
  9. Predatory publishing and scientific misconduct
  10. The ethics of calling bullshit.
  11. Fake news
  12. Refuting bullshit

Week 1. Introduction to bullshit. What is bullshit? Concepts and categories of bullshit. The art, science, and moral imperative of calling bullshit. Brandolini’s Bullshit Asymmetry Principle.

Supplementary readings

  • G. A. Cohen (2002) Deeper into Bullshit. Buss and Overton, eds., Contours of Agency: Themes from the Philosophy of Harry Frankfurt Cambridge, Massachusetts: MIT Press.
  • Philip Eubanks and John D. Schaeffer (2008) A kind word for bullshit: The problem of academic writing. College Composition and Communication 59(3): 372-388
  • J. L. Austin Performative Utterance, in Austin, Urmson, and Warnock (1979) Philosophical Papers. Clarendon.

Week 2. Spotting bullshit. Truth, like liberty, requires eternal vigilance. How do you spot bullshit in the wild? Effect sizes, dimensions, Fermi estimation, and checks on plausibility. Claims and the interests of those who make them. Forensic data analysis: GRIM test, NewcombBenford law.


Week 3. The natural ecology of bullshit. Where do we find bullshit? Why news media provide bullshit. TED talks and the marketplace for upscale bullshit. Why social media provide ideal conditions for the growth and spread of bullshit.


Week 4. Causality One common source of bullshit data analysis arises when people ignore, deliberately or otherwise, the fact that correlation is not causation. The consequences can be hilarious, but this confusion can also be used to mislead. Confusing causality with necessity or sufficiency. Regression to the mean pitched as treatment effect. Milton Friedman’s thermostat. Selection masked as transformation.

Supplementary reading

  • Karl Pearson (1897) On a Form of Spurious Correlation which may arise when Indices are used in the Measurement of Organs. Proceedings of the Royal Society of London 60: 489–498. For context see also Aldrich (1995).

Week 5. Statistical traps and trickery. Bayes rule and conditional probabilities. Base-rate fallacy / prosecutor’s fallacy. Simpson’s paradox. Data censoring. Will Rogers effect, lead-time bias, and length time bias. Means versus medians. Importance of higher moments.


Week 6. Data visualization. Data graphics can be powerful tools for understanding information, but they can also be powerful tools for misleading audiences. We explore the many ways that data graphics can steer viewers toward misleading conclusions.

  • Edward Tufte (1983) The Visual Display of Quantitative Information Chapters 2 (Graphical integrity) and 5 (Chartjunk: vibrations, grids, and ducks).
  • Tools and tricks: Misleading axes
  • Tools and tricks: Proportional Ink

Week 7. Big data. When does any old algorithm work given enough data, and when is it garbage in, garbage out? Use and abuse of machine learning. Misleading metrics. Goodhart’s law.

Supplementary reading

  • Cathy O’Neil (2016) Weapons of Math Destruction Crown Press.
  • Peter Lawrence (2014) The mismeasurement of science. Current Biology17:R583-585

Week 8. Publication bias. Even a community of competent scientists all acting in good faith can generate a misleading scholarly record when — as is the case in the current publishing environment — journals prefer to publish positive results over negative ones. In a provocative and hugely influential 2005 paper, epidemiologist John Ioannides went so far as to argue that this publication biashas created a situation in which most published scientific results are probably false. As a result, it’s not clear that one can safely rely on the results of some random study reported in the scientific literature, let alone on Buzzfeed. Once corporate funders with private agendas become involved, matters become all the more complicated.

Supplementary Reading

eLife

     5:e21451

Week 9. Predatory publishing and scientific misconduct. Predatory publishing. Beall’s list and his anti-Open Access agenda. Publishing economics. Pathologiesof publish-or-perish culture. Pursuit of PR instead of progress.

New York Times


Week 10. The ethics of calling bullshit. Where is the line between deserved criticism and targeted harassment? Is it, as one prominent scholar argued, “methodological terrorism” to call bullshit on a colleague’s analysis? What if you use social media instead of a peer-reviewed journal to do so? How about calling bullshit on a whole field that you know almost nothing about? Pubpeer. Principles for the ethical calling of bullshit. The Dunning-Kruger effect. Differences between being a hard-minded skeptic and being a domineering jerk.


Week 11. Fake news.. Fifteen years ago, nascent social media platforms offered the promise of a more democratic press through decentralized broadcasting and a decoupling of publishing from advertising revenue. Instead, we get sectarian echo chambers and, lately, a serious assault on the very notion of fact. Not only did fake news play a substantive role in the November 2016 US elections, but recently a fake news story actually provoked nuclear threats issued by twitter.

New York Times


Week 12. Refuting bullshit. Refuting bullshit requires different approaches for different audiences. What works for a quantitatively-skilled professional scientist won’t always convince your casually racist uncle on facebook, and vice versa.


Exercises

Exercise 1: A bullshit inventory. How much bullshit are you dealing with, anyway? Keep track of your encounters with bullshit over the course of a week, and come up with a way to visualize your results.


Calling Bullshit has been developed by Carl Bergstrom and Jevin West to meet what we see as a major need in higher education nationwide.

Disclaimer: This website is intended for personal educational use and should be employed for informational purposes only. Accordingly, all warranties and forms of liability from your use of this website are disclaimed to the extent applicable in your jurisdiction. Nothing on this website constitutes guaranteed accuracy of any kind. Calls of bullshit represent the opinions of the instructors and are not intended as definitive judgements of fact. Callingbullshit.org is not liable for any loss of credulity you may suffer as a consequence of reading the information herein. Viewer discretion advised. May cause drowsiness. Void where prohibited. No animals were used during testing. May cause excitability. Not recommended for children under the age of 12. Use only as directed. Any similarity to any person living or dead is merely coincidental. Live, except on West Coast. Do not drive or operate heavy machinery while using this website. Objects on this site may be closer than they appear. Additional taxes may apply in some jurisdictions. Individual results may vary. Not to be used with alcoholic beverages. I bet you think this website is about you, don’t you? Don’t you?

Advertisements