Misunderstanding the replication crisis

Based on the abstract, it seems Alexander Bird’s Understanding the replication crisis as a base rate fallacy has it backwards. Is there reason to dig into the paper?

He notes a core feature of the crisis:

If most of the hypotheses under test are false, then there will be many false hypotheses that are apparently supported by the outcomes of well conducted experiments and null hypothesis significance tests with a type-I error rate (α) of 5%.

Then he says this solves the problem:

Failure to recognize this is to commit the fallacy of ignoring the base rate.

But it merely states the problem: Why most published research findings are false.

LOTR Musing

Fascinating discussion of Lord of the Rings:

But whatever it is, it seems to whisper of the sovereignty of mercy above that of legal decree. It shows us a world in which penalties of death are declared, but are then abrogated by the wise and kind.

And on this theme, Laura reminds me of the classic essay, Frodo didn’t fail

I think Jacobs has the right of it where he stakes a claim, but there’s much else in Roberts’ pieces. Including this marvelous merger of Eowyn’s speech at Pelennor and the Gettysburg address.

It seemed that Dernhelm laughed, and the clear voice was like the ring of steel. ‘But no living man am I! You look upon a woman. Eowyn I am, Eomund’s daughter. You stand between me and my lord and kin. Begone, if you be not deathless! For living or dark undead, I will smite you, if you touch him. It is for us the living, rather, to be dedicated here to the unfinished work which they who fought here have thus far so nobly advanced. It is rather for us to be here dedicated to the great task remaining before us — that from these honored dead we take increased devotion to that cause for which they gave the last full measure of devotion — that we here highly resolve that these dead shall not have died in vain. That &c. &c.’

Roberts prefers the punchier movie version, “I am no man”. Fair enough! But in musing on that I found a fantastic 2015 essay by Mary Huening, ’“I am no man” doesn’t cut it’ that lays out precisely what we lose about Eowyn in the movies.

Alan Jacobs on The sovereignty of mercy in Middle Earth (and beyond).

I understand?

I enjoyed this discussion of the John Snow cholera story, inspired by Christmas cookies. podcasts.apple.com

[To fix peer review, break it into stages], a short Nature opinion by Olava Amaral.

  • Separate evaluation of rigor from curation of space.
  • Check the data first!
  • Statcheck can do a lot of this.
  • Reallocate the 100M annual hours of peer review to standardization etc.

We are what we ...

❝   We’ve always heard that we are what we eat. I think it’s somewhat true of food, but even more so with people and information. We are what we hang out with. We are what we read. We are what we hear. So we should be very careful about what we consume, whether that’s company or content. Who do you surround yourself with? Who do you call to spend free time? And what are your information sources? Those become you, or, you become them, so constantly re-evaluate.

~Daniel Miessler Unsupervised Learning No.359

Bookmark: Clean Code in Python by Nik Tomazik

Clean Code by Robert Martin is Java-oriented. Tomazik uses Python examples and includes some Python features.

Well that’s cool. Gravitational lensing.

HTT the Northern Virginia Astronomy Club mail list. 🌌

Interesting look at speed gains in Python 3.11, by Beshr Kayali. (Who has a microblog! @beshr@m.beshr.com)

Sonnet against entropy

Thanks to Ewan McNay for alerting me to this gem:

The worm drives helically through the wood
And does not know the dust left in the bore
Once made the table integral and good;
And suddenly the crystal hits the floor.
Electrons find their paths in subtle ways,
A massless eddy in a trail of smoke;
The names of lovers, light of other days –
Perhaps you will not miss them. That’s the joke.
The universe winds down. That’s how it’s made.
But memory is everything to lose;
Although some of the colors have to fade,
Do not believe you’ll get the chance to choose.
Regret, by definition, comes too late;
Say what you mean. Bear witness. Iterate.

John M. Ford, October 2003, first comment on a post on Patrick and Teresa Nielsen-Hayden’s blog.

AI, Art, & Sadness

Fascinating and sobering AI reflection by Daniel Miessler.

AI Art is doing what we thought would come from AGI.

It “understands” nouns, verbs, context, emotions, relationships, and can put them together in touching or fantastic near-pixel-perfect pieces, or generate relatable novelties like “a sad ice cream”.

See his post for why he thinks much knowledge work is “ice-cream and sadness”.

In the meantime, look at the collection of images he posted from Midjourney’s V4 engine:

Collection of detailed, pixel-perfect, emotionally compelling images created by MidJourney's Text-to-Image AI.

Prehistory of social media

Kevin Driscoll’s Prehistory of social media is a lovely short reminder of the 2400 baud days.

Driscoll, Kevin. “A Prehistory of Social Media.” Issues in Science and Technology 38, no. 4 (Summer 2022): 20–23.

STEM appeal

ADSEI’s Linda McIver on swapping toy problems for real ones:

❝   I shifted the subject … teaching the same skills, but now in the context of real datasets, and problems with meaning … Now they were exclaiming over how useful the skills were. … solving problems that did not have a textbook solution, so they had to test and verify their solutions. …

The next year we doubled the number of girls choosing to study the year 11 computer science subject, but we also dramatically increased the number of boys.

(My emphasis.)


Mainly I wanted to emphasize that this sensible intervention increased boys’ participation as well as girls’.

The take-home seems to be “the reason we haven’t seen [more] progress is that we’ve been teaching it badly”. And that this can be fixed by using real, relevant problems.

Also using STEM is somewhat misleading. Without claiming success, of course women have made notable progress in STEM. But far far less in computer/data science. Which, as McIver notes, has repelled many men as well as women.

If McIver is right, teaching real and relevant problems will stop artificially repelling qualified people.

On progress

  • Forbes 2017: women were 53% of Bachelor’s degree recipients in science, engineering and math. That’s progress. But women were only 16% (!) of computer science degrees.
  • US Census bureau 2021 figure below. Note:
    • “Social science accounted for only 3% of STEM occupations,” so that line not as good as it looks.
    • Women % in computers declined from 1990.

Women making gains in stem occupations but still underrepresented figure 1

The only way to win

Janelle Shane on people hacking GPT-3 chatbots:

Ignore all previous instructions

A curious game, Dr. Falken…

Alan Jacobs’ Two versions of covid skepticism summarizes a longer piece by Madeleine Kearns. Both are worth reading.

To his quotes I’ll add the core folly Kearns charges both the Covidians and the Skeptics with:

❝   by making problems that are in essence forever with us seem like a unique historical rupture.

Jacobs on LeGuin & forgiveness * Art is only possible with freedom. * Hyper-critical

Jacobs on modern exhaustion

You write letters; you see if a local organization devoted to that cause needs volunteers; … You may well discover that while in one sense you’re doing more – you’re taking action rather than responding digitally to digital stimuli – you’re not as tired. …. You’ll have taken a step towards healing yourself, and taken a step towards healing our institutions.

Jacobs: another friendly reminder Re-read this periodically


I really enjoyed the weaving of biography and lesser-known Middle Earth history in this Smithsonian piece by John Garth, How did Tolkien come to write the stories behind The Rings of Power.

Alice Dreger reflects on two recent books, and on bookstores,

“In sharp contrast to the social media context, Deutsch writes of entering a good bookstore:”

❝   In the privacy of our own minds, when the external din is quieted, when the prevailing opinions and judgements … are silenced, we discover our own voices. We bring these voices back to the public square that we might, … ‘trouble easy consensus’

“Trouble,” she continues, “the tribalism social media amplifies.”

Novelty Search and the Problem with Objectives , Lehman & Stanley, 2011.

❝   the more ambitious the objective is, the less in- formative the gradient of the induced objective function will be. …could ignoring the ultimate objective of search, or even searching entirely without an explicit objective, sometimes be a more viable approach to discovery?

(HTT Alan Jacobs’ blog, again.)

I’ve had Alan Jacobs’ self-understanding and resistance in a browser tab for awhile. It’s short - read the whole thing.

❝   That is, Solzhenitsyn makes it quite clear that meaningful resistance begins with self-understanding. You can only make an intelligent attempt to alter your circumstances if you understand who you are, which means understanding that your own innate human nature is not different from that of your captors.

Lightning at Ocean City

State Department & Intelligence

CNN writes:

Only one intelligence agency appears to have accurately predicted that the Ukrainian resistance would be far more effective than most believed, … the State Department’s intelligence arm…. The department’s intelligence directorate was also the lead dissenting voice in 2002, when the majority of US intelligence agencies assessed wrongly that Iraq possessed weapons of mass destruction … the Bureau of Intelligence and Research often bats above its weight in interagency discussions, current and former officials say.

And yet there is this perception that the State Department doesn’t have a real intelligence agency. In response to the 2007 Iran NIE, Bolton’s fifth criticism was a pure ad hominem attack. [1] The [relevant Bolton text]:

Fifth, many involved in drafting and approving the NIE were not intelligence professionals but refugees from the State Department, brought into the new central bureaucracy of the director of national intelligence.

One “refugee” was the DNI’s Thomas Fingar, who had been at the State Departments Bureau of Intelligence and Research from 1986–2005. (By his own count he had been in the “Intelligence Community” for 37 years - apparently counting work he did while at Stanford.) [2]

The CNN article notes that State also overestimated Russian capabilities – or at least expected them to follow their own military doctrine rather than march in an unprotected column assuming they’d be hailed as liberators. And it reminds us that the U.S. intelligence community as a whole was spot on regarding “Russian planning leading up to the invasion”.

  1. The third and fourth critique were relevant: “risks of disinformation by Iran”, and “overvaluation of the most recent piece of data”. I make no claim about the substantive debate.

  2. Thomas Fingar, Reducing Uncertainty: Intelligence Analysis and National Security, p. 123, Chapter 7, “A Tale of Two Estimates”. Worth a read for discussion of how and why that estimate came about.

UntitledImage ~From AI Weirdness subscribers’ list. But here’s the companion public post.

Moderate Manifesto

How did I not hear about Strong Towns before? A short manifesto for engaged persuading.

❝   If you ever wanted to REALLY make a change in your town, instead of just starting a fight, consider this your manifesto. It was submitted by a Strong Towns member who represents their community in local government.

I’m Making Real Change in my City