Surprisingly, fact-checking prompted Iran’s Supreme Leader to retract his mistaken claim about Iran’s economy.

It’s probably the first time in the past three decades that Ayatolla Khamenei admitted to a mistake and corrected himself.

A glimmer of hope in a social media world that treats fact-checking as evidence for the false claim. (Though before the retraction, state media photoshopped an infographic to try to bolster the claim.)

So…

Anyone know if Biden has corrected any of his (relatively rare) 4-Pinnochio claims from his first 100 days? [List.]

Or the former US President on his 3,294 unique 4-Pinnochio claims, 94 from the first 100 days? [Full dataset here].

…we are left with the problem that… Social scientists think of themselves as explorers and they will continue to sail the world’s oceans shouting “Land!” at every mirage on the horizon even if much of the Earth has already been mapped. ~Jay Greene

Wilderness & Environmental Medicine’s editor Neil Pollock frequently writes an Editor’s note about peer review, publishing, and reproducibility. I wish he were more concise, but he often raises good points, and I expect none of my colleagues read WEM, so here’s a digest. This month’s essay is Avoiding the High Cost of Peer Review Failures.
Skipping a review of the problem, and a note about unintended consequences of open access, we get to some clear caveat emptor:

The legitimacy of journals cannot be confirmed by name or impact factor scores, and often not by promises made regarding peer-review standards…. Many predatory journals have credible and even inspiring names. They can also manufacture or manipulate impact factor scores and blatantly mislead regarding peer-review practices. [including ignoring reviews]

Caveat emptor:

Mindfulness, and more than a small degree of cynicism, is necessary to critically evaluate the legitimacy of any journal.

How you will be tempted to fail:

…getting through “peer review” with no more than trivial editorial comments may seem reasonable for the person or team thinking that their words are gold.

Being invited to review may also confer an aura of legitimacy. Such events could result in additional manuscripts being submitted to the same journal.

Stop being cats:

The inherently independent nature of researchers can lead to avoidance of conversations regarding research publication. [Discuss concerns and establish institutional guidelines to avoid being trapped by predatory journals.]

For example: [breaking his sentences into bullets]

• Did a person or team publish in such a journal inadvertently or to get around research weaknesses?
• Should full (or any) credit be given for publications in journals found to be predatory?
• Should job candidates with a history of publication in predatory journals be considered?
• Should articles published in journals employing predatory practices count in tenure packages?
• What scrutiny of the effort of flagged authors is warranted?

It’s been awhile since I read Shapin, but I’m reminded of early scientific societies and the network of trust built up by personally recommending new members. At this point I can’t see submitting to a journal that isn’t already known to my colleagues and field. But replicability indices (here | here | here…) show even that is not enough - Pollock is right that your department and institution has to ask some hard questions.

Flu has nearly disappeared worldwide durin the COVID pandemic

The distancing measures to prevent COVID are super-effective against flu. (So COVID is remarkably contagious.)

Interview in Bulletin of the Atomic Scientists about his new book 📚 Restricted Data. A thoughtful, nuanced discussion. Here are just a few:

You learn that most of what they’re redacting is really boring.

Explicit information—information you can write down—by itself is rarely sufficient for these kinds of technologies. …That isn’t saying the secrets are worthless, but it is saying that they’re probably much lower value than our system believes them to be.

Once you peel back the layer of secrecy—even in the Eisenhower years—you don’t find a bunch of angry malcontented bureaucrats on the other side. You find rich discussions about what should and shouldn’t be released. You find differences of opinion, …

I was also surprised that so many aspects of the system that we’ve come to take for granted are really determined by a tiny number of people—maybe six or seven people.

HTT Bob Horn

Congratulations to Michael Gordon et al for their paper in PLoS One!

This paper combines the results of four previous replication market studies. Data & scripts are in the pooledmaRket R package.

Key points:

• Combined, it covers 103 replications in behavioral & social sciences.
• Markets were 73% accurate, surveys a bit less.
• p-values predict original findings, though not at the frequencies you’d expect.

Enough summarizing - it’s open access, go read it! 😀🔬📄

~ ~ ~

Coda

We used this knowledge in the Replication Markets project, but it took awhile to get into print, as it does.

It should be possible to get 80-90% accuracy:

• These were one-off markets - no feedback and no learning!
• A simple p-value model does nearly as well, with different predictions.
• Simple NLP models on the PDF of the paper do nearly as well, with different predictions.

Replication Markets probably did worse ☹️, but another team may have done better. TBD.

Sullivan compares COVID to AIDS, Camus, and . One person thinking through what vaccination & far fewer fatalities mean. When to stop masking? Will we? The thinking through is the point, but for summary, he concludes:

So get vaccinated. Then use reason. The point is to get back to normal life, not to perpetuate the damaging patterns of plague life. So take off your masks, if you want. Plan parties for vaccinated friends. Get your vacation plans ready. And stop the constant judging and moralizing of people with masks and those without. Summer is coming. Let’s celebrate it.

Individual thresholds may vary - but I endorse his call not to get attached to plague life, as many of Camus' villagers did.

Note: discusses condoms, and the interesting but imperfect analogy to face masks, esp. as the plague is beaten.

Ben Kuo saved a life using Google Earth and clear thinking.

www.washingtonpost.com/lifestyle…

An image of West Coast data science:

I keep climbing the leaderboard! … Life is pretty sweet. I pick up indoor rock climbing, sign up for wood working classes; I read Proust and books about espresso.

…Kaggle releases the final score. What an embarrassment! …Inevitably, I hike the Pacific Crest Trail and write a novel about it.

In competing w/o the data, a good discussion on fooling yourself using holdout sets.

I continue to be astonished by how much “Alvaro de Menard” reads. See his Q1 2021 links & reading summary.

I was going to write more, but finding this draft two months later, I think I’ll just post.

www.economist.com

A big reason spelling systems never seem to get overhauled in more liberal societies is that those in a position to change the rules have learned the old ones. Put another way, the type of folk who were once good at spelling bees now run the world.

www.economist.com

Neither are face-recognition systems or sentencing software bought by those who suffer because of their failures.

In this letter, Heidi Larson & David Broniatowski argue that,

vaccine hesitancy is not the same as being “anti-vaccine.”

And that conflating the two risks driving the hesitant towards the fringe. Before exploring that, pause to admire the term Emotional Epidemiology which they [adopt from physician Danielle Ofri.]

Anti-vaxx typically shifts goalposts, embraces other conspiracy theories, and pushes explicit agendas. (To say nothing of deliberate disinfo, or shifting among mutually inconsistent theories.)

But vaccine hesitancy is diverse, including:

• Safety concerns
• Individual or community history
• Questions about COVID-19 vaccines specifically

For example, you can be pro-vaxx and know that the replication crisis ⋅ probably ⋅ affects ⋅ big ⋅ clinical ⋅ trials, so you want to be sure about any particular new vaccine. Especially, say, one that set world speed records from research to production.

In this case after enough time without obvious problems, or a properly done study, a trusted person being convinced, or simply an increased risk of pandemic death, you’ll decide it’s safe enough. You may even be able to state those conditions ahead of time.

Finally, there’s the decision theorists who rationally decide to vaccinate only if others don’t. This is a moral problem of collective action, not a failure to understand. It’s why there are often laws about such things.

I have lost more time than I care to admit watching the volcano in Geldingsadalur. It’s fascinating.

~ ~ ~

~ ~ ~

~ ~ ~

~ ~ ~

I’m sorry to say I found this reflection by Adam Briggle almost entirely uninteresting. Of course we feel the absence of our tools. Is there really any philosophy to be done there?

In biographies and brands, after some clear examples Jacobs notes that many accusations of “factual errors” are really brand defenses:

When they said that Jacobs makes many factual errors, they weren’t even really making a truth claim, they were uttering a spell to ward off the stranger. They were placing me outside their Inner Ring.

~ ~ ~

In vendoring culture, I’m fascinated by the parallels he draws between vendoring code, and what Gene Luen Yang has done by incorporating DC Comics' earliest racist caricature into a new comic:

What Yang has done is moral repair through vendoring code &em; in this case ... cultural code. And note that Yang has not ... simply pointed to code created and maintained by someone else. ...he could only correct it by making it his own.

Had I read both, I still wouldn’t have made that connection. Spark.

~ ~ ~

Frictionless note-taking produces notes, but it doesn’t - for me - produce memory.

HTT Nuño Sempere’s January forecasting newsletter. And be sure to check out his marvelous Metaforecast service!

~ ~ ~

So: Metacululs and RootClaim give very different probabilities that COVID-19 originated in a lab (see earlier post summarizing Monk):

• Metaculus: 15% for Hubei lab origins (either accidental or deliberate) - median never went above 30% during the last year.
• RootClaim: 76% accidental release, and 2% deliberate, based on a (simplified?) Bayesian analysis.

Metaculus has ~3K forecasts on that question over the last year+, and over 260 comments, most well-informed. They’ve done well in COVID-19 forecasts vs. experts. (And famously one of their top forecasters nailed the pandemic in late January 2020, as Sempere reminds us.)

Rootclaim, as far as I can tell, begins with some crowdsourcing to formulate hypotheses, get initial probabilities, gather sources, and maybe to help set likelihoods. Then they do a Bayesian update. At one point they used full Bayesian networks. It seems this one treats each evidence-group as independent.

Both are heavily rationalist and Bayesian-friendly, and had access to each other’s forecasts. So the divergence is quite interesting - I wish I had time to dig into it some more.

The Excel row-limit inadvertently prevented contact tracing for two weeks in the UK, causing a natural experiment. Using outcome data the team estimated the gap caused 125,000 extra SARS-COV-2 cases and 1,500 COVID-19 deaths. Ouch.

ncrc.jhsph.edu/research/…

Sullivan on SAT

Killing the SAT Means Hurting Minorities

While I’m thinking of Sullivan, this from his newsletter, regarding a conversation on his podcast:

When you regard debate itself as a form of white supremacy, you tend not to be very good at it.

Psychic AI

FollowTheArgument on Inferring Political Orientation From a Single Picture reviews a recent paper showing algorithms can do this an astonishing 70% of the time, beating humans (55%) and a 100-item personality inventory (66%).

It’s still possible it’s latched onto some irrelevant or fragile feature set, but (a) they cropped close to the face, (b) they controlled for age/sex/race, (c) they cross-validated, and (d) they tested models from facebook on photos from other sites, and verce visa. Given the 2,048 features extracted by the facial image classifier, even regression did as well.

It seems ~60% is easily-named transient features like head-tilt, facial expression, etc. Leaving the rest as-yet-unnamed.

It’s tempting to go to physiognomy, but consider the following things humans do specifically to signal their affiliations:

• Makeup amount and style
• Beard type
• Earrings, other jewelry
• Hair style

What’s fascinating is how poorly humans do. But then, that result might be very different if the subjects were trained cold-readers.

Republicans embracing “class”

FTA also covers Scott Alexander’s Modes Proposal. Good read.

Muse of Seuss

I’ve been musing about all the Seussing. Social media discussions, and reading the spectrum on Flip Side (recommended).

I’m not especially qualified, but it seems to me:

• Liberals are right the board’s decision probably isn’t cancellation. (It depends on motivation, which we’re likely to get wrong.)
• But eBay’s action is cancellation. It’s illiberal, and wrong. It’s also downright silly given what eBay still sells.
• Conservatives make a strong case that most of the alleged offenses really weren’t – see Cathy Young’s take for why anyone thought so. If I Ran the Zoo being a notable exception.

Worth remembering:

“Among the people I work with in medical schools, there’s a strong adherence to what they call the health-literacy model, which is that physicians really believe that if you tell people the right information they will make decisions for themselves you yourself would make. And that’s an incorrect assumption.”

The New Yorker

Despite the title, I found the article reassuring.

Key quotes from this post:

Snell:

incompatibility comes for every abandoned app eventually

Rendle:

on the web we don’t really own any space, we’re just borrowing land temporarily.

Jacobs:

My data I own, my internet presence I rent.

it’s quite likely that I will outlive all my work, and I’m just fine with that. So I’ll write in venues that give me pleasure,

Worth reading in entirety, Alan Jacobs reflects on an essay by Douthat:

Such a system, predictably, was terrible at generating the kind of outward-facing, evangelistic conservatives who had made the Reagan revolution possible.

He ends by saying Antonin Scalia was the sole survivor of the Old Republic, making Amy Coney Barret the Last Jedi.

(OK, Douthat doesn’t use Star Wars. But Sunstein did. I just blended them.)

Where Douthat muses on Trump’s loss, Ian Leslie reflected on Biden’s victory over both Sanders and Trump:

It's worth spending a bit of time on what it means to be moderate.

Turns out Leslie has a new book, Conflicted:

Disagreement is the best way of thinking we have. It weeds out weak arguments, improves decision-making, leads to new ideas, and, counter-intuitively, brings us closer to one another. But only if we do it well - and right now, we’re doing it terribly.

Looking forward to it. Mercier & Sperber demonstrated that disagreement is how groups think, and that under the right conditions, they vastly outthink people.

For some gentle advice on How to Think, try Alan Jacobs' book by that name. We all cover the philosophy & cognitive science. Jacobs tackles the human component:

[We describe] argument as war ... because in many arguments there truly is something to be lost, and most often what's under threat is social affiliation. [My bold.]

What to do?

6. Gravitate, as best you can, in every way you can, toward people who seem to value genuine community and can handle disagreement with equanimity.

Slogan on an ad for scrum training. Featured in the the 2015 talk “Agile is Dead”.

Dave Thomas shows what goes wrong when “Agile” becomes a proper noun, and talks about reclaiming agility.

• Find out where you are.

• Take a small step towards your goal.

• Adjust your understanding based on what you learned.

• Repeat.

• [All things equal], take the path that makes future change easier.

Gelman’s recent short post on Relevance of Bad Science for Good Science includes a handy Top10 junk list:

A Ted talkin’ sleep researcher misrepresenting the literature or just plain making things up; a controversial sociologist drawing sexist conclusions from surveys of N=3000 where N=300,000 would be needed; a disgraced primatologist who wouldn’t share his data; a celebrity researcher in eating behavior who published purportedly empirical papers corresponding to no possible empirical data; an Excel error that may have influenced national economic policy; an iffy study that claimed to find that North Korea was more democratic than North Carolina; a claim, unsupported by data, that subliminal smiley faces could massively shift attitudes on immigration; various noise-shuffling statistical methods that just won’t go away—all of these, and more, represent different extremes of junk science.

And the following sobering reminder why we study failures:

None of us do all these things, and many of us try to do none of these things—but I think that most of us do some of these things much of the time.