Turns out fact checking is hard and costs money. What's surprising is that Meta, which has capable staff and lots of money, can't seem to be bothered to do it properly.
I think what I wanted to say is that it seems to me that a) the resources Meta throws at this are not sufficient to check facts properly, and b) that proper fact checking is a really difficult business.
a) because if they had used staff with minimal domain knowledge they would probably have been aware what BMJ is, that it is a trusted scientific journal that has peer-review in place. I don't know whether the article in question was peer reviewed, but I assume so. Either way, this didn't seem to play a role. Whoever fact-checked the article labeled BMJ a newsblog, so they have near-zero domain knowledge.
b) BMJ and all scientific journals which are worth their money employ experts to do the fact checking in peer review. Funny enough, these experts mainly do that work for free. Turns out that the experts place high value in fact-checked and verified information. Clearly, peer review takes a long time and wouldn't be suitable for facebook's quick turnaround. Or perhaps it would... if Meta designed a peer review process that takes Facebook's specific requirements into account. But that's a whole new approach to things, perhaps better developed in a startup and then bought out by Meta ;)
Not that peer-review is perfect, or that all experts are always right. But Meta needs to acknowledge that fact checkers are sometimes wrong, too. When that happens, there must be a way to correct the error. The BMJ having to write an open letter to Zuckerberg clearly indicates that this error correction doesn't work.
The original BMJ article doesn't draw any conclusions that I can see. It certainly doesn't suggest the Pfizer vaccine is unsafe or ineffective, as the "fact check" heavily implies.
People on Facebook aren't actually reading the journal. They're looking at the title and saying "see, the vaccine was unsafe all along." The fact checkers said "Missing context ... Independent fact-checkers say this information could mislead people" which is absolutely true.
My bet would be they're not willing to pay for the actual expertise they need. I bet the fact-checkers FB employs don't have the qualifications those of us who fact-check academic publications have; the sheer amount of information FB has to wade through tells me that they need a lot of people, and I have an inkling that they would really resist paying for non-tech expertise that doesn't immediately lead to $$$$.
I also wouldn't be surprised if there were perverse incentives for the fact-checkers: Some claims take longer than others to evaluate, but I bet they're on some sort of metric based system. Or that 'I don't know' or 'it's unverifiable' aren't good enough answers even though that happens.
I also don't know if FB could GET that expertise at this point: I view big tech more kindly than most information specialists I know and I wouldn't work for FB even for six-figures.
Brand new facts generally go through a public disclosure process where some established institution (a news outlet, academic journal, whatever) is willing to vouch for it as true, or at least vouch for it as being worthy of consideration and discussion— and in any case, will strive to present it with the appropriate context and framing as far as level of certainty, who the players are and what their motivations are, etc etc.
I think it's reasonably fair for the policy for nobodies posting on social media to be that "new facts" (or "original research" in Wikipedia parlance), particularly those which may be harmful to public safety or marginalized groups should be either unpostable, or considerably reduced in how far they're able to spread organically (shadow-banning).
If you don't like it, go to some other social media network. For my part, I'd be content with a Facebook feed where none of the posts I'm seeing are "new" facts. OTOH, I'd also be content with a FB feed that was exclusively pics of my friends and their kids and very little news or politics at all.
In this case I assume the BMJ article was peer reviewed, so it was already fact-checked by knowledgeable individuals. BMJ is a reputable journal that can be relied to fact-check their articles. The fact that it was labeled a newsblog means that the fact-checkers really had no domain knowledge about the facts they were supposed to check.
Expertise isn't a cure all. An example is the famous Minux/Linux fight. Tanenbaum was a distinguished professor, arguing with some guy on the internet who built a kernel. "Is a monokernel a viable way to build a first rate operating system?"
Fact Check: operating system researchers widely believe that monolithic kernels are a fundamentally flawed way to build a system...
The premise that there are some specially endowed people who, by virtue of their credentials, are the only ones able to mark things as true sounds like the core tenants of a religion, not a secular society!
Well, I wasn't saying that there should be specially endowed people, I was asking about the reverse: how can someone who is not informed fact-check someone who is?
Or to make it less personal, how can someone fact-check information they aren't knowledgeable about?
Tanenbaum was in fact right. Monolithic kernel design has caused no end of problems, which is why modern OS's (including Linux!) are moving a lot into userspace. See Windows moving font parsing out of the kernel, macOS DriverKit, Linux uio, etc.
> While I could go into a long story here about the relative merits of the
two designs, suffice it to say that among the people who actually design
operating systems, the debate is essentially over. Microkernels have won.
Create a bureaucracy that will still largely be corrupt (from one side or there other) and get things massively wrong?
I'm actually more for community tools to do moderation correctly and for content to be moderated by "people I trust" within the community. So for example if someone who I've marked as trustworthy on science reporting marks a science story as fake news it's flagged for those of us who have marked the flagger as an authority.
People will say you are just encouraging echo chambers like this but I actually think it's inevitable and we all live in echo chambers. Having your views challenged is extremely hard work that we don't always have time or the inclination to work through day to day. Everyone is watching a different movie.
I would start with having knowledgeable people doing the fact checking. Next, since it will not always be possible to get every fact check right the first time, allow for fact checking results to be revised. Exactly what BMJ asks for.
