So there is a clear result found and then post hoc there is an explanation made for why this makes sense. That's a bit of a red flag since you preferably hypothesize first, but let's look at the analysis. If I'm not mistaken, then these are the most important parts:
- 467 measured metabolites (features)
- 136 males and 69 females = 205 (data points)
- "To minimize overfitting, only correlations with q values < 0.05 were tallied."
Then the predictive performance in Fig. 3 looks very promising with a 0.88 AUC score. However, according to the methods: "Classifiers of 4–7 metabolites were selected and tested for diagnostic accuracy using area under the receiver operator characteristic curve and random forest analysis."
Then, those AUC scores do not provide so much value since they first used the data to select the best features and then used the same data to fit the model on the best features.
It also looks a bit like the authors have thrown the kitchen sink at these 200 samples since I see R, GraphPad Prism, Python, CIRCOS, Cytoscape, and MetaboAnalyst being used.
And the statement "Fifty (50) random samples at each subsample size were taken to estimate the population statistics based on the central limit theorem." is a bit weird because the central limit theorem shows up throughout many analyses including confidence intervals.
So all in all I'm not 100% sure, but I am skeptical. Anyone here who knows more about "Metabolic network and hub-and-spoke analysis" or "network growth" analysis? Or why it is reasonable to use so many metrics instead of being more selective?
> So there is a clear result found and then post hoc there is an explanation made for why this makes sense. That's a bit of a red flag since you preferably hypothesize first, but let's look at the analysis.
Eh, this doesn’t look to be a P-hacking attempt. Scientific exploration doesn’t always hit the hypothesis first model. Ideally this paper should be a source of future specific hypothesis and experiments. Though, IMHO, their implicit hypothesis is that metabolism is a core feature of ASD.
The points about the predictive power in fig 3 are interesting.
Better to read the paper instead of the pop journals. Most writers don't understand the issues fully and make mistakes, or sensationalize the discovery.
I’ve only ever seen “this ^” refer to the parent comment, not the one directly above it as you mention. So as long as the comment hierarchy is maintained (which it is on HN) then the reference is clear
"Of 50 biochemical pathways and 450 polar and lipid metabolites examined, the developmental regulation of the purine network was most changed. Purine network hub analysis revealed a 17-fold reversal in typically developing children. This purine network reversal did not occur in ASD."
That's a strong statement, and one which can be checked by others. Anyone trying to replicate this yet?
The article was published today, and study like that take months to do and then months again to be published especially in this kind of journals (thankfully they don't accept replications most of the time so it can get published by more reasonable venues).
The paper is a super fascinating read, and lines up with several key features I've noticed in both of my professionally diagnosed autistic children (good luck getting access to a competent diagnostician who understands how the ASD criteria manifest in passing adults). It's really enlightening to see the association between Xanthine, one of the end-stage products of eATP metabolization, and the anxiety that is so prevalent within my own family.
From the Discussion[1] section of the paper:
> These self-calming connections in metabolism failed to develop in ASD. The natural consequence of the loss of these metabolic safeguards to overexcitation is for children with ASD to seek sameness to avoid the anxiety produced by change91, and to be more sensitive to environmental changes across many sensory domains.
And a bit further on:
> In the current study, xanthine was the purine that gained the most stimulatory (+r) correlations in 5-year-olds with ASD. Xanthine is one of the end-products of eATP metabolism97. Xanthine is known to trigger a cascade of events that leads to mitochondrial network fragmentation, reactive oxygen species and reactive nitrogen species (ROS and RNS), eicosanoid (e.g., leukotriene, HETE, and prostaglandin) signaling, immune activation, anxiety-associated behaviors, and consolidates long-term aversive memories that make the animal hypersensitive to future environmental changes that warn of environmental danger, cause fear, and trigger anxiety in mice, and is elevated in the blood of adults with anxiety disorders98. Anxiety is a common but under-recognized problem in autism99.
Another really important observation:
> A major result of this research was that the developmental differences observed in ASD were not the result of an increase or decrease of one causal metabolite, or an isolated change in the gut-brain axis, or neuroendocrine, autonomic, cytokine, or immunologic circuits. Instead, it was the interconnectedness and developmental state of the metabolic network that underlies all these systems that was fundamentally changed.
In the last year, the All Brains Belong VT organization has been working on a collection of information they call "All The Things" [2], which lines up with the paper's observation that there is an underlying metabolic network at play. As a side note, All Brains Belong VT is a fantastic organization that focuses heavily on validation and support in a healthcare industry which often feels incredibly invalidating for individuals who have been reporting symptoms across a wide variety of siloed physician specialties.
