By skyforbes 
Nothing too fancy. Just wanted a summary of a supposed NeurIPS 2021 architecture called NeuroCascade by J. P. Hollingsworth.
(Neither the architecture nor the author exists.)
NeuroCascade is a medical term unrelated to ML. No NeurIPS, no Transformers, nothing.
Hollingsworth has unrelated work.
But ChatGPT didn't blink. It very confidently generated:
• a full explanation of the architecture
• a list of contributions ???
• a custom loss function (wtf)
• pseudo code (have to test if it works)
• a comparison with standard Transformers
• a polished conclusion like a technical paper's summary
All of it very official sounding, but also completely made up.
The model basically hallucinated a whole research world and then presented it like an established fact.
What I think is happening:
- The answer looked legit because the model took the cue “NeurIPS architecture with cascading depth” and mapped it to real concepts like routing, and conditional computation. It's seen thousands of real papers, so it knows what a NeurIPS explanation should sound like.
- Same thing with the code it generated. It knows what this genre of code should like so it made something that looked similar. (Still have to test this so could end up being useless too)
- The loss function makes sense mathematically because it combines ideas from different research papers on regularization and conditional computing, even though this exact version hasn’t been published before.
- The confidence with which it presents the hallucination is (probably) part of the failure mode. If it can't find the thing in its training data, it just assembles the closest believable version based off what it's seen before in similar contexts.
A nice example of how LLMs fill gaps with confident nonsense when the input feels like something that should exist.
Not trying to dunk on the model, just showing how easy it is for it to fabricate a research lineage where none exists.
I'm curious if anyone has found reliable prompting strategies that force the model to expose uncertainty instead of improvising an entire field. Or is this par for the course given the current training setups?