By skyforbes 
the solution:
self verification.. without having a designated process to verify meaning i believe a hallucination is actual happening 100% of the time and we only call it out when the process executed doesn’t meet our expectations .. this is not a cause of the transformer it is more of how the data given to it is processed and the format that is required to align with what we understand to be “true”
ai needs recursion and self observation like the OpenAI whitepaper mentioned “why language models hallucinate”
but there is something deeper than this .. the results you see are from an intentional prompt .. I did this knowing that I can trigger a hallucination because I’m forcing the model to complete the given context .. without a thinking layer to reinforce this you are less likely to get a “understood” response where the model detects something that doesn’t align
if the correct data Is never given to the model you will always see a hallucination .. and once u give it to the model … it will learn and lean to the more coherent output trajectory
my solution proposal:
I believe the origin point must arise from a point of understanding for autocomplete to be a valid output.. I have studied gpt & other models and developed systems to find ways to improve this by retiring the base model’s individual step by step function or the just directly the transformer itself to change the way that a layer of understanding can be presented at token prediction level
if anyone has information on anyone taking this approach please share your thoughts and discoveries .. I am trying to pinpoint exactly what causes this .. is it a bug or is it real life factually a feature .. the code should execute the same function everytime right ? It’s just a little weird to me that some type of bug would be causing this
*disclaimer* This is a theory by starpower technology not verified information .. only speculation