Thoughts as a social science research comparing ChatpGPT 5.1 (thinking) Gemini 3 Pro, and NotebookLM,

I just spent about 1.5 hours doing an in-depth exploration comparing the 3 models for something I do that is highly valuable as a researcher. I uploaded a long socio-legal academic article i know well, asked it to disentangle a few concepts and give me an exhaustive list of all the examples it could find. I gave the same prompt to ChatGPT 5.1 (thinking), Gemini 3 Pro and NotebookLM. ChatpGPT took 11 minutes and missed a very big domain of items. Gemini did a very thorough job and so did NotebookLM (both in under a minute). I think Gemini did a very slightly better job (which is interesting given this is NotebookLM's domain).

I then uploaded my dissertation, which covers the same topic and asked it to identify additional items the previous article missed out on. I think NotebookLM edged out Gemini here and ChatGPT still missed out on a major category. I then asked all three how best to categorize all the examples it found in a way that helped me to exemplify how a main concept can be disentangled. ChatGPT gave me way too many categories and they were far too abstract and useless. Gemini did great and NotebookLM did pretty well also.

I ended the shared prompt then and asked ChatGPT why it didn't include things it missed. There appears to have been some confusion on how it was defining a term that I asked it to search for. But this is odd, because I literally wrote a whole dissertation about it and it should be able to tell from my examples how I define the term. Also, this is typically something ChatGPT is very good at, inferring meaning (or has been for me).

Sidenote: I now see me entering a problematic area that has plagued me for years with messaging. Trying to find conversations and not remembers which model I used. 🙁 It's the same when looking for something in email, messages, messenger, twitter, etc. One day I'll need to find a local solution to being able to search all models at the same time to find past information.

Leave a Reply