Made a Github awesome-list about AI evals, looking for contributions and feedback


As AI grows in popularity, evaluating reliability in a production environments will only become more important.

Saw a some general lists and resources that explore it from a research / academic perspective, but lately as I build I've become more interested in what is being used to ship real software.

Seems like a nascent area, but crucial in making sure these LLMs & agents aren't lying to our end users.

Looking for contributions, feedback and tool / platform recommendations for what has been working for you in the field

Leave a Reply