# Evaluating LLMs and Potential Pitfalls

Intro to AI-Driven Science on Supercomputers @ ALCF 2024

**Contact:** Marieme Ngom ([mngom@anl.gov](mailto:///mngom@anl.gov)), Bethany Lusch ([blusch@anl.gov](mailto:///blusch@anl.gov)), Sandeep Madireddy  ([smadireddy@anl.gov](mailto:///smadireddy@anl.gov)) 


[Overview of LLMs Evaluation](https://github.com/argonne-lcf/ai-science-training-series/blob/main/08_Evaluating_LLMs/LLM_Evaluation_Overview.pdf)

[Potential Pitfalls of LLMs](https://github.com/argonne-lcf/ai-science-training-series/blob/main/08_Evaluating_LLMs/LLM-Pitfalls.pdf)
    
[Link to breakout rooms forms](https://drive.google.com/drive/folders/1BN_aBlNU-7KVIcySntRtbkBXRGpkMSyz)

Other helpful links:
- [OpenAI tokenizer](https://platform.openai.com/tokenizer)
- [Chatbot Arena](https://chat.lmsys.org/)
- [Chatbot Guardrails Arena](https://huggingface.co/spaces/lighthouzai/guardrails-arena)

 
 **Homework**
 
What do you think is a particularly good use case for LLMs for science? How would you evaluate it?
Your answer does not need to be in paragraphs. When you submit your homework form, you can link to a file in your Github repo where you wrote your answer.
