LiveCodeBench

Website: https://livecodebench.github.io/leaderboard.html

Research Paper: https://arxiv.org/abs/2403.07974

LiveCodeBench is an evaluation framework to assess an LLM’s coding ability.

“LiveCodeBench provides holistic and contamination-free evaluation of coding capabilities of LLMs. Particularly, LiveCodeBench continuously collects new problems over time from contests across three competition platforms — LeetCode, AtCoder, and CodeForces. Next,

 

LiveCodeBench also focuses on a broader range of code-related capabilities, such as self-repair, code execution, and test output prediction, beyond just code generation. Currently,

 

LiveCodeBench hosts four hundred high-quality coding problems that were published between May 2023 and March 2024.”