➜ Re‑run with --mode all after adding an API key

<h1 align="center"><img src="assets/icon/opencompass.png" alt="OpenCompass" height="50" style="vertical-align:middle;" />&nbsp;SciEval ToolKit</h1>

<p align="center"><strong>
A unified evaluation toolkit and leaderboard for rigorously assessing the scientific intelligence of large language and vision–language models across the full research workflow.
</strong></p>

<hr style="width:100%;margin:16px 0;border:0;border-top:0.1px solid #d0d7de;" />

<div align="center">

[![Tutorial](https://img.shields.io/badge/Tutorial-SciEval-b8dcff?style=for-the-badge&logo=google-chrome&logoColor=white)](https://scievalkit-docs.readthedocs.io/en/latest)&#160;
[![Leaderboard](https://img.shields.io/badge/LEADERBOARD-Scieval-f6e58d?style=for-the-badge&logo=huggingface)](https://opencompass.org.cn/Intern-Discovery-Eval/rank)&#160;
[![Report](https://img.shields.io/badge/REPORT-Technical-f4c2d7?style=for-the-badge&logo=arxiv)](https://arxiv.org/abs/2512.22334)&#160;
[![GitHub](https://img.shields.io/badge/GitHub-Repository-c7b9e2?style=for-the-badge&logo=github&logoColor=white)](https://github.com/InternScience/SciEvalKit)

<img src="assets/icon/welcome.png" alt="welcome" height="24" style="vertical-align:middle;" />
&nbsp;Welcome to the official repository of <strong>SciEval</strong>!

<div align="center">
  <img src="assets/SciEvalKit.png" alt="SciEval capability radar" width="90%">
</div>

</div>

## <img src="assets/icon/why.png" alt="why" height="28" style="vertical-align:middle;" />&nbsp;Why SciEval?

**SciEval** is an open‑source evaluation framework and leaderboard aimed at measuring the **scientific intelligence** of large language and vision–language models.  
Although modern frontier models often achieve *~90* on general‑purpose benchmarks, their performance drops sharply on rigorous, domain‑specific scientific tasks—revealing a persistent **general‑versus‑scientific gap** that motivates the need for SciEval.
Its design is shaped by following core ideas:

- **Beyond general‑purpose benchmarks ▸** Traditional evaluations focus on surface‑level correctness or broad‑domain reasoning, hiding models’ weaknesses in realistic scientific problem solving.  SciEval makes this **general‑versus‑scientific gap** explicit and supplies the evaluation infrastructure needed to guide the integration of broad instruction‑tuned abilities with specialised skills in coding, symbolic reasoning and diagram understanding.
- **End‑to‑end workflow coverage ▸** SciEval spans the full research pipeline—such as **image interpretation, symbolic reasoning, executable code generation, and hypothesis generation**—instead of isolated subtasks.  
- **Capability‑oriented & reproducible ▸** A unified toolkit for **dataset construction, prompt engineering, inference, and expert‑aligned scoring** ensures transparent and repeatable comparisons.  
- **Grounded in real scenarios ▸** Benchmarks use domain‑specific data and tasks so performance reflects **actual scientific practice**, not synthetic proxies.

For a detailed and systematic introduction to SciEvalKit, please refer to the [SciEvalKit Tutorial](https://scievalkit-docs.readthedocs.io/en/latest).


## <img src="assets/icon/progress.png" alt="progress" height="28" style="vertical-align:middle;" />&nbsp;Progress in Scientific Intelligence

*Realtime updates — scores are synchronized with the [Intern‑Discovery‑Eval](https://opencompass.org.cn/Intern-Discovery-Eval/rank) leaderboard.*

<div align="center">
  <img src="assets/general_scientific_comparison.png" alt="SciEval capability radar" width="100%">
</div>


- **General benchmarks overestimate scientific competence.** Even the strongest frontier models (e.g., **Gemini 3 Pro**) score below **60** on **Scientific Text Capability** , despite scoring near *90* on widely used general‑purpose benchmarks.
- **Multimodal capability is breaking the 60‑point barrier.** **Gemini 3 Pro** leads **Scientific Multimodal Capability** with **62.88**, reflecting strong performance in multimodal perception and reasoning.
- **Open‑source systems are rapidly closing the gap.** *Qwen3‑VL‑235B‑A22B* and *Qwen3‑Max* now match or surpass several proprietary models in symbolic reasoning and code generation, signalling healthy community progress.
- **Symbolic reasoning and code generation remain bottlenecks.** No model exceeds **50** in equation‑level manipulation or **30** in end‑to‑end executable code tasks, indicating that scientific workflows requiring programmatic pipelines still fail frequently.


## <img src="assets/icon/key.png" alt="key" height="28" style="vertical-align:middle;" />&nbsp;Key Features

<div align="center">
  <img src="assets/radar.png" alt="SciEval capability radar" width="70%">
</div>


| Category                                  | Highlights                                                                                                                                                       |
| ----------------------------------------- | --------------

... (truncated)