书生·浦语2-chat-1.8b
介绍
书生·浦语-1.8B (InternLM2-1.8B) 是第二代浦语模型系列的18亿参数版本。为了方便用户使用和研究，书生·浦语-1.8B (InternLM2-1.8B) 共有三个版本的开源模型，他们分别是：
- InternLM2-1.8B: 具有高质量和高适应灵活性的基础模型，为下游深度适应提供了良好的起点。
- InternLM2-Chat-1.8B-SFT：在 InternLM2-1.8B 上进行监督微调 (SFT) 后得到的对话模型。
- InternLM2-Chat-1.8B：通过在线 RLHF 在 InternLM2-Chat-1.8B-SFT 之上进一步对齐。 InternLM2-Chat-1.8B表现出更好的指令跟随、聊天体验和函数调用，推荐下游应用程序使用。
InternLM2 模型具备以下的技术特点
- 有效支持20万字超长上下文：模型在20万字长输入中几乎完美地实现长文“大海捞针”，而且在 LongBench 和 L-Eval 等长文任务中的表现也达到开源模型中的领先水平。
- 综合性能全面提升：各能力维度相比上一代模型全面进步，在推理、数学、代码等方面的能力提升显著。
体验过程
体验使用https://studio.intern-ai.org.cn/进行体验
准备环境
拉取环境
/root/share/install_conda_env_internlm_base.sh internlm-demo # 执行该脚本文件来安装项目实验环境
激活环境
conda activate internlm-demo
安装运行 demo 所需要的依赖
# 升级pip
python -m pip install --upgrade pip
pip install modelscope==1.9.5
pip install transformers==4.35.2
pip install streamlit==1.24.0
pip install sentencepiece==0.1.99
pip install accelerate==0.24.1
模型下载
从魔塔社区下载模型
文件名:download_model.py
import torch
from modelscope import snapshot_download, AutoModel, AutoTokenizer
import os
model_dir = snapshot_download('jayhust/internlm2-chat-1_8b', cache_dir='/root/model', revision='master')
模型会下载到/root/model文件夹下，完整路径：/root/model/jayhust/internlm2-chat-1_8b
克隆InternLM代码
mkdir -p /root/code
cd /root/code
git clone https://gitee.com/internlm/InternLM.git
cd InternLM
运行
使用终端体验代码
import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
model_name_or_path = "/root/model/jayhust/internlm2-chat-1_8b"
tokenizer = AutoTokenizer.from_pretrained(model_name_or_path, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(model_name_or_path, trust_remote_code=True, torch_dtype=torch.bfloat16, device_map='auto')
model = model.eval()
system_prompt = """You are an AI assistant whose name is InternLM (书生·浦语).
- InternLM (书生·浦语) is a conversational language model that is developed by Shanghai AI Laboratory (上海人工智能实验室). It is designed to be helpful, honest, and harmless.
- InternLM (书生·浦语) can understand and communicate fluently in the language chosen by the user such as English and 中文.
"""
messages = [(system_prompt, '')]
print("=============Welcome to InternLM chatbot, type 'exit' to exit.=============")
while True:
input_text = input("User >>> ")
input_text = input_text.replace(' ', '')
if input_text == "exit":
break
response, history = model.chat(tokenizer, input_text, history=messages)
messages.append((input_text, response))
print(f"robot >>> {response}")
运行结果：
OpenCompass模型评测
准备环境
安装conda环境
conda create --name opencompass --clone=/root/share/conda_envs/internlm-base
conda activate opencompass
git clone https://github.com/open-compass/opencompass
cd opencompass
pip install -e .
安装依赖
pip install protobuf
数据准备
cp /share/temp/datasets/OpenCompassData-core-20231110.zip /root/opencompass/
unzip OpenCompassData-core-20231110.zip
启动评测
python run.py --datasets ceval_gen --hf-path /root/model/jayhust/internlm2-chat-1_8b --tokenizer-path /root/model/jayhust/internlm2-chat-1_8b --tokenizer-kwargs padding_side='left' truncation='left' trust_remote_code=True --model-kwargs trust_remote_code=True device_map='auto' --max-seq-len 2048 --max-out-len 16 --batch-size 4 --num-gpus 1 --debug
评测结果:
20240302_122143
tabulate format
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
dataset version metric mode opencompass.models.huggingface.HuggingFace_jayhust_internlm2-chat-1_8b
---------------------------------------------- --------- ------------- ------ ------------------------------------------------------------------------
ceval-computer_network db9ce2 accuracy gen 26.32
ceval-operating_system 1c2571 accuracy gen 36.84
ceval-computer_architecture a74dad accuracy gen 14.29
ceval-college_programming 4ca32a accuracy gen 13.51
ceval-college_physics 963fa8 accuracy gen 36.84
ceval-college_chemistry e78857 accuracy gen 12.5
ceval-advanced_mathematics ce03e2 accuracy gen 10.53
ceval-probability_and_statistics 65e812 accuracy gen 33.33
ceval-discrete_mathematics e894ae accuracy gen 18.75
ceval-electrical_engineer ae42b9 accuracy gen 27.03
ceval-metrology_engineer ee34ea accuracy gen 50
ceval-high_school_mathematics 1dc5bf accuracy gen 5.56
ceval-high_school_physics adf25f accuracy gen 31.58
ceval-high_school_chemistry 2ed27f accuracy gen 42.11
ceval-high_school_biology 8e2b9a accuracy gen 15.79
ceval-middle_school_mathematics bee8d5 accuracy gen 15.79
ceval-middle_school_biology 86817c accuracy gen 57.14
ceval-middle_school_physics 8accf6 accuracy gen 36.84
ceval-middle_school_chemistry 167a15 accuracy gen 60
ceval-veterinary_medicine b4e08d accuracy gen 43.48
ceval-college_economics f3f4e6 accuracy gen 43.64
ceval-business_administration c1614e accuracy gen 36.36
ceval-marxism cf874c accuracy gen 78.95
ceval-mao_zedong_thought 51c7a4 accuracy gen 45.83
ceval-education_science 591fee accuracy gen 68.97
ceval-teacher_qualification 4e4ced accuracy gen 68.18
ceval-high_school_politics 5c0de2 accuracy gen 47.37
ceval-high_school_geography 865461 accuracy gen 36.84
ceval-middle_school_politics 5be3e7 accuracy gen 57.14
ceval-middle_school_geography 8a63be accuracy gen 58.33
ceval-modern_chinese_history fc01af accuracy gen 60.87
ceval-ideological_and_moral_cultivation a2aa4a accuracy gen 68.42
ceval-logic f5b022 accuracy gen 31.82
ceval-law a110a1 accuracy gen 20.83
ceval-chinese_language_and_literature 0f8b68 accuracy gen 17.39
ceval-art_studies 2a1300 accuracy gen 51.52
ceval-professional_tour_guide 4e673e accuracy gen 62.07
ceval-legal_professional ce8787 accuracy gen 30.43
ceval-high_school_chinese 315705 accuracy gen 21.05
ceval-high_school_history 7eb30a accuracy gen 70
ceval-middle_school_history 48ab4a accuracy gen 72.73
ceval-civil_servant 87d061 accuracy gen 38.3
ceval-sports_science 70f27b accuracy gen 47.37
ceval-plant_protection 8941f9 accuracy gen 45.45
ceval-basic_medicine c409d6 accuracy gen 57.89
ceval-clinical_medicine 49e82d accuracy gen 40.91
ceval-urban_and_rural_planner 95b885 accuracy gen 45.65
ceval-accountant 002837 accuracy gen 38.78
ceval-fire_engineer bc23f5 accuracy gen 38.71
ceval-environmental_impact_assessment_engineer c64e2d accuracy gen 38.71
ceval-tax_accountant 3a5e3c accuracy gen 32.65
ceval-physician 6e277d accuracy gen 38.78
ceval-stem - naive_average gen 29.41
ceval-social-science - naive_average gen 54.16
ceval-humanities - naive_average gen 46.1
ceval-other - naive_average gen 42.11
ceval-hard - naive_average gen 23.9
ceval - naive_average gen 40.39
$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
-------------------------------------------------------------------------------------------------------------------------------- THIS IS A DIVIDER --------------------------------------------------------------------------------------------------------------------------------
csv format
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
dataset,version,metric,mode,opencompass.models.huggingface.HuggingFace_jayhust_internlm2-chat-1_8b
ceval-computer_network,db9ce2,accuracy,gen,26.32
ceval-operating_system,1c2571,accuracy,gen,36.84
ceval-computer_architecture,a74dad,accuracy,gen,14.29
ceval-college_programming,4ca32a,accuracy,gen,13.51
ceval-college_physics,963fa8,accuracy,gen,36.84
ceval-college_chemistry,e78857,accuracy,gen,12.50
ceval-advanced_mathematics,ce03e2,accuracy,gen,10.53
ceval-probability_and_statistics,65e812,accuracy,gen,33.33
ceval-discrete_mathematics,e894ae,accuracy,gen,18.75
ceval-electrical_engineer,ae42b9,accuracy,gen,27.03
ceval-metrology_engineer,ee34ea,accuracy,gen,50.00
ceval-high_school_mathematics,1dc5bf,accuracy,gen,5.56
ceval-high_school_physics,adf25f,accuracy,gen,31.58
ceval-high_school_chemistry,2ed27f,accuracy,gen,42.11
ceval-high_school_biology,8e2b9a,accuracy,gen,15.79
ceval-middle_school_mathematics,bee8d5,accuracy,gen,15.79
ceval-middle_school_biology,86817c,accuracy,gen,57.14
ceval-middle_school_physics,8accf6,accuracy,gen,36.84
ceval-middle_school_chemistry,167a15,accuracy,gen,60.00
ceval-veterinary_medicine,b4e08d,accuracy,gen,43.48
ceval-college_economics,f3f4e6,accuracy,gen,43.64
ceval-business_administration,c1614e,accuracy,gen,36.36
ceval-marxism,cf874c,accuracy,gen,78.95
ceval-mao_zedong_thought,51c7a4,accuracy,gen,45.83
ceval-education_science,591fee,accuracy,gen,68.97
ceval-teacher_qualification,4e4ced,accuracy,gen,68.18
ceval-high_school_politics,5c0de2,accuracy,gen,47.37
ceval-high_school_geography,865461,accuracy,gen,36.84
ceval-middle_school_politics,5be3e7,accuracy,gen,57.14
ceval-middle_school_geography,8a63be,accuracy,gen,58.33
ceval-modern_chinese_history,fc01af,accuracy,gen,60.87
ceval-ideological_and_moral_cultivation,a2aa4a,accuracy,gen,68.42
ceval-logic,f5b022,accuracy,gen,31.82
ceval-law,a110a1,accuracy,gen,20.83
ceval-chinese_language_and_literature,0f8b68,accuracy,gen,17.39
ceval-art_studies,2a1300,accuracy,gen,51.52
ceval-professional_tour_guide,4e673e,accuracy,gen,62.07
ceval-legal_professional,ce8787,accuracy,gen,30.43
ceval-high_school_chinese,315705,accuracy,gen,21.05
ceval-high_school_history,7eb30a,accuracy,gen,70.00
ceval-middle_school_history,48ab4a,accuracy,gen,72.73
ceval-civil_servant,87d061,accuracy,gen,38.30
ceval-sports_science,70f27b,accuracy,gen,47.37
ceval-plant_protection,8941f9,accuracy,gen,45.45
ceval-basic_medicine,c409d6,accuracy,gen,57.89
ceval-clinical_medicine,49e82d,accuracy,gen,40.91
ceval-urban_and_rural_planner,95b885,accuracy,gen,45.65
ceval-accountant,002837,accuracy,gen,38.78
ceval-fire_engineer,bc23f5,accuracy,gen,38.71
ceval-environmental_impact_assessment_engineer,c64e2d,accuracy,gen,38.71
ceval-tax_accountant,3a5e3c,accuracy,gen,32.65
ceval-physician,6e277d,accuracy,gen,38.78
ceval-stem,-,naive_average,gen,29.41
ceval-social-science,-,naive_average,gen,54.16
ceval-humanities,-,naive_average,gen,46.10
ceval-other,-,naive_average,gen,42.11
ceval-hard,-,naive_average,gen,23.90
ceval,-,naive_average,gen,40.39
$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
-------------------------------------------------------------------------------------------------------------------------------- THIS IS A DIVIDER --------------------------------------------------------------------------------------------------------------------------------
raw format
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
-------------------------------
Model: opencompass.models.huggingface.HuggingFace_jayhust_internlm2-chat-1_8b
ceval-computer_network: {'accuracy': 26.31578947368421}
ceval-operating_system: {'accuracy': 36.84210526315789}
ceval-computer_architecture: {'accuracy': 14.285714285714285}
ceval-college_programming: {'accuracy': 13.513513513513514}
ceval-college_physics: {'accuracy': 36.84210526315789}
ceval-college_chemistry: {'accuracy': 12.5}
ceval-advanced_mathematics: {'accuracy': 10.526315789473683}
ceval-probability_and_statistics: {'accuracy': 33.33333333333333}
ceval-discrete_mathematics: {'accuracy': 18.75}
ceval-electrical_engineer: {'accuracy': 27.027027027027028}
ceval-metrology_engineer: {'accuracy': 50.0}
ceval-high_school_mathematics: {'accuracy': 5.555555555555555}
ceval-high_school_physics: {'accuracy': 31.57894736842105}
ceval-high_school_chemistry: {'accuracy': 42.10526315789473}
ceval-high_school_biology: {'accuracy': 15.789473684210526}
ceval-middle_school_mathematics: {'accuracy': 15.789473684210526}
ceval-middle_school_biology: {'accuracy': 57.14285714285714}
ceval-middle_school_physics: {'accuracy': 36.84210526315789}
ceval-middle_school_chemistry: {'accuracy': 60.0}
ceval-veterinary_medicine: {'accuracy': 43.47826086956522}
ceval-college_economics: {'accuracy': 43.63636363636363}
ceval-business_administration: {'accuracy': 36.36363636363637}
ceval-marxism: {'accuracy': 78.94736842105263}
ceval-mao_zedong_thought: {'accuracy': 45.83333333333333}
ceval-education_science: {'accuracy': 68.96551724137932}
ceval-teacher_qualification: {'accuracy': 68.18181818181817}
ceval-high_school_politics: {'accuracy': 47.368421052631575}
ceval-high_school_geography: {'accuracy': 36.84210526315789}
ceval-middle_school_politics: {'accuracy': 57.14285714285714}
ceval-middle_school_geography: {'accuracy': 58.333333333333336}
ceval-modern_chinese_history: {'accuracy': 60.86956521739131}
ceval-ideological_and_moral_cultivation: {'accuracy': 68.42105263157895}
ceval-logic: {'accuracy': 31.818181818181817}
ceval-law: {'accuracy': 20.833333333333336}
ceval-chinese_language_and_literature: {'accuracy': 17.391304347826086}
ceval-art_studies: {'accuracy': 51.515151515151516}
ceval-professional_tour_guide: {'accuracy': 62.06896551724138}
ceval-legal_professional: {'accuracy': 30.434782608695656}
ceval-high_school_chinese: {'accuracy': 21.052631578947366}
ceval-high_school_history: {'accuracy': 70.0}
ceval-middle_school_history: {'accuracy': 72.72727272727273}
ceval-civil_servant: {'accuracy': 38.297872340425535}
ceval-sports_science: {'accuracy': 47.368421052631575}
ceval-plant_protection: {'accuracy': 45.45454545454545}
ceval-basic_medicine: {'accuracy': 57.89473684210527}
ceval-clinical_medicine: {'accuracy': 40.909090909090914}
ceval-urban_and_rural_planner: {'accuracy': 45.65217391304348}
ceval-accountant: {'accuracy': 38.775510204081634}
ceval-fire_engineer: {'accuracy': 38.70967741935484}
ceval-environmental_impact_assessment_engineer: {'accuracy': 38.70967741935484}
ceval-tax_accountant: {'accuracy': 32.6530612244898}
ceval-physician: {'accuracy': 38.775510204081634}
ceval-stem: {'naive_average': 29.410892033746723}
ceval-social-science: {'naive_average': 54.161475396956334}
ceval-humanities: {'naive_average': 46.102931026874565}
ceval-other: {'naive_average': 42.10911608938226}
ceval-hard: {'naive_average': 23.89894005847953}
ceval: {'naive_average': 40.387790633140824}
$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$$
