먼저 결론부터 말하자면 유의미한 차이가 있더라
순정 9000Mhz (288GB/s)랑
오버클럭 10500Mhz (337GB/s) 일때로 비교해봤음
벤치마킹은 llm_benchmark라는거로 했음
메모리 클럭 9000Mhz 순정
Checking and pulling the following LLM models
gemma:2b
gemma:7b
mistral:7b
llama2:7b
llama2:13b
llava:7b
llava:13b
----------
model_name = mistral:7b
prompt = Write a step-by-step guide on how to bake a chocolate cake from scratch.
eval rate: 55.94 tokens/s
prompt = Develop a python function that solves the following problem, sudoku game
eval rate: 55.87 tokens/s
prompt = Create a dialogue between two characters that discusses economic crisis
eval rate: 56.36 tokens/s
prompt = In a forest, there are brave lions living there. Please continue the story.
eval rate: 56.35 tokens/s
prompt = I'd like to book a flight for 4 to Seattle in U.S.
eval rate: 56.43 tokens/s
--------------------
Average of eval rate: 56.19 tokens/s
----------------------------------------
model_name = gemma:2b
prompt = Explain Artificial Intelligence and give its applications.
eval rate: 109.66 tokens/s
prompt = How are machine learning and AI related?
eval rate: 109.91 tokens/s
prompt = What is Deep Learning based on?
eval rate: 109.97 tokens/s
prompt = What is the full form of LSTM?
eval rate: 109.87 tokens/s
prompt = What are different components of GAN?
eval rate: 110.09 tokens/s
--------------------
Average of eval rate: 109.9 tokens/s
----------------------------------------
model_name = gemma:7b
prompt = Explain Artificial Intelligence and give its applications.
eval rate: 44.61 tokens/s
prompt = How are machine learning and AI related?
eval rate: 44.68 tokens/s
prompt = What is Deep Learning based on?
eval rate: 44.74 tokens/s
prompt = What is the full form of LSTM?
eval rate: 46.43 tokens/s
prompt = What are different components of GAN?
eval rate: 44.54 tokens/s
--------------------
Average of eval rate: 45.0 tokens/s
----------------------------------------
model_name = llama2:7b
prompt = Explain Artificial Intelligence and give its applications.
eval rate: 57.81 tokens/s
prompt = How are machine learning and AI related?
eval rate: 57.40 tokens/s
prompt = What is Deep Learning based on?
eval rate: 57.99 tokens/s
prompt = What is the full form of LSTM?
eval rate: 59.68 tokens/s
prompt = What are different components of GAN?
eval rate: 58.07 tokens/s
--------------------
Average of eval rate: 58.19 tokens/s
----------------------------------------
model_name = llama2:13b
prompt = Explain Artificial Intelligence and give its applications.
eval rate: 31.95 tokens/s
prompt = How are machine learning and AI related?
eval rate: 32.22 tokens/s
prompt = What is Deep Learning based on?
eval rate: 31.78 tokens/s
prompt = What is the full form of LSTM?
eval rate: 34.19 tokens/s
prompt = What are different components of GAN?
eval rate: 32.22 tokens/s
--------------------
Average of eval rate: 32.472 tokens/s
----------------------------------------
model_name = llava:7b
prompt = Describe the image, /usr/local/lib/python3.11/dist-packages/llm_benchmark/data/img/sample1.jpg
eval rate: 57.31 tokens/s
prompt = Describe the image, /usr/local/lib/python3.11/dist-packages/llm_benchmark/data/img/sample2.jpg
eval rate: 57.62 tokens/s
prompt = Describe the image, /usr/local/lib/python3.11/dist-packages/llm_benchmark/data/img/sample3.jpg
eval rate: 57.15 tokens/s
prompt = Describe the image, /usr/local/lib/python3.11/dist-packages/llm_benchmark/data/img/sample4.jpg
eval rate: 57.18 tokens/s
prompt = Describe the image, /usr/local/lib/python3.11/dist-packages/llm_benchmark/data/img/sample5.jpg
eval rate: 57.02 tokens/s
--------------------
Average of eval rate: 57.256 tokens/s
----------------------------------------
model_name = llava:13b
prompt = Describe the image, /usr/local/lib/python3.11/dist-packages/llm_benchmark/data/img/sample1.jpg
eval rate: 32.80 tokens/s
prompt = Describe the image, /usr/local/lib/python3.11/dist-packages/llm_benchmark/data/img/sample2.jpg
eval rate: 32.99 tokens/s
prompt = Describe the image, /usr/local/lib/python3.11/dist-packages/llm_benchmark/data/img/sample3.jpg
eval rate: 33.20 tokens/s
prompt = Describe the image, /usr/local/lib/python3.11/dist-packages/llm_benchmark/data/img/sample4.jpg
eval rate: 33.27 tokens/s
prompt = Describe the image, /usr/local/lib/python3.11/dist-packages/llm_benchmark/data/img/sample5.jpg
eval rate: 32.96 tokens/s
--------------------
Average of eval rate: 33.044 tokens/s
----------------------------------------
10500Mhz (+1500Mhz) 오버클럭
Checking and pulling the following LLM models
gemma:2b
gemma:7b
mistral:7b
llama2:7b
llama2:13b
llava:7b
llava:13b
----------
model_name = mistral:7b
prompt = Write a step-by-step guide on how to bake a chocolate cake from scratch.
eval rate: 63.90 tokens/s
prompt = Develop a python function that solves the following problem, sudoku game
eval rate: 63.93 tokens/s
prompt = Create a dialogue between two characters that discusses economic crisis
eval rate: 64.44 tokens/s
prompt = In a forest, there are brave lions living there. Please continue the story.
eval rate: 64.52 tokens/s
prompt = I'd like to book a flight for 4 to Seattle in U.S.
eval rate: 64.67 tokens/s
--------------------
Average of eval rate: 64.292 tokens/s
----------------------------------------
model_name = gemma:2b
prompt = Explain Artificial Intelligence and give its applications.
eval rate: 122.47 tokens/s
prompt = How are machine learning and AI related?
eval rate: 121.90 tokens/s
prompt = What is Deep Learning based on?
eval rate: 122.04 tokens/s
prompt = What is the full form of LSTM?
eval rate: 122.45 tokens/s
prompt = What are different components of GAN?
eval rate: 122.08 tokens/s
--------------------
Average of eval rate: 122.188 tokens/s
----------------------------------------
model_name = gemma:7b
prompt = Explain Artificial Intelligence and give its applications.
eval rate: 50.98 tokens/s
prompt = How are machine learning and AI related?
eval rate: 51.24 tokens/s
prompt = What is Deep Learning based on?
eval rate: 50.90 tokens/s
prompt = What is the full form of LSTM?
eval rate: 52.99 tokens/s
prompt = What are different components of GAN?
eval rate: 50.92 tokens/s
--------------------
Average of eval rate: 51.406 tokens/s
----------------------------------------
model_name = llama2:7b
prompt = Explain Artificial Intelligence and give its applications.
eval rate: 66.23 tokens/s
prompt = How are machine learning and AI related?
eval rate: 66.65 tokens/s
prompt = What is Deep Learning based on?
eval rate: 66.25 tokens/s
prompt = What is the full form of LSTM?
eval rate: 68.91 tokens/s
prompt = What are different components of GAN?
eval rate: 66.86 tokens/s
--------------------
Average of eval rate: 66.98 tokens/s
----------------------------------------
model_name = llama2:13b
prompt = Explain Artificial Intelligence and give its applications.
eval rate: 37.07 tokens/s
prompt = How are machine learning and AI related?
eval rate: 37.21 tokens/s
prompt = What is Deep Learning based on?
eval rate: 37.07 tokens/s
prompt = What is the full form of LSTM?
eval rate: 39.45 tokens/s
prompt = What are different components of GAN?
eval rate: 37.09 tokens/s
--------------------
Average of eval rate: 37.578 tokens/s
----------------------------------------
model_name = llava:7b
prompt = Describe the image, /usr/local/lib/python3.11/dist-packages/llm_benchmark/data/img/sample1.jpg
eval rate: 65.52 tokens/s
prompt = Describe the image, /usr/local/lib/python3.11/dist-packages/llm_benchmark/data/img/sample2.jpg
eval rate: 66.54 tokens/s
prompt = Describe the image, /usr/local/lib/python3.11/dist-packages/llm_benchmark/data/img/sample3.jpg
eval rate: 64.87 tokens/s
prompt = Describe the image, /usr/local/lib/python3.11/dist-packages/llm_benchmark/data/img/sample4.jpg
eval rate: 65.32 tokens/s
prompt = Describe the image, /usr/local/lib/python3.11/dist-packages/llm_benchmark/data/img/sample5.jpg
eval rate: 65.30 tokens/s
--------------------
Average of eval rate: 65.51 tokens/s
----------------------------------------
model_name = llava:13b
prompt = Describe the image, /usr/local/lib/python3.11/dist-packages/llm_benchmark/data/img/sample1.jpg
eval rate: 37.83 tokens/s
prompt = Describe the image, /usr/local/lib/python3.11/dist-packages/llm_benchmark/data/img/sample2.jpg
eval rate: 38.16 tokens/s
prompt = Describe the image, /usr/local/lib/python3.11/dist-packages/llm_benchmark/data/img/sample3.jpg
eval rate: 38.43 tokens/s
prompt = Describe the image, /usr/local/lib/python3.11/dist-packages/llm_benchmark/data/img/sample4.jpg
eval rate: 37.94 tokens/s
prompt = Describe the image, /usr/local/lib/python3.11/dist-packages/llm_benchmark/data/img/sample5.jpg
eval rate: 38.45 tokens/s
--------------------
Average of eval rate: 38.162 tokens/s
----------------------------------------
Ai 언어모델 로컬 채널 이용규정
