18:40
LocalAI LLM Testing: Part 2 Network Distributed Inference Llama 3.1 405B Q2 in the Lab!
46:24
LocalAI LLM Testing: Distributed Inference on a network? Llama 3.1 70B on Multi GPUs/Multiple Nodes
36:05
LocalAI LLM Testing: Llama 3.1 8B Q8 Showdown - M40 24GB vs 4060Ti 16GB vs A4500 20GB vs 3090 24GB
21:40
LocalAI LLM Testing: How many 16GB 4060TI's does it take to run Llama 3 70B Q4
10:22
LocalAI LLM Testing: Can 6 Nvidia A4500's Take on the WizardLM 2 8x22b?
32:27
LocalAI LLM Testing: Viewer Questions using mixed GPUs, and what is Tensor Splitting AI lab session
00:25
What's on the Robotf-AI Workbench Today?
31:14
LocalAI LLM Testing: i9 CPU vs Tesla M40 vs 4060Ti vs A4500
32:00
LocalAI Testing: Viewer Question LLM context size, & quant testing with 2x 4060 Ti's 16GB VRAM
20:31
LocalAI LLM Single vs Multi GPU Testing scaling to 6x 4060TI 16GB GPUS
X
About
Just another engineer fooling around in a GPU powered Kubernetes based AI/LLM Lab. We play around with different GPUs, different Large Language Models, run tests and see what we can learn together.
I am not an expert, and make bad decisions, please do not follow anything I do on this channel.
Tags