GeForce RTX 4060 Ti

I understand your point (and definitely want the same), but I do have an almost-AliExpress-LLM-in-a-box: it's an Thunderbolt eGPU dock (that I got from AliE, and it is USB-C...) with a RTX 4060 Ti with 16 GB of VRAM (bought locally for gaming before the price boom) It's been awesome for embeddings and document OCR! 3D printing a case for it is on my todo list.

I have an RTX 4060 Ti with latest drivers installed, KDE on Debian. On the login screen, I can select X11 or Wayland. Selecting X11 lets me log in fine, selecting Wayland and logging in results in a black screen and the only thing I can do to get any video out at all is switching to another TTY.

I’m using Llama-3 8B to classify html files. It’s surprisingly good, and I run it on an RTX 4060 Ti at 8-bit quantization. No complains so far.

I consider the RTX 4060 Ti as the best entry level GPU for running small models. [...] For larger models probably multiple RTX 3090s since you can buy them on the cheap on the second hand market. [...]

This might technically be outside your budget, but if you happen to have a PC, I highly recommend the RTX 4060 Ti 16GB ($450, or less if on sale). [...]

I picked up a couple RTX 4060ti in the 16GB version for $450 each a couple days ago from Bestbuy. Had been looking at the 3060 like yourself. Installed LM Studio and have been trying out a bunch of models with varying levels of quantization, completely pain free.

i got a new graphics card! on my Intel Arc A770, there were these bizarre artifacts all over (a documented issue with Arc) in Aragami 2. on my newly-installed GeForce RTX 4060 Ti, the game looks great!

[...] I am running qwen 3.6 35 a3b on a 16gb vram rtx 4060 ti. I wish I sprung for a 24gb vram card but I never thought the price difference would matter. [...] My set up is not the best but it is working for me and is saving me money.

That's why I keep using Gentoo and X11 to handle my three GPU setup. An Intel iGPU, an AMD dGPU on the same package as the Intel CPU and a RTX 4060 Ti eGPU connected through Thunderbolt.

Oh, this was a great thread. I just investigated a bit which drivers I use, gpu.. And it turns out my arch was using built-in GPU (how?) glxinfo | grep "OpenGL renderer" OpenGL renderer string: AMD Radeon Graphics (radeonsi, raphaelmendocino, LLVM 20.1.8, DRM 3.64, 6.16.0-arch2-1) I've added a few env parameters Claude suggested in /.config/hypr/hyprland.conf, and now it shows: OpenGL renderer string: NVIDIA GeForce RTX 4060 Ti/PCIe/SSE2 Will give it a try few days, to see how it behaves.