new GPU for AI tools: AMD or Nvidia?

Hardware
1

Hello everyone,

thank God pixls is back!

So I want to upgrade my GPU because I am planning to use the new AI tools, especially inpainting/outpainting, denoising, and upscaling.

My current GPU is an Nvidia 1660 Super with 6 GB of RAM. Just tested yesterday Krita AI diffusion with it, sd and sdxl work, no errors, but very slow…

My pc is an approximately 5 years old midi tower by Captiva, with an AMD Ryzen 7 5800X 8-Core processor and 16 GB of RAM (there must even be a thread about it here).

There are two important things to consider: 1. I think I would like to avoid changing the power supply unit, so therefore, the new GPU needs to have 1x8 pins. 2. I think my budget is 550-600 euros.

So far I find these devices interesting:

  1. Sapphire Pulse Radeon RX 9060 XT, 16GB GDDR6, 2x HDMI, DP, lite retail

This one I can just get from a local store, it’s in stock.

  1. ASUS Dual GeForce RTX 5060 Ti OC, DUAL-RTX5060TI-O16G, 16GB GDDR7, HDMI, 3x DP

This one I would need to order from Amazon.

I had been a user of Nvidia GPUs since many years, I also have two laptops with Nvidia GPUs, however, lately I am a bit annoyed about them because they are not perfectly Linux friendly and have some issues with Wayland. E.g., I have a 10-bit-capable screen, my GPU can do 10-bit, however, I cannot find out for sure if my system is actually in 10-bit-mode (probably not), there is apparently no “switch” for this in the Nvidia driver when using Wayland.

On the other hand, Nvidia is much more “AI friendly” than AMD, it’s nice if everything works out of the box. But I think I could make stable diffusion etc work on AMD, too, I am not afraid of the terminal. Besides, since I make my living as an IT journalist, this could be good material for another article? Might be an interesting challenge.

Are there even people here who have experience with things like Krita AI diffusion/inpainting/outpainting and AMD GPUs?

I think I never actually used an AMD GPU, so maybe it’s time?

I think this will be a really difficult decision for me. Who wants to help? What do you think about these thoughts?

Anna

1 Like
2

For me it’s the other way round. So I hardly can compare. I have a RX9070XT running it with the FOSS drivers (Mesa). All in all I’m quite satisfied with it. For me the FOSS part is the important thing on my choice. But I try to avoid AI most of the time, so I don’t have to much expertise on that side. I bet others can give you more relevant information. :stuck_out_tongue_winking_eye:

4 Likes
3

Nvidia is a necessity at the moment if you want to use AI stuff, the exception being LLMs which run well on almost any dGPU as long as you have enough VRAM. For image diffusion afaik almost everything uses pytorch and rocm support is still far behind cuda. You can look up benchmarks for Flux, Wan, Qwen Image, etc, and for the same generation/tier, nvidia always comes out ahead, sadly.

When new models or tech are released, it’s always nvidia first, so if you want to test new stuff, AMD is not a good choice as well.

AMD used to be the best Linux option, both driver wise and also morally, but this is no longer the case in my opinion. AMD is as much to blame for the AI push/bubble as nvidia, and nvidia’s drivers are stable and work very well nowadays.

3 Likes
4

I had a RTX 2060 Super with 8 GB of VRAM and recently upgraded to an RX 9060 XT with 16 GB of VRAM. My experience is very mixed:

  • in the beginning (when drivers and software implementation of AMD was poor) it was almost unusable for linux users
  • with drivers getting better and better most applications run fine but I think there is still a bit of a difference when using Nvidia vs. AMD
  • I have used Krita with the AI tool and it worked. But it is some time ago and I do not remember which GPU I used back then. This is a bit the risk when switching to AMD - does your application support it.
  • All my regular applications work well when using Nobara Linux (DT, digikam, DR, steam, LM studio / VS code, …). I had more difficulties with tumbleweed (since the packages are much more up to date) with some software.
1 Like
5

I can’t comment on AMD vs Nvidia, but can confirm 6GB vram works with sd 1.5 but is quite slow. I also would want much more if were to use AI imaging tools regularly.

2 Likes
6

Actually, at this moment I would wait a tiny bit before jumping if you are not bound to x86.

Nvidia just launch their own CPU, with the a RTX 5070 and ARM cores, but with (up to) 128GB unified memory. No dedicated consumer card has so much memory available.

And since Nvidia other CPU the, dtx spark - more less the same CPU - does support Linux, I have high hopes that we have very nice alternative for Mac in the not so distant future!

1 Like
7

Actually, at this moment I would wait a tiny bit before jumping if you are not bound to x86.

Yes?
Betazoid wants to upgrade an existing system (midi tower) within a certain budget.
RTX Spark systems will certainly not fit into the budget.
Desktop systems are unavailable at the moment.
All the announced notebooks fall into the ultra-slim category. They are usually not good replacements for towers.

And linux support is just not there yes. I’m sure it will be great after some time, but not at the start. Distributions need time.

3 Likes
8

That is why I make that remark. Betazoid’s constraint for now is update (so stay in x86), but nonetheless sometimes such developments can influence decisions.

1 Like
9

Hi, thanks to everybody for the many replies and tips.
I definitely want to upgrade my desktop pc, I wont by any Nvidia pcs or whatever, but I didnt know about this Nvidia thing yet so I am grateful for the info.

Well… apparently there are youtube tutorials about how to set up Krita AI diffusion on systems with AMD cards but overall there does not seem to be very much experience with this?

What do you think? Will AMD cards be better at AI, I mean with new drivers etc?

I mean… it’s not that I want to do AI inpainting/outpainting excessively, I don’t think it will be the main focus of my work in the long run. So it wouldn’t die if I dont have the maximum performance, it should just be significantly better than currently with my Nvidia 1660 super.

Edit: please continue adding your thoughts to this thread.

10

I don’t think so, so much in the AI world is Nvidia focussed and from what I hear and see the last couples years driver support for Nvidia has much improved.

1 Like
11

Well, tbh, if I had the money, I would like to test and own both GPUs, AMD and Nvidia…

I just noticed that the AMD is 150-170watts, whereas the Nvidia is 180, so the AMD might be better for my PSU?

12

It could be, but something is telling me the difference might be insignificat. What PSU have you got at the moment?

1 Like
13

It’s insignificant. They will both have spikes way above that, and the the amd and nvidia card can have a new power limit set with a single command.

2 Likes
14

My rx9060xt hardly ever goes to more than 150 W even if running at 100% over a long period of time. Of course I only see what the software monitor shows (amdtop or similar). The fans on my card ( the cheapest version possible) are only audible very rarely.
As explained before I haven’t used it much for image generation but the performance under comfy ui was as I expected and similar to what I saw others needed on NVIDIA cards.
For Llm tasks there should be no difference between amd and NVIDIA if you use lm studio as your backend.

1 Like
15

AMD requires rocm for such scenarios.

  1. It’s a lot of stuff that needs to be installed/maintained …
  2. rocm stability - still of concern. Also stability varies a lot for different cards.
1 Like
16

Well the one I have in mind is rather new…

Is rocm actually open source or proprietary?

17

ROCm for AMD is a proprietary technology the same way as CUDA for NVIDIA and OpenVINO for Intel are.

1 Like
18

I cant fall asleep so I just ordered the Nvidia

1 Like
19

FWIW I just got a laptop with 32GB. 16 (on my old one) seemed to get tight at times, GPU notwithstanding.

1 Like
20

well, looks like I have to upgrade the PSU afterall. I have never done something similar before.