No one should use ollama. A cursory search of r/localllama gives plenty of occassions where they've proven themselves bad actors. Here's a 'fun' overview
There are multiple (far better) options - eg LM studio if you want GUI, llama.cpp if you want the CLI that ollama ripped off. IMO the only reason ollama is even in the conversation is it was easy to get running on macOS, allowing the SV MBP set to feel included
/r/LocalLlama is a very circle-jerky subreddit. There's a very heavy "I am new to GitHub and have a lot of say"[0] energy. This is really unfortunate because there's also a lot of people doing tons of good work there and posting both cool links and their own projects. The "just give me an EXE types" will brigade causes they do not understand and white knight projects and attack others for no informed logic reason. They're not really a good barometer for the quality of any project, on the whole.
Can't believe you're getting downvoted for one of Ireland's greatest cultural contributions. Behold The Rubberbandits, Horse Outside (helpfully timestamped to the lyric in question)
Nvidia for compatibility, and as much VRAM as you can afford. Shouldn't be hard to find a 3090 / Ti in your price range. I have had decent success with a base 3080 but the 10GB really limits the models you can run
If anyone is interested in using Foliate through WSL, https://opticos.github.io/openinwsl/ is great - it lets you double-click a file in Windows, then Foliate in WSL2 launches to view it
I'm really curious as to why Apple has been unable to reproduce their leap in CPUs in the GPU space.
It's not exactly surprising when Nvidia parts handily beat the M1/M2, but when both Qualcomm and Mediatek have better GPU performance _and_ efficiency [0] something is up, especially given just how far ahead Apple has been in mobile CPU
No, they started designing them earlier. A11 was the first one that they publicly claimed to be fully in-house. They were substantially (but not wholly) Apple-designed as early as A8, and generations prior to that they did significant tweaking to.
I wonder how closely related their GPU is to PowerVR these days as well. With both PowerVR and the Asahi GPU driver it would be interesting to see if any of the design still resembles PowerVR.
> I'm really curious as to why Apple has been unable to reproduce their leap in CPUs in the GPU space.
GPUs are highly parallelized and specialized systems. The workloads are already being optimized for the GPU, rather than having a CPU which is being optimized to deal with more arbitrary workloads (with things like branch prediction, superscalar architecture, etc).
So you could say, without creating new instructions to represent the workflow better, there is a fixed amount of digital logic needed to perform the given work, and that translates to a fixed amount of power draw needed on a particular fabrication process.
So Apple could throw more transistors at the problem (with a memory bus that can support the extra need), but the same amount of work still would take the same amount of power and generate the same amount of heat. It is usually far easier and more efficient to create dedicated logic for particular common problems, such as certain ML operations or toward hardware video encoding/decoding.
> It's not exactly surprising when Nvidia parts handily beat the M1/M2, but when both Qualcomm and Mediatek have better GPU performance _and_ efficiency [0]
Benchmarks are highly subjective, so I'd wait for more reviews (preferably by people with more established reputations, and perhaps a website). Reviewers who might try to determine _why_ one platform is doing better than another.
GPU benchmarks are even more so, because again the workloads are targeted toward the GPU, while the GPU is also optimized for handling particular workloads. This means that benchmarks can be apple-to-oranges comparisons - even before you find out that a given benchmark was optimized differently for different platforms.
There is also of course the reality that some vendors will optimize their code for the benchmarks specifically, going as far as to overclock the chip or to skip requested instructions when a particular benchmark is detected.
Does Apple need to catch up with Qualcomm and MediaTek in terms of raw gpu performance when Apple can optimize software and apis given to developers to work on its hardware? Or am I really out of date and is their public evidence of Qualcomm and Mediattek outperforming apple's hardware in real world workloads?
Nvidia primarily makes add on GPU's, if I understand their business correctly. Apple integrated a GPU onto its m2 (or whichever chip is used in their studio) that performs comparably to the 3060, and even beat the the 3090 in some benchmarks/workloads. I think that's pretty impressive.
This isn’t at all true despite Apple’s marketing. The M2 gets trounced by the 3060 in any graphical benchmark other than power draw, comparing it to a 3090 is just laughable.
> Nvidia primarily makes add on GPU's, if I understand their business correctly. Apple integrated a GPU onto its m2 (or whichever chip is used in their studio) that performs comparably to the 3060, and even beat the the 3090 in some benchmarks/workloads. I think that's pretty impressive.
Its more the fact that we're talking about Apple catching up at all. Android SOCs have been generationally behind Apple for a long time (and MediaTek in particular as a "budget" option), but now in the GPU space that is reversed.
The situation on the desktop/laptop is muddied by CUDA and other Nvidia-exclusive tech - while the M1/M2s indeed trade blows with laptop parts like the 3060 in some specific tasks, once CUDA comes into play Nvidia walks it (unfortunately IMO, even AMD can't compete there and its holding the industry back)
i think m1 was a big boost because of risc--compilers have gotten really good, and cpu pipelining has been well researched/developed, so there was a lot of performance to be harvested by putting everything together.
gpus, on the otherhand, are already risc. so where is apple going to improve? not by integrating everything: lots of companies have done this for years and years. if you want to do more with the same transistors, you'll need an even more clever execution model...
- Win+Z brings up the layout menu
- number keys select the layout, then where in the chosen layout the current window should go
- arrow keys then let you select the other windows to complete the tiling
So if I want Firefox (current window) and Slack side by side
- Win-Z inside FF
- 1 selects side-by-side
- 1 again snaps FF to the left (2 would snap to right, and so on for more complex layouts)
- Slack is the first suggested window (MRU), so Space to snap right
It's not as quick as the mouse interface yet - especially with the mouse improvements MS made - but seems like it could be easily automated with eg Autohotkey
https://www.reddit.com/r/LocalLLaMA/comments/1kg20mu/so_why_...
There are multiple (far better) options - eg LM studio if you want GUI, llama.cpp if you want the CLI that ollama ripped off. IMO the only reason ollama is even in the conversation is it was easy to get running on macOS, allowing the SV MBP set to feel included