The great thing about Framework is that you can bring your own RAM and SSD. If you need/want a lot of RAM (>= 32 GB) and or SSD (>= 2 TB), you can save a lot of money.
I second @jamesliudotcc's suggestion (ASUS ROG Flow Z13). One of the key differentiators vs other laptops is the 128GB unified RAM which runs at 8000 MT/s, which should be excellent for LLMs - I'm not aware of any laptop that can do better, in this form-factor and this level of portability.
Specs aside, this machine can also run Linux well [1], which makes it worth buying, IMO. And if it can run Linux well, then using it for development should be a breeze.
Asus makes a gaming tablet with a 13 inch screen with a 128GB of unified RAM on the new AMD Strix Point.
I agree that MacBook is not an option. I have a work MacBook and it has better silicon, ok, but I hate being on a Mac. Just not for me.
I'd wait until more manufacturers announce what they will do with that AMD SOC.
- Asus Zenbook Duo (2024) is a very interesting form factor. https://www.asus.com/ca-en/laptops/for-home/zenbook/asus-zen...
- Framework reviews are important to figure out the current generation. I tried one out and it had few issues other than me not having the time to tweak it how I wanted. Hardware was great, battery wasn't great at that time (needed simple software optimizations), but Framework now has larger batteries too that seem to have resolved that. The folks I know with them have mostly switched from Mac or X1 Carbons to Frameworks and quite happy with it.
- Specifically for travel, the recently announced Framework 12" looks really inviting if travel looks like the way to go.
To run an LLM on either, some amount of extra ram might help, depending on the model... if you want something heavier it might be cheaper to run a model privately in the cloud as needed, or use an eGPU and plug it in when you want. Local LLM use seems to be fun, but having something at home running it so it's accessible seems serviceable too.
I like the Lenovo P16 gen2.
It's not cheap and rather heavy, but very powerful. You can get it with lots of RAM, a powerful graphics card (for LLMs), and a CPU with 24 cores (32 threads). And you can get it shipped with Fedora or Ubuntu (although wiped it immediately and installed my own Fedora).
Curious what the bad Lenovo experience was. I've had various Thinkpads since the early 2000's and they all worked for me.
I will be buying a new laptop soon and have been researching this topic recently. I agree that MacBook is not a great option as a dev machine (great laptop, just not for devs, pretty much just because of that ridiculous OS).
Since I'm in the EU, I ended up with 2 possible options: ThinkPad T14 (non-s model), Gen.5 or newer. Or Framework 13 AMD.
System76 also make some interesting laptops, but it's just too expensive to get one in Europe.
I suggest that you get a backend box to run LLMs etc and get a laptop that suits you. Fiddle up a VPN to connect them.
Your backend box can be as ugly as you like whilst your front end laptop is good enough for normal usage.
If you're willing to consider Arm64 this laptop is pretty good: https://www.lenovo.com/us/en/p/laptops/yoga/yoga-slim-series...
There is no configuration that meets both high-quality local LLM performance and excellent portability. A robust discrete GPU in a thin, portable laptop does not offer enough memory bandwidth or thermals for full-scale models, and a laptop with abundant DDR5 memory and high-end CPU power isn’t designed for the throughput that local LLM inference demands. The practical solution is to use a Linux laptop optimized for everyday development and use hosted LLM services when you need more power.
Were your Lenovo experiences with Thinkpads or something else? I've heard that their non-Thinkpad laptops are not up to the same quality as Thinkpads - personally except for a couple years using a MacBook Pro, I've had great experiences with top of the line Thinkpads (x-series mostly) over the past 15 years. My current x13 (gen 5) w/ 32GB of memory can handle local LLMs (not terribly quickly, but it works).
Buy any laptop with following specs - 1. 32GB RAM (stock) + Extra slot with >= 32GB support (higher freq. = better perf.) 2. SSD 1TB 3. AMD Ryzen CPU (6x2 or 8x2 threads) 4. Nvidia RTX series GPU with atleast 8GB VRAM (not sure if 12GB VRAM comes with any laptop).
Quantized LLMs upto 8b parameters can be easily run on above specs. Quantized models are getting better and better. I use them for code suggestion and code gen.
I can personally recommend the "Tuxedo Infinity Book 14 Pro" maxed out on options with Linux/Windows Dual Boot preinstalled. AMD and Intel are offered both (personally I would choose Intel, because it comes with Thunderbolt 4). Also: up to 96GB RAM and PCIe4 SSDs.
Instead of Dual Boot they also offer a preinstalled VM with a licensed Windows. Tuxedo OS Linux is a slightly modified Ubuntu optimized for their Hardware.
I just upgraded from a 7-year old XPS 15 - technically it was a Precision 5520 from when Dell still had a Developer Edition that would ship with Ubuntu but it was hands down the best Linux laptop I had. The things I loved about that one was:
- 45W TDP processor - none of the U series 15W slowness
- 4K screen that scaled to 1080P perfectly at 200% scaling - so X11 and XWayland apps still looked good. Also the screen quality was amazing
- Integrated Graphics - no messing around with NVIDIA drivers
Pros:
- 45W TDP processor is fast
- They let you load it up with memory and disk for reasonable prices unlike most mainstream manufacturers
- 2nd M2 slot is nice for copying over old SSD
- They provide Windows Drivers in a GitHub repo
Cons:
- Battery life is pretty bad but I stay plugged in most of the time so not a huge issue for me
- Fans spin up pretty often
- USB-C charging only. There is an A/C Adapter barrel jack but they don't ship a charger and I can't find a suitable aftermarket charger. I worry about wear and tear on the USB-C port damaging it one day and that will be the end of the laptop effectively if I can't plug in.
- The screen needs around 133% fractional scaling which is not even an option. So I have to use a combination of 125% scaling and tweaks. XWayland apps look terrible so had to go through and force Wayland on all Electron apps and JetBrains and as you can probably guess there are random odd bugs
- Random issue (rarely) where I power it on and it does not start up, then I have to close the lid, hold the power button, etc before it starts up. Support told me to try booting with one memory stick but it's so hard to recreate I don't think I'll be messing around with it unless the issue starts cropping up more.
You would want a laptop with RTX 4090 or RTX 5090. Anything under 16GB VRAM is going to be too limited for playing around with local LLMs.
Franework is nice but no Nvidia option which is a no-go for LLM.
Thinkpad P1 Gen 6 with RTX 4090 is a solid one. Older generation but Thinkpad seems like no longer slapping top GPU into its notebook.
Other laptops with 4090/5090 are too huge (Titan, Raider, Vector, Strix Scar 18,…)
I’m using Arch on the Thinkpad above.
Hope this helps
These requirements and history just remind me of a thread earlier today bemoaning why MacBook competitors are so badly outmatched. There was plenty of discussion of the alternatives and their weaknesses.
A laptop with Ada 5000 or 3500 seems pretty good in terms of value/price ratio for this use case. Dell and Lenovo have decent offerings.
I was just researching this for my own use.
Framework 13 (AMD). It’s not even a question. I’ve had mine for a year and it’s the best laptop I’ve ever owned.
So many ultrabooks and even some high-end productivity laptops cap out at 32GB of soldered RAM, while portable gaming devices are packing 64GB of upgradeable RAM. I feel like that dam is going to break soon.
Some thing to consider: physical home/end/page up/page down buttons. I certainly miss them on my current one, not even a "function" key variant for them.
I prefer laptops to be great at being mobile devices.
Running LLMs on a local mobile device doesn't make sense to me. Sure it sounds great but at what cost & what trade-offs? This goes for laptops & phones.
You can have the best of both worlds by setting up a separate computer in your local network or rent one on the cloud if data privacy & messing around with LLMs is important to you. Then you don't sacrifice the mobile advantage of a great laptop. It'll be cheaper & can be more focused too.
Asus ROG Z13, X13 (not your size, but also X16). You should consider getting used, tremendous value. There are some of the usual gaming laptop quirks (normal usbc chargers cant keep it charged on full load, QA issues like squeaky touchpad) but the versatility is unmatched.
Thinkpads are reliable and great QA/support, but the HW is underwhelming, bad thermals, dim screens, and just too expensive.
ASUS Proart P16 with the 4070/64gb configuration. Has usb a and c, hdmi, sd card reader, and can be powered over usb c though if you are using the 4070 you’ll want the ac adapter. The build quality is only matched by the screen which is awesome. Battery life is also excellent. If I was to have a laptop for 6 years this is the one I’d invest in.
Easy choices with system76: https://system76.com/
I have been having good experiences with Windows on ARM. Better performance, lower power, thinner, and cheaper at the same time.
Where Lenovo is unbeatable for me, is the extended warranty cost. You can get (usually on discount) full 4 year coverage next-day in-home with accident protection for only a couple hundred bucks. I like a company that can stand behind their product in that manner.
Framework with AMD HX "Ai" APU (Strix point) and a ton of ram.
My framework experience has been great (3.5 years). Would never go back to dell or Lenovo.
Why is Macbook not an option? Even if it's running Asahi Linux?
Why is the Macbook not an option? They are the best at making laptop hardware, and to my knowledge you can run what you like on it. Mac OS is good, but I also like Fedora's Atomic distros, such as Bluefin-dx.
A new Dell.
You could buy a MacBook and scratch away the logo, then replace it with a logo of your preferred brand.
HP zbook ultra G1A
You have two requirements that are at odds:
* Macbook is not an option
* Want to be able to mess around with some local LLMs.
Your choices for a Window laptop that can run a local LLM is either to get a large amount of system RAM and have it be abysmally slow, or to run a very tiny model on a discrete GPU which will (a) not be very good due to its small size and high quantization and (b) evaporate your battery life.
If you want to run local LLMs on a laptop and actually have them be useful, a Mac is currently the only real choice.
That said, with the money you save buying a Linux laptop instead, you can pay for a lot of tokens for whatever hosted LLM you want and it will be higher quality than what you could potentially run locally on a Mac.