Table of Contents
As recently as two years ago, “in-house AI” meant one thing for most businesses: a ChatGPT or Copilot subscription. Today, more and more organisations are reaching the point where the cloud is no longer enough. There are usually three reasons: costs that rise with the number of queries, data that cannot be sent outside the company, and the need for AI to work with their own documents, databases and processes rather than general knowledge from the internet.
This raises the question: which AI server should you actually buy, and how much does it cost?
The answer is more interesting than it might seem, because the price range is enormous. From a book-sized device costing just over £4,000 to a machine priced at more than £383,788.19. Importantly, each of these solutions makes sense, just for a different type of buyer. Below, I outline three investment levels, the calculation worth making before deciding, and five questions that will help you select the right hardware without buying too much or too little.
How does an AI server differ from a standard server?
A standard server is built around the CPU. An AI server is built around GPUs – they perform all the work involved in running models, while the remaining hardware exists to support them. This results in three practical differences.
- GPUs and their memory. Such a machine uses between four and eight accelerators – typically PCIe cards in rack servers, or modules soldered onto a shared board in the most powerful platforms. Their combined memory is the most important factor in the entire purchase decision – more on that shortly.
- Power. Eight cards under load consume 5–6 kW, equivalent to dozens of office computers. This requires a rack with adequate power provision and efficient heat removal from the data centre.
- Networking. A standard high-speed network is sufficient for a single machine. When several servers are combined into one system, much faster interconnects are required, as GPUs in different chassis must exchange data with latency comparable to operation within a single chassis.
This is why it is not simply a “more powerful server”. It is a different hardware category, selected according to different criteria.
GPU memory: the one figure that brings the entire selection process into focus
There is one parameter more important than price: the total GPU memory available in the server.
Why? The entire model must fit into that memory, just as a file must be loaded into an application before it can be used. If it does not fit, it simply will not start.
However, the model itself is only the start of the calculation. Every logged-in user and every analysed document requires additional memory for an active conversation. The same model therefore needs noticeably more capacity for one hundred users than for one – and it is the number of users, rather than model size alone, that most often determines the hardware tier required.
The guide explains the basic memory requirements of models Which AI computer should you choose. The table below shows the second dimension of this calculation: how requirements change with deployment scale. The figures are indicative and apply to a typical production configuration.
| Model size | 5 users | 50 users | 200 users | Hardware tier |
|---|---|---|---|---|
| small (8B class) | approx. 8 GB | approx. 20 GB | approx. 50 GB | DGX Spark |
| medium (70B class) | approx. 45 GB | approx. 70 GB | approx. 130 GB | rack server, 1–2 cards |
| large (200B+ class) | approx. 130 GB | approx. 200 GB | approx. 350 GB | eight-card server |
| training a proprietary model | several times more than running the model alone | – | – | HGX B200 platform |
Designations such as 8B and 70B indicate how many billions of parameters a model contains – 8B means eight billion. The more parameters, the better the response quality and the more GPU memory the model requires.
There is also one thing that is easy to overlook.
Headroom. The figures above apply to normal operation. At peak times, allow an additional 20–30% margin; otherwise, the system will start rejecting queries under heavier traffic.
Tier 1: NVIDIA DGX Spark, the starting point
NVIDIA DGX Spark is a book-sized desktop device with 128 GB of unified memory and AI computing performance of around 1 petaFLOPS. It enables models in the class powering popular assistants to run locally, without sending a single document to the cloud. Several variants are available, differing mainly in storage capacity (1, 2 or 4 TB) and support package. The full range in this hardware class, including models from eight manufacturers, is available in the AI computers category.
Who is it for? For a company that wants to establish whether AI will improve customer service, contract analysis or quotation preparation before anyone commits a major budget. The investment is around £4,605.45 net, less than an annual subscription to AI tools for a team of more than a dozen people. Two devices can be connected into a single system.
Limitation. This hardware is for a single team, not the entire organisation. It cannot be installed in a rack, has no redundant power supply and cannot be covered by the service policy used in the data centre. When one hundred people need to use AI simultaneously, you need to move up a tier.
Tier 2: rack-mounted GPU server, or production AI for business
In my view, this is the tier most often overlooked today, yet it is the most sensible choice for a company with a real AI use case.
This refers to a dedicated rack-mounted GPU platform (5U), designed for eight double-width cards. It is not tied to a single accelerator model – card selection depends on the use case. The base configuration includes two AMD EPYC processors, up to 6 TB of DDR5 memory across 24 slots, and six 2700 W power supplies operating redundantly.
The number of cards is key, as it determines GPU memory capacity. Two RTX PRO 6000 cards with 96 GB each provide 192 GB and can support an internal assistant for several dozen users. A full eight-card configuration provides 768 GB, enough for several hundred users or very large models. Additional cards can be installed in the same chassis without replacing the server — the key benefit at this level is that you do not need to start with a full configuration.
Specific model: Supermicro AS-5126GS-TNRT Gold Series
Sold as a complete, configured system. An entry configuration with two cards represents an investment of approximately £95,947.96 net. Check the product page for the exact configuration and current price.
What does this mean in practice? This type of server can run large language models locally, analyse corporate document archives, automate customer service, generate content in multiple languages and analyse images from production lines. RTX PRO cards are also versatile: the same server can be used for rendering, engineering simulations or virtual workstation infrastructure. This is often the deciding factor: the hardware does not sit idle between AI projects.
Who is it for? In its entry configuration, it is suited to organisations with several dozen employees that have completed the testing stage and want to give teams access to an internal assistant familiar with company procedures. In a full configuration, it is intended for large organisations, hundreds of concurrent users and regulated industries such as finance, healthcare and the public sector, where data must remain on site.
What to keep in mind. Power consumption increases with the number of cards: from approximately 2 kW for two cards to 5–6 kW for eight. The rack must supply that power, and the cooling system must remove the heat. If you are planning to expand, calculate power and air-conditioning capacity for the target configuration from the outset, not the initial one.
Level 3: NVIDIA HGX B200, the infrastructure used to train models
At the very top end is the kind of hardware featured in reports on the technology race between the world’s largest companies. A server based on the NVIDIA HGX B200 platform includes eight Blackwell processors, currently the most powerful AI accelerators on the market, with a combined 1,44 TB of HBM3e memory — the fastest available to buy today.
The machine occupies 10U in a rack and has eight 400 Gb/s network interfaces, as it is designed to be clustered with further identical servers. Power is supplied by six power supplies rated at approximately 5 kW, operating redundantly in a 3+3 configuration — the machine’s actual power draw under load is around 10 kW, with the remainder providing headroom and fault protection. It includes a BlueField-3 processor, which offloads networking and security tasks from the main system, plus three years of next-business-day on-site support.
Specific model: Supermicro AS-A126GS-TNBR
Designed for scientific computing, AI, industrial automation and data analytics. Indicative investment: approximately £383,791.84 net per server; see the product page for the current price.
The difference compared with a level-two GPU server is not that B200 “does the same thing, only faster”. It is that it can perform tasks the other system cannot complete in a reasonable time: training proprietary models from scratch, fine-tuning models with hundreds of billions of parameters, serving hundreds of users simultaneously, scientific research and industrial-scale simulations.
Who is it for? For companies building AI-based products that create their own models rather than relying on ready-made ones. For research institutions and universities. For telecommunications operators, banks, major industrial groups and pharmaceutical companies.
In practice, organisations rarely buy just one such machine. It also requires network infrastructure, data-centre power and cooling — at 10 kW per chassis, liquid cooling is usually required. This is not simply a hardware purchase; it is a strategic decision.
How do you choose an AI server? Five questions worth asking
Do you already know what you want AI to do in your organisation?
If not, choose level one. For the price of one premium laptop, you get an environment in which you can test several ideas within two weeks and make decisions based on facts rather than presentations.
How many people will use it, and how often?
If the answer is “the whole company, every day”, a desktop device will not be enough, and a rack-mounted GPU server will be the right choice. It is worth calculating how much you currently spend on cloud APIs and subscriptions. At scale, on-premises hardware is less expensive over a two- to three-year period. When this type of investment starts to pay for itself is explained in the article about an on-premises LLM for business.
Are you building your own models or using ready-made ones?
Using ready-made models, even very large ones, is the domain of level two. Building proprietary models requires B200. The dividing line here is clear, and there is rarely any doubt about which side a company belongs on.
How large a model do you need to run, and for how many people?
Return to the memory table. If the selected model requires 60 GB and fifty people need to use it simultaneously, one card with 96 GB of memory is the absolute minimum, while two provide headroom and room for growth.
What can your data centre support?
The most common surprise in AI projects is not the hardware but the building: how much power can be supplied, whether the air conditioning can remove the additional heat, and whether there is space in the rack. It is better to check this before placing the order than after delivery.
What else should be included in the implementation budget?
The server price typically accounts for 70–85% of the project budget. The remaining items should be considered at the planning stage.
- Power and rack – A suitably rated PDU, phase distribution, uninterruptible power supply and, for some installations, a new power connection.
- Cooling – up to approximately 5–6 kW per rack, precision air conditioning is usually sufficient. Above 10 kW, liquid cooling is used, which requires facility adaptations and a separate service procedure. If the server room was not designed for this, this cost should be included in the estimate from the outset.
- RAM – AI configurations start at several hundred gigabytes, and server memory module prices rose significantly in 2026. The difference between 512 GB and 1 TB can be a significant item in the cost estimate.
- Networking – a switch with sufficient bandwidth, transceivers and cabling. Cluster configurations also require a separate GPU network.
- Storage – models and document collections occupy terabytes, and read throughput should match the performance of the cards.
- Deployment – software installation, model deployment, and integration with your corporate login system and the systems you use.
- Support and warranty – next-business-day on-site service is the standard for production environments.
Where to start
The most common mistake in purchases like this is starting with the catalogue. The effective order is the reverse: first, define a single use case; then establish the number of users; then compare it with current cloud costs; and only finally select the configuration.
If you do not yet know exactly what AI will be used for, level one is sufficient for several months of testing and costs about the same as a single laptop. If testing is already complete and the assistant is to serve the entire company, level two is currently the most sensible choice for most organisations. Level three is chosen when you are building your own models—and in that case, the decision is usually clear.
If the configuration needs to be tailored to your server room—whether in terms of the number of cards, memory, networking or cooling—our team will prepare a proposal for your specific use case.
FAQ
The range is wide: from approximately £4,605.47 net for a DGX Spark-class desktop device, through approximately £95,946.04 for an entry-level rack-mounted GPU server, to approximately £383,784.18 for an HGX B200 platform. Add 15–30% to the hardware price for power, cooling, networking and deployment.
It depends on the scale. Cloud costs increase with the number of requests, while the cost of an in-house server is fixed—after purchase, you do not pay for each call. It is therefore worth comparing current API bills and subscriptions with hardware instalments spread over three years and energy costs. With intensive use, in-house infrastructure comes out ahead; for occasional use, the cloud remains less expensive.
The entire model must fit in the memory of the cards. As a guideline: a 8B-class model requires several dozen gigabytes, a 70B-class model requires from 70 to 140 GB, depending on the model’s numerical precision, while models with more than 200 billion parameters start at several hundred gigabytes. Additional memory is also needed for ongoing conversations, increasing with the number of users.
For testing and individual use cases—yes. For production workloads, usually not, because the limiting factors are the combined memory of the cards, as well as the power supply and cooling of a chassis not designed for eight accelerators.
If AI is to run in production for everyone, the most sensible option is a rack-mounted GPU server configured with two cards and the option to add more. If you are still at the testing stage or it is used by a single team, a DGX Spark-class desktop device is sufficient, or alternatively two units connected into a single system.
A desktop device uses around 240 W. A rack-mounted GPU server uses from approximately 2 kW with two cards to 5–6 kW with eight, while an HGX B200 platform is around 10 kW. Calculate the cost based on your own kWh rate and planned operating time—when running continuously, the difference between the levels is several dozen times over.
