The data never leaves.
Prompts, documents and outputs stay on infrastructure you control. That matters where contracts, regulators or residency rules restrict where client and company data may be processed.
A working tool · Every figure is an estimate
Choose the departments and workflows you would automate and set your monthly volumes. The estimator breaks each workflow into units of judgement and prices them three ways: on GPUs you rent or own, on Claude or OpenAI, and on DeepSeek, Kimi or hosted open-weight models.
Prices were looked up on 15 November 2026 and each one links to its source below. Token counts, throughput and staff time are stated assumptions that you can change. Nothing here is a quotation.
Enter your domain and industry for a starting point, or drag departments onto the canvas. Tick the workflows and set how many you handle in a month. To match your own process, edit a workflow's steps or build one from the units of judgement.
The running cost of each route: models or GPUs, a shared platform and the engineering time to keep it going. Your own staff's review time is added underneath each total.
Add a department above to see the estimate.
Shared platform is the ordinary server that runs everything around the models: the gateway that every model call passes through, the workflow engine that moves work from step to step (n8n, for example), and the logs that record each run. Every route needs it, so the same amount appears in all three. It does not include your ERP, which you already run, or backups, a test environment and disaster recovery. The default of $100 a month is our assumption for one small cloud server, not a quoted price, and you can change it below.
Engineering time is the hours a technical person spends keeping the automation working after it is live, costed at the engineer rate in the assumptions. It is not the work of building it, which is a one-off cost, and it is not your staff reviewing the output. It has two parts. Upkeep applies on every route: watching for failures, updating prompts and test cases, and re-testing when a provider changes or retires a model. Running your own GPUs adds a second part on the self-hosted route only: the serving software, drivers, security patches, capacity and being on call. The hours are our assumptions, and you can change them below.
The comparison assumes every route does the work equally well. That is not established: the same tier does not mean the same accuracy, and a smaller model may need more retries and more review. A test set built from your own documents settles it. This page compares three ways of running the automation; it does not show whether automating costs less than the way you do the work today, which depends on your current handling time. Figures are US dollars at list price (AED is pegged at 3.6725), before UAE VAT and before any volume or committed-use discount.
API cost rises with every item. A GPU costs the same whether it is busy or idle. The chart scales your volumes up and down to show where the lines cross.
At low volume an API nearly always costs less. Three reasons can still justify your own models, and none of them appears in the totals above.
The reasons run the other way too. The largest hosted models are more capable than any model that fits on one GPU, an API needs no on-call engineer, and a middle path exists: open-weight models from a hosted provider in a jurisdiction you accept. How we approach private AI →
Every default below is our working assumption, not a measurement of your business. A pilot replaces them with real figures.
The monthly figures leave out the work of getting there. QCS charges $100 an hour for it. As a guide, automating one department, Sales for example, takes around 50 hours with AI assistance, which is about $5,000. The hours depend on the department, its workflows and your systems, and are agreed at scoping. The self-hosted route adds work that is scoped separately.
| Item | Applies to | Estimate |
|---|---|---|
| Workflow design, integration and the action gate | Every route, for each workflow | Within the 50 hours for each department |
| Test sets and evaluation, for each workflow | Every route | Within the 50 hours for each department |
| Data-protection review and processing agreement | Every route; heavier when data leaves your boundary | Within the 50 hours for each department |
| Serving stack, gateway and security hardening | Self-hosted | Extra; agreed at scoping |
| Fine-tuning a small model on your approved examples | Self-hosted and hosted open-weight, where evaluation supports it | Extra; agreed at scoping |
| Hardware purchase | Owned GPUs only | From the GPUs your estimate needs |
Not included on any route: management time, network egress, backups and disaster recovery beyond the shared platform, insurance, the cost of an error that reaches a customer or the ledger, migration away from a provider, and price changes after 15 November 2026. Each open-weight model's licence needs checking before use.
A workflow is a chain of small decisions. Each is done by a model, by ordinary code or by a person. Only model-backed units carry a model cost. Select a unit to see its size, tier and the model each route would use.
ModelCode, no model costPerson
Links an outside description to the right internal record. Code shortlists candidates with search and embeddings. The model picks one and gives the reason.
Examples: Supplier item descriptions to item codes, bank lines to invoices, duplicate customers.
Assumed size: 2,500 input and 200 output tokens per line item. An estimate, not a measurement.
Models: Claude (Anthropic): Claude Sonnet 5.5; OpenAI: gpt-6.1-sol; DeepSeek: deepseek-flash; Kimi (Moonshot AI): kimi-k3.1-pro; Open-weight models, hosted: Qwen3.8-27B on DeepInfra (list price); self-hosted: Qwen3.8-27B (FP8) on a 48 GB GPU.
Turns a document into structured fields. A model with a fixed output schema. The ERP validates every field.
Examples: Invoices, emailed purchase orders, RFQs, contracts, CVs, delivery notes.
Assumed size: 4,000 input and 800 output tokens per document. An estimate, not a measurement.
Models: Claude (Anthropic): Claude Haiku 5.5; OpenAI: gpt-6-luna; DeepSeek: deepseek-flash; Kimi (Moonshot AI): kimi-k3.1-pro; Open-weight models, hosted: Qwen3.5-9B on DeepInfra; self-hosted: Gemma 4 E4B (8-bit), fine-tuned on a 24 GB GPU.
Decides what something is and where it goes. A small model. Usually the cheapest unit.
Examples: Inbound email and tickets, expense categories, the language and intent of an enquiry.
Assumed size: 1,200 input and 50 output tokens per message. An estimate, not a measurement.
Models: Claude (Anthropic): Claude Haiku 5.5; OpenAI: gpt-6-luna; DeepSeek: deepseek-flash; Kimi (Moonshot AI): kimi-k3.1-pro; Open-weight models, hosted: Qwen3.5-9B on DeepInfra; self-hosted: Gemma 4 E4B (8-bit), fine-tuned on a 24 GB GPU.
Cleans and completes records to a standard. Code handles units and formats. A model handles free text.
Examples: Units of measure, item naming, addresses, missing attributes.
Assumed size: 1,000 input and 200 output tokens per record. An estimate, not a measurement.
Models: Claude (Anthropic): Claude Haiku 5.5; OpenAI: gpt-6-luna; DeepSeek: deepseek-flash; Kimi (Moonshot AI): kimi-k3.1-pro; Open-weight models, hosted: Qwen3.5-9B on DeepInfra; self-hosted: Gemma 4 E4B (8-bit), fine-tuned on a 24 GB GPU.
Spots what is wrong or unusual. Mostly code: thresholds and duplicate checks. A model sees only the records code cannot settle.
Examples: Duplicate invoices, price deviations from the order, odd expense claims.
Assumed size: 1,500 input and 150 output tokens per record flagged by code. An estimate, not a measurement.
Models: Claude (Anthropic): Claude Haiku 5.5; OpenAI: gpt-6-luna; DeepSeek: deepseek-flash; Kimi (Moonshot AI): kimi-k3.1-pro; Open-weight models, hosted: Qwen3.5-9B on DeepInfra; self-hosted: Gemma 4 E4B (8-bit), fine-tuned on a 24 GB GPU.
Weighs options against criteria. A top-tier model drafts the comparison. A person decides. Output includes the model's reasoning tokens, which are billed as output.
Examples: Supplier quote comparison, contract terms against a standard, candidate against a role.
Assumed size: 12,000 input and 3,000 output tokens per comparison. An estimate, not a measurement.
Models: Claude (Anthropic): Claude Opus 5.5; OpenAI: gpt-6-astra; DeepSeek: deepseek-v4-pro; Kimi (Moonshot AI): kimi-k3.1-pro; Open-weight models, hosted: gpt-oss-120b on Groq, Together or Fireworks; self-hosted: gpt-oss-120b on an 80 GB GPU.
Applies rules to approve, reject or escalate. Code and people. The rules are written or configured, so the same facts always give the same answer. A model may gather the facts, but it does not decide, so there is no model cost.
Examples: Lead qualification against criteria you set, credit limits, discount approvals, leave and expense claims.
Predicts a number or a date. Statistical code. Language models are poor at numeric forecasting.
Examples: Demand, cash flow, delivery dates.
Works out a feasible sequence under constraints. Solver code and people.
Examples: Reordering, shift rosters, delivery routes.
Answers from company records and documents, with sources. Search code finds the passages. A model answers and cites them.
Examples: What did we quote this client last year? Policy and SOP questions.
Assumed size: 6,000 input and 500 output tokens per question. An estimate, not a measurement.
Models: Claude (Anthropic): Claude Sonnet 5.5; OpenAI: gpt-6.1-sol; DeepSeek: deepseek-flash; Kimi (Moonshot AI): kimi-k3.1-pro; Open-weight models, hosted: Qwen3.8-27B on DeepInfra (list price); self-hosted: Qwen3.8-27B (FP8) on a 48 GB GPU.
Produces a first version for review. A model drafts. A person reviews before anything is sent.
Examples: Quotations, customer replies, requirement documents, reports.
Assumed size: 5,000 input and 1,500 output tokens per document drafted. An estimate, not a measurement.
Models: Claude (Anthropic): Claude Sonnet 5.5; OpenAI: gpt-6.1-sol; DeepSeek: deepseek-flash; Kimi (Moonshot AI): kimi-k3.1-pro; Open-weight models, hosted: Qwen3.8-27B on DeepInfra (list price); self-hosted: Qwen3.8-27B (FP8) on a 48 GB GPU.
Condenses activity and flags what needs attention. Code detects. A model writes the digest.
Examples: Meeting notes to actions, daily exception digest, overdue items.
Assumed size: 10,000 input and 600 output tokens per digest or meeting. An estimate, not a measurement.
Models: Claude (Anthropic): Claude Sonnet 5.5; OpenAI: gpt-6.1-sol; DeepSeek: deepseek-flash; Kimi (Moonshot AI): kimi-k3.1-pro; Open-weight models, hosted: Qwen3.8-27B on DeepInfra (list price); self-hosted: Qwen3.8-27B (FP8) on a 48 GB GPU.
Gathers missing information from people. About six turns. The history is sent again on each turn, so input grows, and most of it can be read from the prompt cache.
Examples: Chasing a supplier for a missing spec, collecting customer requirements.
Assumed size: 15,000 input and 1,200 output tokens per conversation. An estimate, not a measurement.
Models: Claude (Anthropic): Claude Sonnet 5.5; OpenAI: gpt-6.1-sol; DeepSeek: deepseek-flash; Kimi (Moonshot AI): kimi-k3.1-pro; Open-weight models, hosted: Qwen3.8-27B on DeepInfra (list price); self-hosted: Qwen3.8-27B (FP8) on a 48 GB GPU.
Reads images and drawings. A vision model. Current Claude and OpenAI models bill a full-page image at about 2,600 to 4,800 tokens, depending on resolution. Resizing photos before sending lowers this.
Examples: Item photo to catalogue item, damage on receipt, receipts.
Assumed size: 4,000 input and 200 output tokens per image. An estimate, not a measurement.
Models: Claude (Anthropic): Claude Sonnet 5.5; OpenAI: gpt-6.1-sol; DeepSeek: deepseek-flash; Kimi (Moonshot AI): kimi-k3.1-pro; Open-weight models, hosted: Qwen3.8-27B on DeepInfra (list price); self-hosted: Qwen3.8-27B (FP8) on a 48 GB GPU.
Sources for the size: Anthropic: vision (image tokens) · OpenAI: images and vision (image tokens)
Works across languages and vocabularies. Input and output are about the same size.
Examples: Multilingual descriptions, supplier jargon against internal terms.
Assumed size: 1,500 input and 1,500 output tokens per message. An estimate, not a measurement.
Models: Claude (Anthropic): Claude Haiku 5.5; OpenAI: gpt-6-luna; DeepSeek: deepseek-flash; Kimi (Moonshot AI): kimi-k3.1-pro; Open-weight models, hosted: Qwen3.5-9B on DeepInfra; self-hosted: Gemma 4 E4B (8-bit), fine-tuned on a 24 GB GPU.
Carries a multi-step task across systems to completion. Workflow code such as n8n, calling the other units. Orchestration, not a model.
Examples: Reminders, retries, hand-offs, escalation when stuck.
Supporting steps
Turns a scanned page into text, tables and layout. On an API a small vision model reads the page image: an A4 scan at 150 dpi is about 2,600 to 2,800 input tokens, and more at higher resolution. Self-hosted, a specialist OCR model does. Output of about 1,500 tokens a page matches the PaddleOCR-VL paper.
Examples: Scanned invoices, delivery notes, forms.
Assumed size: 3,000 input and 1,500 output tokens per scanned page. An estimate, not a measurement.
Models: Claude (Anthropic): Claude Haiku 5.5; OpenAI: gpt-6-luna; DeepSeek: deepseek-flash; Kimi (Moonshot AI): kimi-k3.1-pro; Open-weight models, hosted: Qwen3.5-9B on DeepInfra; self-hosted: PaddleOCR-VL on a 24 GB GPU.
Sources for the size: Anthropic: PDF support (tokens per page) · Anthropic: vision (image tokens) · OpenAI: images and vision (image tokens) · PaddleOCR-VL paper, table 13 (pages per second)
Transcribes calls and voice notes. Priced per audio minute on an API, and by GPU time when self-hosted.
Examples: Sales calls, WhatsApp voice notes.
Priced as: Claude (Anthropic): OpenAI gpt-transcribe, $0.0045 a minute; OpenAI: gpt-transcribe, $0.0045 a minute; DeepSeek: Whisper large-v3-turbo on Groq, $0.00067 a minute; Kimi (Moonshot AI): Whisper large-v3-turbo on Groq, $0.00067 a minute; Open-weight models, hosted: Whisper large-v3-turbo on Groq, $0.00067 a minute. Self-hosted: Whisper large-v3-turbo on a 24 GB GPU.
Screens what comes in and what goes out. A small model, separate from the models that do the work. The size covers screening the input and the output of one item.
Examples: Prompt injection hidden in a document, unsafe output.
Assumed size: 2,000 input and 20 output tokens per item screened. An estimate, not a measurement.
Models: Claude (Anthropic): Claude Haiku 5.5; OpenAI: gpt-6-luna; DeepSeek: deepseek-flash; Kimi (Moonshot AI): kimi-k3.1-pro; Open-weight models, hosted: Qwen3.5-9B on DeepInfra; self-hosted: Qwen3Guard (smallest size) on a 24 GB GPU.
Plans several steps and calls approved tools. The costliest unit: each tool call sends the whole history again, so most input can be read from the prompt cache. Sized for a short run of about eight model calls with reasoning kept low; a long conversation with many tool calls can use four times as much. The action gate in code still checks every write.
Examples: The website enquiry agent that logs a lead or opens a requirements session.
Assumed size: 50,000 input and 5,000 output tokens per run. An estimate, not a measurement.
Models: Claude (Anthropic): Claude Opus 5.5; OpenAI: gpt-6-astra; DeepSeek: deepseek-v4-pro; Kimi (Moonshot AI): kimi-k3.1-pro; Open-weight models, hosted: gpt-oss-120b on Groq, Together or Fireworks; self-hosted: gpt-oss-120b on an 80 GB GPU.
Reviews the proposal and approves, amends or rejects. Staff time, shown separately. Assumed the same on every route; a weaker model may need more.
Examples: Posting an invoice, sending a quotation.
List prices in US dollars, before tax and volume discounts. Providers change prices often, so check the source before relying on a figure.
| Provider | Tier | Model | Input | Cached input | Output | Source |
|---|---|---|---|---|---|---|
| Claude (Anthropic) | Small | Claude Haiku 5.5 | $0.10 | $0.01 | $0.50 | Anthropic: Claude API pricing |
| Claude (Anthropic) | Mid | Claude Sonnet 5.5 | $2.00 | $0.1 | $10.00 | Anthropic: Claude API pricing |
| Claude (Anthropic) | Top | Claude Opus 5.5 | $4.00 | $0.2 | $20.00 | Anthropic: Claude API pricing |
| Claude (Anthropic) | Embeddings | OpenAI text-embedding-3-small | $0.02 | OpenAI: API pricing | ||
| Claude (Anthropic) | Speech to text | OpenAI gpt-transcribe | $0.0045 per audio minute | OpenAI: API pricing | ||
| OpenAI | Small | gpt-6-luna | $0.10 | $0.01 | $0.50 | OpenAI: API pricing |
| OpenAI | Mid | gpt-6.1-sol | $2.00 | $0.1 | $10.00 | OpenAI: API pricing |
| OpenAI | Top | gpt-6-astra | $10.00 | $1.0 | $50.00 | OpenAI: API pricing |
| OpenAI | Embeddings | text-embedding-3-small | $0.02 | OpenAI: API pricing | ||
| OpenAI | Speech to text | gpt-transcribe | $0.0045 per audio minute | OpenAI: API pricing | ||
| DeepSeek | Small | deepseek-flash | $0.30 | $0.006 | $1.20 | DeepSeek: models and pricing |
| DeepSeek | Mid | deepseek-flash | $0.30 | $0.006 | $1.20 | DeepSeek: models and pricing |
| DeepSeek | Top | deepseek-v4-pro | $1.32 | $0.044 | $3.96 | DeepSeek: models and pricing |
| DeepSeek | Embeddings | BGE-M3 on DeepInfra | $0.01 | DeepInfra: pricing | ||
| DeepSeek | Speech to text | Whisper large-v3-turbo on Groq | $0.00067 per audio minute | Groq: models and pricing | ||
| Kimi (Moonshot AI) | Small | kimi-k3.1-pro | $2.00 | $0.2 | $10.00 | Moonshot AI: Kimi API pricing (international) |
| Kimi (Moonshot AI) | Mid | kimi-k3.1-pro | $2.00 | $0.2 | $10.00 | Moonshot AI: Kimi API pricing (international) |
| Kimi (Moonshot AI) | Top | kimi-k3.1-pro | $2.00 | $0.2 | $10.00 | Moonshot AI: Kimi API pricing (international) |
| Kimi (Moonshot AI) | Embeddings | BGE-M3 on DeepInfra | $0.01 | DeepInfra: pricing | ||
| Kimi (Moonshot AI) | Speech to text | Whisper large-v3-turbo on Groq | $0.00067 per audio minute | Groq: models and pricing | ||
| Open-weight models, hosted | Small | Qwen3.5-9B on DeepInfra | $0.10 | $0.1 | $0.15 | DeepInfra: pricing |
| Open-weight models, hosted | Mid | Qwen3.8-27B on DeepInfra (list price) | $0.20 | $0.05 | $2.50 | DeepInfra: pricing |
| Open-weight models, hosted | Top | gpt-oss-120b on Groq, Together or Fireworks | $0.15 | $0.15 | $0.60 | Groq: models and pricing · Together AI: pricing · Fireworks AI: serverless pricing |
| Open-weight models, hosted | Embeddings | BGE-M3 on DeepInfra | $0.01 | DeepInfra: pricing | ||
| Open-weight models, hosted | Speech to text | Whisper large-v3-turbo on Groq | $0.00067 per audio minute | Groq: models and pricing | ||
| Class | Provider | GPU | Per hour | Always on, per month | Source |
|---|---|---|---|---|---|
| 24 GB GPU | RunPod Secure Cloud | RTX 4090 | $0.89 | $650 | RunPod: GPU pricing |
| 24 GB GPU | RunPod Secure Cloud | L4 | $0.69 | $504 | RunPod: GPU pricing |
| 24 GB GPU | RunPod Secure Cloud | RTX A5000 | $0.27 | $197 | RunPod: GPU pricing |
| 24 GB GPU | Vast.ai (median) | RTX 4090 | $0.50 | $368 | Vast.ai: live marketplace offers (median of verified single-GPU offers) |
| 24 GB GPU | Lambda | A10 | $1.29 | $942 | Lambda: cloud pricing (rates effective 16 Nov 2026) |
| 24 GB GPU | AWS, US East | g6.xlarge (L4) | $0.80 | $588 | AWS EC2 on-demand price list |
| 24 GB GPU | AWS, UAE | g6.xlarge (L4) | $0.99 | $721 | AWS EC2 on-demand price list |
| 24 GB GPU | Azure, East US | NV36ads A10 v5 | $3.20 | $2,336 | Azure retail prices API |
| 48 GB GPU | RunPod Secure Cloud | L40S | $1.09 | $796 | RunPod: GPU pricing |
| 48 GB GPU | RunPod Secure Cloud | RTX A6000 | $0.69 | $504 | RunPod: GPU pricing |
| 48 GB GPU | Hyperstack | RTX A6000 | $0.50 | $365 | Hyperstack: GPU pricing |
| 48 GB GPU | Vast.ai (median) | L40S | $0.82 | $600 | Vast.ai: live marketplace offers (median of verified single-GPU offers) |
| 48 GB GPU | Lambda | RTX A6000 | $1.09 | $796 | Lambda: cloud pricing (rates effective 16 Nov 2026) |
| 48 GB GPU | Crusoe | L40S | $1.50 | $1,095 | Crusoe: cloud pricing |
| 48 GB GPU | Verda | L40S | $2.02 | $1,473 | Verda (formerly DataCrunch): pricing |
| 48 GB GPU | AWS, US East | g6e.xlarge (L40S) | $1.86 | $1,359 | AWS EC2 on-demand price list |
| 48 GB GPU | AWS, UAE | g6e.xlarge (L40S) | $2.28 | $1,667 | AWS EC2 on-demand price list |
| 80 GB GPU | RunPod Secure Cloud | A100 80 GB PCIe | $2.09 | $1,526 | RunPod: GPU pricing |
| 80 GB GPU | RunPod Secure Cloud | H100 PCIe | $2.89 | $2,110 | RunPod: GPU pricing |
| 80 GB GPU | Hyperstack | A100 80 GB | $1.35 | $986 | Hyperstack: GPU pricing |
| 80 GB GPU | Hyperstack | H100 | $2.50 | $1,825 | Hyperstack: GPU pricing |
| 80 GB GPU | Lambda | H100 PCIe | $3.79 | $2,767 | Lambda: cloud pricing (rates effective 16 Nov 2026) |
| 80 GB GPU | Vast.ai (median) | RTX PRO 6000 96 GB | $1.74 | $1,269 | Vast.ai: live marketplace offers (median of verified single-GPU offers) |
| 80 GB GPU | Crusoe | H100 (rises to 4.50 on 1 Dec 2026) | $3.90 | $2,847 | Crusoe: cloud pricing |
| 80 GB GPU | Verda | H100 SXM | $5.24 | $3,826 | Verda (formerly DataCrunch): pricing |
| 80 GB GPU | CoreWeave | H100 | $6.16 | $4,493 | CoreWeave: pricing (8-GPU nodes, shown per GPU) |
| 80 GB GPU | Core42 (UAE) | H100, 'from' price | $2.50 | $1,825 | Core42 AI Cloud (UAE): 'from' prices |
| 80 GB GPU | AWS, US East | p5.4xlarge (H100) | $6.88 | $5,022 | AWS EC2 on-demand price list |
| 80 GB GPU | Azure, East US | NC40ads H100 v5 | $6.98 | $5,095 | Azure retail prices API |
| 80 GB GPU | Azure, UAE North | NC40ads H100 v5 | $9.98 | $7,285 | Azure retail prices API |
A month is 730 hours. Each rented GPU also carries $20 a month for disk: 200 gb of disk per gpu at runpod's $0.10 per gb a month, applied to every price list. Committed and reserved prices are lower and are not shown. The Vast.ai figures are medians of live marketplace offers from independent hosts. The AWS and Azure rows name the instance type; the GPU inside is from the providers' instance documentation.
| Class | Runs | Card price | Rated power | Throughput we assume, and why |
|---|---|---|---|---|
| 24 GB GPU | small models, OCR, speech, embeddings and the guard | $3,250 to $6,183 RTX 4090 24 GB on the resale market to RTX 5090 32 GB. gpupoet: resale price tracker | 450 W NVIDIA product pages: rated power | 350 output and 5,000 input tokens a second. 8B models on a dedicated A10 24 GB reached 349 output tokens a second at 16 concurrent requests, and a classifier workload about 10,000 input tokens a second (TrueFoundry); we use 350 and half that input rate. When three or more of these models share the card, all rates are halved. Sharing five models needs the small model at 8-bit and leaves no room for long prompts, so it suits a pilot. OCR: PaddleOCR-VL ran 1.35 pages a second on an A100 under vLLM; we assume 0.5 on a 24 GB card. Speech: published real-time factors run from 46x (faster-whisper, 8 GB card) to 145x (A100); we assume 40x. Embeddings: BGE-M3 at about 97,000 tokens a second on an RTX A5000 (RunPod). TrueFoundry: vLLM benchmark, 8B models on an A10 24 GB · PaddleOCR-VL paper, table 13 (pages per second) · faster-whisper README benchmarks · Open ASR Leaderboard paper (real-time factors) · RunPod: embedding throughput benchmark (BGE-M3) · Google: Gemma 4 memory requirements |
| 48 GB GPU | the mid-tier model | $7,072 to $8,965 RTX 6000 Ada to L40S, 48 GB. Reseller listings (CDW, Amazon, Saitech), read from search results | 350 W NVIDIA product pages: rated power | 500 output and 4,000 input tokens a second. We found no published benchmark for a quantised 27B model on one 48 GB card. Nearest: Gemma 3 27B unquantised gave about 1,000 output tokens a second on two A100 40 GB cards with 100-token prompts (Database Mart), and 4.17 requests a second with 1,500-token prompts on one 80 GB-class card (Google Cloud), from which we derive about 830 output tokens a second. We assume 500 output and 4,000 input tokens a second on one L40S, which puts that 1,500-token case at about 1.3 requests a second, a third of the Google figure. With the model loaded, roughly 17 GB is left for requests in flight, which limits it to about six to ten concurrent 6,000-token prompts. Database Mart: vLLM benchmark on two A100 40 GB (Gemma 3 27B) · Google Cloud: vLLM performance tuning (Gemma 3 27B sizing case) · Google: Gemma 4 memory requirements |
| 80 GB GPU | the top-tier model | $12,550 to $27,295 A100 80 GB, refurbished, to H100 80 GB at list price. Network Outlet: A100 80 GB, refurbished · Supermicro store: H100 80 GB PCIe (listed, out of stock) | 350 W NVIDIA product pages: rated power | 1,500 output and 9,000 input tokens a second. gpt-oss-120b fits one 80 GB GPU (OpenAI model card). It gave about 1,500 output tokens a second at 50 concurrent requests on an RTX Pro 6000 96 GB with 100-token prompts (Database Mart), which we use as measured. With 8,000-token prompts an H100 handled about 5,900 to 9,000 input and output tokens a second, depending on the speed each user gets (InferenceX); our input rate is set so the formula stays below that. OpenAI gpt-oss-120b model card (fits one 80 GB GPU) · Database Mart: vLLM benchmarks on RTX Pro 6000 and A100 · InferenceX: gpt-oss-120b on H100 |
Card prices are wide ranges because the market is unsettled: several come from resale trackers and reseller listings, not manufacturer price lists, and an owned-hardware figure needs a supplier quotation. Each GPU is assumed to need one host machine per GPU, from a desktop base to a rackmount base (BIZON starting prices), $3,788 to $9,415 (BIZON: workstation and rackmount base prices). Power is gpu rated power plus half again for the rest of the machine and cooling. Throughput figures are taken from, or derated from, the nearest published benchmarks, which were run on other hardware and workloads; input rates are our own derivation. GPUs are sized to run at 65% of these rates in the busiest hour. A consumer card such as an RTX 4090 may not be licensed or built for a server room, and a desktop host cannot cool a data-centre card.
The throughput figures above are for one reference card in each class. Rental lists and hardware prices name other cards, so each is counted at a multiple of the reference card's speed, and a slower card needs more hours or more cards for the same work.
| Class | Card | Speed we assume | Why |
|---|---|---|---|
| 24 GB GPU | A10 (reference) | 1× | The card the throughput figures were measured on. |
| 24 GB GPU | L4 | 0.55× | On the same llama.cpp test an L4 instance generated 53.5 tokens a second against 94.2 on an A10G instance (7B model), and 25.2 against 47.8 (14B): 0.53 to 0.57. Single requests, not a loaded server. Spare Cores: llama.cpp speed on cloud servers (AWS g5.xlarge with an A10G, g6.xlarge with an L4) |
| 24 GB GPU | RTX 4090 | 1.4× | No publisher we found tested both cards. Two llama.cpp tests of 4-bit 7B and 8B models give 127.7 tokens a second on an RTX 4090 and 94.2 on an A10G, about 1.4. Chaining Runpod's vLLM figures through the L4 gives about 2. We use the lower figure. XiongjieDai: GPU benchmarks on LLM inference (llama.cpp, May 2024) · Spare Cores: llama.cpp speed on cloud servers (AWS g5.xlarge with an A10G, g6.xlarge with an L4) · Runpod: 21 GPUs under vLLM, Qwen2.5-7B at one request a second (measured 15 Sep 2026) |
| 24 GB GPU | RTX 5090 | 1.9× | Published tests put an RTX 5090 at 1.35 to 1.66 times an RTX 4090: 101 against 61 tokens a second per user at light load (Runpod), and 1.35 under concurrent load (Trooper.ai). We use 1.4 times the RTX 4090 figure, rounded down. Runpod: 21 GPUs under vLLM, Qwen2.5-7B at one request a second (measured 15 Sep 2026) · Trooper.ai: RTX 5090 against RTX 4090 |
| 48 GB GPU | L40S (reference) | 1× | The card the throughput figures are assumed for. |
| 48 GB GPU | RTX A6000 | 0.9× | 44 against 48 tokens a second per user under vLLM at light load (Runpod), and 102 against 114 on llama.cpp (XiongjieDai): 0.90 to 0.92. Neither is a loaded-server test. Runpod: 21 GPUs under vLLM, Qwen2.5-7B at one request a second (measured 15 Sep 2026) · XiongjieDai: GPU benchmarks on LLM inference (llama.cpp, May 2024) |
| 48 GB GPU | RTX 6000 Ada | 1.15× | 58 against 48 tokens a second per user under vLLM at light load (Runpod), and 131 against 114 on llama.cpp (XiongjieDai): 1.15 to 1.21. We use the lower figure. Runpod: 21 GPUs under vLLM, Qwen2.5-7B at one request a second (measured 15 Sep 2026) · XiongjieDai: GPU benchmarks on LLM inference (llama.cpp, May 2024) |
| 80 GB GPU | H100 (reference) | 1× | The card the input rate was checked against. PCIe and SXM versions are counted the same, though Runpod measured the SXM version about half as fast again. |
| 80 GB GPU | A100 80 GB | 0.8× | Under vLLM at batch 32 an A100 ran at 0.70 to 0.85 of an H100 on 7B and 8B models (Koyeb), and at 0.86 per user at light load (Runpod, PCIe cards). A model that nearly fills the card's memory, as the top-tier model does, may do worse. Koyeb: GPU benchmarks (vLLM, batch 32) · Runpod: 21 GPUs under vLLM, Qwen2.5-7B at one request a second (measured 15 Sep 2026) |
| 80 GB GPU | RTX PRO 6000 | 1× | Our output rate was measured on this card. Published comparisons with an H100 disagree, from 0.82 (Runpod, light load) to above 1 (Database Mart), so we count it the same. Runpod: 21 GPUs under vLLM, Qwen2.5-7B at one request a second (measured 15 Sep 2026) · Database Mart: vLLM benchmarks on RTX Pro 6000 and A100 |
These multiples are estimates from published tests of smaller models, mostly single requests or light load, not from tests of the models priced here on a loaded server. A pilot on the chosen card replaces them.
Add a department to build your estimate.
We also receive a link that reopens this estimate exactly as you built it. Nothing is sent until you press the button.