Throughput Intermediate

Throughput

Intermediate

How many requests or tokens can be processed per unit time; affects scalability and cost.

AdvertisementAd space — term-top

Definition

Full Definition

How many requests or tokens can be processed per unit time; affects scalability and cost.

Keywords

QPS tokens/sec

Domains

Transformers & LLMs

Related Terms

Attention related to

Mechanism that computes context-aware mixtures of representations; scales well and captures long-range dependencies.

 Self-Attention related to

Attention where queries/keys/values come from the same sequence, enabling token-to-token interactions.

 Vocabulary related to

The set of tokens a model can represent; impacts efficiency, multilinguality, and handling of rare strings.

 Context Window related to

Maximum number of tokens the model can attend to in one forward pass; constrains long-document reasoning.

Welcome to AI Glossary

The free, self-building AI dictionary. Help us keep it free—click an ad once in a while!

Search

Type any question or keyword into the search bar at the top.

Browse

Tap a letter in the A–Z bar to browse terms alphabetically, or filter by domain, industry, or difficulty level.

3D WordGraph

Fly around the interactive 3D graph to explore how AI concepts connect. Click any word to read its full definition.