d-Matrix

d-Matrix was founded in 2019 by Sid Sheth, Sudeep Bhoja, and Jayram Bhat, semiconductor industry veterans who identified a specific and increasingly important gap in the AI chip market: while companies like Nvidia have dominated the market for training large AI models, a computationally intensive process performed relatively infrequently to create a model, the cost of running, or "inferencing," those already-trained models to actually serve user requests in production has grown into an enormous and continuously recurring computational expense for companies deploying AI at scale, and many existing AI chips designed primarily with training workloads in mind are not necessarily the most cost- or energy-efficient option for this different inference-specific workload profile. d-Matrix's chip architecture is specifically optimized for inference workloads, employing a technique called "digital in-memory compute" that performs certain computations directly within memory rather than requiring data to be repeatedly shuttled back and forth between separate memory and processing units, a common performance and energy efficiency bottleneck in traditional chip architectures, particularly relevant to the memory-intensive process of running large language models where model weights must be rapidly accessed to generate each new output token. The company has argued that this specialized inference-focused architecture can offer meaningfully better cost and energy efficiency for running large language models in production compared to using general-purpose GPUs originally designed with training workloads as the primary consideration, an increasingly significant argument as the costs of serving popular AI applications and chatbots to millions of users have become a major and continuously growing expense for AI companies. d-Matrix has raised substantial venture funding from investors including Microsoft, Nvidia's competitor ecosystem investors, Playground Global, and Temasek, reaching a valuation in the billions of dollars, positioning itself within a growing category of specialized AI inference chip startups competing against both incumbent GPU makers and other emerging inference-focused chip designers.

News

No news available for this company.