Connexion
SK hynix and TSMC announced early on Friday that they had signed a memorandum of understanding to collaborate on developing the next-generation HBM4 memory and advanced packaging technology. The initiative is designed to speed up the adoption of HBM4 memory and solidify SK hynix's and TSMC's leading positions in high-bandwidth memory and advanced processor applications.
The primary focus of SK hynix's and TSMC's initial efforts will be to enhance the performance of the HBM4 stack's base die, which (if we put it very simply) acts like an ultra-wide interface between memory devices and host processors. With HBM4, SK hynix plans to use one of TSMC's advanced logic process technologies to build base dies to pack additional features and I/O pins within the confines of existing spatial constraints.
This collaborative approach also enables SK hynix to customize HBM solutions to satisfy diverse customer performance and energy efficiency requirements. HBM has been touting custom HBM solutions for a while, and teaming up with TSMC will undoubtedly help with this.
"TSMC and SK hynix have already established a strong partnership over the years. We've worked together in integrating the most advanced logic and state-of-the art HBM in providing the world's leading AI solutions," said Dr. Kevin Zhang, Senior Vice President of TSMC's Business Development and Overseas Operations Office, and Deputy Co-Chief Operating Officer. "Looking ahead to the next-generation HBM4, we're confident that we will continue to work closely in delivering the best-integrated solutions to unlock new AI innovations for our common customers."
Furthermore, the collaboration extends to optimizing the integration of SK hynix's HBM with TSMC's CoWoS advanced packaging technology. CoWoS is among the most popular specialized 2.5D packaging process technologies for integrating logic chips and stacked HBM into a unified module.
For now, it is expected that HBM4 memory will be integrated with logic processors using direct bonding. However, some of TSMC's customers might prefer to use an ultra-advanced version of CoWoS to integrate HBM4 with their processors.
"We expect a strong partnership with TSMC to help accelerate our efforts for open collaboration with our customers and develop the industry's best-performing HBM4," said Justin Kim, President and the Head of AI Infra at SK hynix. "With this cooperation in place, we will strengthen our market leadership as the total AI memory provider further by beefing up competitiveness in the space of the custom memory platform."
Corsair has introduced a family of registered memory modules with ECC that are designed for AMD's Ryzen Threadripper 7000 and Intel's Xeon W-2400/3400-series processors. The new Corsair WS DDR5 RDIMMs with AMD EXPO and Intel XMP 3.0 profiles will be available in kits of up to 256 GB capacity and at speeds of up to 6400 MT/s.
Corsair's family of WS DDR5 RDIMMs includes 16 GB modules operating at up to 6400 MT/s with CL32 latency as well as 32 GB modules functioning at 5600 MT/s with CL40 latency. At present, Corsair offers a quad-channel 64 GB kit (4×16GB, up to 6400 MT/s), a quad-channel 128GB kit (4×32GB, 5600 MT/s), an eight-channel 128 GB kit (8×16GB, 5600 MT/s), and an eight-channel 256 GB kit (8×32GB, 5600 MT/s) and it remains to be seen whether the company will expand the lineup.
Corsair's WS DDR5 RDIMMs are designed for AMD's TRX50 and WRX90 platforms as well as Intel's W790 platform and are therefore compatible with AMD's Ryzen Threadripper Pro 7000 and 7000WX-series as well as Intel's Xeon W-2400/3400-series CPUs. The modules feature both AMD EXPO and Intel XMP 3.0 profiles to easily set their beyond-JEDEC-spec settings and come with thin heat spreaders made of pyrolytic graphite sheet (PGS), which thermal conductivity than that of copper and aluminum of the same thickness. For now, Corsair does not disclose which RCD and memory chips its registered memory modules use.
Unlike many of its rivals among leading DIMM manufacturers, Corsair did not introduce its enthusiast-grade RDIMMs when AMD and Intel released their Ryzen Threadripper and Xeon W-series platforms for extreme workstations last year. It is hard to tell what the reason for that is, but perhaps the company wanted to gain experience working with modules featuring registered clock drivers (RCDs) as well as AMD's and Intel's platforms for extreme workstations.
The result of the delay looks to be quite rewarding: unlike modules from its competitors that either feature AMD EXPO or Intel XMP 3.0 profiles, Corsair's WS DDR5 RDIMMs come with both. While this may not be important on the DIY market where people know exactly what they are buying for their platform, this is a great feature for system integrators, which can use Corsair WS DDR5 RDIMMs both for their AMD Ryzen Threadripper and Intel Xeon W-series builds, something that greatly simplifies their inventory management.
Since Corsair's WS DDR5 RDIMMs are aimed at workstations and are tested to offer reliable performance beyond JEDEC specifications, they are quite expensive. The cheapest 64 GB DDR5-5600 CL40 kit costs $450, the fastest 64 GB DDR5-6400 CL32 kit is priced at $460, whereas the highest end 256 GB DDR5-5600 CL40 kit is priced at $1,290.
Following the magnitude 7.2 earthquake that struck Taiwan on April 3, 2024, there was immediate concern over what impact this could have on chip production within the country. Even for a well-prepared country like Taiwan, the tremor was the strongest quake to hit the region in 25 years, making it no small matter. But, according to research compiled by TrendForce, the impact on the production of DRAM will not be significant. The market tracking company believes that Taiwanese DRAM industry has remained largely unaffected, primarily due to their robust earthquake preparedness measures.
There are four memory makers in Taiwan: Micron, the sole member of the "big three" memory manufacturers on the island, runs two fabs. Meanwhile among the smaller players is Nanya (which has one fab), Winbond (which makes specialty memory at one fab), and PSMC (which produces specialty memory at one plant). The study found that these DRAM producers quickly resumed full operations, but had to throw away some wafers. The earthquake is estimated to have a minor effect on Q2 DRAM production, with a negligible 1% impact, TrendForce claims
In fact, as Micron is ramping up production of DRAM on its 1alpha and 1beta nm process technologies, it increases bit production of memory, which will positively affect supply of commodity DRAM in Q2 2025.
Following the earthquake, there was a temporary halt in quotations for both the contract and spot DRAM markets. However, the spot market quotations have already largely resumed, while contract prices have not fully restarted. Notably, Micron and Samsung ceased issuing quotes for mobile DRAM immediately after the earthquake, with no updates provided as of April 8th. In contrast, SK hynix resumed quotations for smartphone customers on the day of the earthquake and proposed more moderate price adjustments for Q2 mobile DRAM.
TrendForce anticipates a seasonal contract price increase for Q2 mobile DRAM of between 3% and 8%. This moderate increase is partly due to SK hynix's more restrained pricing strategy, which is likely to influence overall pricing strategies across the industry. The earthquake's impact on server DRAM primarily affected Micron's advanced fabrication nodes, potentially leading to a rise in final sale prices for Micron's server DRAM, according to TrendForce. However, the exact direction of future prices remains to be seen.
Meanwhile, DRAM fabs outside of Taiwan have none been directly affected by the quake. This includes Micron's HBM production line in Hiroshima, Japan, and Samsung's and SK hynix's HBM lines in South Korea, all of which are apparently operating with business as usual.
In general, the DRAM industry has shown resilience in the face of the earthquake, with minimal disruptions and a quick recovery. The abundant inventory levels for DDR4 and DDR5, coupled with weak demand, suggest that any slight price elevations caused by the earthquake are expected to normalize quickly. The only potential outlier here is DDR3, which is nearing the end of its commercial lifetime and production is already decreasing.
Composable disaggregated data center infrastructure promises to change the way data centers for modern workloads are built. However, to fully realize the potential of new technologies, such as CXL, the industry needs brand-new hardware. Recently, Samsung introduced its CXL Memory Module Box (CMM-B), a device that can house up to eight CXL Memory Module – DRAM (CMM-D) devices and add plenty of memory connected using a PCIe/CXL interface.
Samsung's CXL Memory Module Box (CMM-B) is the first device of this type to accommodate up to eight 2 TB E3.S CMM-D memory modules and add up to 16 TB of memory to up to three modern servers with appropriate connectors. As far as performance is concerned, the box can offer up to 60 GB/s of bandwidth (which aligns with what a PCIe 5.0 x16 interface offers) and 596 ns latency.
From a pure performance point of view, one CXL Memory Module—Box is slower than a dual-channel DDR5-4800 memory subsystem. Yet, the unit is still considerably faster than even advanced SSDs. At the same time, it provides very decent capacity, which is often just what the doctor ordered for many applications.
The Samsung CMM-B is compatible with the CXL 1.1 and CXL 2.0 protocols. It consists of a rack-scale memory bank (CMM-B), several application hosts, Samsung Cognos management console software, and a top-of-rack (ToR) switch. The device was developed in close collaboration with Supermicro, so expect this server maker to offer the product first.
Samsung's CXL Memory Module – Box is designed for applications that need a lot of memory, such as AI, data analytics, and in-memory databases, albeit not at all times. CMM-B allows the dynamic allocation of necessary memory to a system when it needs this memory and then uses DRAM with other machines. As a result, operators of datacenters can spend money on procuring expensive memory (16 TB of memory costs a lot), reduce power consumption, and add flexibility to their setups.
Offering some rare insight into the scale of HBM memory sales – and on its growth in the face of unprecedented demand from AI accelerator vendors – the company recently disclosed that it expects HBM sales to make up "a double-digit percentage of its DRAM chip sales" this year. Which if it comes to pass, would represent a significant jump in sales for the high-bandwidth, high-priced memory.
As first reported by Reuters, SK hynix CEO Kwak Noh-Jung has commented that he expects HBM sales will constitute a double-digit percentage of its DRAM chip sales in 2024. This prediction corroborate with estimates from TrendForce, who believe that, industry-wide, HBM will account for 20.1% of DRAM revenue in 2024, more than doubling HBM's 8.4% revenue share in 2023.
And while SK hynix does not break down its DRAM revenue by memory type on a regular basis, a bit of extrapolation indicates that they're on track to take in billions in HBM revenue for 2024 – having likely already crossed the billion dollar mark itself in 2023. Last year, SK hynix's DRAM revenue $15.941 billion, according to Statista and TrendForce. So SK hynix only needs 12.5% of its 2024 revenues to come from HBM (assuming flat or positive revenue overall) in order to pass 2 billion in HBM sales. And even this is a low-ball estimate.
Overall, SK hynix currently commands about 50% of HBM market, having largely split the market with Samsung over the last couple of years. Given that share, and that DRAM industry revenue is expected to increase to $84.150 billion in 2024, SK hynix could earn as much as $8.45 billion on HBM in 2024 if TrendForce's estimates prove accurate.
It should be noted that with demand for AI servers at record levels, all three leading makers of DRAM are poised to increase their HBM production capacity this year. Most notable here is a nearly-absent Micron, who was the first vendor to start shipping HBM3E memory to NVIDIA earlier this year. So SK hynix's near-majority of the HBM market may falter some this year, though with a growing pie they'll have little reason to complain. Ultimately, if sales of HBM reach $16.9 billion as projected, then all memory makers will be enjoying significant HBM revenue growth in the coming months.
Sources: Reuters, TrendForce
Now that JEDEC has published specification of GDDR7 memory, memory manufacturers are beginning to announce their initial products. The first out of the gate for this generation is Samsung, which has has quietly added its GDDR7 products to its official product catalog.
For now, Samsung lists two GDDR7 devices on its website: 16 Gbit chips rated for an up to 28 GT/s data transfer rate and a faster version running at up to 32 GT/s data transfer rate (which is in line with initial parts that Samsung announced in mid-2023). The chips feature a 512M x32 organization and come in a 266-pin FBGA packaging. The chips are already sampling, so Samsung's customers – GPU vendors, AI inference vendors, network product vendors, and the like – should already have GDDR7 chips in their labs.
The GDDR7 specification promises the maximum per-chip capacity of 64 Gbit (8 GB) and data transfer rates of 48 GT/s. Meanwhile, first generation GDDR7 chips (as announced so far) will feature a rather moderate capacity of 16 Gbit (2 GB) and a data transfer rate of up to 32 GT/s.
Performance-wise, the first generation of GDDR7 should provide a significant improvement in memory bandwidth over GDDR6 and GDDR6X. However capacity/density improvements will not come until memory manufacturers move to their next generation EUV-based process nodes. As a result, the first GDDR7-based graphics cards are unlikely to sport any memory capacity improvements. Though looking a bit farther down the road, Samsung and SK Hynix have previously told Tom's Hardware that they intend to reach mass production of 24 Gbit GDDR7 chips in 2025.
Otherwise, it is noteworthy that SK Hynix also demonstrated its GDDR7 chips at NVIDIA's GTC last week. So Samsung's competition should be close behind in delivering samples, and eventually mass production memory.
Source: Samsung (via @harukaze5719)
SK hynix is considering whether to build an advanced packaging facility in Indiana, reports the Wall Street Journal. If the company proceeds with the plan, it intends to invest $4 billion in it and construct one of the world's largest advanced packaging facilities. But to accomplish the project, SK hynix expects it will need help from the U.S. government.
Acknowledging the report but stopping short of confirming the company's plans, a company spokeswoman told the WSJ that SK hynix "is reviewing its advanced chip packaging investment in the U.S., but hasn’t made a final decision yet."
Companies like TSMC and Intel spend billions on advanced packaging facilities, but so far, no company has announced a chip packaging plant worth quite as much as SH hynix's $4 billion. The field of advanced packaging – CoWoS, passive silicon interposers, redistribution layers, die-to-die bonding, and other cutting edge technologies – has seen an explosion in demand in the last half-decade. As bandwidth advances with traditional organic packaging are largely played out, chip designers have needed to turn to more complex (and difficult to assemble) technologies in order to wire up an ever larger number of signals at ever-higher transfer rates. Which has turned advanced packaging into a bottleneck for high-end chip and accelerator production, driving a need for additional packaging facilities.
If SK hynix approves the project, the advanced packaging facility is expected to begin operations in 2028 and could create as many as 1,000 jobs. With an estimated cost of $4 billion, the plant is poised to become one of the largest advanced packaging facilities in the world.
Meanwhile, government backing is thought to be essential for investments of this scale, with potential state and federal tax incentives, according to the report. These incentives form part of a broader initiative to bolster the U.S. semiconductor industry and decrease dependence on memory produced in South Korea.
SK hynix is the world's leading producer of HBM memory, and is one of the key HBM suppliers to NVIDIA. Next generations of HBM memory (including HBM4 and HBM4E) will require even closer collaboration between chip designers, chipmakers, and memory makers. Therefore, packaging HBM in America could be a significant benefit for NVIDIA, AMD, and other U.S. chipmakers.
Investing in the Indiana facility will be a strategic move by SK hynix to enhance its advanced chip packaging capabilities in general and demonstrating dedication to the U.S. semiconductor industry.
When a major industry slowdown occurs, big companies tend to slowdown their mid-term and long-term capacity related investments. This is exactly what happened to SK hynix's Yongin Semiconductor Cluster, a major project announced in April 2021 and valued at $106 billion. While development of the site has been largely completed, only 35% of the initial shell building has been constructed, according to the Korean Ministry of Trade, Industry, and Energy.
"Approximately 35% of Fab 1 has been completed so far and site renovation is in smooth progress," a statement by the Korean Ministry of Trade, Industry, and Energy reads. "By 2046, over KRW 120 trillion ($90 billion today, $106 billion in 2021) in investment will be poured to complete Fabs 1 through 4, and construction of Fab 1's production line will commence in March next year. Once completed, the infrastructure will rank as the world's largest three-story fab."
The new semiconductor fabrication cluster by SK hynix announced almost exactly three years ago is primarily meant to be used to make DRAM for PCs, mobile devices, and servers using advanced extreme ultraviolet lithography (EUV) process technologies. The cluster, located near Yongin, South Korea, is intended to consist of four large fabs situated on a 4.15 million m2 site. With a planned capacity of approximately 800,000 wafer starts per month (WSPMs), it is set to be one of the world's largest semiconductor production hubs.
With that said, SK hynix's construction progress has been slower than the company first projected. The first fab in the complex was originally meant to come online in 2025, with construction starting in the fourth quarter of 2021. However, SK hynix began to cut its capital expenditures in the second half of 2022, and the Yongin Semiconductor Cluster project fell a victim of that cut. To be sure, the site continues to be developed, just at a slower pace; which is why some 35% of the first fab shell has been built at this point.
If completed as planned in 2021, the first phase of SK hynix Yongin operations would have been a major memory production facility costing $25 billion, equipped with EUV tools, and capable of 200,000-WSPM, according to reports from 2021.
Sources: Korean Ministry of Trade, Industry, and Energy; ComputerBase
Micron this week announced that it had begun sampling of its 256 GB multiplexer combined (MCR) DIMMs, the company's highest-capacity memory modules to date. These brand-new DDR5-based MCRDIMMs are aimed at next-generation servers, particularly those powered by Intel's Xeon Scalable 'Granite Rapids' processors that are set to support 12 or 24 memory slots per socket. Usage of these modules can enable datacenter machines with 3 TB or 6 TB of memory, with the combined ranks allowing for effect data rates of DDR5-8800.
"We also started sampling our 256 GB MCRDIMM module, which further enhances performance and increases DRAM content per server," said Sanjay Mehrotra, chief executive of Micron, in prepared remarks for the company's earnings call this week.
In addition to announcing sampling of these modules, Micron also demonstrated them at NVIDIA's GTC conference, where server vendors and customers alike are abuzz at building new servers for the next generation of AI accelerators. Our colleagues from Tom's Hardware have managed to grab a couple of pictures of Micron's 256 GB DDR5-8800 MCR DIMMs.
Image Credit: Tom's Hardware
Apparently, Micron's 256 GB DDR5-8800 MCRDIMMs come in two variants: a taller module with 80 DRAM chips distributed on both sides, and a standard-height module using 2Hi stacked packages. Both are based on monolithic 32 Gb DDR5 ICs and are engineered to cater to different server configurations with the standard-height MCRDIMM adressing 1U servers.The taller version consumes about 20W of power, which is in line with expectations as a 128 GB DDR5-8000 RDIMM consumes around 10W in DDR5-4800 mode. I have no idea about power consumption of the version that uses 2Hi packages, though expect it to be a little bit hotter and harder to cool down.
Image Credit: Tom's Hardware
Multiplexer Combined Ranks (MCR) DIMMs are dual-rank memory modules featuring a specialized buffer that allows both ranks to operate simultaneously. This buffer enables the two physical ranks to operate as though they were separate modules working in parallel, which allows for concurrent retrieval of 128 bytes of data from both ranks per clock cycle (compared to 64 bytes per cycle when it comes to regular memory modules), effectively doubling performance of a single module. Of course, since the modules retains physical interface of standard DDR5 modules (i.e., 72-bits), the buffer works with host at a very high data transfer rate to transfer that fetched data to the host CPU. These speeds exceed the standard DDR5 specifications, reaching 8800 MT/s in this case.
While MCR DIMMs make memory modules slightly more complex than regular RDIMMs, they increase performance and capacity of memory subsystem without increasing the number of memory modules involved, which makes it easier to build server motherboards. These modules are poised to play a crucial role in enabling the next generation of servers to handle increasingly demanding applications, particularly in the AI field.
Sources: Tom's Hardware, Micron
Being the first company to ship HBM3E memory has its perks for Micron, as the company has revealed that is has managed to sell out the entire supply of its advanced high-bandwidth memory for 2024, while most of their 2025 production has been allocated, as well. Micron's HBM3E memory (or how Micron alternatively calls it, HBM3 Gen2) was one of the first to be qualified for NVIDIA's updated H200/GH200 accelerators, so it looks like the DRAM maker will be a key supplier to the green company.
"Our HBM is sold out for calendar 2024, and the overwhelming majority of our 2025 supply has already been allocated," said Sanjay Mehrotra, chief executive of Micron, in prepared remarks for the company's earnings call this week. "We continue to expect HBM bit share equivalent to our overall DRAM bit share sometime in calendar 2025."
Micron's first HBM3E product is an 8-Hi 24 GB stack with a 1024-bit interface, 9.2 GT/s data transfer rate, and a total bandwidth of 1.2 TB/s. NVIDIA's H200 accelerator for artificial intelligence and high-performance computing will use six of these cubes, providing a total of 141 GB of accessible high-bandwidth memory.
"We are on track to generate several hundred million dollars of revenue from HBM in fiscal 2024 and expect HBM revenues to be accretive to our DRAM and overall gross margins starting in the fiscal third quarter," said Mehrotra.
The company has also began sampling its 12-Hi 36 GB stacks that offer a 50% more capacity. These KGSDs will ramp in 2025 and will be used for next generations of AI products. Meanwhile, it does not look like NVIDIA's B100 and B200 are going to use 36 GB HBM3E stacks, at least initially.
Demand for artificial intelligence servers set records last year, and it looks like it is going to remain high this year as well. Some analysts believe that NVIDIA's A100 and H100 processors (as well as their various derivatives) commanded as much as 80% of the entire AI processor market in 2023. And while this year NVIDIA will face tougher competition from AMD, AWS, D-Matrix, Intel, Tenstorrent, and other companies on the inference front, it looks like NVIDIA's H200 will still be the processor of choice for AI training, especially for big players like Meta and Microsoft, who already run fleets consisting of hundreds of thousands of NVIDIA accelerators. With that in mind, being a primary supplier of HBM3E for NVIDIA's H200 is a big deal for Micron as it enables it to finally capture a sizeable chunk of the HBM market, which is currently dominated by SK Hynix and Samsung, and where Micron controlled only about 10% as of last year.
Meanwhile, since every DRAM device inside an HBM stack has a wide interface, it is physically bigger than regular DDR4 or DDR5 ICs. As a result, the ramp of HBM3E memory will affect bit supply of commodity DRAMs from Micron, the company said.
"The ramp of HBM production will constrain supply growth in non-HBM products," Mehrotra said. "Industrywide, HBM3E consumes approximately three times the wafer supply as DDR5 to produce a given number of bits in the same technology node."
SK Hynix and AMD were at the forefront of the memory industry with the first generation of high bandwidth memory (HBM) back in 2013 – 2015, and SK Hynix is still leading this market in terms of share. In a bid to maintain and grow its position, SK Hynix has to adapt to the requirements of its customers, particularly in the AI space, and to do so it's mulling over how to make 'differentiated' HBM products for large customers.
"Developing customer-specific AI memory requires a new approach as the flexibility and scalability of the technology becomes critical," said Hoyoung Son, the head of Advanced Package Development at SK Hynix in the status of a vice president
When it comes to performance, HBM memory with a 1024-bit interface has been evolving fairly fast: it started with a data transfer rate of 1 GT/s in 2014 – 2015 and reached upwards of 9.2 GT/s – 10 GT/s with the recently introduced HBM3E memory devices. With HBM4, the memory is set to transit to a 2048-bit interface, which will ensure steady bandwidth improvement over HBM3E.
But there are customers which may benefit from differentiated (or semi-custom) HBM-based solutions, according to the vice president.
"For implementing diverse AI, the characteristics of AI memory also need to become more varied," Hoyoung Son said in an interview with BusinessKorea. "Our goal is to have a variety of advanced packaging technologies capable of responding to these changes. We plan to provide differentiated solutions that can meet any customer needs."
With a 2048-bit interface, many (if not the vast majority) of HBM4 solutions will likely be custom or at least semi-custom based on what we know from official and unofficial information about the upcoming standard. Some customers might want to keep using interposers (but this time they are going to get very expensive) and others will prefer to install HBM4 modules directly on logic dies using direct bonding techniques, which are also expensive.
Making differentiated HBM offerings requires sophisticated packaging techniques, including (but certainly not limited to) SK Hynix's Advanced Mass Reflow Molded Underfill (MR-RUF) technology. Given the company's vast experience with HBM, it may well come up with something else, especially for differentiated offerings.
"For different types of AI to be realized, the characteristics of AI memory also need to be more diverse," the VP said. "Our goal is to have a range of advanced packaging technologies to respond to the shifting technological landscape. Looking ahead, we plan to provide differentiated solutions to meet all customer needs."
Sources: BusinessKorea, SK Hynix
Samsung announced late on Monday the completion of the development of its 12-Hi 36 GB HBM3E memory stacks, just hours after Micron said it had kicked off mass production of its 8-Hi 24 GB HBM3E memory products. The new memory packages, codenamed Shinebolt, increase peak bandwidth and capacity compared to their predecessors, codenamed Icebolt, by over 50% and are currently the world's fastest memory devices.
As the description suggests, Samsung's Shinebolt 12-Hi 36 GB HBM3E stacks pack 12 24Gb memory devices on top of a logic die featuring a 1024-bit interface. The new 36 GB HBM3E memory modules feature a data transfer rate of 10 GT/s and thus offer a peak bandwidth of 1.28 TB/s per stack, the industry's highest per-device (or rather per-module) memory bandwidth.
Meanwhile, keep in mind that developers of HBM-supporting processors tend to be cautious, so they will use Samsung's HBM3E at much lower data transfer rates to some degree because of power consumption and to some degree to ensure ultimate stability for artificial intelligence (AI) and high-performance computing (HPC) applications.
| Samsung HBM Memory Generations | ||||||
| HBM3E (Shinebolt) |
HBM3 (Icebolt) |
HBM2E (Flashbolt) |
HBM2 (Aquabolt) |
|||
| Max Capacity | 36GB | 24 GB | 16 GB | 8 GB | ||
| Max Bandwidth Per Pin | 9.8 Gb/s | 6.4 Gb/s | 3.6 Gb/s | 2.0 Gb/s | ||
| Number of DRAM ICs per Stack | 12 | 12 | 8 | 8 | ||
| Effective Bus Width | 1024-bit | |||||
| Voltage | ? | 1.1 V | 1.2 V | 1.2 V | ||
| Bandwidth per Stack | 1.225 TB/s | 819.2 GB/s | 460.8 GB/s | 256 GB/s | ||
To make its Shinebolt 12-Hi 36 GB HBM3E memory stacks, Samsung had to use several advanced technologies. First, the 36 GB HBM3E memory products are based on memory devices made on Samsung's 4th generation 10nm-class (14nm) fabrication technology, which is called and uses extreme ultraviolet (EUV) lithography.
Secondly, to ensure that 12-Hi HBM3E stacks have the same z-height as 8-Hi HBM3 products, Samsung used its advanced thermal compression non-conductive film (TC NCF), which allowed it to achieve the industry's smallest gap between memory devices at seven micrometers (7 µm). By shrinking gaps between DRAMs, Samsung increases vertical density and mitigates chip die warping. Furthermore, Samsung uses bumps of various sizes between the DRAM ICs; smaller bumps are used in areas for signaling. In contrast, larger ones are placed in spots that require heat dissipation, which improves thermal management.
Samsung estimates that its 12-Hi HBM3E 36 GB modules can increase the average speed for AI training by 34% and expand the number of simultaneous users of inference services by more than 11.5 times. However, the company has not elaborated on the size of the LLM.
Samsung has already begun providing samples of the HBM3E 12H to customers, with mass production scheduled to commence in the first half of this year.
Source: Samsung
Micron Technology on Monday said that it had initiated volume production of its HBM3E memory. The company's HBM3E known good stack dies (KGSDs) will be used for Nvidia's H200 compute GPU for artificial intelligence (AI) and high-performance computing (HPC) applications, which will ship in the second quarter of 2024.
Micron has announced it is mass-producing 24 GB 8-Hi HBM3E devices with a data transfer rate of 9.2 GT/s and a peak memory bandwidth of over 1.2 TB/s per device. Compared to HBM3, HBM3E increases data transfer rate and peak memory bandwidth by a whopping 44%, which is particularly important for bandwidth-hungry processors like Nvidia's H200.
Nvidia's H200 product relies on the Hopper architecture and offers the same computing performance as the H100. Meanwhile, it is equipped with 141 GB of HBM3E memory featuring bandwidth of up to 4.8 TB/s, a significant upgrade from 80 GB of HBM3 and up to 3.35 TB/s bandwidth in the case of the H100.
Micron's memory roadmap for AI is further solidified with the upcoming release of a 36 GB 12-Hi HBM3E product in March 2024. Meanwhile, it remains to be seen where those devices will be used.
Micron uses its 1β (1-beta) process technology to produce its HBM3E, which is a significant achievement for the company as it uses its latest production node for its data center-grade products, which is a testament to the manufacturing technology.
Starting mass production of HBM3E memory ahead of competitors SK Hynix and Samsung is a significant achievement for Micron, which currently holds a 10% market share in the HBM sector. This move is crucial for the company, as it allows Micron to introduce a premium product earlier than its rivals, potentially increasing its revenue and profit margins while gaining a larger market share.
"Micron is delivering a trifecta with this HBM3E milestone: time-to-market leadership, best-in-class industry performance, and a differentiated power efficiency profile," said Sumit Sadana, executive vice president and chief business officer at Micron Technology. "AI workloads are heavily reliant on memory bandwidth and capacity, and Micron is very well-positioned to support the significant AI growth ahead through our industry-leading HBM3E and HBM4 roadmap, as well as our full portfolio of DRAM and NAND solutions for AI applications."
Source: Micron
Rambus has introduced its fourth-generation registering clock driver (RCD) chip for server-grade DDR5 memory modules. The updated RCD chip brings support for higher clockspeeds on DDR5 RDIMMs, allowing for future RDIMMs to run as fast as DDR5-7200, a step ahead of their current third-generation/DDR5-6400 RCD. Faster DDR5 RDIMMs will eventually go hand-in-hand with upcoming server platforms, with AMD, Intel, and others all eyeing significantly higher memory speeds for their next round of products.
An RCD functions as a buffer between the memory controller and DRAM chips in RDIMMs, redistributing command and address signals across the module – and making up the "Registered" in "Registered DIMM". This enhances signal integrity and enables more memory devices to be connected to a single DRAM channel. To work properly. the RCD must support a specific data transfer rate, so the new RCD buffer from Rambus will enable memory vendors to build server-grade DDR5-7200 modules.
In addition to performing regular functions of RCD, the new Rambus chip also packs a serial presence detect (SPD) hub and temperature sensors, which are vital for DDR5 memory sticks on general and DDR5 datacenter memory modules in particular. Such functionality somewhat reduces costs of datacenter-grade DRAM sticks, though the costs of such devices are depend on the number of memory devices rather than by tiny chips like SPDs and temperature sensors.
RCDs are crucial for contemporary server-grade memory modules, as these devices can only reach their full memory capacity with registered DIMMs. Rambus is the first company to officially introduce DDR5 RCDs that support a 7200 MT/s data transfer rate, which will be useful for next-generation server platforms.
According to Rambus, their latest RCD chip is now shipping. Though given the long lead times in servers and server parts due to validation requirements, it's likely still some time off before it starts appearing in commercial servers and DIMMs.
UPDATE 11/24: Demu, our reader with sharp eyes, notified us that Gigabyte had published a list of overclockable RDIMMs with ECC and AMD EXPO profiles supported by its TRX50 motherboards for AMD Ryzen Threadripper 7000-series processors. As it turns out, Kingston is also prepping such memory modules, whereas v-color is planning quad-channel DDR5-7800 RDIMM kits for AMD Ryzen Threadripper 7000-series CPUs. We have added specifications of all upcoming overclockable memory modules for AMD Ryzen Threadripper 7000 platforms into the table below the original story.
With the launch of AMD new Ryzen Threadripper 7000-series processors and associated TRX50 platform comes the need for new memory kits. Thanks to UDIMMs and RDIMMs requiring different slot designs under DDR5, AMD has opted to go exclusively with RDIMMs for their latest generation of Threadripper processors. Which for memory makers has included putting together overclockable registered memory modules with ECC that support EXPO timings, such as the kits G.Skill and V-Color have announced this week with their Zeta R5 Neo and v-color DDR5 OC R-DIMM memory kits.
G.Skill's Zeta R5 Neo product family and v-color's OC R-DIMM are exclusively designed for AMD's Ryzen Threadripper 7000-series processors. Both vendors are currently offering two types of kits: a 64 GB quad-channel kit consisting of four 16 GB RDIMMs, and a 128 GB quad-channel kit featuring four 32 GB RDIMMs. For systems using the TRX50 platform, a single kit suffices, while the high-end workstation-class WRX90 platform requires two of these quad-channel kits to fully populate all eight channels.
G.Skill's modules with ECC are rated for of DDR5-6400 CL32 39-39-102 at massive 1.40 Volts, whereas v-color promises data transfer rates of up to 7200 MT/s, never mentions timings or voltages, but stresses that its kits are aimed at TRX50-based machines.
G.Skill and v-color note that their overclockable RDIMMs, similar to other premium memory modules, are built using hand-selected memory ICs, making them better suited for overclocking. G.Skill's modules come with very basic heat spreaders, which in case of DDR5 may become a limiting factor for overclocking potential. By contrast, v-color's modules come with rather serious heat spreaders akin to those used for some server-grade modules.
All the overclockable RDIMM modules from G.Skill and v-color come with AMD's EXPO profiles for simplified setup, with G.Skill using DDR5-6400 CL32 timings, while v-color runs at DDR5-7200 at unspecified timings.
The exact overclocking potential of G.Skill's and v-color's Threadripper memory kits remain to be seen. The actual I/O die powering Threadripper is the same I/O die as AMD's other server processors, which is to say it's been optimized for stability over performance – not to mention supporting a much larger amount of memory. Still, as we've seen on AMD's consumer CPU platforms, the company's memory controllers are no slouches. All the while memory vendors are already offering high-clocked RDIMM kits for Intel's rival Sapphire Rapids Xeon platform.
G.Skill's Zeta R5 Neo memory kits designed for AMD's Ryzen Threadripper 7000-series processors are already available at Newegg: the 64 GB kit is priced at $530, whereas the 128GB kit costs $1,070. This is of course a huge price premium, but this can be explained by the fact that we are talking about unique memory kits designed for very specific high-end CPUs.
As for v-color's DDR5 OC R-DIMM memory kits for AMD's Ryzen Threadripper TRX50 platform, the company says that they will be available in November on its website.
| Overclockable RDIMMs for AMD Ryzen Threadripper 7000 CPUs | ||||||||||
| Brand | Speed | Capacity | Timings | Voltage | DRAM IC | Native | P/N | |||
| G.Skill | 6400MT/s | 16GB | 32-39-39-102 | 1.4v | Hynix A | 4800MT/s | F5-6400R3239G32GQ4-ZR5NK | |||
| 32GB | F5-6400R3239G16GQ4-ZR5NK | |||||||||
| Kingston | 6000MT/s | 16GB | 32-38-38-80 | 1.35v | Skhynix | 4800MT/s | KF560R32RBE-16 | |||
| Skhynix | KF560R32RBE-16 | |||||||||
| Hynix A | KF560R32RBEK4-64 | |||||||||
| 32GB | Hynix A | KF560R32RBEK4-128 | ||||||||
| 6400MT/s | 16GB | 32-39-39-80 | 1.4v | Skhynix | 4800MT/s | KF564R32RBE-16 | ||||
| Hynix A | KF564R32RBEK4-64 | |||||||||
| V-Color | 4800MT/s | 24GB | 36-38-38-70 | 1.1V | Hynix M | 4800MT/s | TRA524G48S836 | |||
| 16GB | Hynix A | 5600MT/s | TRA516G48S836 | |||||||
| 24GB | Hynix M | 4800MT/s | TRA524G48S836 | |||||||
| 32GB | Hynix M | TRA532G48S436 | ||||||||
| 5200MT/s | 16GB | 36-40-40-80 | 1.25V | Hynix A | 5600MT/s | TRA516G52S836 | ||||
| 24GB | Hynix M | 4800MT/s | TRA524G52S836 | |||||||
| 32GB | Hynix M | TRA532G52S436 | ||||||||
| 5600MT/s | 16GB | 36-38-38-80 | Hynix A | 5600MT/s | TRA516G56S836 | |||||
| 24GB | Hynix M | 4800MT/s | TRA524G56S836 | |||||||
| 32GB | Hynix M | TRA532G56S436 | ||||||||
| 16GB | 32-39-39-102 | 1.4V | Hynix A | 5600MT/s | TRA516G60S832 | |||||
| 24GB | Hynix M | 4800MT/s | TRA524G60S832 | |||||||
| 32GB | 32-38-38-96 | 1.25V | Hynix M | 4800MT/s | TRA532G60S432 | |||||
| 6400MT/s | 24GB | 32-39-39-102 | 1.4V | Hynix M | 4800MT/s | TRA524G64S832 | ||||
| 16GB | Hynix A | 5600MT/s | TRA516G64S832 | |||||||
| 6600MT/s | 24GB | 34-46-46-92 | Hynix M | 4800MT/s | TRA524G66S834 | |||||
| 16GB | Hynix A | 5600MT/s | TRA516G66S834 | |||||||
| 24GB | Hynix M | 4800MT/s | TRA524G68S834 | |||||||
| 16GB | Hynix A | 5600MT/s | TRA516G68S834 | |||||||
| 7000MT/s | 24GB | 34-42-42-102 | Hynix M | 4800MT/s | TRA524G70S834R3 | |||||
| 16GB | Hynix A | 5600MT/s | TRA516G70S834 | |||||||
| 7200MT/s | 24GB | 34-45-45-112 | Hynix M | 4800MT/s | TRA524G72S834R3 | |||||
| 16GB | Hynix A | 5600MT/s | TRA516G72S834 | |||||||
| 7600MT/s | 24GB | 38-48-48-122 | Hynix M | 4800MT/s | TRA524G76S838R3 | |||||
| 16GB | Hynix A | 5600MT/s | TRA516G76S838 | |||||||
| 7800MT/s | 24GB | 38-48-48-126 | Hynix M | 4800MT/s | TRA524G78S838R3 | |||||
| 16GB | Hynix A | 5600MT/s | TRA516G78S838 | |||||||
SK hynix had started volume shipments of its LPDDR5T-9600 memory for high-end smartphones, the company announced this week. So far, the company's LPDDR6 'Turbo' memory with a 9600 MT/s data transfer speed has been certified to work with two range-topping mobile application processors from Qualcomm and MediaTek.
SK hynix's LPDDR5T-9600 memory is available in 16 GB packages with a VDD voltage range of 1.01V to 1.12V and a VDDQ of 0.5v. Notably, this VDD range is just slightly over the LPDDR5X specfication (1.00V to 1.1V), which shouldn't be a serious problem, but likely warrants some extra compatibility testing with existing chips.
Smartphone and SoC manufacturers have a great incentive to validate SK hynix's LPDDR5T-9600 and Micron's LPDDR5X-9600 memory as these modules offer a 76.8 GB/s peak bandwidth, up 12.5% from 68.2 GB/s offered by LPDDR5X-8533.
So far, SK Hynix's LPDDR6T-9600 modules have been certified by Qualcomm for its Snapdragon 8 Gen 3 mobile SoC as well as MediaTek for its Dimensity 9300 and some subsequent processors. Meanwhile, SK Hynix confirmed that it had begun shipments of its LPDDR6T-9600 devices to Vivo, which will use it for X100 and X100 Pro mobile application processors.
Following the latest trends, SK hynix mentions that its very fast LPDDR5T-9600 memory will be particularly useful for on-device AI applications. And, of course, faster DRAM is always welcomed for graphics intensive mobile applications, such as games.
"Smartphones are becoming essential devices for implementing on-device AI technology as the AI era kicks into full swing," said Myoungsoo Park, Vice President and Head of DRAM Marketing at SK hynix. "There is a growing demand for high-performing, high-capacity mobile DRAMs in the market. We will continue to lead the premium DRAM market based on our technological leadership in AI memories, while staying in tune with market demands."
The path to high-capacity RDIMMs for servers has primarily been through 3D stacking (3DS) of DRAM dies using Through-Silicon Vias (TSVs). However, this has presented significant challenges in packaging (driving up the cost), and has also not been efficient in terms of energy consumption. The demand for large memory capacity RDIMMs is being primarily driven by the sudden emergence of large-language models (LLMs) for generative AI and increasing CPU core counts. Both of these require significant amount of DRAM to keep pace with performance requirements. Keeping these in mind, Micron is introducing 128 GB DDR5 RDIMMs capable of operating at up to 8000 MT/s today, with mass-production slated for 2024.
Micron has recently started fabricating 32 Gb monolithic DDR5 dies using its proven and mature 1β technology. The new dies have a 45%+ increase in bit density, and are capable of reaching up to 8000 MT/s while also operating with much more aggressive timing latencies compared to the standard JEDEC specs. The company is claiming that it improves energy efficiency by as much as 24% compared to the competition's 3DS TSV offerings, and the faster operation can also help in faster AI training times. Avoiding 3DS TSV allows Micron to optimize the data input buffers and critical I/O circuits better, while also reducing the pin capacitance on the data lines. These contribute to the reduced power and improved speeds.
Micron has been doubling its monolithic die density every 3 years or so, thanks to advancements in CMOS process as well as improvements in array efficiency. The company sees a clear path to 48 Gb and 64 Gb monolithic dies in the future with continued technological progress. Micron is also claiming that its 1β node has reached mass production ahead of the competition, and that it has had the fastest yield maturity in the company's history. Dual-die packages and tall form-factor (TFF) modules using 1β DRAM are expected to enable 1TB modules in the near future.
Along with the announcement of the 128 GB RDIMMs using 1β technology, the company also laid out its roadmap for upcoming products. HDM and GDDR7 are expected to dominate bandwidth-hungry applications, while RDIMMs, MCRDIMMs, and CXL solutions are in the pipeline for systems requiring massive capacity. LPDDR5X, and LPCAMM2 solutions going up to 192 GB are expected to make an appearance in power-sensitive systems as early as 2026.
Fast memory is crucial for the performance of high-end system-on-chips that are getting more sophisticated every year. When it comes to smartphones, the most obvious way to boost memory performance is to push its data transfer rate. Apparently, this is what Micron and SK Hynix are doing with their new LPDDR5X and LPDDR5T DRAMs that boast a data transfer rate of 9.6 GT/s.
Micron's LPDDR5X-9600 memory devices are made on the company's latest 1β (1-beta) process technology. They are offered in up to 16 GB x64 packages (though it is unclear how many actual memory devices these packages integrate). Micron says that its LPDDR5X made on its latest production node boasts up to 30% lower power consumption compared to competing LPDDR5X ICs made on 1α (1-alpha) technology, though this is something to be expected.
Micron does not disclose how it managed to increase the data transfer rate of its LPDDR5X to 9.6 GT/s, which is a 12% increase compared to 8.53 GT/s, which was once considered the highest speed of LPDDR5X memory. The only thing that the company discloses is that these ICs boast 'enhanced' dynamic voltage and frequency scaling, although DVFS is a part of LPDDR5X specification.
"Generative AI is poised to unleash unprecedented productivity, ease of use, and personalization for smartphone users by delivering the power of large language models to flagship mobile phones," said Mark Montierth, corporate vice president and general manager of Micron's Mobile Business Unit. "Micron's 1β LPDDR5X combined with Qualcomm Technologies' AI-optimized Snapdragon 8 Gen 3 Mobile Platform empowers smartphone manufacturers with the next-generation performance and power efficiency essential to enabling revolutionary AI technology at the edge."
SK Hynix is another company to start shipping LPDDR5-9600 memory today, which calls its fastest LPDDR5 DRAMs LPDDR5T (T stands for Turbo). The new memory will be available in 16 GB packages with a VDD voltage range of 1.01V to 1.12V and a VDDQ of 0.5v. By contrast, LPDDR5X should have a maximum VDD voltage of 1.1V, so LPDDR5T is slightly out of LPDDR5X spec.
Meanwhile, both Micron's LPDDR5X-9600 and SK Hynix's LPDDR5T-9600 are compatible with Qualcomm's Snapdragon 8 Gen 3 system-on-chip for smartphones, the two compares announced on Tuesday. Micron is already shipping its 16 GB LPDDR 9.6 GT/s modules featuring a 76.8 GB/s peak bandwidth, so expect some of Qualcomm's partners to use the world's fastest mobile memory shortly. SK Hynix's module has been validated by Qualcomm, so the South Korean company will likely begin commercial shipments of its LPDDR5T-9600 product soon.
"We are thrilled that we have met our customers' needs for the ultra-high performance mobile DRAM with the provision of the LPDDR5T," said Sungsoo Ryu, head of DRAM Product Planning at SK Hynix.
Samsung’s annual Memory Tech Day is taking place in San Jose this morning, and as part of the event, the company is making a couple of notable memory technology announcements/disclosures. The highlight of Samsung’s event is the introduction of Shinebolt, Samsung’s HBM3E memory that will set new marks for both memory bandwidth and memory capacity for high-end processors. The company is also disclosing a bit more on their GDDR7 memory, which will mark a significant technological update to the GDDR family of memory standards.
High-bandwidth memory has been around for about a decade, and throughout its its continued development it has steadily increased in speed, starting at a data transfer rate from 1 GT/s (the original HBM) and reaching upwards of 9 GT/s with the forthcoming HBM3E. This has made for an impressive jump in bandwidth in less than 10 years, making HBM an important cornerstone for whole new classes of HPC accelerators that have since hit the market. But it's also a pace that's getting harder to sustain as memory transfer rates increase, especially as the underlying physics of DRAM cells have not changed. As a result, for HBM4 the major memory manufacturers behind the spec are planning on making a more substantial change to the high-bandwidth memory technology, starting with an even wider 2048-bit memory interface.
Designed as a wide-but-slow memory technology that utilizes an ultra-wide interface running at a relatively modest clockspeed, HBM's current 1024-bit memory interface has been a defining characteristic of the technology. Meanwhile its modest clockspeeds have become increasingly less modest in order to keep improving memory bandwidth. This has worked thus far, but as clockspeeds increase, the highly parallel memory is risking running into the same signal integrity and energy efficiency issues that challenge GDDR and other highly serial memory technologies.
Consequently, for the next generation of the technology, organizers are looking at going wider once more, expanding the width of the HBM memory interface even further to 2048-bits. And, equally as important for multiple technical reasons, they intend to do this without increasing the footprint of HBM memory stacks, essentially doubling the interconnection density for the next-generation HBM memory. The net result would be a memory technology with an even wider memory bus than HBM today, giving memory and device vendors room to further improve bandwidth without further increasing clock speeds.
As planned, this would make HBM4 a major technical leap forward on multiple levels. On the DRAM stacking side of matters, a 2048-bit memory interface is going to require a significant increase in the number of through-silicon vias routed through a memory stack. Meanwhile the external chip interface will require shrinking the bump pitch to well below 55 um, all the while increasing the total number of micro bumps significantly from the current count of (around) 3982 bumps for HBM3.
Adding some additional complexity to the technology, memory makers have indicated that they are also going to stack up to 16 memory dies in one module; so-called 16-Hi stacking. (HBM3 technically supports 16-Hi stacks as well, but so far no manufacturer is actually using it) This will allow memory vendors to significantly increase the capacity of their HBM stacks, but it brings new complexity in wiring up an even larger number of DRAM dies without defects, and then keeping the resulting HBM stack suitably and consistently short.
All of this, in turn will require even closer collaboration between chip makers, memory makers, and chip packaging firms in order to make everything come together smoothly.
Speaking at TSMC's OIP 2023 conference in Amsterdam, Dan Kochpatcharin, TSMC's Head of Design Infrastructure Management had this to say: "Because instead of doubling the speed, they doubled the [interface] pins [with HBM4]. That is why we are pushing to make sure that we work with all three partners to qualify their HBM4 [with our advanced packaging methods] and also make sure that either RDL or interposer or whatever in between can support the layout and the speed [of HBM4]. So, [we work with] Samsung, SK Hynix, and Micron."
Since system-in-package (SiP) designs are getting larger, and the number of HBM stacks supported by advanced chip packages is increasing (e.g. 6x reticle size interposers and chips with 12 HBM stacks on-package), chip packages are getting more complex. To ensure that everything continues to work together, TSMC is pushing chip and memory designers to embrace Design Technology Co-Optimization (DTCO). This being a big part of the reason why the world's largest foundry recently organized 3DFabric Memory Alliance, a program designed to enable close collaboration between DRAM makers and TSMC in a bid to enable next-generation solutions that will pack huge amounts of logic transistors and advanced memory.
Among other things, TSMC's 3DFabric Memory Alliance is currently working on ensuring that HBM3E/HBM3 Gen2 memory works with CoWoS packaging, 12-Hi HBM3/HBM3E packages are compatible with advanced packages, UCIe for HBM PHY, and buffer-less HBM (a technology spearheaded by Samsung).
Overall, TSMC's comments last week give us our best look yet at the next generation of high-bandwidth memory. Still, additional technical details about HBM4 remain rather scarce for the moment. Micron said earlier this year that 'HBMNext' memory set to arrive around 2026 will offer capacities between 36 GB and 64 GB per stack and peak bandwidth of 2 TB/s per stack or higher. All of which indicates that memory makers won't be backing off on memory interface clockspeeds for HBM4, even with the move to a wider memory bus.
Micron has reaffirmed plans to start shipments of its HBM3E memory in high volume in early 2024, while also revealing that NVIDIA is one of its primary customers for the new RAM. Meanwhile, the company stressed that its new product has been received with great interest by the industry at large, hinting that NVIDIA will likely not be the only customer to end up using Micron's HBM3E.
"The introduction of our HBM3E product offering has been met with strong customer interest and enthusiasm," said Sanjay Mehrotra, president and chief executive of Micron, at the company's earnings call.
Introducing HBM3E, which the company also calls HBM3 Gen2, ahead of its rivals Samsung and SK Hynix is a big deal for Micron, which is an underdog on the HBM market with a 10% market share. The company obviously pins a lot of hopes on its HBM3E since this will likely enable it to offer a premium product (to drive up its revenue and margins) ahead of its rivals (to win market share).
Typically, memory makers tend not to reveal names of their customers, but this time around Micron emphasized that its HBM3E is a part of its customer's roadmap, and specifically mentioned NVIDIA as its ally. Meanwhile, the only HBM3E-supporting product that NVIDIA has announced so far is its Grace Hopper GH200 compute platform, which features an H100 compute GPU and a Grace CPU.
"We have been working closely with our customers throughout the development process and are becoming a closely integrated partner in their AI roadmaps," said Mehrotra. "Micron HBM3E is currently in qualification for NVIDIA compute products, which will drive HBM3E-powered AI solutions."
Micron's 24 GB HBM3E modules are based on eight stacked 24Gbit memory dies made using the company's 1β (1-beta) fabrication process. These modules can hit data rates as high as 9.2 GT/second, enabling a peak bandwidth of 1.2 TB/s per stack, which is a 44% increase over the fastest HBM3 modules available. Meanwhile, the company is not going to stop with its 8-Hi 24 Gbit-based HBM3E assemblies. The company has announced plans to launch superior capacity 36 GB 12-Hi HBM3E stacks in 2024 after it initiates mass production of 8-Hi 24GB stacks.
"We expect to begin the production ramp of HBM3E in early calendar 2024 and to achieve meaningful revenues in fiscal 2024," added chief executive of Micron.
Micron is sampling 128 GB DDR5 memory modules, the company said at its earnings call this week. The modules are based on the company's latest single die, non-stacked 32 Gb DDR5 memory devices, which the company announced earlier this summer and which will eventually open doors for 1 TB memory modules for servers.
"We expanded our high-capacity D5 DRAM module portfolio with a monolithic die-based 128 GB module, and we have started shipping samples to customers to help support their AI application needs," said Sanjay Mehrotra, president and chief executive of Micron. "We expect revenue from this product in Q2 of calendar 2024."
Micron's 32 Gb DDR5 dies are made on the company's 1β (1-beta) manufacturing process, which is the last production node that solely relies on multi-patterning using deep ultraviolet (DUV) lithography and does not use extreme ultraviolet (EUV) lithography tools. This is all that we know about Micron's 32 Gb DDR5 ICs at this point, though: the company does not disclose its maximum speed bin, though we can expect a drop in power consumption compared to two 16 Gb DDR5 ICs operating at the same voltage and data transfer rate.
Micron's new 32 Gb memory chips pave the way for creating a standard 32 GB module for personal computers with just eight individual memory chips and a server-oriented 128 GB module based on 32 of such ICs. Moreover, these chips make producing memory modules with a 1 TB capacity feasible, deemed unattainable today. These 1 TB modules might seem excessive for now, but they benefit fields like artificial intelligence, Big Data, and server databases. Such modules can enable servers to support up to 12 TB of DDR5 memory per socket (in the case of a 12-channel memory subsystem).
Speaking of DDR5 memory in general, it is noteworthy that the company expects that its bit production of DDR5 will exceed that of DDR4 in early 2024, placing it a bit ahead of the industry.
"Micron also has a strong position in the industry transition to D5," said Mehrotra. "We expect Micron D5 volume to cross over D4 in early calendar 2024, ahead of the industry."
Corsair has started sales of its Dominator Titanium memory modules that were formally introduced this May. The new modules bring together luxurious look, customizable design, and extreme data transfer rates of up to 8000 MT/s. Speaking of performance, the company implied that it intends to introduce Dominator Titanium with speed bins beyond DDR5-8000 when the right platform arrives.
Corsair's Dominator Titanium family is based around 16 GB, 24 GB, 32 GB, and 48 GB memory modules that come in kits ranging from 32GB (2 x 16GB) up to 192GB (4x 48GB). As for performance, the lineup listed at the company's website includes DDR5-6000 CL30, DDR5-6400 CL32, DDR5-6600 CL32, DDR5-7000 CL34, DDR5-7000 CL36, DDR5-7200 CL34, and DDR5-7200 CL36 with voltages of 1.40 V – 1.45V.
Although Corsair claims that Dominator Titanium with data transfer speeds beyond 8000 MT/s are coming, it is necessary to note that they will be supported by next generation platforms from AMD and Intel. For now, the company only offers 500 First Edition Dominator Titanium kits rated for DDR5-8266 mode for its loyal fans.
To address demand from different types of users, Corsair offers Dominator Titanium with XMP 3.0 SPD settings for Intel's 12th and 13th Generation Core CPUs with black and white heat spreaders as well as with AMD EXPO SPD profiles for AMD's Ryzen processors with grey finish on heat spreaders.
In terms of design of heat spreaders, Corsair remained true to aesthetics. The modules are equipped with 11 customizable Capellix RGB LEDs, offering users a personalized touch. This can be easily adjusted using Corsair's proprietary software. For enthusiasts who lean towards a more traditional aesthetic, Corsair provides an alternative design with fins, reminiscent of their classic memory modules.
Speaking of heat spreaders, it is necessary to note that despite the name of the modules, they do not come with titanium radiators and keep using aluminum, which is a good thing since titanium has a rather low thermal conductivity of 11.4 W/mK and will therefore heat up memory chips rather than distribute heat away from them. Traditionally, Corsair's Dominator memory modules use cherry-picked DRAM chips and the company's proprietary printed circuit boards enhanced with internal cooling planes and external thermal pads to improve cooling.
Corsair's Dominator Titanium memory products are now available both directly from the company and from its resellers. The cheapest Dominator Titanium DDR5-6000 CL30 32 GB kit (2 x 16 GB) costs $175, whereas faster and higher-capacity kits are priced higher.
Although Low Power DDR(LPDDR) memory has played a pivotal role in reducing PC laptop power usage, the drawback to the mobile-focused memory has always been its tight signaling and power delivery requirements. Designed to be placed close to its host CPU in order to minimize power expenditures and maximize clockspeeds, LPDDR memory is unsuitable for use in traditional DIMMs and SO-DIMMs – instead requiring that it be soldered down on a device in advance. But it looks like the days of soldered-down LPDDR memory are soon at an end, as this evening Samsung is announcing a new standard for removable and modular LPDDR memory: LPCAMM.
Epos has announced that it will be exiting the gaming headphone business and will instead focus on enterprise communications products. The company's gaming products division, which was formerly part of the legendary Sennheiser, was responsible for shipping a number of notable gaming headsets over the past decade. However, the continuing weakness of the wired consumer audio market – and especially the gaming market – has taken its toll.
Established in 2020 when Sennheiser and its partner Demant decided to part ways from their joint venture of nearly 20 years, Demant-owned Epos became the new home for what were Sennheiser's enterprise and gaming product divisions. While hopes were high for Epos as the gaming market has been growing in the recent years, Epos's gaming business performed weaker-than-expected in 2022 – 2023. And, facing a situation where additional investments would be required to keep the gaming division alive, Demant has decided to cut its losses and focus on enterprise communications.
According to social media posts from Epos staff members, the company has already laid off all of its gaming division employees. Though the company has stated that it will continue to support customers and sell off inventory of existing products in the coming months. That self-off process is expected to take a decent bit of time, with Epos expecting it to stretch in to 2024.
"Since the demerger of our joint venture with the Sennheiser group, our Gaming business has faced a volatile market environment," said Søren Nielsen, President & CEO of Demant. "Following extraordinary demand sparked by the pandemic in 2020, the gaming market has slowed down significantly due to weak consumer sentiment, and we do not see a viable path to creating a profitable business without significant investments in products, brand and distribution. We of course regret the impact our decision will have on affected employees and would like to thank all employees who have worked very hard to build the Gaming business under difficult circumstances."
Looking ahead, Demant's primary focus will be on fortifying its position in the enterprise solutions market. The company aims to expand its product range for businesses and grow its distribution partnerships.
Financially, Demant anticipates some minor one-time costs in 2023 due to this shift. However, the overall financial forecast for the year and beyond remains stable. The Gaming division accounted for approximately 15% of the Communications segment's revenue in the first half of 2023. By 2024, with the combination of this phase-out and previous cost-saving measures, Demant expects its operational expenses in Communications to drop to DKK 450-500 million ($65 - $72 million) annually. This reduction is anticipated to lead to improved margins and a more favorable financial position in the near future.
Samsung early on Friday revealed the world's first 32 Gb DDR5 DRAM die. The new memory die is made on the company's 12 nm-class DRAM fabrication process and not only offers increased density, but also lowers power consumption. The chip will allow Samsung to build record 1 TB RDIMMs for servers as well as lower costs of high-capacity memory modules.
“With our 12nm-class 32 Gb DRAM, we have secured a solution that will enable DRAM modules of up to 1 TB, allowing us to be ideally positioned to serve the growing need for high-capacity DRAM in the era of AI (Artificial Intelligence) and big data,” said SangJoon Hwang, executive vice president of DRAM product & technology at Samsung Electronics.
32 Gb memory dies not only enable Samsung to build a regular, single-rank 32 GB module for client PCs using only eight single-die memory chips, but they also allow for higher capacity DIMMs that were not previously possible. We are talking about 1 TB memory modules using 40 8-Hi 3DS memory stacks based on eight 32 Gb memory devices. Such modules may sound overkill, but for artificial intelligence (AI), Big Data, and database servers, more DRAM capacity can easily be put to good use. Eventually, 1TB RDIMMs would allow for up to 12 TB of memory in a single socket server (e.g. AMD's EPYC 9004 platform), something that cannot be done now.
With regards to power consumption, Samsung says that using the new dies they can build 128 GB DDR5 RDIMMs for servers that consume 10% less power than current-generation modules built around 16 Gb dies. This drop of power consumption can be attributed to both 12 nm-class DRAM production node as well as avoiding the use of 3D stacked (3DS) chips that pack two 16 Gb dies into a single package.
Samsung is not disclosing the speed bins of its 32 Gb memory dies, but finished 16 Gb modules made on the same 12 nm-class technology offer a 7200 MT/s data transfer rate.
Samsung intends to start mass production of 32 Gb memory dies by the end of 2023, but for now the company isn't detailing when it plans to offer finished chips to customers. it's likely that the company will start with client PCs first, though whether that translasts into any cost savings remains to be seen.
Otherwise, for servers it usually takes a while for server platform developers and vendors to validate and qualify new memory components. So while Samsung has 1 TB RDIMMs in its future, it will take some time before we see them in shipping servers.
SK hynix on Monday announced that it had completed initial development of its first HBM3E memory stacks, and has begun sampling the memory to a customer. The updated ("extended") version of the high bandwidth memory technology is scheduled to begin shipping in volume in the first half of next year, with hardware vendors such as NVIDIA already lining up to incorporate the memory into their HPC-grade compute products.
First revealed by SK hynix back at the end of May, HBM3E is an updated version of HBM3 that is designed to clock higher than current HBM3, though specific clockspeed targets seem to vary by manufacturer. For SK hynix, as part of today's disclosure the company revealed that their HBM3E memory modules will be able to hit data transfer rates as high as 9 GT/sec, which translates to a peak bandwidth of 1.15 TB/sec for a single memory stack.
Curiously, SK hynix has yet to reveal anything about the planned capacity for their next-gen memory. Previous research from TrendForce projected that SK hynix would mass produce 24 GB HBM3E modules in Q1 2024 (in time to address applications like NVIDIA's GH200 with 144GB of HBM3E memory), boosting capacity over today's 16GB HBM3 stacks. And while this still seems likely (especially with the NV announcement), for now it remains unconfirmed.
TrendForce HBM Market Projections (Source: TrendForce)
Meanwhile, the SK hynix also confirms that its HBM3E stacks are set to use its Advanced Mass Reflow Molded Underfill (MR-RUF) technology to reduce their heat dissipation by 10%. But thermals is not the only benefit MR-RUF can provide. MR-RUF implies the usage of an improved underfill between layers, which improves thermals and reduces thickness of HBM stacks, which allows the construction of 12-Hi HBM stacks that are only as tall as 8-Hi modules. This does not automatically mean that we are dealing with 12-Hi HBM3E stacks here, of course.
At present, SK hynix is the only high volume manufacturer of HBM3 memory, giving the company a very lucrative position, especially with the explosion in demand for NVIDIA's H100 and other accelerators for generative AI. And while the development of HBM3E is meant to help SK hynix keep that lead, they will not be the only memory vendor offering faster HBM next year. Micron also threw their hat into the ring last month, and where those two companies are, Samsung is never too far behind. In fact, all three companies seem to be outpacing JEDEC, the organization that is responsible for standardizing DRAM technologies and various memory interfaces, as that group has still not published finalized specifications for the new memory.
Samsung, MemVerge, H3 Platform, and XConn have jointly unveiled their 2 TB Pooled CXL Memory System at the Flash Memory Summit. The device can be connected to up to eight hosts, allowing them to use its memory when needed. The 2 TB Pooled CXL Memory system has software enabling it to visualize, pool, tier, and dynamically allocate memory to connected hosts.
The 2 TB Pooled CXL Memory system is a 2U rack-mountable machine built by H3 with eight 256 GB Samsung CXL memory modules connected using XConn's XC50256 CXL 2.0 switch supporting 256 PCIe Gen5 lanes and 32 ports. The firmware of the 2 TB Pooled CXL Memory system allows you to connect it to up to eight hosts that can dynamically use CXL memory when they need it, thanks to software by MemVerge.
The Pooled CXL Memory system was developed to overcome limitations in memory capacity and composability in today's system architecture, which involves tight coupling between CPU and DRAM. Such architecture leads to performance challenges in highly distributed AI/ML applications, such as spilling memory to slow storage, excessive memory copying, I/O to storage, serialization/deserialization, and Out-of-Memory errors that can crash applications.
Attaching 2 TB of fast, low-latency memory using a PCIe 5.0 interface with the CXL 2.0 protocol on top to eight host systems and using it dynamically between eight hosts saves a lot of money while providing loads of performance benefits. According to companies, the initiative represents a significant step towards creating a more robust and flexible memory-centric data infrastructure for modern AI applications.
"Modern AI applications require a new memory-centric data infrastructure that can meet the performance and cost requirements of its data pipeline," said Charles Fan, CEO and co-founder of MemVerge. "Hardware and software vendors in the CXL Community are co-engineering such memory-centric solutions that will deeply impact our future."
The jointly developed demonstration system can be pooled, tiered with main memory, and dynamically provisioned to applications with Memory Machine X software from MemVerge and its elastic memory service. Viewer service showcases the system's physical layout and provides a heat map indicating memory capacity and bandwidth consumption per application.
"The concept system unveiled at Flash Memory Summit is an example of how we are aggressively expanding its usage in next-generation memory architectures," said JS Choi, Vice President of New Business Planning Team at Samsung Electronics. "Samsung will continue to collaborate across the industry to develop and standardize CXL memory solutions, while fostering an increasingly solid ecosystem."
On Friday, SK Hynix said it had started mass production of 24 GB LPDDR5X memory stacks that can be used for ultra-high-end smartphones and PCs. The company's LPDDR5X-8500 devices combine ultra-high-performance with high density, thus enabling fast systems with sufficient memory capacity. SK Hynix says such modules could be used well beyond smartphones, PCs, and even servers.
SK Hynix's 24 GB LPDDR5X package features an 8500 MT/s data transfer rate and a wide 64-bit interface, thus offering a peak bandwidth of 68 GB/s in the ultra-low voltage range of 1.01 to 1.12V. From a typical PC perspective, this is comparable to bandwidth provided by a dual-channel DDR5-4800 memory subsystem (76.8 GB/s), but at considerably lower power and an orders of magnitude smaller footprint.
An interesting wrinkle of the SK Hynix's announcement is that the company started to supply these 24 GB LPDDR5X modules well before the announcement as the devices are already used in Oppo's Oneplus Ace 2 Pro smartphone launched on August 10, 2023.
Oppo is not the only maker of high-end Android smartphones out there, so I would expect more companies to follow suit in the coming months as they roll out handsets based on the Qualcomm Snapdragon 8 Gen 2 system-on-chip.
But SK Hynix envisions its LPDDR5X devices to be used beyond smartphones, so think PCs. Apple was the first company to use LPDDR for desktops and laptops. Still, now that PC SoCs from AMD and Intel support LPDDR5X expect other leading makers of notebooks to adopt LPDDR5X in general and SK Hynix's 24 GB packages in particular.
Meanwhile, 64-bit LPDDR5X-8500 devices look particularly compelling for the automotive industry, combining performance, capacity, and a very compact form factor. Given the fact that modern infotainment systems require high memory bandwidth, such memory devices will be pretty beneficial. SK Hynix says these memory stacks could be used for servers and even high-performance computing (HPC) applications.
"Along with a faster advancement in broader IT industry, our LPDDR products will be able to support a growing list of applications such as PC, server, high-performance computing (HPC) and automotive vehicles," said Myoungsoo Park, Vice President and Head of DRAM Marketing at SK Hynix. "The company will cement our leadership in the premium memory market by providing the highest performance products that meet customers' needs."
Source: SK Hynix
TrendForce projects a remarkable 105% increase in annual bit shipments of high-bandwidth memory (HBM) this year. This boost comes in response to soaring demands from AI and high-performance computing processor developers, notably Nvidia, and cloud service providers (CSPs). To fulfill demand, Micron, Samsung, and SK Hynix are reportedly increasing their HBM capacities, but new production lines will likely start operations only in Q2 2022.
Memory makers managed to more or less match the supply and demand of HBM in 2022, a rare occurrence in the market of DRAM. However, an unprecedented demand spike for AI servers in 2023 forced developers of appropriate processors (most notably Nvidia) and CSPs to place additional orders for HBM2E and HBM3 memory. This made DRAM makers use all of their available capacity and start placing orders for additional tools to expand their HBM production lines to meet the demand for HBM2E, HBM3, and HBM3E memory in the future.
However, meeting this HBM demand is not something straightforward. In addition to making more DRAM devices in their cleanrooms, DRAM manufacturers need to assemble these memory devices into intricate 8-Hi or 12-Hi stacks, and here they seem to have a bottleneck since they do not have enough TSV production tools, according to TrendForce. To produce enough HBM2, HBM2E, and HBM3 memory, leading DRAM producers have to procure new equipment, which takes 9 to 12 months to be made and installed into their fabs. As a result, a substantial hike in HBM production is anticipated around Q2 2024, the analysts claim.
A noteworthy trend pinpointed by TrendForce analysts is the shifting preference from HBM2e (Used by AMD's Instinct MI210/MI250/MI250X, Intel's Sapphire Rapids HBM and Ponte Vecchio, and Nvidia's H100/H800 cards) to HBM3 (incorporated in Nvidia's H100 SXM and GH200 supercomputer platform and AMD's forthcoming Instinct MI300-series APUs and GPUs). TrendForce believes that HBM3 will account for 50% of all HBM memory shipped in 2023, whereas HBM2E will account for 39%. In 2024, HBM3 is poised to account for 60% of all HBM shipments. This growing demand, when combined with its higher price point, promises to boost HBM revenue in the near future.
Just yesterday, Nvidia launched a new version of its GH200 Grace Hopper platform for AI and HPC that uses HBM3E memory instead of HBM3. The new platform consisting of a 72-core Grace CPU and GH100 compute GPU, boasts higher memory bandwidth for the GPU, and it carries 144 GB of HBM3E memory, up from 96 GB of HBM3 in the case of the original GH200. Considering the immense demand for Nvidia's offerings for AI, Micron — which will be the only supplier of HBM3E in 1H 2024 — stands a high chance to benefit significantly from the freshly released hardware that HBM3E powers.
TrendForce also noted a consistent decline in HBM product ASPs each year. To invigorate interest and offset decreasing demand for older HBM models, prices for HBM2e and HBM2 are set to drop in 2023, according to the market tracking firm. With 2024 pricing still undecided, further reductions for HBM2 and HBM2e are expected due to increased HBM production and manufacturers' growth aspirations.
In contrast, HBM3 prices are predicted to remain stable, perhaps because, at present, it is exclusively available from SK Hynix, and it will take some time for Samsung to catch up. Given its higher price compared to HBM2e and HBM2, HBM3 could push HBM revenue to an impressive $8.9 billion by 2024, marking a 127% YoY increase, according to TrendForce.
SK Hynix commanded 50% of the HBM memory market in 2022, followed by Samsung with 40% and Micron with a 10% share. Between 2023 and 2024, Samsung and SK Hynix will continue to dominate the market, holding nearly identical stakes that sum up to about 95%, TrendForce projects. On the other hand, Micron's market share is expected to hover between 3% and 6%.
Meanwhile, (for now) SK Hynix seems to have an edge over its rivals. SK Hynix is the primary producer of HBM3, the only company to supply memory for Nvidia's H100 and GH200 products. In comparison, Samsung predominantly manufactures HBM2E, catering to other chip makers and CSPs, and is gearing up to start making HBM3. Micron, which does not have HBM3 in the roadmap, produces HBM2E (which Intel reportedly uses for its Sapphire Rapids HBM CPU) and is getting ready to ramp up production of HBM3E in 1H 2024, which will give it a significant competitive advantage over its rivals that are expected to start making HBM3E only in 2H 2024.
This week, Micron announced the sample availability of its first CXL 2.0 memory expansion modules for servers that promise easy and cheap DRAM subsystem expansions.
Modern server platforms from AMD and Intel boast formidable 12- and 8-channel DDR5 memory subsystems offering bandwidth of up to 460.8 – 370.2 GB/s and capacities of up to 6 – 4 TB per socket. But some applications consume all DRAM they can get and demand more. To satisfy the needs of such applications, Micron has developed its CZ120 CXL 2.0 memory expansion modules that carry 128 GB and 256 GB of DRAM and connect to a CPU using a PCIe 5.0 x8 interface.
"Micron is advancing the adoption of CXL memory with this CZ120 sampling milestone to key customers," said Siva Makineni, vice president of the Micron Advanced Memory Systems Group.
Micron's CZ120 memory expansion modules use Microchip's SMC 2000-series smart memory controller that supports two 64-bit DDR4/DDR5 channels as well as Micron's DRAM chips made on the company's 1α (1-alpha) memory production node. Every CZ120 module delivers bandwidth up to 36 GB/s (measured by running an MLC workload with a 2:1 read/write ratio on a single module), putting it only slightly behind a DDR5-4800 RDIMM (38.4 GB/s) but orders of magnitude ahead of a NAND-based storage device.
Micron asserts that adding four of its 256 GB CZ120 CXL 2.0 Type 3 expansion modules to a server running 12 64GB DDR5 RDIMMs can increase memory bandwidth by 24%, which is significant. Perhaps more significant is that adding an extra 1 TB of memory enables such a server to handle nearly double the number of database queries daily.
Of course, such an expansion means using PCIe lanes and thus reducing the number of SSDs that can be installed into such a machine. But the reward seems quite noticeable, especially if Micron's CZ120 memory expansion modules are cheaper than actual RDIMMs or have comparable costs.
For now, Micron has announced sample availability, and it is unclear when the company will start to ship its CZ120 memory expansion modules commercially. Micron claims that it has already tested its modules with major server platform developers, so right now, its customers are probably validating and qualifying the modules with their machines and workloads, so it is reasonable to expect CZ120 to be deployed already in 2024.
"We have been developing and testing our CZ120 memory expansion modules utilizing both Intel and AMD platforms capable of supporting the CXL standard," added Makineni. "Our product innovation coupled with our collaborative efforts with the CXL ecosystem will enable faster acceptance of this new standard, as we work collectively to meet the ever-growing demands of data centers and their memory-intensive workloads."
While DDR5 memory has been out and in use for a couple of years now, so far we haven't seen the memory reach its full potential – at least, not for rank-and-file standards-compliant DIMMs. The specification allows for speeds as high as DDR5-6400, but to date we've only seen on-spec kits (and processors) as fast as DDR5-5600. But at last, it looks like things are about to change and DDR5 is set to live up to its full potential, going by a new memory kit announcement from TeamGroup.
The memory kit vendor on Monday introduced its new ElitePlus-series DDR5-6400 memory modules, the first DDR5-6400 kit to be announced as JEDEC specification compliant. This means their new kit not only hits 6400 MT/s with standards-compliant timings, but arguably more importantly, it does so at DDR5's standard voltage of 1.1V as well. And while there are no platforms on the market at this time that are validated for JEDEC DDR5-6400 speeds, TeamGroup's product page already lists compatibility with Intel's yet-to-be-announced "Z790 Refresh" platform – so suitable processors seem to be due soon.
TeamGroup's Elite and ElitePlus DDR5-6400 memory modules come in 16 GB and 32 GB capacities (32 GB and 64 GB dual-channel kits) and feature JEDEC-standard CL52 52-52-103 timings as well as 1.1V voltage, as specified by the organization overseeing DRAM specs. For the moment, at least, TeamGroup's DDR5-6400 modules are the industry's fastest UDIMMs that are fully compliant with the JEDEC specifications.
And while DDR5-6400 speeds (and far higher) are available today with factory overclocked XMP/EXPO, the announcement of a JEDEC standards-compliant kit is still significant for a few different reasons. Being able to hit DDR5-6400B speeds and timings at 1.1V means DDR5 memory has improved to the point to make higher speeds at low voltages more viable, which has potential payoffs for memory at every speed grade by allowing for improved speeds and reduced power consumption/heat. And for OEM and other warrantied systems that only use JEDEC-complaint RAM, this allows for a straightforward improvement in memory speeds and bandwidth. About the only downside to faster on-spec kits is that they lack XMP or EXPO serial presence detect (SPD) profiles, which makes their configuration slightly more complicated on existing platforms from AMD and Intel, as they don't officially support DDR5-6400.
Meanwhile, on their product pages TeamGroup notes that the new RAM is compatible with Intel's "Z790 Refresh" platform, a platform that has yet to be officially announced, but is rumored to go hand-in-hand with Intel "Raptor Lake Refresh" processors. Despite the lack of formal announcements from Intel there, TeamGroup seems to have let the cat out of the bag. So, prospective owners of Z790 Refresh systems can look forward to having access to specs-compliant 1.1V DDR5-6400 memory when that platform launches later this year.
As for the modules at hand, traditionally, TeamGroup's Elite and ElitePlus memory modules are minimalistic and are aimed both at system integrators and at enthusiasts who are not after fancy designs of heat spreaders, RGB lighting, and maximum performance. In fact, TeamGroup's Elite modules do not have heat spreaders at all, whereas ElitePlus modules have a minimalistic heat spreader that will not interfere with large CPU coolers.
TeamGroup says its Elite and ElitePlus DDR5-6400 memory modules will be available separately and in dual-channel kits starting from August in North America and Taiwan. And from that, we'd assume, Raptor Lake Refresh will not be far behind.
In addition to unveiling its first HBM3 memory products yesterday, Micron also published a fresh DRAM roadmap for its AI customers for the coming years. Being one of the world's largest memory manufacturers, Micron has a lot of interesting things planned, including high-capacity DDR5 memory devices and modules, GDDR7 chips for graphics cards and other bandwidth-hungry devices, as well as HBMNext for artificial intelligence and high-performance computing applications.
We all love inexpensive high-capacity memory modules, and it looks like Micron has us covered. Sometimes in the late first half of 2024, the company plans to roll-out its first 32 Gb DDR5 memory dies, which will be produced on the company's 1β (1-beta) manufacturing process. This is Micron's latest process node and which does not use extreme ultraviolet lithography, but rather relies on multipatterning.
32 Gb DRAM dies will enable Micron to build 32 GB DDR5 modules using just eight memory devices on one side of the module. Such modules can be made today with Micron's current 16 Gb dies, but this requires either placing 16 DRAM packages over both sides of a memory module – driving up production costs – or by placing two 16 Gb dies within a single DRAM package, which incurs its own costs due to the packaging required. 32 Gb ICs, by comparison, are easier to use, so 32 GB modules based on denser DRAM dies will eventually lead to lower costs compared to today's 32 GB memory sticks.
But desktop matters aside, Micron's initial focus with their higher density dies will be to build even higher capacity data center-class parts, including RDIMMs, MRDIMMs, and CXL modules. Current high performance AI models tend to be very large and memory constrained, so larger memory pools open the door both to even larger models, or in bringing down inference costs by being able to run additional instances on a single server.
For 2024, Micron is planning to release 128GB DDR5 modules based on these new dies. In addition, the company announced plans for 192+ GB and 256+ GB DDR5 modules for 2025, albeit without disclosing which chips these are set to use.
Meanwhile, Micron's capacity-focused roadmap doesn't have much to say about bandwidth. While it would be unusual for newer DRAM dies not to clock at least somewhat higher, memory manufacturers as a whole have not offered much guidance about future DDR5 memory speeds. Especially with MRDIMMs in the pipeline, the focus is more on gaining additional speed through parallelism, rather than running individual DRAM cells faster. Though with this roadmap in particular, it's clear that Micron is more focused on promoting DDR5 capacity than promoting DDR5 performance.
Micron was the first larger memory maker to announce plans to roll out its GDDR7 memory in the first half of 2024. And following up on that, the new roadmap has the the company prepping 16 Gb and 24 Gb GDDR7 chips for late Q2 2024.
As with Samsung, Micron's plans for their first generation GDDR7 modules do not have them reaching the spec's highest transfer rates right away (36 GT/sec), and instead Micron is aiming for a more modest and practical 32 GT/sec. Which is still good enough to enable upwards of 50% greater bandwidth for next-generation graphics processors from AMD, Intel, and NVIDIA. And perhaps especially NVIDIA, since this roadmap also implies that we won't be seeing a GDDR7X from Micron, meaning that for the first time since 2018, NVIDIA won't have access to a specialty GDDR DRAM from Micron.
In addition to GDDR7, which will be used by graphics cards, game consoles, and lower-end high-bandwidth applications like accelerators and networking equipment, Micron is also working on the forthcoming generations of its HBM memory for heavy-duty artificial intelligence (AI) and high-performance computing (HPC) applications.
Micron expects its HBMNext (HBM4?) to be available in 36 GB and 64 GB capacities, which points to a variety of configurations, such as 12-Hi 24 Gb stacks (36 GB) or 16-Hi 32 Gb stacks (64 GB), though these are pure speculations at this point. As for performance, Micron is touting 1.5 TB/s – 2+ TB/s of bandwidth per stack, which points to data transfer rates in excess of 11.5 GT/s/pin.
Micron today is introducing its first HBM3 memory products, becoming the latest of the major memory manufacturers to start building the high bandwidth memory that's widely used in server-grade GPUs and other high-end processors. Aiming to make up for lost time against its Korean rivals, Micron intends to essentially skip "vanilla" HBM3 and move straight on to even higher bandwidth versions of the memory they're dubbing "HBM3 Gen2", developing 24 GB stacks that run at over 9 GigaTransfers-per-second. These new HBM3 memory stacks from Micron will target primarily AI and HPC datacenter, with mass production kicking off for Micron in early 2024.
Micron's 24 GB HBM3 Gen2 modules are based on stacking eight 24Gbit memory dies made using the company's 1β (1-beta) fabrication process. Notably, Micron is the first of the memory vendors to announce plans to build HBM3 memory with these higher-density dies, while SK hynix offers their own 24 GB stacks, the company is using a 12-Hi configuration of 16Gbit dies. So Micron is on track to be the first vendor to offer 24 GB HBM3 modules in the more typical 8-Hi configuration. And Micron is not going to stop at 8-Hi 24Gbit-based HBM3 Gen2 modules, either, with the company saying that they plan to introduce even higher capacity class-leading 36 GB 12-Hi HBM3 Gen2 stacks next year.
Besides taking the lead in density, Micron is also looking to take a lead in speed. The company expects its HBM3 Gen2 parts to hit date rates as high as 9.2 GT/second, 44% higher than the top speed grade of the base HBM3 specification, and 15% faster than the 8 GT/second target for SK hynix's rival HBM3E memory. The increased data transfer rate enables each 24 GB memory module to offer peak bandwidth of 1.2 TB/sec per stack.
Micron says that 24GB HBM3 Gen2 stacks will enable 4096-bit HBM3 memory subsystems with a bandwidth of 4.8 TB/s and 6096-bit HBM3 memory subsystems with a bandwidth of 7.2 TB/s. To put the numbers into context, Nvidia's H100 SXM features a peak memory bandwidth of 3.35 TB/s.
| HBM Memory Comparison | ||||||
| "HBM3 Gen2" | HBM3 | HBM2E | HBM2 | |||
| Max Capacity | 24 GB | 24 GB | 16 GB | 8 GB | ||
| Max Bandwidth Per Pin | 9.2 GT/s | 6.4 GT/s | 3.6 GT/s | 2.0 GT/s | ||
| Number of DRAM ICs per Stack | 8 | 12 | 8 | 8 | ||
| Effective Bus Width | 1024-bit | |||||
| Voltage | 1.1 V? | 1.1 V | 1.2 V | 1.2 V | ||
| Bandwidth per Stack | 1.2 TB/s | 819.2 GB/s | 460.8 GB/s | 256 GB/s | ||
High frequencies aside, Micron's HBM3 Gen2 stacks are otherwise drop-in compatible with current HBM3-compliant applications (e.g., compute GPUs, CPUs, FPGAs, accelerators). So device manufacturers will finally have the option of tapping Micron as an HBM3 memory supplier as well, pending the usual qualification checks.
Under the hood, Micron's goal to jump into an immediate performance leadership position within the HBM3 market means that they need to one-up their competition from a technical level. Among other changes and innovations to accomplish that, the company increased the number of through-silicon vias (TSVs) by two times compared to shipping HBM3 products. In addition, Micron shrunk the distance between DRAM devices in its HBM3 Gen2 stacks. These two changes to packaging reduced thermal impendence of these memory modules and made it easier to cool them down. Yet, the increased number of TSVs can bring other advantages too.
Given that Micron uses 24 Gb memory devices (rather than 16 Gb memory devices) for its HBM3 Gen2 stacks, it is inevitable that it had to increase the number of TSVs to ensure proper connectivity. Yet, doubling the number of TSVs in an HBM stack can enhance overall bandwidth (and shrink latency), power efficiency, and scalability by facilitating more parallel data transfers. It also improves reliability by mitigating the impact of single TSV failures through data rerouting. However, these benefits come with challenges such as increased manufacturing complexity and increased potential for higher defect rates (already an ongoing concern for HBM), which can translate to higher costs.
Just like other HBM3 memory modules, Micron's HBM3 Gen2 stacks feature Reed-Solomon on-die ECC, soft repair of memory cells, hard-repair of memory cells as well as auto error check and scrub support.
Micron says it will mass produce its 24 GB HBM3 modules starting in Q1 2024, and will start sampling its 12-Hi 36GB HBM3 stacks around this time as well. The latter will enter high volume production in the second half of 2024.
To date, the JEDEC has yet to approve a post-6.4GT/second HBM3 standard. So Micron's HBM3 Gen2 memory, as well as SK hynix's rival HBM3E memory, are both off-roadmap standards for the moment. Given the interest in higher bandwidth HBM memory and the need for standardization, we'd be surprised if the group didn't eventually release an updated version of the HBM3 standard that Micron's devices will conform to. Though as the group tends to shy away from naming battles ("HBM2E" was never a canonical product name for faster HBM2, despite its wide use), it's anyone's guess how this latest kerfuffle over naming will play out.
Beyond their forthcoming HBM3 Gen2 products, Micron is also making it known that the company already working on HBMNext (HBM4?) memory. That iteration of HBM will provide 1.5 TB/s – 2+ TB/s of bandwidth per stack with capacities ranging from 36 GB to 64 GB.
In a surprising turn of events, Cadence and Rambus entered into a definitive agreement late last week for Cadence to buy memory physical interface IP and SerDes businesses from Rambus. As a result, Cadence will get a comprehensive portfolio of memory PHY IP and an established client base. Meanwhile, with the sale of its PHY and SerDes assets, Rambus will now solely focus on licensing digital IP.
Historically, Rambus developed memory technologies, including RDRAM and XDR DRAM. At some point, the company patented fundamental technologies enabling SDRAM, DDR SDRAM, and their successors. Doing this allowed them to effectively sue virtually all memory makers and designers of memory controllers (including AMD and Nvidia) and make them pay license fees.
Over time the company began to license memory controllers and PHY. It became a one-stop shop for chip developers needing a turnkey memory, PCIe, or MIPI solution for their designs. Nowadays, it is possible to come to Rambus and get one of the industry's best memory controllers and silicon-proven interfaces. But while Rambus plans to retain memory and interface controllers and everything logic-related, it intends to get rid of its PHY and SerDes IP assets and sell them to Cadence.
Getting silicon-proven PHY and SerDes IP assets and clients for Cadence makes perfect sense.
"The acquisition of the Rambus PHY IP broadens Cadence's well-established enterprise IP portfolio and expands its reach across geographies and vertical markets, such as the aerospace and defense market, providing complete subsystem solutions that meet the demands of our worldwide customers," said Boyd Phelps, senior vice president and general manager of the IP Group at Cadence.
But the rationale behind Rambus's decision to sell PHY and SerDes business is less obvious. On the one hand, memory PHY and SerDes businesses require Rambus to invest in expensive tape-outs on the latest nodes, and this requires capital and increases risks as Rambus has to compete against companies like Cadence and Synopsys that are larger and have more money. On the other hand, Rambus can be a one-stop shop for memory controllers and PHY, which has advantages (i.e., Rambus can charge a premium).
Meanwhile, without needing to keep its physical IP assets up to date, Rambus can now focus on licensing pure technologies and no longer invest in physical IP like PHY or SerDes.
"With this transaction, we will increase our focus on market-leading digital IP and chips and expand our roadmap of novel memory solutions to support the continued evolution of the data center and AI," said Sean Fan, senior vice president and chief operating officer at Rambus.
The transaction is projected to have a negligible impact on the revenue and earnings of each company for this year. The anticipated closing date is in the third quarter of 2023, subject to specific closing conditions.
Source: Cadence
Samsung has announced this evening that they have completed development on their first generation of GDDR7 memory. The next iteration of the high bandwidth memory technology, which has been under industry-wide development, is expected to hit the market in 2024, with Samsung in prime position to be one of the first memory vendors out of the gate. With their first generation of GDDR7 parts slated to hit up to 32Gbps/pin of bandwidth – 33% more than their best GDDR6 parts today – the company is looking to deliver a sizable increase in GDDR memory bandwidth on the back of the technology’s adoption of PAM-3 signaling.
Samsung’s announcement comes as we’ve been seeing an increase in disclosures and announcements around the next iteration of the widely-used memory technology. While a finished specification for the memory has yet to be released by JEDEC, Samsung rival Micron has previously announced that it plans to introduce its own GDDR7 memory in 2024 – a similar timeline as to Samsung’s current schedule. Meanwhile, EDA tool firm Cadence disclosed a significant amount of technical details earlier this year as part of announcing their GDDR7 verification tools, revealing that the memory would use PAM-3 signaling and reach data rates of up to 36Gbps/pin.
With today’s announcement, Samsung has become the first of the major memory manufacturers to publicly announce that they’ve completed development of their first generation of GDDR7. And while the company tends to make these sorts of memory announcements relatively early in the bring-up process – well before memory is ready for commercial mass product – it’s none the less an important milestone in the development of GDDR7, as it means that memory and device manufacturers can begin validation work against functional hardware. As for Samsung itself, the announcement gives the Korean conglomerate a very visible opportunity to reinforce their claim of leadership within the GDDR memory industry.
Besides offering an update on the development process for GDDR7, Samsung’s announcement also provides some high-level technical details about the company’s first generation of GDDR7 – though “high-level” is the operative word as this is not by any means a technical deep dive.
| GPU Memory Math | |||||
| GDDR7 | GDDR6X | GDDR6 | |||
| B/W Per Pin | 32 Gbps (Projected) | 24 Gbps (Shipping) | 24 Gbps (Sampling) | ||
| Chip Density | 2 GB (16 Gb) | 2 GB (16 Gb) | 2 GB (16 Gb) | ||
| Total B/W (256-bit bus) | 1024 GB/sec | 768 GB/ssec | 768 GB/ssec | ||
| DRAM Voltage | 1.2 V | 1.35 V | 1.35 V | ||
| Data Rate | QDR | QDR | QDR | ||
| Signaling | PAM-3 | PAM-4 | NRZ (Binary) | ||
| Packaging | 266 FBGA | 180 FBGA | 180 FBGA | ||
According to Samsung’s announcement, they’re expecting to reach data rates as high as 32Gbps/pin. That’s 33% higher than the 24Gbps data rate the company’s top GDDR6 products can hit today. Samsung and Cadence have both previously disclosed that they expect GDDR7 memory to eventually hit 36Gbps/pin, though as with the development of GDDR6 – a full 50% faster than GDDR6 – this is likely going to take multiple generations of products.
Interestingly, this is starting much closer to projected limits of GDDR7 than we’ve seen in past generations of the memory technology. Whereas GDDR6 launched at 14Gbps and eventually scaled up to 24Gbps, Samsung wants to start at 32Gbps. At the same time, however, GDDR7 is going to be a smaller generational leap than we saw for GDDR6 or GDDR5; rather than doubling the signaling bandwidth of the memory technology over its predecessor, GDDR7 is only a 50% increase, owing to the switch from NRZ (2 state) signaling to PAM-3 (3 state) signaling.
It’s also worth noting that, at present, the fastest GDDR6 memory we see used video cards is only running at 20Gbps.Samsung’s own 24Gbps GDDR6, though announced just over a year ago, is still only “sampling” at this time. So the multitude of other GDDR6-using products notwithstanding, the effective jump in bandwidth for video cards in 2024/2025 could be more significant, depending on just what speed grades are available at the time.
As for capacity, Samsung’s first GDDR7 chips are 16Gb, matching the existing density of today’s top GDDR6(X) chips. So memory capacities on final products will not be significantly different from today’s products, assuming identical memory bus widths. DRAM density growth as a whole has been slowing over the years due to scaling issues, and GDDR7 will not be immune to that.
Samsung is also claiming that their GDDR7 technology offers a “20%-improvement in power efficiency versus existing 24Gbps GDDR6 DRAM,” though this is a broad claim where the devil is in the details. As power efficiency for DRAM is normally measured on a per-bit basis (picojoules-per-bit/pJpb), then our interpretation is that this is the figure Samsung is referencing in that claim.
Update (7/19, 3pm ET): Following a round of Q&A, Samsung has provided some additional technical details on their first-generation GDDR7 memory, including voltages (1.2v), how they are measuring energy efficiency, and the process node being used (D1z). The rest of the article has been updated accordingly.
At a high level, the good news is that Samsung’s GDDR7 is slated to deliver a tangible increase in energy efficiency. With a nominal voltage of 1.2v, GDDR7 requires less voltage to drive it than its predecessor. But with only a 20% overall improvement in energy efficiency for a memory technology that is delivering up to 33% more bandwidth, this means that the absolute power consumption of the memory is going up versus the previous generation. As Samsung’s energy efficiency figures are for GDDR7@32Gbps versus GDDR6@24Gbps, we're looking at around a 7% increase in total energy consumption. Which, thankfully, is not a huge increase in power consumption. But it is an increase none the less.
Broadly speaking, this is the same outcome as we saw with the introduction of GDDR6(X), where despite the energy efficiency gains there, overall power consumption increased from one generation to the next, as energy efficiency gains are not keeping pace with bandwidth demands. Not to say that any of this is unexpected, but it means that good cooling will be even more critical for GDDR7 memory.
But for clients with strict power/cooling needs, Samsung is also announcing that they will be making a low-voltage version of their GDDR7 memory available. This will be a 1.1v version of their GDDR7 chips, and while clockspeeds haven't been disclosed, we'd expect something closer to their GDDR6 clockspeeds. With current-generation GDDR6, high-end laptops are typically paired with low-voltage memory running no faster than 18Gbps, so low-voltage GDDR7 running at around 24Gbps would still represent a significant step forward.
Samsung has also confirmed that they're going to be using their D1z memory fab process for their first generation GDDR7. This is the same EUV-based process the company is using for their most recent GDDR6 – but curiously, it's older than the 12nm process Samsung revealed they'll be using elsewhere in a recent DDR5 memory announcement. The fact that the company isn't using a newer node here is a bit surprising, but from an architectural perspective, it also means that Samsung has achieved a 20% increase in energy efficiency without a newer node. Architecturally-driven energy gains are few and far between in both the memory and logic spaces these days, all of which makes for a promising sign for the future of GDDR7.
Speaking of architecture and design, the company notes that their GDDR7 memory employs “IC architecture optimization” to keep power and heat generation in check. Though at this point we don't have any further details on just what that means.
Electronics production aside, the final major innovation with Samsung’s GDDR7 will be decidedly physical: epoxy. Clearly mindful of the already high heat loads generated by existing GDDR memory clocked at its highest speeds, Samsung’s press release notes that they’re using a new epoxy molding compound (EMC) for GDDR7, which is designed to better transfer heat. All told, Samsung is claiming a 70% reduction in thermal resistance versus their GDDR6 memory, which should help ensure that a good cooler can still pull enough heat from the memory chips, despite the overall increase in heat generation.
Wrapping things up, now that initial development on their GDDR7 memory has been completed, Samsung is moving on to verification testing with their partners. According to the company, they’ll be working with key customers on verification this year; though at this point, the company isn’t saying when they expect to kick off mass production of the new memory.
Given the timing of Samsung’s announcement (as well as Micron’s), the initial market for GDDR7 seems to be AI and networking accelerators, rather than the video cards that GDDR7 gets its name from. With both AMD and NVIDIA barely a quarter of the way through their current architectural release cycles, neither company is likely be in a position to use GDDR7 in 2024 when it’s ready. Instead, it’s going to be the other users of GDDR memory such as networking products and high-performance accelerators that are likely to be first to use the technology.
Micron late on Wednesday revealed plans to introduce its first GDDR7 memory devices in the first half of 2024. The memory is expected to be used by next generation of graphics cards, and deliver performance that will be considerably higher than that of GDDR6 and GDDR6X.
"We plan to introduce our next-generation G7 product on our industry-leading 1ß node in the first half of calendar year 2024," said Sanjay Mehrotra, chief executive of Micron, as part of the company's earnings call.
Micron did not provide any additional specifics about its GDDR7 SGRAM devices that are set to be introduced in the first half of calendar 2024, though some general things about the technology have already been revealed by Cadence and Samsung in the recent months.
Samsung expects next generation GDDR to hit data transfer rates of 36 GT/s, though it's unclear whether they're talking about initial speeds for the new memory, some time later on. In any case, any increase over current 22 ~ 23 GT/s offered by GDDR6X will make the new type of memory more preferable for bandwidth-hungry devices like high-end graphics cards.
Meanwhile, Cadence has previously disclosed that GDDR7 will use PAM3 signaling, a three-level pulse amplitude modulation (which includes -1, 0, and +1 signaling levels), which allows it to transfer three bits of data over a span of two cycles. PAM3 provides a more efficient data transmission rate per cycle compared to two-level NRZ used by GDDR6, thereby reducing the need to upgrade to higher memory interface clocks and the subsequent signal loss challenges that this might cause. GDDR6X currently does something similar with PAM4 (four states), so GDDR7 will be a bit different still. PAM3 ultimately transmits a bit less data per clock edge (1.5 bits vs. 2 bits), but it trades off with less stringent signal-to-noise ratio requirements.
While Micron plans to introduce its first GDDR7 product in the first half of 2024, an official launch of a new memory device indicates conclusion of its development and not its immediate use in commercial products. As GDDR7 employs a new encoding mechanism, it requires brand-new new memory controllers, and hence, graphics processors. While it is reasonable to anticipate AMD, Intel, and NVIDIA to introduce their next generation GPUs in the 2024 – 2025 timeframe, the exact timing of these graphics processor releases remains completely unclear.
For now, Cadence has GDDR7 verification solution for chip designers that need to ensure that their controllers and PHY are compliant with the upcoming specification while they finalize design of their GPUs and other processors.
G.Skill has quietly started selling its 24 GB DDR5 memory modules with AMD EXPO profiles for single-click overclocking. G.Skill's Trident Z5 Neo RGB modules are among the first EXPO-profiled DIMMs larger than 16GB to hit the market, with G.Skill offering kits as large as 48GB (2 x 24GB).
G.Skill's 24 GB Trident Z5 Neo RGB memory modules with AMD EXPO profiles support are designed for a 6000 MT/s data transfer rate, which is considered to be a sweet spot for AMD's Ryzen 7000-series processors based on the Zen 4 microarchitecture. As for timings, the manufacturer recommends CL40 48-48-96 settings at 1.35V, which is a rather significant (22%) overvoltage for DDR5 memory.
Traditional for G.Skill's Trident-series modules for PC hardware enthusiasts and overclockers, 24 GB Trident Z5 Neo RGB DIMMs come equipped with aluminum heat spreaders and, as their name suggests, addressable RGB bars. Keeping in mind that these memory sticks are overvolted and also carry a power management IC and voltage regulating module onboard, these heat spreaders promise to come handy.
The key selling point of G.Skill's 24GB Trident Z5 Neo RGB modules and associated 48GB dual-channel kits is support for AMD's EXPO memory technology, which enables single-click overclocking profiles for modules rated to operate at beyond standard settings.
24GB DIMMs have been available in the market for a bit, but the first products were aimed at Intel systems, and shipped with XMP 3.0 profiles. Initial media reports have demonstrated that there have been some compatibility issues with these new 24GB XMP DIMMs and AMD systems, so having DIMMs that are formally tested for AMD platforms is a welcome step forward. Though most of the heavy lifting is coming from UEFI BIOS updates to account for the relatively novel, non-power-of-two organization of these new DIMMs.
G.Skill's Trident Z5 Neo RGB 48GB dual-channel (2 x 24GB) DDR5-6000 kit F5-6000J4048F24GX2-TZ5NR is now available from Newegg for $159.99.
This week, Realtek sued rival MediaTek in Northern California federal court, accusing it of conspiring against it with a patent holder in a bid to drive Realtek out of business. Reuters reports that Realtek asserts that MediaTek intends to monopolize the markets of smart TVs and set-top-boxes and asks the court to protect itself from the ongoing injury and the market from unfair competition practices.
It is not a secret that various patent assertion entities (PAE), a juridical term for patent hoarders, acquire patents and then sue different designers for royalties. In 2019, MediaTek entered into a patent license agreement with Future Link and its owner IPValue to license some of the patents issued initially to NXP Semiconductor and Philips. While this would be a reasonably typical license agreement, it contained a secret provision, which included MediaTek's demanding Future Link act against its rivals, including Realtek.
Future Link sued Realtek for alleged patent infringement on April 13, 2021, accusing the chip designer of infringing two of its patents related to TV SoCs. By the end of 2021, Future Link reportedly expanded the list of its allegations against Realtek with two more patents and then dropped some of the claims. The important thing here is that Future Link alleges that all of Realtek's products that use Arm's AXI interface for multi-core SoCs infringe its patents, which in turn means that virtually all of Realtek's TV chips use its IP illegally and have to be barred from the market, which will eliminate a rival for Mediatek, which already controls 60% of the TV SoC market, according to Reuters citing the lawsuit.
At the time, Realtek was not aware that the action brought by Future Link was inspired by its rival MediaTek, the company says.
"In April 2022, there was a shocking revelation regarding that seemingly innocuous agreement," Realtek states. "Discovery revealed that before the litigation against Realtek began, the license agreement among MediaTek, IPValue, and Future Link included a secret litigation 'bounty' provision previously hidden from the public and Realtek. […]. Specific details of this scheme remain hidden from the public even today because Future Link has managed to keep the arrangement buried under confidentiality obligations and protective orders."
On April 12, 2022, an Administrative Law Judge at the ITC said that the license agreement between Future Link and MediaTek contained a demand for action against Realtek.
"At a minimum, it would seem to warrant an action by Realtek against either Future Link or its counterparty for unfair competition," the ALJ wrote.
Realtek says its move aims to counter a modern-day monopolist and its allies to protect itself, uphold competition in the semiconductor industry, and hold the offenders — MediaTek, Future Link, and IPValue — accountable. Any damages recovered by Realtek from this action will be donated to charity.
"Because Realtek seeks to protect the public interest with this action, Realtek will donate the amount of damages that it recovers to charity," the company said.
MediaTek, Future Link, and IPValue did not comment on the story.
Sources: Reuters, Reuters (PDF)
At Computex 2023, TeamGroup unveiled two additions to its extensive family of memory products. Available with ARGB heatsinks or subtle black non-RGB heatsinks, the TeamGroup T-Force Xtreem DDR5 memory promises high performance for memory overclockers and gamers.
Looking to rival the high-speed DDR5 kits from companies such as G.Skill, the T-Force Xtreem DDR5 memory is available up to DDR5-8266, with overclocked XMP 3.0 profiles designed to be used with Intel's 13th Gen Core series processors. Regarding the competition, the T-Force Xtreme DDR5-8266 will be one of, if not the fastest, fastest kit available at retail upon its launch, as G.Skill has a kit of their Trident Z5 DDR5-8000 at retail.
The most glamorous looking of the two kits is the T-Force Xtreem ARGB DDR5 memory, with dual light pipes that sit between black translucent acrylic panels. Underneath the acrylic is a 2 mm thick aluminum alloy heatsink which has been anodized black. Between the memory ICs and the heatsink, TeamGroup claims they use a 'highly thermally conductive silicone gel' to provide optimal heat dissipation.
Despite TeamGroup not disclosing the height of the DRAM modules, they look pretty tall, so they might not be compatible with all air coolers, especially those large dual/triple designs. TeamGroup claims that the heat spreaders themselves have a texture similar to black sea sand, although the feel of memory is less important than other things, such as style and actual performance.
The second kit has the same overall shape and design as the ARGB variant but without the actual ARGB LED lighting. This gives the T-Force Xtreem DDR5 modules a classier look with a subtle matte black color scheme throughout, while one side of the heat spreader has a T-Force logo badge which reminds me of something Star Trek characters have on their uniforms.
Due to the speed of these kits, the T-Force Xtreem kits are primarily for Intel's current 13th Gen Core series processors, as AMD's Ryzen 7000 series just cannot hit the memory speeds that Intel's 13th Gen chips can. The TeamGroup T-Force Xtreem ARGB DDR5 and T-Force Xtreem DDR5 kits will come in kits ranging between DDR5-7000 and a blazingly fast DDR5-8266 kit.
As it stands, TeamGroup hasn't unveiled its expected pricing for the T-Force Xtreem ARGB and non-ARGB DDR5 memory kits, nor has it provided what memory ICs they are using or the latency timings. These are expected to be unveiled closer to their launch, which at the moment is unknown.
Dynamic random access memory is an indispensable part of all computers, and requirements for DRAM — such as performance, power, density, and physical implementation — tend to change now and then. In the coming years, we will see new types of memory modules for laptops and servers as traditional SO-DIMMs and RDIMMs/LRDIMMs seem to run out of steam in terms of performance, efficiency, and density.
ADATA demonstrated potential candidates to replace SO-DIMMs and RDIMMs/LRDIMMs from client and server machines, respectively, in the coming years, at Computex 2023 in Taipei, Taiwan, reports Tom's Hardware. These include Compression Attached Memory Modules (CAMMs) for at least ultra-thin notebooks, compact desktops, and other small form-factor applications; Multi-Ranked Buffered DIMMs (MR-DIMMs) for servers; and CXL memory expansion modules for machines that need extra system memory at a cost that is below that of commodity DRAM.
The CAMM specification is slated to be finalized by JEDC later in 2023. Still, ADATA demonstrated a sample of such a module at the trade show to highlight its readiness to adopt the upcoming technology.
Image Courtesy Tom's Hardware
The key benefits CAMMs include shortened connections between memory chips and memory controllers (which simplifies topology and therefore enables higher transfer rates and lowers costs), usage of modules based on DDR5 or LPDDR5 chips (LPDDR has traditionally used point-to-point connectivity), dual-channel connectivity on a single module, higher DRAM density, and reduced thickness when compared to dual-sided SO-DIMMs.
While the transition to an all-new type of memory module will require tremendous effort from the industry, the benefits promised by CAMMs will likely justify the change.
Last year, Dell was the first PC maker to adopt CAMM in its Precision 7670 notebook. Meanwhile, ADATA's CAMM module differs significantly from Dell's version, although this is not unexpected as Dell has been using pre-JEDEC-standardized modules.
Datacenter-grade CPUs are increasing their core count rapidly and therefore need to support more memory with each generation. But it is hard to increase DRAM device density at a high pace due to costs, performance, and power consumption concerns, which is why along with the number of cores, processors add memory channels, which results in an abundant number of memory slots per CPU socket and increased complexity of motherboards.
This is why the industry is developing two types of memory modules to replace RDIMMs/LRDIMMs used today.
On the one hand, there is the Multiplexer Combined Ranks DIMM (MCR DIMM) technology backed by Intel and SK Hynix, which are dual-rank buffered memory modules with a multiplexer buffer that fetches 128 bytes of data from both ranks that work simultaneously and works with memory controller at high speed (we are talking about 8000 MT/s for now). Such modules promise to increase performance and somewhat simplify building dual-rank modules significantly.
Image Courtesy Tom's Hardware
On the other hand, there is the Multi-Ranked Buffered DIMM (MR DIMM) technology which seems to be supported by AMD, Google, Microsoft, JEDEC, and Intel (at least based on information from ADATA). MR DIMM uses the same concept as MCR DIMM (a buffer that allows the memory controller to access both ranks simultaneously and interact with the memory controller at an increased data transfer rate). This specification promises to start at 8,800 MT/s with Gen1, then evolve to 12,800 MT/s with Gen2, and then skyrocket to 17,600 MT/s in its Gen3.
ADATA already has MR DIMM samples supporting an 8,400 MT/s data transfer rate that can carry 16GB, 32GB, 64GB, 128GB, and 192GB of DDR5 memory. These modules will be supported by Intel's Granite Rapids CPUs, according to ADATA.
But while both MR DIMMs and MCR DIMMs promise to increase module capacity, some servers need a lot of system memory at a relatively low cost. Today such machines have to rely on Intel's Optane DC Persistent Memory modules based on now obsolete 3D XPoint memory that reside in standard DIMM slots. Still, in the future, they will use memory on modules featuring a Compute Express Link (CXL) specification and connected to host CPUs using a PCIe interface.
Image Courtesy Tom's Hardware
ADATA displayed a CXL 1.1-compliant memory expansion device at Computex with an E3.S form factor and a PCIe 5.0 x4 interface. The unit is designed to expand system memory for servers cost-effectively using 3D NAND yet with significantly reduced latencies compared to even cutting-edge SSDs.
Corsair has introduced its new Dominator Titanium series of DDR5 memory modules that will combine performance, capacity, and style. The new lineup of memory modules and kits will offer DRAM kits up to 192 GB in capacity at data transfer rates as high as DDR5-8000.
The Dominator Titanium DIMMs are based on cherry-picked memory chips and Corsair's own printed circuit boards to ensure signal quality and integrity. Also, these PCBs are supplemented with internal cooling planes and external thermal pads that transfer heat to aluminum heat spreaders, with an aim on keeping the heavily overclocked DRAM sufficiently cooled.
With regards to performance, the retail versions of the Titanium kits will run at speeds ranging from DDR5-6000 to DDR5-8000. Which, at the moment, would make the top-end SKUs of the highest clocked DDR5 RAM on the market. Corsair is also promissing kits with CAS latencies as low as CL30, though absent a full product matrix, it's likely those kits will be clocked lower. The DIMMs come equipped with AMD's EXPO (AMD version) and Intel's XMP 3.0 (Intel version) SPD profiles for easier overclocking.
As for capacity, the Titanium DIMMs will be available in 16GB, 24GB, 32GB, and 48GB configurations, allowing for kits ranging from 32GB (2 x 16GB) up to 192GB (4x 48GB). Following the usual rule curve for DDR5 memory kits, we'll wager that DDR5-8000 kits won't be avaialble in 192GB capacities – even Intel's DDR5 memory controller has a very hard time with running 4 DIMMs anywhere near that fast – so we're expecting that the fastest kits will be limited to smaller capacities; likely 48GB (2 x 24GB).
Corsair is not disclosing whose memory chips it uses for its Dominator Titanium memory modules, but there is a good chance that it uses Micron's latest generation of DDR5 chips, which are available in both 16Gbit and 24Gbit capacities. Micron was the first DRAM vendor to publicly start shipping 24Gbit DRAM chips, so they are the most likely candidate for the first 24GB/48GB DIMMs such as Corsair's. And if that's the case, that would mark an interesting turn-around for Micron; the company's first-generation DDR5 modules are not known for overclocking very well, which is why we haven't been seeing them on current high-end DDR5 kits.
Image Credit: Future/TechRadar
Corsair has also taken into account aesthetic preferences by incorporating 11 addressable Capellix RGB LEDs into the modules. Users can customize and control these LEDs using Corsair's iCue software. For those favoring minimalism, Corsair offers separate Fin Accessory Kits. These kits replace the RGB top bars with fins, bringing a classic look reminiscent of the original Dominator memory.
While Corsair's new Dominator Titanium memory modules are already very fast, to commemorate their debut Corsair plans to release a limited run of First-Edition kits. These exclusive kits will feature even higher clocks and tighter timings – likely running at DDR5-8266 speeds, which Corsair is showing off at Computex. Corsair intends to offer only 500 individually numbered First-Edition kits.
Corsair plans to start selling its Dominator Titanium kits in July. Pricing will depend on market conditions, but expect these DIMMs to carry a premium price tags.
SK Hynix was one of the key developers of the original HBM memory back in 2014, and the company certainly hopes to stay ahead of the industry with this premium type of DRAM. On Tuesday, buried in a note about qualifying the company's 1bnm fab process, the the manufacturer remarked for the first time that it is working on next-generation HBM3E memory, which will enable speeds of up to 8 Gbps/pin and will be available in 2024.
Contemporary HBM3 memory from SK Hynix and other vendors supports data transfer rates up to 6.4Gbps/pin, so HBM3E with an 8 Gbpis/pin transfer rate will provide a moderate, 25% bandwidth advantage over existing memory devices.
To put this in context, with a single HBM stack using a 1024-bit wide memory bus, this would give a known good stack die (KGSD) of HBM3E around 1 TB/sec of bandwidth, up from 819.2 GB/sec in case of HBM3 today. Which, with modern HPC-class processors employing half a dozen stacks (or more), would work out to several TB/sec of bandwidth for those high-end processors.
According to the company's note, SK Hynix intends to start sampling its HBM3E memory in the coming months, and initiate volume production in 2024. The memory maker did not reveal much in the way of details about HBM3E (in fact, this is the first public mention of its specifications at all), so we do not know whether these devices will be drop-in compatible with existing HBM3 controllers and physical interfaces.
| HBM Memory Comparison | ||||||
| HBM3E | HBM3 | HBM2E | HBM2 | |||
| Max Capacity | ? | 24 GB | 16 GB | 8 GB | ||
| Max Bandwidth Per Pin | 8 Gb/s | 6.4 Gb/s | 3.6 Gb/s | 2.0 Gb/s | ||
| Number of DRAM ICs per Stack | ? | 12 | 8 | 8 | ||
| Effective Bus Width | 1024-bit | |||||
| Voltage | ? | 1.1 V | 1.2 V | 1.2 V | ||
| Bandwidth per Stack | 1 TB/s | 819.2 GB/s | 460.8 GB/s | 256 GB/s | ||
Assuming SK hynix's HBM3E development goes according to plan, the company should have little trouble lining up customers for even faster memory. Especially with demand for GPUs going through the roof for use in building AI training and inference systems, NVIDIA and other processor vendors are more than willing to pay premium for advanced memory they need to produce ever faster processors during this boom period in the industry.
SK Hynix will be producing HBM3E memory using its 1b nanometer fabrication technology (5th Generation 10nm-class node), which is currently being used to make DDR5-6400 memory chips that are set to be validated for Intel’s next generation Xeon Scalable platform. In addition, the manufacturing technology will be used to make LPDDR5T memory chips that will combine high performance with low power consumption.
Micron this week officially said that it would equip its fab in Hiroshima, Japan, to produce DRAM chips on its 1γ (1-gamma) process technology, its first node to use extreme ultraviolet lithography, in 2025. The company will be the first chipmaker to use EUV for volume production in Japan and its fabs in Hiroshima and Taiwan will be its first sites to use the upcoming 1γ technology.
As the only major DRAM maker that has not adopted extreme ultraviolet lithography, Micron planned to start using it with its 1γ process (its 3rd Generation 10nm-class node) in 2024. But due to PC market slump and its spending cuts, the company had to delay the plan to 2025. Micron's 1γ process technology is set to use EUV for several layers, though it does not disclose how many layers will use it.
What the company does say is that its 1γ node will enable the world's smallest memory cell, which is bold claim considering the fact that Micron cannot possibly know what its rivals are going to have in 2025.
Last year the 1-gamma technology was at the 'yield enablement' stage, which means that the company was testing samples of DRAMs extensive testing and quality control procedures. At this point, the company may implement innovative inspection to tools to identify defects and then introduce certain improvements to certain process steps (e.g., lithography, etching) to maximize yields.
“Micron’s Hiroshima operations have been central to the development and production of several industry-leading technologies for memory over the past decade,” Micron President and CEO Sanjay Mehrotra said. “We are proud to be the first to use EUV in Japan and to be developing and manufacturing 1-gamma at our Hiroshima fab.
To produce memory chips on its 1-gamma node at its Hiroshima fab, Micron needs to install ASML's Twinscan NXE scanners, which cost about $200 million per unit. To equip its fab with advanced tools, Micron secured ¥46.5 billion ($320 million) grant from the Japanese government last September. Meanwhile, Micron says it will invest ¥500 billion ($3.618 billion) in the technology 'over the next few years, with close support from the Japanese government.'
“Micron is the only company that manufactures DRAM in Japan and is critical to setting the pace for not only the global DRAM industry but our developing semiconductor ecosystem,” said Satoshi Nohara, METI Director-General of the Commerce and Information Policy Bureau. “We are pleased to see our collaboration with Micron take root in Hiroshima with state-of-the-art EUV to be introduced on Japanese soil. This will not only deepen and advance the talent and infrastructure of our semiconductor ecosystem, it will also unlock exponential growth and opportunity for our digital economy.”
Samsung on Thursday said it had started high volume production DRAM chips on its latest 12nm fabrication process. The new manufacturing node has allowed Samsung to reduce the power consumption of its DRAM devices, as well as decrease their costs significantly compared to its previous-generation node.
According to Samsung's announcement, the company's 12nm fabrication process is being used to produce 16Gbit DDR5 memory chips. And while the company is already producing DDR5 chips with that capacity (e.g. K4RAH086VB-BCQK), the switch to the newer and smaller 12nm process has paid off both in terms of power consumption and die size. As compared to DDR5 dies made on the company's previous-generation node (14nm), the new 12nm dies offer up to 23% lower power consumption, and Samsung is able to produce 20% more dies per wafer (i.e., the DDR5 dies are tangibly smaller).
Samsung says that the key innovation of its 12nm DRAM fabrication process is usage of new high-k material for DRAM cell capacitors that enabled it to increase cell's capacitance to boost performance, but without increasing their dimensions and die sizes. Higher DRAM cell capacitance means a DRAM cell can store more data and reduce power-draining refresh cycles, hence increasing performance. However, larger capacitors typically result in increased cell and die size, which makes the resulting dies more expensive.
DRAM makers have been addressing this by using high-k materials for years, but finding these materials is getting trickier with each new node as memory makers also have to take into account yields and production infrastructure they have. Apparently, Samsung has succeeded in doing so with its 12nm node, though it does not make any disclosures on the matter. That Samsung has succeeded in reducing their die size by a meaningful amount at all is quite remarkable, as analog components like capacitors were some of the first parts of chips to stop scaling down further with finer process nodes.
In addition to introducing a new high-k material, Samsung also reduced operating voltage and noise for its 12nm DDR5 ICs to offer a better balance of performance and power consumption compared to predecessors.
One of the aspects about Samsung's 12nm DRAM technology is that it looks to be the company's 3rd Generation production node for memory that uses extreme ultraviolet lithography. The first D1x node was purely designed as a proof of concept and its successor D1a, which has been in use since 2021, used EUV for five layers. Meanwhile, it is unclear to what degree Samsung's 12nm node is using EUV tools.
"Using differentiated process technology, Samsung’s industry-leading 12nm-class DDR5 DRAM delivers outstanding performance and power efficiency," said Jooyoung Lee, Executive Vice President of DRAM Product & Technology at Samsung Electronics.
Meanwhile, Samsung is also eyeing faster memory speeds with their new 12nm DDR5 dies. According to the company, these dies can run as fast as DDR5-7200 (i.e. 7.2Gbps/pin), which is well ahead of what the official JEDEC specification currently allows for. The voltage required isn't being stated, but if nothing else, it offers some promise for future XMP/EXPO memory kits.
Memory module producers have been shipping unbuffered DDR5 memory modules for desktop and laptop computers running Intel's 12th Generation Core 'Alder Lake' processors in high volumes since September, 2021, without any major issues. But DDR5 is just now entering the datacenter world, and according to a recent report, it looks like power management ICs (PMICs) for registered DIMMs have become a constraining factor due to compatibility issues.
In a report published by TrendForce discussing the current state of the market for server-grade DDR5 memory, the semiconductor analyst firm noted that there is an issue with PMIC compatibility for DDR5 RDIMMs, with both DRAM suppliers and PMIC vendors are collaborating to resolve the problem. The analysts do not reveal the exact root cause of the problem, but claim that PMICs from Monolithic Power Systems (MPS) do not have any issues, leading them to expect MPS PMICs to be in high demand for the foreseeable future.
Although DRAM makers have been distributing samples of their server grade modules to CPU and server makers since early 2022, practical issues only emerged recently when producers began to ramp up production of their machines running AMD's EPYC 9004 'Genoa' and Intel's 4th Generation Xeon Scalable 'Sapphire Rapids' processors. As a result, the demand for PMICs from a single supplier has created a bottleneck in production, claims TrendForce. This will have a knock-on effect on the server market, which is already suffering from a demand drop.
Neither analysts nor DRAM producers are currently disclosing the precise reason for the PMIC issue. But it is evident that, as both client and server DDR5 DIMMs require PMICs, it is turning out to be harder to make server-grade modules than client-aimed DIMMs.
As part of the changes that came with the DDR5 specification, DDR5 memory modules now come with their own voltage regulating modules (VRMs) and PMIC. Moving these components on to DIMMs is intended to minimize voltage fluctuation ranges (DDR5's allowable range is about 3% (±0.033V) for a 1.1 volt supply), as well as decrease power consumption and improve performance. But doing so adds complexity to individual DIMMs, as well.
Unbuffered DDR5 DIMMs for client PCs are relatively simple since they are all single or dual-rank and carry at most 16 single-die memory chips. High-capacity Registered DDR5 memory modules for servers use more chips and those chips can pack in multiple DRAM dies each, which greatly increases complexity.
As a result of the PMIC bottleneck as well as a slower ramping of DDR5 manufacturing capacity, TrendForce predicts that prices of server-grade 32GB DDR5 modules will drop to around $80 - $90 in April and May, due to the lower fulfillment rates of DDR5 server DRAM in the short term. As a result, DDR5 prices are expected to fall more slowly than DDR4 for the next couple of quarters, with DDR5 prices only finally catching up (or rather, down) with DDR4 once production picks up.
When SK hynix initially announced its HBM3 memory portfolio in late 2021, the company said it was developing both 8-Hi 16GB memory stacks as well as even more technically complex 12-Hi 24GB memory stacks. Now, almost 18 months after that initial announcement, SK hynix has finally begun sampling its 24GB HBM3 stacks to multiple customers, with an aim towards going into mass production and market availability in the second half of the year. All of which should be a very welcome development for SK hynix's downstream customers, many of whom are chomping at the bit for additional memory capacity to meet the needs of large language models and other high-end computing uses.
Based on the same technology as SK hynix's existing 16GB HBM3 memory modules, the 24GB stacks are designed to further improve on the density of the overall HBM3 memory module by increasing the number of DRAM layers from 8 to 12 – adding 50% more layers for 50% more capacity. This is something that's been in the HBM specification for quite some time, but it's proven difficult to pull off as it requires making the extremely thin DRAM dies in a stack even thinner in order to squeeze more in.
Standard HBM DRAM packages are typically 700 – 800 microns high (Samsung claims its 8-Hi and 12-Hi HBM2E are 720 microns high), and, ideally, that height needs to be maintained in order for these denser stacks to be physically compatible with existing product designs, and to a lesser extent to avoid towering over the processors they're paired with. As a result, to pack 12 memory devices into a standard KGSD, memory producers must either shrink the thickness of each DRAM layer without compromising performance or yield, reduce the space between layers, minimize the base layer, or introduce a combination of all three measures.
While SK hynix's latest press release offers limited details, the company has apparently gone for thinning out the DRAM dies and the space between them with an improved underfill material. For the DRAM dies themselves, SK hynix has previously stated that they've been able to shave their die thickness down to 30 microns. Meanwhile, the improved underflow material on their 12-Hi stacks is being provided via as part of the company's new Mass Reflow Molded Underfill (MR-MUF) packing technology. This technique involves bonding the DRAM dies together all at once via the reflow process, while simultaneously filling the gaps between the dies with the underfill material.
SK hynix calls their improved underfill material "liquid Epoxy Molding Compound", or "liquid EMC", which replaces the older non conductive film (NCF) used in older generations of HBM. Of particular interest here, besides the thinner layers this allows, according to SK hynix liquid EMC offers twice the thermal conductivity of NCF. Keeping the lower layers of stacked chips reasonably cool has been one of the biggest challenges with chip stacking technology of all varieties, so doubling the thermal conductivity of their fill material marks a significant improvement for SK hynix. It should go a long way towards making 12-Hi stacks more viable by better dissipating heat from the well-buried lowest-level dies.
Assembly aside, the performance specifications for SK hynix's 24GB HBM3 stacks are identical to their existing 16GB stacks. That means a maximum data transfer speed of 6.4Gbps/pin running over a 1024-bit interface, providing a total bandwidth of 819.2 GB/s per stack.
Ultimately, all the assembly difficulties with 12-Hi HBM3 stacks should be more than justified by the benefits that the additional memory capacity brings. SK hynix's major customers are already employing 6+ HBM3 stacks on a single product in order to deliver the total bandwidth and memory capacities they deem necessary. A 50% boost in memory capacity, in turn, will be a significant boon to products such as GPUs and other forms of AI accelerators, especially as this era of large language models has seen memory capacity become bottlenecking factor in model training. NVIDIA is already pushing the envelope on memory capacity with their H100 NVL – a specialized, 96GB H100 SKU that enables previously-reserved memory – so it's easy to see how they would be eager to be able to provide 120GB/144GB H100 parts using 24GB HBM3 stacks.
Source: SK Hynix
The demand for high bandwidth memory is set to explode in the coming quarters and years due to the broader adoption of artificial intelligence in general and generative AI in particular. SK Hynix will likely be the primary beneficiary of the HBM rally as it leads shipments of this type of memory, holding a 50% share in 2022, according to TrendForce.
Analysts from TrendForce believe that shipments of AI servers equipped with compute GPUs like Nvidia's A100 or H100 will increase by roughly 9% year-over-year in 2022. However, they do not elaborate on whether they mean unit shipments or dollar shipments. They now estimate that the rise of generative AI will catalyze demand for AI servers, and this market will grow by 15.4% in 2023 and continue growing at a compound annual growth rate of 12.2% through 2027.
The upsurge in AI server usage will also increase demand for all types of memory, including commodity DDR5 SDRAM, HBM2e as well as HBM3 for compute GPUs, and 3D NAND memory for high-performance and high-capacity storage devices.
TrendForce estimates that while general-purpose servers pack 500 GB – 600 GB of commodity memory, an AI server uses 1.2 TB – 1.7 TB. In addition, such machines use compute GPUs equipped with 80 GB or more of HBM2e/HBM3 memory. Since each AI machine comes with multiple compute GPUs, the total content of HBM per box is now 320 GB – 640 GB, and it is only set to grow further as accelerators like AMD's Instinct MI300 and Nvidia H100 NVL carry more HBM3 memory.
Speaking of HBM3 adoption, it is necessary to note that SK Hynix is currently the only maker that mass produces this type of memory, according to TrendForce. As a result, as demand for this type of memory grows, it will benefit the most. Last year SK Hynix commanded 50% of HBM shipments, followed by Samsung with 40% and Micron with 10%. This year the company will solidify its positions and control 53% of HBM shipments, whereas shares of Samsung and Micron will decline to 38% and 9%, respectively, TrendForce claims.
Nowadays, AI servers are used primarily by the leading U.S. cloud service providers, including AWS, Google, Meta, and Microsoft. As more companies launch their generative AI products, they will inevitably have to use AI servers either on-prem or at AWS or Microsoft. For example, Baidu and ByteDance plan to introduce generative AI products and services in the coming quarters.
Source: TrendForce
Following an ongoing slump in demand that has impacted the entire memory industry over the last several months, Samsung this week has become the latest memory foundry to announce production cuts for NAND and DRAM. The "meaningful" cuts planned by Samsung make it the latest memory fab – and last of the big three – to undertake cuts in light of a significantly weaker market, while also marking an end to Samsung's efforts hold off on production cuts amidst the current slump to try to gain market share.
Kingston Fury, the gaming and high-performance division of Kingston Technology Company, Inc., has expanded the aesthetics of the company's Fury DDR5 memory portfolio. The Fury Beast and Fury Renegade DDR5 memory lineups now arrive with a white heat spreader design. As a result, consumers of both AMD and Intel platforms can take advantage of the new memory kits when putting together a PC with a white theme.
The Fury Beast and Fury Renegade memory kits arrive in vanilla and RGB variants. In the case of the Fury Beast, the non-RGB version measures 34.9 mm, whereas the RGB version stands at 42.23 mm. The memory sticks to a single color, either black or white. On the other hand, the Fury Renegade is slightly taller at 39.2 mm. The RGB-illuminated trim is 44 mm in height. Unlike the Fury Beast, the Fury Renegade rocks a dual-tone exterior in either black and silver or the more recent white and silver combination.
Included within the RGB variations of the Fury Beast and Fury Renegade is Kingston's patented Infrared Sync technology, which, as the name implies, keeps the illumination on the memory module in sync. Kingston provides the company's proprietary Fury CTRL software for users to control the lighting, or they can use the included RGB software from their memory vendors.
Kingston Fury Renegade DDR5 memory with white heatsink
Kingston commercializes the Fury Beast and Fury Renegade as individual memory modules and dual-DIMM memory kits. Unfortunately, consumers that want a quad-DIMM memory kit are out of luck until next month. Kingston still uses standard 16 gigabit dies with the brand's DDR5 memory kits. As a result, the company cannot match other vendors who have hit 192 GB (4 x 48 GB) capacity with non-binary memory modules.
| Fury Beast Specifications | ||
| Frequency | Latency Timings | Capacities |
| DDR5-6000 | 40-40-40 (1.35 V) 36-38-38 (1.35 V) |
8 GB (1 x 8 GB) 16 GB (1 x 16 GB) 16 GB (2 x 8 GB) 32 GB (2 x 16 GB) 64 GB (2 x 32 GB) |
| DDR5-5600 | 40-40-40 (1.25 V) 36-38-38 (1.25 V) |
8 GB (1 x 8 GB) 16 GB (1 x 16 GB) 16 GB (2 x 8 GB) 32 GB (2 x 16 GB) 64 GB (2 x 32 GB) |
| DDR5-5200 | 40-40-40 (1.25 V) 36-40-40 (1.25 V) |
8 GB (1 x 8 GB) 16 GB (1 x 16 GB) 16 GB (2 x 8 GB) 32 GB (2 x 16 GB) 64 GB (2 x 32 GB) |
| DDR5-4800 | 38-38-38 (1.10 V) | 8 GB (1 x 8 GB) 16 GB (1 x 16 GB) 16 GB (2 x 8 GB) 32 GB (2 x 16 GB) 64 GB (2 x 32 GB) |
The Fury Beast portfolio caters to mainstream consumers and offers more varieties. The speeds span from 4,800 MT/s to 6,000 MT/s, with memory kit capacities starting at 16 GB. There are Intel XMP 3.0- and AMD EXPO memory kits. The DDR5-4800 memory kit has CL 38-38-38 timings and is plug-and-play friendly. The higher-grade memory kits come with either Intel XMP 3.0 or AMD EXPO support. The Intel version of the Fury Beast DDR5-6000 memory kit sports 40-40-40 timings and requires 1.35 volts. On the contrary, the AMD version possesses better memory timings (CL 36-38-38) while using the same voltage.
| Fury Renegade Specifications | ||
| Frequency | Latency Timings | Capacities |
| DDR5-7200 | 38-44-44 (1.45 V) | 16 GB (1 x 16 GB) 32 GB (2 x 16 GB) |
| DDR5-6800 | 36-42-42 (1.40 V) | 16 GB (1 x 16 GB) 32 GB (2 x 16 GB) |
| DDR5-6400 | 32-39-39 (1.40 V) | 16 GB (1 x 16 GB) 32 GB (2 x 16 GB) |
| DDR5-6000 | 32-38-38 (1.35 V) | 16 GB (1 x 16 GB) 32 GB (2 x 16 GB) 32 GB (1 x 32 GB) 64 GB (2 x 32 GB) |
The Fury Renegade series targets gamers and enthusiasts. The memory kits start where the Fury Beast left off. The slowest Fury Renegade memory kit clock in at 6,000 MT/s, and the fastest option maxes out at 7,200 MT/s. Kingston only sells the Fury Renegade in 32 GB and 64 GB kit capacities. All Fury Renegade memory kits are Intel XMP 3.0-certified. The DDR5-7200 memory kit, available only in 32 GB (2 x 16 GB), has the memory timings configured to CL 38-44-44 and pulls 1.45 volts.
In addition, Kingston backs its Fury Beast and Fury Renegade products with a limited lifetime warranty. The Fury Beast memory kits start at $69, $119, and $228 for the 16 GB, 32 GB, and 64 GB options, respectively. Meanwhile, the starting prices for the Fury Renegade 32 GB and 64 GB memory kits are $159 and $368, respectively.
Lateo.net - Flux RSS en pagaille (pour en ajouter : @ moi) 