Connexion
G.Skill on Tuesday introduced its ultra-low-latency DDR5-6400 memory modules that feature a CAS latency of 30 clocks, which appears to be the industry's most aggressive timings yet for DDR5-6400 sticks. The modules will be available for both AMD and Intel CPU-based systems.
With every new generation of DDR memory comes an increase in data transfer rates and an extension of relative latencies. While for the vast majority of applications, the increased bandwidth offsets the performance impact of higher timings, there are applications that favor low latencies. However, shrinking latencies is sometimes harder than increasing data transfer rates, which is why low-latency modules are rare.
Nonetheless, G.Skill has apparently managed to cherry-pick enough DDR5 memory chips and build appropriate printed circuit boards to produce DDR5-6400 modules with CL30 timings, which are substantially lower than the CL46 timings recommended by JEDEC for this speed bin. This means that while JEDEC-standard modules have an absolute latency of 14.375 ns, G.Skill's modules can boast a latency of just 9.375 ns – an approximately 35% decrease.
G.Skill's DDR5-6400 CL30 39-39-102 modules have a capacity of 16 GB and will be available in 32 GB dual-channel kits, though the company does not disclose voltages, which are likely considerably higher than those standardized by JEDEC.
The company plans to make its DDR5-6400 modules available both for AMD systems with EXPO profiles (Trident Z5 Neo RGB and Trident Z5 Royal Neo) and for Intel-powered PCs with XMP 3.0 profiles (Trident Z5 RGB and Trident Z5 Royal). For AMD AM5 systems that have a practical limitation of 6000 MT/s – 6400 MT/s for DDR5 memory (as this is roughly as fast as AMD's Infinity Fabric can operate at with a 1:1 ratio), the new modules will be particularly beneficial for AMD's Ryzen 7000 and Ryzen 9000-series processors.
G.Skill notes that since its modules are non-standard, they will not work with all systems but will operate on high-end motherboards with properly cooled CPUs.
The new ultra-low-latency memory kits will be available worldwide from G.Skill's partners starting in late August 2024. The company did not disclose the pricing of these modules, but since we are talking about premium products that boast unique specifications, they are likely to be priced accordingly.
Samsung is announcing today that it has begun mass production of 12 GB and 16 GB LPDDR5X modules in the industry's thinnest package. Samsung's shrunken memory packages measure approximately 0.65 mm in thickness, making them 0.06 mm (~9%) thinner than standard LPDDR5X packages. The company expects the new DRAM devices to be used to make for thinner smartphones, or improve their performance by enabling better airflow inside.
According to the company's press release, Samsung achieved this ultra-thin design by employing new packaging methods, such as optimized printed circuit boards (PCBs) and epoxy molding compound (EMC). Additionally, an optimized back-lapping process was used to further reduce the height of the packages. The newly developed DRAM packages are not only thinner by 9% compared to previous models but also offer a 21.2% improvement in heat resistance.
Thinner LPDDR5X packaging help enhance airflow within smartphones, significantly improving thermal management, which means higher performance and longer battery life. Also, better thermal management help to prolong device's lifespan.
"Samsung's LPDDR5X DRAM sets a new standard for high-performance on-device AI solutions, offering not only superior LPDDR performance but also advanced thermal management in an ultra-compact package," said YongCheol Bae, Executive Vice President of Memory Product Planning at Samsung Electronics. "We are committed to continuous innovation through close collaboration with our customers, delivering solutions that meet the future needs of the low-power DRAM market."
While Samsung's thinner LPDDR5X DRAM packages contribute to making smartphones slimmer, they are just one part of the overall design strategy. Other components, such as thinner protective glass, PCBs, and batteries, play considerably more significant roles in reducing device thickness. Meanwhile, the primary benefit of these new memory modules may be in improving airflow inside smartphones.
Samsung is looking to further expand its LPDDR5X product lineups by developing even more compact packages, including 6-layer 24 GB and 8-layer 32 GB modules. Specific details about the thickness of these future memory modules have not yet been disclosed, though making high-capacity DRAMs thinner in general is an important thing.
Following a relative lull in the desktop memory industry in the previous decade, the past few years have seen a flurry of new memory standards and form factors enter development. Joining the traditional DIMM/SO-DIMM form factors, we've seen the introduction of space-efficient DDR5 CAMM2s, their LPDDR5-based counterpart the LPCAMM2, and the high-clockspeed optimized CUDIMM. But JEDEC, the industry organization behind these efforts, is not done there. In a press release sent out at the start of the week, the group announced that it is working on standards for DDR5 Multiplexed Rank DIMMs (MRDIMM) for servers, as well as an updated LPCAMM standard to go with next-generation LPDDR6 memory.
Just last week Micron introduced the industry's first DDR5 MRDIMMs, which are timed to launch alongside Intel's Xeon 6 server platforms. But while Intel and its partners are moving full steam ahead on MRDIMMs, the MRDIMM specification has not been fully ratified by JEDEC itself. All told, it's not unusual to see Intel pushing the envelope here on new memory technologies (the company is big enough to bootstrap its own ecosystem). But as MRDIMMs are ultimately meant to be more than just a tool for Intel, a proper industry standard is still needed – even if that takes a bit longer.
Under the hood, MRDIMMs continue to use DDR5 components, form-factor, pinout, SPD, power management ICs (PMICs), and thermal sensors. The major change with the technology is the introduction of multiplexing, which combines multiple data signals over a single channel. The MRDIMM standard also adds RCD/DB logic in a bid to boost performance, increase capacity of memory modules up to 256 GB (for now), shrink latencies, and reduce power consumption of high-end memory subsystems. And, perhaps key to MRDIMM adoption, the standard is being implemented as a backwards-compatible extension to traditional DDR5 RDIMMs, meaning that MRDIMM-capable servers can use either RDIMMs or MRDIMMs, depending on how the operator opts to configure the system.
The MRDIMM standard aims to double the peak bandwidth to 12.8 Gbps, increasing pin speed and supporting more than two ranks. Additionally, a "Tall MRDIMM" form factor is in the works (and pictured above), which is designed to allow for higher capacity DIMMs by providing more area for laying down memory chips. Currently, ultra high capacity DIMMs require using expensive, multi-layer DRAM packages that use through-silicon vias (3DS packaging) to attach the individual DRAM dies; a Tall MRDIMM, on the other hand, can just use a larger number of commodity DRAM chips. Overall, the Tall MRDIMM form factor enables twice the number of DRAM single-die packages on the DIMM.
Meanwhile, this week's announcement from JEDEC offers the first significant insight into what to expect from LPDDR6 CAMMs. And despite LPDDR5 CAMMs having barely made it out the door, some significant shifts with LPDDR6 itself means that JEDEC will need to make some major changes to the CAMM standard to accommodate the newer memory type.
JEDEC Presentation: The CAMM2 Journey and Future Potential
Besides the higher memory clockspeeds allowed by LPDDR6 – JEDEC is targeting data transfer rates of 14.4 GT/s and higher – the new memory form-factor will also incorporate an altogether new connector array. This is to accommodate LPDDR6's wider memory bus, which sees the channel width of an individual memory chip grow from 16-bits wide to 24-bits wide. As a result, the current LPCAMM design, which is intended to match the PC standard of a cumulative 128-bit (16x8) design needs to be reconfigured to match LPDDR6's alterations.
Ultimately, JEDEC is targeting a 24-bit subhannel/48-bit channel design, which will result in a 192-bit wide LPCAMM. While the LPCAMM connector itself is set to grow from 14 rows of pins to possibly as high as 20. New memory technologies typically require new DIMMs to begin with, so it's important to clarify that this is not unexpected, but at the end of the day it means that the LPCAMM will be undergoing a bigger generational change than what we usually see.
JEDEC is not saying at this time when they expect either memory module standard to be completed. But with MRDIMMs already shipping for Intel systems – and similar AMD server parts due a bit later this year – the formal version of that standard should be right around the corner. Meanwhile, LPDDR6 CAMMs will be a bit farther out, particularly as the memory standard itself is still under development.
Samsung has successfully validated its new LPDDR5X-10700 memory with MediaTek's upcoming Dimensity platform. At present, 10.7 GT/s is the highest performing speed grade of LPDDR5X DRAM slated to be released this year, so the upcoming Dimensity 9400 system-on-chip will get the highest memory bandwidth available for a mobile application processor.
The verification process involved Samsung's 16 GB LPDDR5X package and MediaTek's soon-to-be-announced Dimensity 9400 SoC for high-end 5G smartphones. Usage of LPDDR5X-10700 provides a memory bandwidth of 85.6 GB/second over a 64-bit interface, which will be available for bandwidth-hungry applications like graphics and generative AI.
"Working together with Samsung Electronics has made it possible for MediaTek's next-generation Dimensity chipset to become the world's first to be validated at LPDDR5X operating speeds up to 10.7Gbps, enabling upcoming devices to deliver AI functionality and mobile performance at a level we have never seen before," said JC Hsu, Corporate Senior Vice President at MediaTek. "This updated architecture will make it easier for developers and users to leverage more AI capabilities and take advantage of more features with less impact on battery life."
Samsung's LPDDR5X 10.7 GT/s memory in made on the company's 12nm-class DRAM process technology and is said to provide a more than 25% improvement in power efficiency over previous-generation LPDDR5X, in addition to extra performance. This will positively affect improved user experience, including enhanced on-device AI capabilities, such as faster voice-to-text conversion, and better quality graphics.
Overall, the two companies completed this process in just three months. Though it remains to be seen when smartphones based on the Dimensity 9400 application processor and LPDDR5X memory are set to be available on the market, as MediaTek has not yet even formally announced the SoC itself.
"Through our strategic cooperation with MediaTek, Samsung has verified the industry's fastest LPDDR5X DRAM that is poised to lead the AI smartphone market," said YongCheol Bae, Executive Vice President of Memory Product Planning at Samsung Electronics. "Samsung will continue to innovate through active collaboration with customers and provide optimum solutions for the on-device AI era."
The compute market has always been hungry for memory bandwidth, particularly for high-performance applications in servers and datacenters. In recent years, the explosion in core counts per socket has further accentuated this need. Despite progress in DDR speeds, the available bandwidth per core has unfortunately not seen a corresponding scaling.
The stakeholders in the industry have been attempting to address this by building additional technology on top of existing widely-adopted memory standards. With DDR5, there are currently two technologies attempting to increase the peak bandwidth beyond the official speeds. In late 2022, SK hynix introduced MCR-DIMMs meant for operating with specific Intel server platforms. On the other hand, JEDEC - the standards-setting body - also developed specifications for MR-DIMMs with a similar approach. Both of them build upon existing DDR5 technologies by attempting to combine multiple ranks to improve peak bandwidth and latency.
The MR-DIMM standard is conceptually simple - there are multiple ranks of memory modules operating at standard DDR5 speeds with a data buffer in front. The buffer operates at 2x the speed on the host interface side, allowing for essentially double the transfer rates. The challenges obviously lie in being able to operate the logic in the host memory controller at the higher speed and keeping the power consumption / thermals in check.
The first version of the JEDEC MR-DIMM standard specifies speeds of 8800 MT/s, with the next generation at 12800 MT/s. JEDEC also has a clear roadmap for this technology, keeping it in sync with the the improvements in the DDR5 standard.
Micron and Intel have been working closely in the last few quarters to bring their former's first-generation MR-DIMM lineup to the market. Intel's Xeon 6 Family with P-Cores (Granite Rapids) is the first platform to bring MR-DIMM support at 8800 MT/s on the host side. Micron's standard-sized MR-DIMMs (suitable for 1U servers) and TFF (tall form-factor) MR-DIMMs (for 2U+ servers) have been qualified for use with the same.
The benefits offered by MR-DIMMs are evident from the JEDEC specifications, allowing for increased data rates and system bandwidth, with improvements in latency. On the capacity side, allowing for additional ranks on the modules has enabled Micron to offer a 256 GB capacity point. It must be noted that some vendors are also using TSV (through-silicon vias) technology to to increase the per-package capacity at standard DDR5 speeds, but this adds additional cost and complexity that are largely absent in the MR-DIMM manufacturing process.
The tall form-factor (TFF) MR-DIMMs have a larger surface area compared to the standard-sized ones. For the same airflow configuration, this allows the DIMM to have a better thermal profile. This provides benefits for energy efficiency as well by reducing the possibility of thermal throttling.
Micron is launching a comprehensive lineup of MR-DIMMs in both standard and tall form-factors today, with multiple DRAM densities and speed options as noted above.
Micron and Intel hosted a media / analyst briefing recently to demonstrate the benefits of MR-DIMMs for Xeon 6 with P-Cores (Granite Rapids). Using a 2P configuration with 96-core Xeon 6 processors, benchmarks for different workloads were processed with both 8800 MT/s MR-DIMMs and 6400 MT/s RDIMMs. The chosen workloads are particularly notorious for being limited in performance by memory bandwidth.
OpenFOAM is a widely-used CFD workload that benefits from MR-DIMMs. For the same memory capacity, the 8800 MT/s MR-DIMM shows a 1.31x speedup based on higher average bandwidth and IPC improvements, along with lower last-level cache miss latency.
The performance benefits are particularly evident with more cores participating the workload.
Apache Spark is a commonly used big-data platform operating on large datasets. Depending on the exact dataset in the picture, the performance benefits of MR-DIMMs can vary. Micron and Intel used a 2.4TB set from Intel's Hibench benchmark suite for this benchmark, showing a 1.2x speedup at the same capacity and 1.7x speedup with doubled-capacity TFF MR-DIMMs.
Avoiding the need to push data back to the permanent storage also contributes to the speedup.
The higher speed offered by MR-DIMMs also helps in AI inferencing workloads, with Micron and Intel showing a 1.31x inference performance improvement along with reduced time to first token for a Llama 3 8B parameter model. Obviously, purpose-built inferencing solutions based on accelerators will perform better. However, this was offered as a demonstration of the type of CPU workloads that can benefit from MR-DIMMs.
As the adage goes, there is no free lunch. At 8800 MT/s, MR-DIMMs are definitely going to guzzle more power compared to 6400 MT/s RDIMMs. However, the faster completion of workloads mean that the the energy consumption for a given workload will be lower for the MR-DIMM configurations. We would have liked Micron and Intel to quantify this aspect for the benchmarks presented in the demonstration. Additionally, Micron indicated that the energy efficiency (in terms of pico-joules per bit transferred) is largely similar for both the 6400 MT/s RDIMMs and 8800 MT/s MR-DIMMs.
The standardization of MR-DIMMs by JEDEC allows multiple industry stakeholders to participate in the market. Customers are not vendor-locked and can compare and contrast options from different vendors to choose the best fit for their needs.
At Computex, we saw MR-DIMMs from ADATA on display. As a Tier-2 vendor without its own DRAM fab, ADATA's play is on cost benefits with the possibility of the DRAM die being sourced from different fabs. The MR-DIMM board layout is dictated by JEDEC specifications, and this allows Tier-2 vendors to have their own play with pricing flexibility. Modules are also built based on customer orders. Micron, on the other hand, has a more comprehensive portfolio / lineup of SKUs for different use-cases with the pros and cons of vertical integration in the picture.
Micron is also not the first to publicly announce MR-DIMM sampling. Samsung announced their own lineup (based on 16Gb DRAM dies) last month. It must be noted that Micron's MR-DIMM portfolio uses 16 Gb, 24 Gb, and 32 Gb dies fabricated in 1β technology. While Samsung's process for the 16 Gb dies used in their MR-DIMMs is not known, Micron believes that their MR-DIMM technology will provide better power efficiency compared to the competition while also offering customers a wider range of capacities and configurations.
When Micron announced plans to build two new fabs in the U.S. in 2022, the company vaguely said both would come online by the decade's end. Then, in 2023, it began to optimize its spending, which pushed production at these fabrication facilities. This week, the company outlined more precise timeframes for when its fabs in Idaho and New York will start operations: this will happen from calendar 2026 to calendar 2029.
"These fab construction investments are necessary to support supply growth for the latter half of this decade," a statement by Micron in its Q3 FY2024 financial results report reads. "This Idaho fab will not contribute to meaningful bit supply until fiscal 2027 and the New York construction capex is not expected to contribute to bit supply growth until fiscal 2028 or later. The timing of future [wafer fab equipment] spend in these fabs will be managed to align supply growth with expected demand growth."
Micron's fiscal year 2027 starts in September 2026, so the new fab near Boise, Idaho, is set to start operations between September 2026 and September 2027. The company's fiscal 2028 starts in September 2027, so the fab will likely begin operations in calendar 2028 or later, probably depending on the demand for DRAM memory in the coming years. That said, Micron's U.S. memory fabs will begin operations between late 2026 and 2029, which aligns with the company's original plans.
Construction of the fab in Idaho is well underway. In contrast, construction of the New York facility has yet to begin as the company is working on regulatory and permitting processes in the state.
Micron's capital expenditure (CaPex) plan for FY2024 is approximately $8.0 billion, with a decrease in year-over-year spending on wafer fabrication equipment (WFE). In Q4 FY2024, the company will spend around $3 billion on fab construction, new wafer fab tools, and various expansions/upgrades.
Looking ahead to FY2025, the company plans a substantial increase in capex, targeting a mid-30s percentage of revenue to support various technological and facility advancements. In particular, it expects its quarterly CapEx to average above the $3 billion level seen in the fourth quarter of FY2024, which means that it plans to spend about $12 billion in its fiscal 2025, which begins in late September.
Half or more of the total CapEx increase in FY2025 (i.e., over $2 billion) will be allocated to constructing new fabs in Idaho and New York. Meanwhile, the FY2025 CapEx will significantly rise to fund high-bandwidth memory (HBM) assembly and testing and the construction of fabrication and back-end facilities. This increase also includes investments in technology transitions to meet growing demand.
"Fab construction in Idaho is underway, and we are working diligently to complete the regulatory and permitting processes in New York," said Sanjay Mehrotra, chief executive officer of Micron, at the company's conference call with investors and financial analysts (via SeekingAlpha). "This additional leading-edge greenfield capacity, along with continued technology transition investments in our Asia facilities, is required to meet long-term demand in the second half of this decade and beyond. These investments support our objective to maintain our current bit share over time and to grow our memory bit supply in line with long-term industry bit demand."
While the new CAMM and LPCAMM memory modules for laptops have garnered a great deal of attention in recent months, it's not just the mobile side of the PC memory industry that is looking at changes. The desktop memory market is also coming due for some upgrades to further improve DIMM performance, in the form of a new DIMM variety called the Clocked Unbuffered DIMM (CUDIMM). And while this memory isn't in use quite yet, several memory vendors had their initial CUDIMM products on display at this year's Computex trade show, offering a glimpse into the future of desktop memory.
A variation on traditional Unbuffered DIMMs (UDIMMs), Clocked UDIMMs (and Clocked SODIMMs) have been created as another solution to the ongoing signal integrity challenges presented by DDR5 memory. DDR5 allows for rather speedy transfer rates with removable (and easily installed) DIMMs, but further performance increases are running up against the laws of physics when it comes to the electrical challenges of supporting memory on a stick – particularly with so many capacity/performance combinations like we see today. And while those challenges aren't insurmountable, if DDR5 (and eventually, DDR6) are to keep increasing in speed, some changes appear to be needed to produce more electrically robust DIMMs, which is giving rise to the CUDIMM.
Standardized by JEDEC earlier this year as JESD323, CUDIMMs tweak the traditional unbuffered DIMM by adding a clock driver (CKD) to the DIMM itself, with the tiny IC responsible for regenerating the clock signal driving the actual memory chips. By generating a clean clock locally on the DIMM (rather than directly using the clock from the CPU, as is the case today), CUDIMMs are designed to offer improved stability and reliability at high memory speeds, combating the electrical issues that would otherwise cause reliability issues at faster memory speeds. In other words, adding a clock driver is the key to keeping DDR5 operating reliably at high clockspeeds.
All told, JEDEC is proposing that CUDIMMs be used for DDR5-6400 speeds and higher, with the first version of the specification covering speeds up to DDR5-7200. The new DIMMs will also be drop-in compatible with existing platforms (at least on paper), using the same 288-pin connector as today's standard DDR5 UDIMM and allowing for a relatively smooth transition towards higher DDR5 clockspeeds.
UPDATE 6/12: Micron notified us that it expects its HBM market share to rise to mid-20% in the middle of calendar 2025, not in the middle of fiscal 2025.
For Computex week, Micron was at the show in force in order to talk about its latest products across the memory spectrum. The biggest news for the memory company was that it has kicked-off sampling of it's next-gen GDDR7 memory, which is expected to start showing up in finished products later this year and was being demoed on the show floor. Meanwhile, the company is also eyeing taking a much larger piece of the other pillar of the high-performance memory market – High Bandwidth Memory – with aims of capturing around 25% of the premium HBM market.
Micron's first GDDR7 chip is a 16 Gb memory device with a 32 GT/sec (32Gbps/pin) transfer rate, which is significantly faster than contemporary GDDR6/GDDR6X. As outlined with JEDEC's announcement of GDDR7 earlier this year, the latest iteration of the high-performance memory technology is slated to improve on both memory bandwidth and capacity, with bandwidths starting at 32 GT/sec and potentially climbing another 50% higher to 48 GT/sec by the time the technology reaches its apex. And while the first chips are starting off at the same 2GByte (16Gbit) capacity as today's GDDR6(X) chips, the standard itself defines capacities as high as 64Gbit.
Of particular note, GDDR7 brings with it the switch to PAM3 (3-state) signal encoding, moving from the industry's long-held NRZ (2-state) signaling. As Micron was responsible for the bespoke GDDR6X technology, which was the first major DRAM spec to use PAM signaling (in its case, 4-state PAM4), Micron reckons they have a leg-up with GDDR7 development, as they're already familiar with working with PAM.
The GDDR7 transition also brings with it a change in how chips are organized, with the standard 32-bit wide chip now split up into four 8-bit sub-channels. And, like most other contemporary memory standards, GDDR7 is adding on-die ECC support to hold the line on chip reliability (though as always, we should note that on-die ECC isn't meant to be a replacement for full, multi-chip ECC). The standard also implements some other RAS features such as error checking and scrubbing, which although are not germane to gaming, will be a big deal for compute/AI use cases.
The added complexity of GDDR7 means that the pin count is once again increasing as well, with the new standard adding a further 86 pins to accommodate the data transfer and power delivery changes, bringing it to a total of 266 pins. With that said, the actual package size is remaining unchanged from GDDR5/GDDR6, maintaining that familiar 14mm x 12mm package. Memory manufacturers are instead using smaller diameter balls, as well as decreasing the pitch between the individual solder balls – going from GDDR6's 0.75mm x 0.75mm pitch to a slightly shorter 0.75mm x 0.73mm pitch. This allows the same package to fit in another 5 rows of contacts.
As for Micron's own production plans, the company is using its latest 1-beta (1β) fabrication process. While the major memory manufacturers don't readily publish the physical parameters of their processes these days, Micron believes that they have the edge on density with 1β, and consequently will be producing the densest GDDR7 at launch. And, while more nebulous, the company company believes that 1β will give them an edge in power efficiency as well.
Micron says that the first devices incorporating GDDR7 will be available this year. And while video card vendors remain a major consumer of GDDR memory, in 2024 the AI accelerator market should not be overlooked. With AI accelerators still bottlenecked by memory capacity and bandwidth, GDDR7 is expected to pair very well with inference accelerators, which need a more cost-effective option than HBM.
Speaking of HBM, Micron was the first company to formally announce its HBM3E memory last year, and it was among the first to start its volume shipments earlier this year. For now, Micron commands a 'mid-single digit' share of this lucrative market, but the company has said that it plans to rapidly gain share. If all goes well, by the middle of its calendar 2025 (i.e., the end of calendar Q1 2025) the company hopes to capture a mid-20% share of the HBM market.
"As we go into fiscal year 2025, we expect our share of HBM to be very similar to our overall share on general DRAM market," said Praveen Vaidyanathan, vice president and general manager of the Compute Products Group at Micron. "So, I would say mid-20%. […] We believe we have a very strong product as [we see] a lot of interest from various GPU and ASIC vendors, and continuing to engage with customers […] for the next, say 12 to 15 months."
When asked whether Micron can accelerate output of HBM3E at such a rapid pace in terms of manufacturing capacity, Vaidyanathan responded that the company has a roadmap for capacity expansion and that the company would meet the demand for its HBM3E products.
Ultra-high performance memory modules are a staple of of Computex, and it looks like this year G.Skill is showing off the highest performance dual-channel memory module kit to date. The company is demonstrating a DDR5 kit capable of 10,600 MT/s data transfer rate, which is a considerably higher speed compared to memory modules available today.
The dual-channel kit that G.Skill is demonstrating is a 32 GB Trident Z5 RGB kit that uses cherry-picked DDR5 memory devices and which can work in a DDR5-10600 mode with CL56 62-62-126 timings at voltages that are way higher than standard. The demoed DIMMs are running the whole day in a fairly warm room, though it does not really run demanding applications or stress tests.
Traditionally, memory module makers like G.Skill use Intel processors to demonstrate their highest-performing kits. But with the DDR5-10600 kit, the company uses AMD's Ryzen 5 8500G processor, which is a monolithic Zen 4-based APU with integrated graphics that's normally sold for budget systems. The motherboard is a high-end Asus ROG Crosshair X670E Gene and the APU is cooled down using a custom liquid cooling system The Asus ROG Crosshair X670E Gene motherboard has only two memory slots, which greatly helps to enable high data transfer rates, so it is a very good fit for the DDR5-10600 dual-channel kit.
Though I have sincere doubts that someone is going to use an ultra-expensive DDR5-10600 memory kit and related gate with this inexpensive processor, it is interesting (and unexpected) to see an AMD APU as a good fit to demonstrate performance potential of G.Skill's upcoming modules.
Speaking of availability of G.Skill's DDR5-10600 memory, it does not look like this kit is around the corner. The fastest DDR5 kit that G.Skill has today is its DDR5-8400 offering, so the DDR5-10600 will come to market a few speed bins later as G.Skill certainly needs to test the kit with various CPUs and ensure its stability.
One other thing to keep in mind is that both AMD and Intel are about to release new desktop processors this year, with the Ryzen 9000-series and Arrow Lake processors respectively. So G.Skill will undoubtedly focus on tuning its DDR5-10600 and other high-end kits primarily with those new CPUs.
As LPCAMM2 adoption begins, the first retail memory modules are finally starting to hit the retail market, courtesy of Micron. The memory manufacturer has begun selling their LPDDR5X-based LPCAMM2 memory modules under their in-house Crucial brand, making them available on the latter's storefront. Timed to coincide with the release of Lenovo's ThinkPad P1 Gen 7 laptop – the first retail laptop designed to use the memory modules – this marks the de facto start of the eagerly-awaited modular LPDDR5X memory era.
Micron's Low Power Compression Attached Memory Module 2 (LPCAMM2) modules are available in capacities of 32 GB and 64 GB. These are dual-channel modules that feature a 128-bit wide interface, and are based around LPDDR5X memory running at data rates up to 7500 MT/s. This gives a single LPCAMM2 a peak bandwidth of 120 GB/s. Micron is not disclosing the latencies of its LPCAMM2 memory modules, but it says that high data transfer rates of LPDDR5X compensate for the extended timings.
Micron says that LPDDR5X memory offers significantly lower power consumption, with active power per 64-bit bus being 43-58% lower than DDR5 at the same speed, and standby power up to 80% lower. Meanwhile, similar to DDR5 modules, LPCAMM2 modules include a power management IC and voltage regulating circuitry, which provides module manufacturers additional opportunities to reduce power consumption of their products.
Source: Micron LPDDR5X LPCAMM2 Technical Brief
It's worth noting, however, that at least for the first generation of LPCAMM2 modules, system vendors will need to pick between modularity and performance. While soldered-down LPDDR5X memory is available at speeds up to 8533 MT/sec – and with 9600 MT/sec on the horizon – the fastest LPCAMM2 modules planned for this year by both Micron and rival Samsung will be running at 7500 MT/sec. So vendors will have to choose between the flexibility of offering modular LPDDR5X, or the higher bandwidth (and space savings) offered by soldering down their memory.
Micron, for its part, is projecting that 9600 MT/sec LPCAMM2 modules will be available by 2026. Though it's all but certain that faster memory will also be avaialble in the same timeframe.
Micron's Crucial LPDDR5X 32 GB module costs $174.99, whereas a 64 GB module costs $329.99.
Demand for high-performance processors for AI training is skyrocketing, and consequently so is the demand for the components that go into these processors. So much so that SK hynix this week is very publicly announcing that the company's high-bandwidth memory (HBM) production capacity has already sold out for the rest of 2024, and even most of 2025 has already sold out as well.
SK hynix currently produces various types of HBM memory for customers like Amazon, AMD, Facebook, Google (Broadcom), Intel, Microsoft, and, of course, NVIDIA. The latter is an especially prolific consumer of HBM3 and HBM3E memory for its H100/H200/GH200 accelerators, as NVIDIA is also working to fill what remains an insatiable (and unmet) demand for its accelerators.
As a result, HBM memory orders, which are already placed months in advance, are now backlogging well into 2025 as chip vendors look to secure supplies of the memory stacks critical to their success.
This has made SK hynix the secnd HBM memory vendor in recent months to announce that they've sold out into 2025, following an earlier announcement from Micron regarding its HBM3E production. But of the two announcements, SK hynix's is arguably the most significant yet, as the South Korean firm's HBM production capacity is far greater than Micron's. So while things were merely "interesting" with the smallest of the Big Three memory manufacturers being sold out into 2025, things are taking a more concerning (and constrained) outlook now that SK hynix is as well.
SK hynix currently controls roughly 46% - 49% of HBM market, and its share is not expected to drop significantly in 2025, according to market tracking firm TrendForce. By contrast, Micron's share on HBM memory market is between 4% and 6%. Since HBM supply of both companies is sold out through the most of 2025, we're likely looking at a scenario where over 50% of the industry's total HBM3/HBM3E supply for the coming quarters is already sold out.
This leaves Samsung as the only member of the group not to comment on HBM demand so far. Though with memory being a highly fungible commodity product, it would be surprising if Samsung wasn't facing similar demand. And, ultimately, all of this is pointing towards the indusry entering an HBM3 memory shortage.
Separately, SK hynix said that it is sampling 12-Hi 36GB HBM3E stacks with customers and will begin volume shipments in the third quarter.
When JEDEC released its DDR5 specification (JESD79) back in 2020, the standard setting organization defined precise specs for modules with speed bins of up to 6400 MT/s, while leaving the spec open to further expansions with faster memory as technology progressed. Now, a bit more than three-and-a-half years later, and the standards body and its members are gearing up to release a faster generation of DDR5 memory, which is being laid out in the newly updated JESD79-JC5 specification. The latest iteration of the DDR5 spec defines official DDR timing specifications up to 8800 MT/s, as well as adding some new features when it comes to security.
Diving in, the new specification outlines settings for memory chips (on all types of memory modules) with data transfer rates up to 8800 MT/s (AKA DDR5-8800). This suggests that all members of the JESD79 committee that sets the specs for DDR5 — including memory chip makers and memory controller designers — agree that DDR5-8800 is a viable extension of the DDR5 specification both from performance and cost point of view. Meanwhile, the addition of higher speed bins is perhaps enabled by another JEDEC feature introduced in this latest specification, which is the Self-Refresh Exit Clock Sync for I/O training optimization.
| JEDEC DDR5-A Specifications | |||||||
| AnandTech | Data Rate MT/s |
CAS Latency (cycles) | Absolute Latency (ns) | Peak BW GB/s |
|||
| DDR5-3200 | A | 3200 | 22 | 22 | 22 | 13.75 | 25.6 |
| DDR5-3600 | A | 3600 | 26 | 26 | 26 | 14.44 | 28.8 |
| DDR5-4000 | A | 4000 | 28 | 28 | 28 | 14 | 32 |
| DDR5-4400 | A | 4400 | 32 | 32 | 32 | 14.55 | 35.2 |
| DDR5-4800 | A | 4800 | 34 | 34 | 34 | 14.17 | 38.4 |
| DDR5-5200 | A | 5200 | 38 | 38 | 38 | 14.62 | 41.6 |
| DDR5-5600 | A | 5600 | 40 | 40 | 40 | 14.29 | 44.8 |
| DDR5-6000 | A | 6000 | 42 | 42 | 42 | 14 | 48 |
| DDR5-6400 | A | 6400 | 46 | 46 | 46 | 14.38 | 51.2 |
| DDR5-6800 | A | 6800 | 48 | 48 | 48 | 14.12 | 54.4 |
| DDR5-7200 | A | 7200 | 52 | 52 | 52 | 14.44 | 57.6 |
| DDR5-7600 | A | 7600 | 54 | 54 | 54 | 14.21 | 60.8 |
| DDR5-8000 | A | 8000 | 56 | 56 | 56 | 14 | 64.0 |
| DDR5-8400 | A | 8400 | 60 | 60 | 60 | 14.29 | 67.2 |
| DDR5-8800 | A | 8800 | 62 | 62 | 62 | 14.09 | 70.4 |
When it comes to the JEDEC standard for DDR5-8800, it sets relatively loose timings of CL62 62-62 for A-grade devices and CL78 77-77 for lower-end C-grade ICs. Unfortunately, the laws of physics driving DRAM cells have not improved much over the last couple of years (or decades, for that matter), so memory chips still must operate with similar absolute latencies, driving up the relative CAS latency. In this case 14ns remains the gold standard, with CAS latencies at the new speeds being set to hold absolute latencies around that mark. But in exchange for systems willing to wait a bit longer (in terms of cycles) for a result, the new spec improves the standard's peak memory bandwidth by 37.5%.
This of course is just the timings set in the JEDEC specification, which is primarily of concern for server vendors. So we'll have to see just how much harder consumer memory manufacturers can push things for their XMP/EXPO-profiled memory. Extreme overclockers are already hitting speeds as high as 11,240 MT/s with current-generation DRAM chips and CPUs, so there may be some more headroom to play with in the next generation.
Meanwhile, on the security front, the updated spec makes a couple of changes that have been put in place seemingly to address rowhammer-style exploits. The big item here is Per-Row Activation Counting (PRAC), which true to its name, enables DDR5 to keep a count of how often a row has been activated. Using this information, memory controllers can then determine if a memory row has been excessively activated and is at risk of causing a neighboring row's bits to flip, at which point they can back off to let the neighboring row properly refresh and the data re-stabilize.
Notably here, the JEDEC press release doesn't use the rowhammer name at any point (unfortunately, we haven't been able to see the specification itself). But based on the description alone, this is clearly intended to thwart rowhammer attacks, since these normally operate by forcing a bit flip between refreshes through a large number of activations.
Digging a bit deeper, PRAC seems to be based on a recent Intel patent, Perfect Row Hammer Tracking with Multiple Count Increments (US20220121398A1), which describes a very similar mechanism under the name "Perfect row hammer tracking" (PRHT). Notably, the Intel paper calls out that this technique has a performance cost associated with it because it increases the overall row cycle time. Ultimately, as the vulnerability underpinning rowhammer is a matter of physics (cell density) rather than logic, it's not too surprising to see that any mitigation of it comes with a cost.
The updated DDR5 specification also deprecates support for Partial Array Self Refresh (PASR) within the standard, citing security concerns. PASR is primarily aimed at power efficiency for mobile memory to begin with, and as a refresh-related technology, presumably overlaps some with rowhammer – be it a means to attack memory, or an obstruction to defending against rowhammer. Either way, with mobile devices increasingly moving to low-power optimized LPDDR technologies anyhow, the depreciation of PASR does not immediately look like a major concern for consumer devices.
SK hynix and TSMC announced early on Friday that they had signed a memorandum of understanding to collaborate on developing the next-generation HBM4 memory and advanced packaging technology. The initiative is designed to speed up the adoption of HBM4 memory and solidify SK hynix's and TSMC's leading positions in high-bandwidth memory and advanced processor applications.
The primary focus of SK hynix's and TSMC's initial efforts will be to enhance the performance of the HBM4 stack's base die, which (if we put it very simply) acts like an ultra-wide interface between memory devices and host processors. With HBM4, SK hynix plans to use one of TSMC's advanced logic process technologies to build base dies to pack additional features and I/O pins within the confines of existing spatial constraints.
This collaborative approach also enables SK hynix to customize HBM solutions to satisfy diverse customer performance and energy efficiency requirements. SK hynix has been touting custom HBM solutions for a while, and teaming up with TSMC will undoubtedly help with this.
"TSMC and SK hynix have already established a strong partnership over the years. We've worked together in integrating the most advanced logic and state-of-the art HBM in providing the world's leading AI solutions," said Dr. Kevin Zhang, Senior Vice President of TSMC's Business Development and Overseas Operations Office, and Deputy Co-Chief Operating Officer. "Looking ahead to the next-generation HBM4, we're confident that we will continue to work closely in delivering the best-integrated solutions to unlock new AI innovations for our common customers."
Furthermore, the collaboration extends to optimizing the integration of SK hynix's HBM with TSMC's CoWoS advanced packaging technology. CoWoS is among the most popular specialized 2.5D packaging process technologies for integrating logic chips and stacked HBM into a unified module.
For now, it is expected that HBM4 memory will be integrated with logic processors using direct bonding. However, some of TSMC's customers might prefer to use an ultra-advanced version of CoWoS to integrate HBM4 with their processors.
"We expect a strong partnership with TSMC to help accelerate our efforts for open collaboration with our customers and develop the industry's best-performing HBM4," said Justin Kim, President and the Head of AI Infra at SK hynix. "With this cooperation in place, we will strengthen our market leadership as the total AI memory provider further by beefing up competitiveness in the space of the custom memory platform."
Corsair has introduced a family of registered memory modules with ECC that are designed for AMD's Ryzen Threadripper 7000 and Intel's Xeon W-2400/3400-series processors. The new Corsair WS DDR5 RDIMMs with AMD EXPO and Intel XMP 3.0 profiles will be available in kits of up to 256 GB capacity and at speeds of up to 6400 MT/s.
Corsair's family of WS DDR5 RDIMMs includes 16 GB modules operating at up to 6400 MT/s with CL32 latency as well as 32 GB modules functioning at 5600 MT/s with CL40 latency. At present, Corsair offers a quad-channel 64 GB kit (4×16GB, up to 6400 MT/s), a quad-channel 128GB kit (4×32GB, 5600 MT/s), an eight-channel 128 GB kit (8×16GB, 5600 MT/s), and an eight-channel 256 GB kit (8×32GB, 5600 MT/s) and it remains to be seen whether the company will expand the lineup.
Corsair's WS DDR5 RDIMMs are designed for AMD's TRX50 and WRX90 platforms as well as Intel's W790 platform and are therefore compatible with AMD's Ryzen Threadripper Pro 7000 and 7000WX-series as well as Intel's Xeon W-2400/3400-series CPUs. The modules feature both AMD EXPO and Intel XMP 3.0 profiles to easily set their beyond-JEDEC-spec settings and come with thin heat spreaders made of pyrolytic graphite sheet (PGS), which thermal conductivity than that of copper and aluminum of the same thickness. For now, Corsair does not disclose which RCD and memory chips its registered memory modules use.
Unlike many of its rivals among leading DIMM manufacturers, Corsair did not introduce its enthusiast-grade RDIMMs when AMD and Intel released their Ryzen Threadripper and Xeon W-series platforms for extreme workstations last year. It is hard to tell what the reason for that is, but perhaps the company wanted to gain experience working with modules featuring registered clock drivers (RCDs) as well as AMD's and Intel's platforms for extreme workstations.
The result of the delay looks to be quite rewarding: unlike modules from its competitors that either feature AMD EXPO or Intel XMP 3.0 profiles, Corsair's WS DDR5 RDIMMs come with both. While this may not be important on the DIY market where people know exactly what they are buying for their platform, this is a great feature for system integrators, which can use Corsair WS DDR5 RDIMMs both for their AMD Ryzen Threadripper and Intel Xeon W-series builds, something that greatly simplifies their inventory management.
Since Corsair's WS DDR5 RDIMMs are aimed at workstations and are tested to offer reliable performance beyond JEDEC specifications, they are quite expensive. The cheapest 64 GB DDR5-5600 CL40 kit costs $450, the fastest 64 GB DDR5-6400 CL32 kit is priced at $460, whereas the highest end 256 GB DDR5-5600 CL40 kit is priced at $1,290.
Following the magnitude 7.2 earthquake that struck Taiwan on April 3, 2024, there was immediate concern over what impact this could have on chip production within the country. Even for a well-prepared country like Taiwan, the tremor was the strongest quake to hit the region in 25 years, making it no small matter. But, according to research compiled by TrendForce, the impact on the production of DRAM will not be significant. The market tracking company believes that Taiwanese DRAM industry has remained largely unaffected, primarily due to their robust earthquake preparedness measures.
There are four memory makers in Taiwan: Micron, the sole member of the "big three" memory manufacturers on the island, runs two fabs. Meanwhile among the smaller players is Nanya (which has one fab), Winbond (which makes specialty memory at one fab), and PSMC (which produces specialty memory at one plant). The study found that these DRAM producers quickly resumed full operations, but had to throw away some wafers. The earthquake is estimated to have a minor effect on Q2 DRAM production, with a negligible 1% impact, TrendForce claims
In fact, as Micron is ramping up production of DRAM on its 1alpha and 1beta nm process technologies, it increases bit production of memory, which will positively affect supply of commodity DRAM in Q2 2025.
Following the earthquake, there was a temporary halt in quotations for both the contract and spot DRAM markets. However, the spot market quotations have already largely resumed, while contract prices have not fully restarted. Notably, Micron and Samsung ceased issuing quotes for mobile DRAM immediately after the earthquake, with no updates provided as of April 8th. In contrast, SK hynix resumed quotations for smartphone customers on the day of the earthquake and proposed more moderate price adjustments for Q2 mobile DRAM.
TrendForce anticipates a seasonal contract price increase for Q2 mobile DRAM of between 3% and 8%. This moderate increase is partly due to SK hynix's more restrained pricing strategy, which is likely to influence overall pricing strategies across the industry. The earthquake's impact on server DRAM primarily affected Micron's advanced fabrication nodes, potentially leading to a rise in final sale prices for Micron's server DRAM, according to TrendForce. However, the exact direction of future prices remains to be seen.
Meanwhile, DRAM fabs outside of Taiwan have none been directly affected by the quake. This includes Micron's HBM production line in Hiroshima, Japan, and Samsung's and SK hynix's HBM lines in South Korea, all of which are apparently operating with business as usual.
In general, the DRAM industry has shown resilience in the face of the earthquake, with minimal disruptions and a quick recovery. The abundant inventory levels for DDR4 and DDR5, coupled with weak demand, suggest that any slight price elevations caused by the earthquake are expected to normalize quickly. The only potential outlier here is DDR3, which is nearing the end of its commercial lifetime and production is already decreasing.
Composable disaggregated data center infrastructure promises to change the way data centers for modern workloads are built. However, to fully realize the potential of new technologies, such as CXL, the industry needs brand-new hardware. Recently, Samsung introduced its CXL Memory Module Box (CMM-B), a device that can house up to eight CXL Memory Module – DRAM (CMM-D) devices and add plenty of memory connected using a PCIe/CXL interface.
Samsung's CXL Memory Module Box (CMM-B) is the first device of this type to accommodate up to eight 2 TB E3.S CMM-D memory modules and add up to 16 TB of memory to up to three modern servers with appropriate connectors. As far as performance is concerned, the box can offer up to 60 GB/s of bandwidth (which aligns with what a PCIe 5.0 x16 interface offers) and 596 ns latency.
From a pure performance point of view, one CXL Memory Module—Box is slower than a dual-channel DDR5-4800 memory subsystem. Yet, the unit is still considerably faster than even advanced SSDs. At the same time, it provides very decent capacity, which is often just what the doctor ordered for many applications.
The Samsung CMM-B is compatible with the CXL 1.1 and CXL 2.0 protocols. It consists of a rack-scale memory bank (CMM-B), several application hosts, Samsung Cognos management console software, and a top-of-rack (ToR) switch. The device was developed in close collaboration with Supermicro, so expect this server maker to offer the product first.
Samsung's CXL Memory Module – Box is designed for applications that need a lot of memory, such as AI, data analytics, and in-memory databases, albeit not at all times. CMM-B allows the dynamic allocation of necessary memory to a system when it needs this memory and then uses DRAM with other machines. As a result, operators of datacenters can spend money on procuring expensive memory (16 TB of memory costs a lot), reduce power consumption, and add flexibility to their setups.
Offering some rare insight into the scale of HBM memory sales – and on its growth in the face of unprecedented demand from AI accelerator vendors – the company recently disclosed that it expects HBM sales to make up "a double-digit percentage of its DRAM chip sales" this year. Which if it comes to pass, would represent a significant jump in sales for the high-bandwidth, high-priced memory.
As first reported by Reuters, SK hynix CEO Kwak Noh-Jung has commented that he expects HBM sales will constitute a double-digit percentage of its DRAM chip sales in 2024. This prediction corroborate with estimates from TrendForce, who believe that, industry-wide, HBM will account for 20.1% of DRAM revenue in 2024, more than doubling HBM's 8.4% revenue share in 2023.
And while SK hynix does not break down its DRAM revenue by memory type on a regular basis, a bit of extrapolation indicates that they're on track to take in billions in HBM revenue for 2024 – having likely already crossed the billion dollar mark itself in 2023. Last year, SK hynix's DRAM revenue $15.941 billion, according to Statista and TrendForce. So SK hynix only needs 12.5% of its 2024 revenues to come from HBM (assuming flat or positive revenue overall) in order to pass 2 billion in HBM sales. And even this is a low-ball estimate.
Overall, SK hynix currently commands about 50% of HBM market, having largely split the market with Samsung over the last couple of years. Given that share, and that DRAM industry revenue is expected to increase to $84.150 billion in 2024, SK hynix could earn as much as $8.45 billion on HBM in 2024 if TrendForce's estimates prove accurate.
It should be noted that with demand for AI servers at record levels, all three leading makers of DRAM are poised to increase their HBM production capacity this year. Most notable here is a nearly-absent Micron, who was the first vendor to start shipping HBM3E memory to NVIDIA earlier this year. So SK hynix's near-majority of the HBM market may falter some this year, though with a growing pie they'll have little reason to complain. Ultimately, if sales of HBM reach $16.9 billion as projected, then all memory makers will be enjoying significant HBM revenue growth in the coming months.
Sources: Reuters, TrendForce
Now that JEDEC has published specification of GDDR7 memory, memory manufacturers are beginning to announce their initial products. The first out of the gate for this generation is Samsung, which has has quietly added its GDDR7 products to its official product catalog.
For now, Samsung lists two GDDR7 devices on its website: 16 Gbit chips rated for an up to 28 GT/s data transfer rate and a faster version running at up to 32 GT/s data transfer rate (which is in line with initial parts that Samsung announced in mid-2023). The chips feature a 512M x32 organization and come in a 266-pin FBGA packaging. The chips are already sampling, so Samsung's customers – GPU vendors, AI inference vendors, network product vendors, and the like – should already have GDDR7 chips in their labs.
The GDDR7 specification promises the maximum per-chip capacity of 64 Gbit (8 GB) and data transfer rates of 48 GT/s. Meanwhile, first generation GDDR7 chips (as announced so far) will feature a rather moderate capacity of 16 Gbit (2 GB) and a data transfer rate of up to 32 GT/s.
Performance-wise, the first generation of GDDR7 should provide a significant improvement in memory bandwidth over GDDR6 and GDDR6X. However capacity/density improvements will not come until memory manufacturers move to their next generation EUV-based process nodes. As a result, the first GDDR7-based graphics cards are unlikely to sport any memory capacity improvements. Though looking a bit farther down the road, Samsung and SK Hynix have previously told Tom's Hardware that they intend to reach mass production of 24 Gbit GDDR7 chips in 2025.
Otherwise, it is noteworthy that SK Hynix also demonstrated its GDDR7 chips at NVIDIA's GTC last week. So Samsung's competition should be close behind in delivering samples, and eventually mass production memory.
Source: Samsung (via @harukaze5719)
SK hynix is considering whether to build an advanced packaging facility in Indiana, reports the Wall Street Journal. If the company proceeds with the plan, it intends to invest $4 billion in it and construct one of the world's largest advanced packaging facilities. But to accomplish the project, SK hynix expects it will need help from the U.S. government.
Acknowledging the report but stopping short of confirming the company's plans, a company spokeswoman told the WSJ that SK hynix "is reviewing its advanced chip packaging investment in the U.S., but hasn’t made a final decision yet."
Companies like TSMC and Intel spend billions on advanced packaging facilities, but so far, no company has announced a chip packaging plant worth quite as much as SH hynix's $4 billion. The field of advanced packaging – CoWoS, passive silicon interposers, redistribution layers, die-to-die bonding, and other cutting edge technologies – has seen an explosion in demand in the last half-decade. As bandwidth advances with traditional organic packaging are largely played out, chip designers have needed to turn to more complex (and difficult to assemble) technologies in order to wire up an ever larger number of signals at ever-higher transfer rates. Which has turned advanced packaging into a bottleneck for high-end chip and accelerator production, driving a need for additional packaging facilities.
If SK hynix approves the project, the advanced packaging facility is expected to begin operations in 2028 and could create as many as 1,000 jobs. With an estimated cost of $4 billion, the plant is poised to become one of the largest advanced packaging facilities in the world.
Meanwhile, government backing is thought to be essential for investments of this scale, with potential state and federal tax incentives, according to the report. These incentives form part of a broader initiative to bolster the U.S. semiconductor industry and decrease dependence on memory produced in South Korea.
SK hynix is the world's leading producer of HBM memory, and is one of the key HBM suppliers to NVIDIA. Next generations of HBM memory (including HBM4 and HBM4E) will require even closer collaboration between chip designers, chipmakers, and memory makers. Therefore, packaging HBM in America could be a significant benefit for NVIDIA, AMD, and other U.S. chipmakers.
Investing in the Indiana facility will be a strategic move by SK hynix to enhance its advanced chip packaging capabilities in general and demonstrating dedication to the U.S. semiconductor industry.
When a major industry slowdown occurs, big companies tend to slowdown their mid-term and long-term capacity related investments. This is exactly what happened to SK hynix's Yongin Semiconductor Cluster, a major project announced in April 2021 and valued at $106 billion. While development of the site has been largely completed, only 35% of the initial shell building has been constructed, according to the Korean Ministry of Trade, Industry, and Energy.
"Approximately 35% of Fab 1 has been completed so far and site renovation is in smooth progress," a statement by the Korean Ministry of Trade, Industry, and Energy reads. "By 2046, over KRW 120 trillion ($90 billion today, $106 billion in 2021) in investment will be poured to complete Fabs 1 through 4, and construction of Fab 1's production line will commence in March next year. Once completed, the infrastructure will rank as the world's largest three-story fab."
The new semiconductor fabrication cluster by SK hynix announced almost exactly three years ago is primarily meant to be used to make DRAM for PCs, mobile devices, and servers using advanced extreme ultraviolet lithography (EUV) process technologies. The cluster, located near Yongin, South Korea, is intended to consist of four large fabs situated on a 4.15 million m2 site. With a planned capacity of approximately 800,000 wafer starts per month (WSPMs), it is set to be one of the world's largest semiconductor production hubs.
With that said, SK hynix's construction progress has been slower than the company first projected. The first fab in the complex was originally meant to come online in 2025, with construction starting in the fourth quarter of 2021. However, SK hynix began to cut its capital expenditures in the second half of 2022, and the Yongin Semiconductor Cluster project fell a victim of that cut. To be sure, the site continues to be developed, just at a slower pace; which is why some 35% of the first fab shell has been built at this point.
If completed as planned in 2021, the first phase of SK hynix Yongin operations would have been a major memory production facility costing $25 billion, equipped with EUV tools, and capable of 200,000-WSPM, according to reports from 2021.
Sources: Korean Ministry of Trade, Industry, and Energy; ComputerBase
Micron this week announced that it had begun sampling of its 256 GB multiplexer combined (MCR) DIMMs, the company's highest-capacity memory modules to date. These brand-new DDR5-based MCRDIMMs are aimed at next-generation servers, particularly those powered by Intel's Xeon Scalable 'Granite Rapids' processors that are set to support 12 or 24 memory slots per socket. Usage of these modules can enable datacenter machines with 3 TB or 6 TB of memory, with the combined ranks allowing for effect data rates of DDR5-8800.
"We also started sampling our 256 GB MCRDIMM module, which further enhances performance and increases DRAM content per server," said Sanjay Mehrotra, chief executive of Micron, in prepared remarks for the company's earnings call this week.
In addition to announcing sampling of these modules, Micron also demonstrated them at NVIDIA's GTC conference, where server vendors and customers alike are abuzz at building new servers for the next generation of AI accelerators. Our colleagues from Tom's Hardware have managed to grab a couple of pictures of Micron's 256 GB DDR5-8800 MCR DIMMs.
Image Credit: Tom's Hardware
Apparently, Micron's 256 GB DDR5-8800 MCRDIMMs come in two variants: a taller module with 80 DRAM chips distributed on both sides, and a standard-height module using 2Hi stacked packages. Both are based on monolithic 32 Gb DDR5 ICs and are engineered to cater to different server configurations with the standard-height MCRDIMM adressing 1U servers.The taller version consumes about 20W of power, which is in line with expectations as a 128 GB DDR5-8000 RDIMM consumes around 10W in DDR5-4800 mode. I have no idea about power consumption of the version that uses 2Hi packages, though expect it to be a little bit hotter and harder to cool down.
Image Credit: Tom's Hardware
Multiplexer Combined Ranks (MCR) DIMMs are dual-rank memory modules featuring a specialized buffer that allows both ranks to operate simultaneously. This buffer enables the two physical ranks to operate as though they were separate modules working in parallel, which allows for concurrent retrieval of 128 bytes of data from both ranks per clock cycle (compared to 64 bytes per cycle when it comes to regular memory modules), effectively doubling performance of a single module. Of course, since the modules retains physical interface of standard DDR5 modules (i.e., 72-bits), the buffer works with host at a very high data transfer rate to transfer that fetched data to the host CPU. These speeds exceed the standard DDR5 specifications, reaching 8800 MT/s in this case.
While MCR DIMMs make memory modules slightly more complex than regular RDIMMs, they increase performance and capacity of memory subsystem without increasing the number of memory modules involved, which makes it easier to build server motherboards. These modules are poised to play a crucial role in enabling the next generation of servers to handle increasingly demanding applications, particularly in the AI field.
Sources: Tom's Hardware, Micron
Being the first company to ship HBM3E memory has its perks for Micron, as the company has revealed that is has managed to sell out the entire supply of its advanced high-bandwidth memory for 2024, while most of their 2025 production has been allocated, as well. Micron's HBM3E memory (or how Micron alternatively calls it, HBM3 Gen2) was one of the first to be qualified for NVIDIA's updated H200/GH200 accelerators, so it looks like the DRAM maker will be a key supplier to the green company.
"Our HBM is sold out for calendar 2024, and the overwhelming majority of our 2025 supply has already been allocated," said Sanjay Mehrotra, chief executive of Micron, in prepared remarks for the company's earnings call this week. "We continue to expect HBM bit share equivalent to our overall DRAM bit share sometime in calendar 2025."
Micron's first HBM3E product is an 8-Hi 24 GB stack with a 1024-bit interface, 9.2 GT/s data transfer rate, and a total bandwidth of 1.2 TB/s. NVIDIA's H200 accelerator for artificial intelligence and high-performance computing will use six of these cubes, providing a total of 141 GB of accessible high-bandwidth memory.
"We are on track to generate several hundred million dollars of revenue from HBM in fiscal 2024 and expect HBM revenues to be accretive to our DRAM and overall gross margins starting in the fiscal third quarter," said Mehrotra.
The company has also began sampling its 12-Hi 36 GB stacks that offer a 50% more capacity. These KGSDs will ramp in 2025 and will be used for next generations of AI products. Meanwhile, it does not look like NVIDIA's B100 and B200 are going to use 36 GB HBM3E stacks, at least initially.
Demand for artificial intelligence servers set records last year, and it looks like it is going to remain high this year as well. Some analysts believe that NVIDIA's A100 and H100 processors (as well as their various derivatives) commanded as much as 80% of the entire AI processor market in 2023. And while this year NVIDIA will face tougher competition from AMD, AWS, D-Matrix, Intel, Tenstorrent, and other companies on the inference front, it looks like NVIDIA's H200 will still be the processor of choice for AI training, especially for big players like Meta and Microsoft, who already run fleets consisting of hundreds of thousands of NVIDIA accelerators. With that in mind, being a primary supplier of HBM3E for NVIDIA's H200 is a big deal for Micron as it enables it to finally capture a sizeable chunk of the HBM market, which is currently dominated by SK Hynix and Samsung, and where Micron controlled only about 10% as of last year.
Meanwhile, since every DRAM device inside an HBM stack has a wide interface, it is physically bigger than regular DDR4 or DDR5 ICs. As a result, the ramp of HBM3E memory will affect bit supply of commodity DRAMs from Micron, the company said.
"The ramp of HBM production will constrain supply growth in non-HBM products," Mehrotra said. "Industrywide, HBM3E consumes approximately three times the wafer supply as DDR5 to produce a given number of bits in the same technology node."
SK Hynix and AMD were at the forefront of the memory industry with the first generation of high bandwidth memory (HBM) back in 2013 – 2015, and SK Hynix is still leading this market in terms of share. In a bid to maintain and grow its position, SK Hynix has to adapt to the requirements of its customers, particularly in the AI space, and to do so it's mulling over how to make 'differentiated' HBM products for large customers.
"Developing customer-specific AI memory requires a new approach as the flexibility and scalability of the technology becomes critical," said Hoyoung Son, the head of Advanced Package Development at SK Hynix in the status of a vice president
When it comes to performance, HBM memory with a 1024-bit interface has been evolving fairly fast: it started with a data transfer rate of 1 GT/s in 2014 – 2015 and reached upwards of 9.2 GT/s – 10 GT/s with the recently introduced HBM3E memory devices. With HBM4, the memory is set to transit to a 2048-bit interface, which will ensure steady bandwidth improvement over HBM3E.
But there are customers which may benefit from differentiated (or semi-custom) HBM-based solutions, according to the vice president.
"For implementing diverse AI, the characteristics of AI memory also need to become more varied," Hoyoung Son said in an interview with BusinessKorea. "Our goal is to have a variety of advanced packaging technologies capable of responding to these changes. We plan to provide differentiated solutions that can meet any customer needs."
With a 2048-bit interface, many (if not the vast majority) of HBM4 solutions will likely be custom or at least semi-custom based on what we know from official and unofficial information about the upcoming standard. Some customers might want to keep using interposers (but this time they are going to get very expensive) and others will prefer to install HBM4 modules directly on logic dies using direct bonding techniques, which are also expensive.
Making differentiated HBM offerings requires sophisticated packaging techniques, including (but certainly not limited to) SK Hynix's Advanced Mass Reflow Molded Underfill (MR-RUF) technology. Given the company's vast experience with HBM, it may well come up with something else, especially for differentiated offerings.
"For different types of AI to be realized, the characteristics of AI memory also need to be more diverse," the VP said. "Our goal is to have a range of advanced packaging technologies to respond to the shifting technological landscape. Looking ahead, we plan to provide differentiated solutions to meet all customer needs."
Sources: BusinessKorea, SK Hynix
Samsung announced late on Monday the completion of the development of its 12-Hi 36 GB HBM3E memory stacks, just hours after Micron said it had kicked off mass production of its 8-Hi 24 GB HBM3E memory products. The new memory packages, codenamed Shinebolt, increase peak bandwidth and capacity compared to their predecessors, codenamed Icebolt, by over 50% and are currently the world's fastest memory devices.
As the description suggests, Samsung's Shinebolt 12-Hi 36 GB HBM3E stacks pack 12 24Gb memory devices on top of a logic die featuring a 1024-bit interface. The new 36 GB HBM3E memory modules feature a data transfer rate of 10 GT/s and thus offer a peak bandwidth of 1.28 TB/s per stack, the industry's highest per-device (or rather per-module) memory bandwidth.
Meanwhile, keep in mind that developers of HBM-supporting processors tend to be cautious, so they will use Samsung's HBM3E at much lower data transfer rates to some degree because of power consumption and to some degree to ensure ultimate stability for artificial intelligence (AI) and high-performance computing (HPC) applications.
| Samsung HBM Memory Generations | ||||||
| HBM3E (Shinebolt) |
HBM3 (Icebolt) |
HBM2E (Flashbolt) |
HBM2 (Aquabolt) |
|||
| Max Capacity | 36GB | 24 GB | 16 GB | 8 GB | ||
| Max Bandwidth Per Pin | 9.8 Gb/s | 6.4 Gb/s | 3.6 Gb/s | 2.0 Gb/s | ||
| Number of DRAM ICs per Stack | 12 | 12 | 8 | 8 | ||
| Effective Bus Width | 1024-bit | |||||
| Voltage | ? | 1.1 V | 1.2 V | 1.2 V | ||
| Bandwidth per Stack | 1.225 TB/s | 819.2 GB/s | 460.8 GB/s | 256 GB/s | ||
To make its Shinebolt 12-Hi 36 GB HBM3E memory stacks, Samsung had to use several advanced technologies. First, the 36 GB HBM3E memory products are based on memory devices made on Samsung's 4th generation 10nm-class (14nm) fabrication technology, which is called and uses extreme ultraviolet (EUV) lithography.
Secondly, to ensure that 12-Hi HBM3E stacks have the same z-height as 8-Hi HBM3 products, Samsung used its advanced thermal compression non-conductive film (TC NCF), which allowed it to achieve the industry's smallest gap between memory devices at seven micrometers (7 µm). By shrinking gaps between DRAMs, Samsung increases vertical density and mitigates chip die warping. Furthermore, Samsung uses bumps of various sizes between the DRAM ICs; smaller bumps are used in areas for signaling. In contrast, larger ones are placed in spots that require heat dissipation, which improves thermal management.
Samsung estimates that its 12-Hi HBM3E 36 GB modules can increase the average speed for AI training by 34% and expand the number of simultaneous users of inference services by more than 11.5 times. However, the company has not elaborated on the size of the LLM.
Samsung has already begun providing samples of the HBM3E 12H to customers, with mass production scheduled to commence in the first half of this year.
Source: Samsung
Micron Technology on Monday said that it had initiated volume production of its HBM3E memory. The company's HBM3E known good stack dies (KGSDs) will be used for Nvidia's H200 compute GPU for artificial intelligence (AI) and high-performance computing (HPC) applications, which will ship in the second quarter of 2024.
Micron has announced it is mass-producing 24 GB 8-Hi HBM3E devices with a data transfer rate of 9.2 GT/s and a peak memory bandwidth of over 1.2 TB/s per device. Compared to HBM3, HBM3E increases data transfer rate and peak memory bandwidth by a whopping 44%, which is particularly important for bandwidth-hungry processors like Nvidia's H200.
Nvidia's H200 product relies on the Hopper architecture and offers the same computing performance as the H100. Meanwhile, it is equipped with 141 GB of HBM3E memory featuring bandwidth of up to 4.8 TB/s, a significant upgrade from 80 GB of HBM3 and up to 3.35 TB/s bandwidth in the case of the H100.
Micron's memory roadmap for AI is further solidified with the upcoming release of a 36 GB 12-Hi HBM3E product in March 2024. Meanwhile, it remains to be seen where those devices will be used.
Micron uses its 1β (1-beta) process technology to produce its HBM3E, which is a significant achievement for the company as it uses its latest production node for its data center-grade products, which is a testament to the manufacturing technology.
Starting mass production of HBM3E memory ahead of competitors SK Hynix and Samsung is a significant achievement for Micron, which currently holds a 10% market share in the HBM sector. This move is crucial for the company, as it allows Micron to introduce a premium product earlier than its rivals, potentially increasing its revenue and profit margins while gaining a larger market share.
"Micron is delivering a trifecta with this HBM3E milestone: time-to-market leadership, best-in-class industry performance, and a differentiated power efficiency profile," said Sumit Sadana, executive vice president and chief business officer at Micron Technology. "AI workloads are heavily reliant on memory bandwidth and capacity, and Micron is very well-positioned to support the significant AI growth ahead through our industry-leading HBM3E and HBM4 roadmap, as well as our full portfolio of DRAM and NAND solutions for AI applications."
Source: Micron
Rambus has introduced its fourth-generation registering clock driver (RCD) chip for server-grade DDR5 memory modules. The updated RCD chip brings support for higher clockspeeds on DDR5 RDIMMs, allowing for future RDIMMs to run as fast as DDR5-7200, a step ahead of their current third-generation/DDR5-6400 RCD. Faster DDR5 RDIMMs will eventually go hand-in-hand with upcoming server platforms, with AMD, Intel, and others all eyeing significantly higher memory speeds for their next round of products.
An RCD functions as a buffer between the memory controller and DRAM chips in RDIMMs, redistributing command and address signals across the module – and making up the "Registered" in "Registered DIMM". This enhances signal integrity and enables more memory devices to be connected to a single DRAM channel. To work properly. the RCD must support a specific data transfer rate, so the new RCD buffer from Rambus will enable memory vendors to build server-grade DDR5-7200 modules.
In addition to performing regular functions of RCD, the new Rambus chip also packs a serial presence detect (SPD) hub and temperature sensors, which are vital for DDR5 memory sticks on general and DDR5 datacenter memory modules in particular. Such functionality somewhat reduces costs of datacenter-grade DRAM sticks, though the costs of such devices are depend on the number of memory devices rather than by tiny chips like SPDs and temperature sensors.
RCDs are crucial for contemporary server-grade memory modules, as these devices can only reach their full memory capacity with registered DIMMs. Rambus is the first company to officially introduce DDR5 RCDs that support a 7200 MT/s data transfer rate, which will be useful for next-generation server platforms.
According to Rambus, their latest RCD chip is now shipping. Though given the long lead times in servers and server parts due to validation requirements, it's likely still some time off before it starts appearing in commercial servers and DIMMs.
UPDATE 11/24: Demu, our reader with sharp eyes, notified us that Gigabyte had published a list of overclockable RDIMMs with ECC and AMD EXPO profiles supported by its TRX50 motherboards for AMD Ryzen Threadripper 7000-series processors. As it turns out, Kingston is also prepping such memory modules, whereas v-color is planning quad-channel DDR5-7800 RDIMM kits for AMD Ryzen Threadripper 7000-series CPUs. We have added specifications of all upcoming overclockable memory modules for AMD Ryzen Threadripper 7000 platforms into the table below the original story.
With the launch of AMD new Ryzen Threadripper 7000-series processors and associated TRX50 platform comes the need for new memory kits. Thanks to UDIMMs and RDIMMs requiring different slot designs under DDR5, AMD has opted to go exclusively with RDIMMs for their latest generation of Threadripper processors. Which for memory makers has included putting together overclockable registered memory modules with ECC that support EXPO timings, such as the kits G.Skill and V-Color have announced this week with their Zeta R5 Neo and v-color DDR5 OC R-DIMM memory kits.
G.Skill's Zeta R5 Neo product family and v-color's OC R-DIMM are exclusively designed for AMD's Ryzen Threadripper 7000-series processors. Both vendors are currently offering two types of kits: a 64 GB quad-channel kit consisting of four 16 GB RDIMMs, and a 128 GB quad-channel kit featuring four 32 GB RDIMMs. For systems using the TRX50 platform, a single kit suffices, while the high-end workstation-class WRX90 platform requires two of these quad-channel kits to fully populate all eight channels.
G.Skill's modules with ECC are rated for of DDR5-6400 CL32 39-39-102 at massive 1.40 Volts, whereas v-color promises data transfer rates of up to 7200 MT/s, never mentions timings or voltages, but stresses that its kits are aimed at TRX50-based machines.
G.Skill and v-color note that their overclockable RDIMMs, similar to other premium memory modules, are built using hand-selected memory ICs, making them better suited for overclocking. G.Skill's modules come with very basic heat spreaders, which in case of DDR5 may become a limiting factor for overclocking potential. By contrast, v-color's modules come with rather serious heat spreaders akin to those used for some server-grade modules.
All the overclockable RDIMM modules from G.Skill and v-color come with AMD's EXPO profiles for simplified setup, with G.Skill using DDR5-6400 CL32 timings, while v-color runs at DDR5-7200 at unspecified timings.
The exact overclocking potential of G.Skill's and v-color's Threadripper memory kits remain to be seen. The actual I/O die powering Threadripper is the same I/O die as AMD's other server processors, which is to say it's been optimized for stability over performance – not to mention supporting a much larger amount of memory. Still, as we've seen on AMD's consumer CPU platforms, the company's memory controllers are no slouches. All the while memory vendors are already offering high-clocked RDIMM kits for Intel's rival Sapphire Rapids Xeon platform.
G.Skill's Zeta R5 Neo memory kits designed for AMD's Ryzen Threadripper 7000-series processors are already available at Newegg: the 64 GB kit is priced at $530, whereas the 128GB kit costs $1,070. This is of course a huge price premium, but this can be explained by the fact that we are talking about unique memory kits designed for very specific high-end CPUs.
As for v-color's DDR5 OC R-DIMM memory kits for AMD's Ryzen Threadripper TRX50 platform, the company says that they will be available in November on its website.
| Overclockable RDIMMs for AMD Ryzen Threadripper 7000 CPUs | ||||||||||
| Brand | Speed | Capacity | Timings | Voltage | DRAM IC | Native | P/N | |||
| G.Skill | 6400MT/s | 16GB | 32-39-39-102 | 1.4v | Hynix A | 4800MT/s | F5-6400R3239G32GQ4-ZR5NK | |||
| 32GB | F5-6400R3239G16GQ4-ZR5NK | |||||||||
| Kingston | 6000MT/s | 16GB | 32-38-38-80 | 1.35v | Skhynix | 4800MT/s | KF560R32RBE-16 | |||
| Skhynix | KF560R32RBE-16 | |||||||||
| Hynix A | KF560R32RBEK4-64 | |||||||||
| 32GB | Hynix A | KF560R32RBEK4-128 | ||||||||
| 6400MT/s | 16GB | 32-39-39-80 | 1.4v | Skhynix | 4800MT/s | KF564R32RBE-16 | ||||
| Hynix A | KF564R32RBEK4-64 | |||||||||
| V-Color | 4800MT/s | 24GB | 36-38-38-70 | 1.1V | Hynix M | 4800MT/s | TRA524G48S836 | |||
| 16GB | Hynix A | 5600MT/s | TRA516G48S836 | |||||||
| 24GB | Hynix M | 4800MT/s | TRA524G48S836 | |||||||
| 32GB | Hynix M | TRA532G48S436 | ||||||||
| 5200MT/s | 16GB | 36-40-40-80 | 1.25V | Hynix A | 5600MT/s | TRA516G52S836 | ||||
| 24GB | Hynix M | 4800MT/s | TRA524G52S836 | |||||||
| 32GB | Hynix M | TRA532G52S436 | ||||||||
| 5600MT/s | 16GB | 36-38-38-80 | Hynix A | 5600MT/s | TRA516G56S836 | |||||
| 24GB | Hynix M | 4800MT/s | TRA524G56S836 | |||||||
| 32GB | Hynix M | TRA532G56S436 | ||||||||
| 16GB | 32-39-39-102 | 1.4V | Hynix A | 5600MT/s | TRA516G60S832 | |||||
| 24GB | Hynix M | 4800MT/s | TRA524G60S832 | |||||||
| 32GB | 32-38-38-96 | 1.25V | Hynix M | 4800MT/s | TRA532G60S432 | |||||
| 6400MT/s | 24GB | 32-39-39-102 | 1.4V | Hynix M | 4800MT/s | TRA524G64S832 | ||||
| 16GB | Hynix A | 5600MT/s | TRA516G64S832 | |||||||
| 6600MT/s | 24GB | 34-46-46-92 | Hynix M | 4800MT/s | TRA524G66S834 | |||||
| 16GB | Hynix A | 5600MT/s | TRA516G66S834 | |||||||
| 24GB | Hynix M | 4800MT/s | TRA524G68S834 | |||||||
| 16GB | Hynix A | 5600MT/s | TRA516G68S834 | |||||||
| 7000MT/s | 24GB | 34-42-42-102 | Hynix M | 4800MT/s | TRA524G70S834R3 | |||||
| 16GB | Hynix A | 5600MT/s | TRA516G70S834 | |||||||
| 7200MT/s | 24GB | 34-45-45-112 | Hynix M | 4800MT/s | TRA524G72S834R3 | |||||
| 16GB | Hynix A | 5600MT/s | TRA516G72S834 | |||||||
| 7600MT/s | 24GB | 38-48-48-122 | Hynix M | 4800MT/s | TRA524G76S838R3 | |||||
| 16GB | Hynix A | 5600MT/s | TRA516G76S838 | |||||||
| 7800MT/s | 24GB | 38-48-48-126 | Hynix M | 4800MT/s | TRA524G78S838R3 | |||||
| 16GB | Hynix A | 5600MT/s | TRA516G78S838 | |||||||
SK hynix had started volume shipments of its LPDDR5T-9600 memory for high-end smartphones, the company announced this week. So far, the company's LPDDR6 'Turbo' memory with a 9600 MT/s data transfer speed has been certified to work with two range-topping mobile application processors from Qualcomm and MediaTek.
SK hynix's LPDDR5T-9600 memory is available in 16 GB packages with a VDD voltage range of 1.01V to 1.12V and a VDDQ of 0.5v. Notably, this VDD range is just slightly over the LPDDR5X specfication (1.00V to 1.1V), which shouldn't be a serious problem, but likely warrants some extra compatibility testing with existing chips.
Smartphone and SoC manufacturers have a great incentive to validate SK hynix's LPDDR5T-9600 and Micron's LPDDR5X-9600 memory as these modules offer a 76.8 GB/s peak bandwidth, up 12.5% from 68.2 GB/s offered by LPDDR5X-8533.
So far, SK Hynix's LPDDR6T-9600 modules have been certified by Qualcomm for its Snapdragon 8 Gen 3 mobile SoC as well as MediaTek for its Dimensity 9300 and some subsequent processors. Meanwhile, SK Hynix confirmed that it had begun shipments of its LPDDR6T-9600 devices to Vivo, which will use it for X100 and X100 Pro mobile application processors.
Following the latest trends, SK hynix mentions that its very fast LPDDR5T-9600 memory will be particularly useful for on-device AI applications. And, of course, faster DRAM is always welcomed for graphics intensive mobile applications, such as games.
"Smartphones are becoming essential devices for implementing on-device AI technology as the AI era kicks into full swing," said Myoungsoo Park, Vice President and Head of DRAM Marketing at SK hynix. "There is a growing demand for high-performing, high-capacity mobile DRAMs in the market. We will continue to lead the premium DRAM market based on our technological leadership in AI memories, while staying in tune with market demands."
The path to high-capacity RDIMMs for servers has primarily been through 3D stacking (3DS) of DRAM dies using Through-Silicon Vias (TSVs). However, this has presented significant challenges in packaging (driving up the cost), and has also not been efficient in terms of energy consumption. The demand for large memory capacity RDIMMs is being primarily driven by the sudden emergence of large-language models (LLMs) for generative AI and increasing CPU core counts. Both of these require significant amount of DRAM to keep pace with performance requirements. Keeping these in mind, Micron is introducing 128 GB DDR5 RDIMMs capable of operating at up to 8000 MT/s today, with mass-production slated for 2024.
Micron has recently started fabricating 32 Gb monolithic DDR5 dies using its proven and mature 1β technology. The new dies have a 45%+ increase in bit density, and are capable of reaching up to 8000 MT/s while also operating with much more aggressive timing latencies compared to the standard JEDEC specs. The company is claiming that it improves energy efficiency by as much as 24% compared to the competition's 3DS TSV offerings, and the faster operation can also help in faster AI training times. Avoiding 3DS TSV allows Micron to optimize the data input buffers and critical I/O circuits better, while also reducing the pin capacitance on the data lines. These contribute to the reduced power and improved speeds.
Micron has been doubling its monolithic die density every 3 years or so, thanks to advancements in CMOS process as well as improvements in array efficiency. The company sees a clear path to 48 Gb and 64 Gb monolithic dies in the future with continued technological progress. Micron is also claiming that its 1β node has reached mass production ahead of the competition, and that it has had the fastest yield maturity in the company's history. Dual-die packages and tall form-factor (TFF) modules using 1β DRAM are expected to enable 1TB modules in the near future.
Along with the announcement of the 128 GB RDIMMs using 1β technology, the company also laid out its roadmap for upcoming products. HDM and GDDR7 are expected to dominate bandwidth-hungry applications, while RDIMMs, MCRDIMMs, and CXL solutions are in the pipeline for systems requiring massive capacity. LPDDR5X, and LPCAMM2 solutions going up to 192 GB are expected to make an appearance in power-sensitive systems as early as 2026.
Fast memory is crucial for the performance of high-end system-on-chips that are getting more sophisticated every year. When it comes to smartphones, the most obvious way to boost memory performance is to push its data transfer rate. Apparently, this is what Micron and SK Hynix are doing with their new LPDDR5X and LPDDR5T DRAMs that boast a data transfer rate of 9.6 GT/s.
Micron's LPDDR5X-9600 memory devices are made on the company's latest 1β (1-beta) process technology. They are offered in up to 16 GB x64 packages (though it is unclear how many actual memory devices these packages integrate). Micron says that its LPDDR5X made on its latest production node boasts up to 30% lower power consumption compared to competing LPDDR5X ICs made on 1α (1-alpha) technology, though this is something to be expected.
Micron does not disclose how it managed to increase the data transfer rate of its LPDDR5X to 9.6 GT/s, which is a 12% increase compared to 8.53 GT/s, which was once considered the highest speed of LPDDR5X memory. The only thing that the company discloses is that these ICs boast 'enhanced' dynamic voltage and frequency scaling, although DVFS is a part of LPDDR5X specification.
"Generative AI is poised to unleash unprecedented productivity, ease of use, and personalization for smartphone users by delivering the power of large language models to flagship mobile phones," said Mark Montierth, corporate vice president and general manager of Micron's Mobile Business Unit. "Micron's 1β LPDDR5X combined with Qualcomm Technologies' AI-optimized Snapdragon 8 Gen 3 Mobile Platform empowers smartphone manufacturers with the next-generation performance and power efficiency essential to enabling revolutionary AI technology at the edge."
SK Hynix is another company to start shipping LPDDR5-9600 memory today, which calls its fastest LPDDR5 DRAMs LPDDR5T (T stands for Turbo). The new memory will be available in 16 GB packages with a VDD voltage range of 1.01V to 1.12V and a VDDQ of 0.5v. By contrast, LPDDR5X should have a maximum VDD voltage of 1.1V, so LPDDR5T is slightly out of LPDDR5X spec.
Meanwhile, both Micron's LPDDR5X-9600 and SK Hynix's LPDDR5T-9600 are compatible with Qualcomm's Snapdragon 8 Gen 3 system-on-chip for smartphones, the two compares announced on Tuesday. Micron is already shipping its 16 GB LPDDR 9.6 GT/s modules featuring a 76.8 GB/s peak bandwidth, so expect some of Qualcomm's partners to use the world's fastest mobile memory shortly. SK Hynix's module has been validated by Qualcomm, so the South Korean company will likely begin commercial shipments of its LPDDR5T-9600 product soon.
"We are thrilled that we have met our customers' needs for the ultra-high performance mobile DRAM with the provision of the LPDDR5T," said Sungsoo Ryu, head of DRAM Product Planning at SK Hynix.
Samsung’s annual Memory Tech Day is taking place in San Jose this morning, and as part of the event, the company is making a couple of notable memory technology announcements/disclosures. The highlight of Samsung’s event is the introduction of Shinebolt, Samsung’s HBM3E memory that will set new marks for both memory bandwidth and memory capacity for high-end processors. The company is also disclosing a bit more on their GDDR7 memory, which will mark a significant technological update to the GDDR family of memory standards.
High-bandwidth memory has been around for about a decade, and throughout its its continued development it has steadily increased in speed, starting at a data transfer rate from 1 GT/s (the original HBM) and reaching upwards of 9 GT/s with the forthcoming HBM3E. This has made for an impressive jump in bandwidth in less than 10 years, making HBM an important cornerstone for whole new classes of HPC accelerators that have since hit the market. But it's also a pace that's getting harder to sustain as memory transfer rates increase, especially as the underlying physics of DRAM cells have not changed. As a result, for HBM4 the major memory manufacturers behind the spec are planning on making a more substantial change to the high-bandwidth memory technology, starting with an even wider 2048-bit memory interface.
Designed as a wide-but-slow memory technology that utilizes an ultra-wide interface running at a relatively modest clockspeed, HBM's current 1024-bit memory interface has been a defining characteristic of the technology. Meanwhile its modest clockspeeds have become increasingly less modest in order to keep improving memory bandwidth. This has worked thus far, but as clockspeeds increase, the highly parallel memory is risking running into the same signal integrity and energy efficiency issues that challenge GDDR and other highly serial memory technologies.
Consequently, for the next generation of the technology, organizers are looking at going wider once more, expanding the width of the HBM memory interface even further to 2048-bits. And, equally as important for multiple technical reasons, they intend to do this without increasing the footprint of HBM memory stacks, essentially doubling the interconnection density for the next-generation HBM memory. The net result would be a memory technology with an even wider memory bus than HBM today, giving memory and device vendors room to further improve bandwidth without further increasing clock speeds.
As planned, this would make HBM4 a major technical leap forward on multiple levels. On the DRAM stacking side of matters, a 2048-bit memory interface is going to require a significant increase in the number of through-silicon vias routed through a memory stack. Meanwhile the external chip interface will require shrinking the bump pitch to well below 55 um, all the while increasing the total number of micro bumps significantly from the current count of (around) 3982 bumps for HBM3.
Adding some additional complexity to the technology, memory makers have indicated that they are also going to stack up to 16 memory dies in one module; so-called 16-Hi stacking. (HBM3 technically supports 16-Hi stacks as well, but so far no manufacturer is actually using it) This will allow memory vendors to significantly increase the capacity of their HBM stacks, but it brings new complexity in wiring up an even larger number of DRAM dies without defects, and then keeping the resulting HBM stack suitably and consistently short.
All of this, in turn will require even closer collaboration between chip makers, memory makers, and chip packaging firms in order to make everything come together smoothly.
Speaking at TSMC's OIP 2023 conference in Amsterdam, Dan Kochpatcharin, TSMC's Head of Design Infrastructure Management had this to say: "Because instead of doubling the speed, they doubled the [interface] pins [with HBM4]. That is why we are pushing to make sure that we work with all three partners to qualify their HBM4 [with our advanced packaging methods] and also make sure that either RDL or interposer or whatever in between can support the layout and the speed [of HBM4]. So, [we work with] Samsung, SK Hynix, and Micron."
Since system-in-package (SiP) designs are getting larger, and the number of HBM stacks supported by advanced chip packages is increasing (e.g. 6x reticle size interposers and chips with 12 HBM stacks on-package), chip packages are getting more complex. To ensure that everything continues to work together, TSMC is pushing chip and memory designers to embrace Design Technology Co-Optimization (DTCO). This being a big part of the reason why the world's largest foundry recently organized 3DFabric Memory Alliance, a program designed to enable close collaboration between DRAM makers and TSMC in a bid to enable next-generation solutions that will pack huge amounts of logic transistors and advanced memory.
Among other things, TSMC's 3DFabric Memory Alliance is currently working on ensuring that HBM3E/HBM3 Gen2 memory works with CoWoS packaging, 12-Hi HBM3/HBM3E packages are compatible with advanced packages, UCIe for HBM PHY, and buffer-less HBM (a technology spearheaded by Samsung).
Overall, TSMC's comments last week give us our best look yet at the next generation of high-bandwidth memory. Still, additional technical details about HBM4 remain rather scarce for the moment. Micron said earlier this year that 'HBMNext' memory set to arrive around 2026 will offer capacities between 36 GB and 64 GB per stack and peak bandwidth of 2 TB/s per stack or higher. All of which indicates that memory makers won't be backing off on memory interface clockspeeds for HBM4, even with the move to a wider memory bus.
Micron has reaffirmed plans to start shipments of its HBM3E memory in high volume in early 2024, while also revealing that NVIDIA is one of its primary customers for the new RAM. Meanwhile, the company stressed that its new product has been received with great interest by the industry at large, hinting that NVIDIA will likely not be the only customer to end up using Micron's HBM3E.
"The introduction of our HBM3E product offering has been met with strong customer interest and enthusiasm," said Sanjay Mehrotra, president and chief executive of Micron, at the company's earnings call.
Introducing HBM3E, which the company also calls HBM3 Gen2, ahead of its rivals Samsung and SK Hynix is a big deal for Micron, which is an underdog on the HBM market with a 10% market share. The company obviously pins a lot of hopes on its HBM3E since this will likely enable it to offer a premium product (to drive up its revenue and margins) ahead of its rivals (to win market share).
Typically, memory makers tend not to reveal names of their customers, but this time around Micron emphasized that its HBM3E is a part of its customer's roadmap, and specifically mentioned NVIDIA as its ally. Meanwhile, the only HBM3E-supporting product that NVIDIA has announced so far is its Grace Hopper GH200 compute platform, which features an H100 compute GPU and a Grace CPU.
"We have been working closely with our customers throughout the development process and are becoming a closely integrated partner in their AI roadmaps," said Mehrotra. "Micron HBM3E is currently in qualification for NVIDIA compute products, which will drive HBM3E-powered AI solutions."
Micron's 24 GB HBM3E modules are based on eight stacked 24Gbit memory dies made using the company's 1β (1-beta) fabrication process. These modules can hit data rates as high as 9.2 GT/second, enabling a peak bandwidth of 1.2 TB/s per stack, which is a 44% increase over the fastest HBM3 modules available. Meanwhile, the company is not going to stop with its 8-Hi 24 Gbit-based HBM3E assemblies. The company has announced plans to launch superior capacity 36 GB 12-Hi HBM3E stacks in 2024 after it initiates mass production of 8-Hi 24GB stacks.
"We expect to begin the production ramp of HBM3E in early calendar 2024 and to achieve meaningful revenues in fiscal 2024," added chief executive of Micron.
Micron is sampling 128 GB DDR5 memory modules, the company said at its earnings call this week. The modules are based on the company's latest single die, non-stacked 32 Gb DDR5 memory devices, which the company announced earlier this summer and which will eventually open doors for 1 TB memory modules for servers.
"We expanded our high-capacity D5 DRAM module portfolio with a monolithic die-based 128 GB module, and we have started shipping samples to customers to help support their AI application needs," said Sanjay Mehrotra, president and chief executive of Micron. "We expect revenue from this product in Q2 of calendar 2024."
Micron's 32 Gb DDR5 dies are made on the company's 1β (1-beta) manufacturing process, which is the last production node that solely relies on multi-patterning using deep ultraviolet (DUV) lithography and does not use extreme ultraviolet (EUV) lithography tools. This is all that we know about Micron's 32 Gb DDR5 ICs at this point, though: the company does not disclose its maximum speed bin, though we can expect a drop in power consumption compared to two 16 Gb DDR5 ICs operating at the same voltage and data transfer rate.
Micron's new 32 Gb memory chips pave the way for creating a standard 32 GB module for personal computers with just eight individual memory chips and a server-oriented 128 GB module based on 32 of such ICs. Moreover, these chips make producing memory modules with a 1 TB capacity feasible, deemed unattainable today. These 1 TB modules might seem excessive for now, but they benefit fields like artificial intelligence, Big Data, and server databases. Such modules can enable servers to support up to 12 TB of DDR5 memory per socket (in the case of a 12-channel memory subsystem).
Speaking of DDR5 memory in general, it is noteworthy that the company expects that its bit production of DDR5 will exceed that of DDR4 in early 2024, placing it a bit ahead of the industry.
"Micron also has a strong position in the industry transition to D5," said Mehrotra. "We expect Micron D5 volume to cross over D4 in early calendar 2024, ahead of the industry."
Corsair has started sales of its Dominator Titanium memory modules that were formally introduced this May. The new modules bring together luxurious look, customizable design, and extreme data transfer rates of up to 8000 MT/s. Speaking of performance, the company implied that it intends to introduce Dominator Titanium with speed bins beyond DDR5-8000 when the right platform arrives.
Corsair's Dominator Titanium family is based around 16 GB, 24 GB, 32 GB, and 48 GB memory modules that come in kits ranging from 32GB (2 x 16GB) up to 192GB (4x 48GB). As for performance, the lineup listed at the company's website includes DDR5-6000 CL30, DDR5-6400 CL32, DDR5-6600 CL32, DDR5-7000 CL34, DDR5-7000 CL36, DDR5-7200 CL34, and DDR5-7200 CL36 with voltages of 1.40 V – 1.45V.
Although Corsair claims that Dominator Titanium with data transfer speeds beyond 8000 MT/s are coming, it is necessary to note that they will be supported by next generation platforms from AMD and Intel. For now, the company only offers 500 First Edition Dominator Titanium kits rated for DDR5-8266 mode for its loyal fans.
To address demand from different types of users, Corsair offers Dominator Titanium with XMP 3.0 SPD settings for Intel's 12th and 13th Generation Core CPUs with black and white heat spreaders as well as with AMD EXPO SPD profiles for AMD's Ryzen processors with grey finish on heat spreaders.
In terms of design of heat spreaders, Corsair remained true to aesthetics. The modules are equipped with 11 customizable Capellix RGB LEDs, offering users a personalized touch. This can be easily adjusted using Corsair's proprietary software. For enthusiasts who lean towards a more traditional aesthetic, Corsair provides an alternative design with fins, reminiscent of their classic memory modules.
Speaking of heat spreaders, it is necessary to note that despite the name of the modules, they do not come with titanium radiators and keep using aluminum, which is a good thing since titanium has a rather low thermal conductivity of 11.4 W/mK and will therefore heat up memory chips rather than distribute heat away from them. Traditionally, Corsair's Dominator memory modules use cherry-picked DRAM chips and the company's proprietary printed circuit boards enhanced with internal cooling planes and external thermal pads to improve cooling.
Corsair's Dominator Titanium memory products are now available both directly from the company and from its resellers. The cheapest Dominator Titanium DDR5-6000 CL30 32 GB kit (2 x 16 GB) costs $175, whereas faster and higher-capacity kits are priced higher.
Although Low Power DDR(LPDDR) memory has played a pivotal role in reducing PC laptop power usage, the drawback to the mobile-focused memory has always been its tight signaling and power delivery requirements. Designed to be placed close to its host CPU in order to minimize power expenditures and maximize clockspeeds, LPDDR memory is unsuitable for use in traditional DIMMs and SO-DIMMs – instead requiring that it be soldered down on a device in advance. But it looks like the days of soldered-down LPDDR memory are soon at an end, as this evening Samsung is announcing a new standard for removable and modular LPDDR memory: LPCAMM.
Samsung early on Friday revealed the world's first 32 Gb DDR5 DRAM die. The new memory die is made on the company's 12 nm-class DRAM fabrication process and not only offers increased density, but also lowers power consumption. The chip will allow Samsung to build record 1 TB RDIMMs for servers as well as lower costs of high-capacity memory modules.
“With our 12nm-class 32 Gb DRAM, we have secured a solution that will enable DRAM modules of up to 1 TB, allowing us to be ideally positioned to serve the growing need for high-capacity DRAM in the era of AI (Artificial Intelligence) and big data,” said SangJoon Hwang, executive vice president of DRAM product & technology at Samsung Electronics.
32 Gb memory dies not only enable Samsung to build a regular, single-rank 32 GB module for client PCs using only eight single-die memory chips, but they also allow for higher capacity DIMMs that were not previously possible. We are talking about 1 TB memory modules using 40 8-Hi 3DS memory stacks based on eight 32 Gb memory devices. Such modules may sound overkill, but for artificial intelligence (AI), Big Data, and database servers, more DRAM capacity can easily be put to good use. Eventually, 1TB RDIMMs would allow for up to 12 TB of memory in a single socket server (e.g. AMD's EPYC 9004 platform), something that cannot be done now.
With regards to power consumption, Samsung says that using the new dies they can build 128 GB DDR5 RDIMMs for servers that consume 10% less power than current-generation modules built around 16 Gb dies. This drop of power consumption can be attributed to both 12 nm-class DRAM production node as well as avoiding the use of 3D stacked (3DS) chips that pack two 16 Gb dies into a single package.
Samsung is not disclosing the speed bins of its 32 Gb memory dies, but finished 16 Gb modules made on the same 12 nm-class technology offer a 7200 MT/s data transfer rate.
Samsung intends to start mass production of 32 Gb memory dies by the end of 2023, but for now the company isn't detailing when it plans to offer finished chips to customers. it's likely that the company will start with client PCs first, though whether that translasts into any cost savings remains to be seen.
Otherwise, for servers it usually takes a while for server platform developers and vendors to validate and qualify new memory components. So while Samsung has 1 TB RDIMMs in its future, it will take some time before we see them in shipping servers.
SK hynix on Monday announced that it had completed initial development of its first HBM3E memory stacks, and has begun sampling the memory to a customer. The updated ("extended") version of the high bandwidth memory technology is scheduled to begin shipping in volume in the first half of next year, with hardware vendors such as NVIDIA already lining up to incorporate the memory into their HPC-grade compute products.
First revealed by SK hynix back at the end of May, HBM3E is an updated version of HBM3 that is designed to clock higher than current HBM3, though specific clockspeed targets seem to vary by manufacturer. For SK hynix, as part of today's disclosure the company revealed that their HBM3E memory modules will be able to hit data transfer rates as high as 9 GT/sec, which translates to a peak bandwidth of 1.15 TB/sec for a single memory stack.
Curiously, SK hynix has yet to reveal anything about the planned capacity for their next-gen memory. Previous research from TrendForce projected that SK hynix would mass produce 24 GB HBM3E modules in Q1 2024 (in time to address applications like NVIDIA's GH200 with 144GB of HBM3E memory), boosting capacity over today's 16GB HBM3 stacks. And while this still seems likely (especially with the NV announcement), for now it remains unconfirmed.
TrendForce HBM Market Projections (Source: TrendForce)
Meanwhile, the SK hynix also confirms that its HBM3E stacks are set to use its Advanced Mass Reflow Molded Underfill (MR-RUF) technology to reduce their heat dissipation by 10%. But thermals is not the only benefit MR-RUF can provide. MR-RUF implies the usage of an improved underfill between layers, which improves thermals and reduces thickness of HBM stacks, which allows the construction of 12-Hi HBM stacks that are only as tall as 8-Hi modules. This does not automatically mean that we are dealing with 12-Hi HBM3E stacks here, of course.
At present, SK hynix is the only high volume manufacturer of HBM3 memory, giving the company a very lucrative position, especially with the explosion in demand for NVIDIA's H100 and other accelerators for generative AI. And while the development of HBM3E is meant to help SK hynix keep that lead, they will not be the only memory vendor offering faster HBM next year. Micron also threw their hat into the ring last month, and where those two companies are, Samsung is never too far behind. In fact, all three companies seem to be outpacing JEDEC, the organization that is responsible for standardizing DRAM technologies and various memory interfaces, as that group has still not published finalized specifications for the new memory.
Samsung, MemVerge, H3 Platform, and XConn have jointly unveiled their 2 TB Pooled CXL Memory System at the Flash Memory Summit. The device can be connected to up to eight hosts, allowing them to use its memory when needed. The 2 TB Pooled CXL Memory system has software enabling it to visualize, pool, tier, and dynamically allocate memory to connected hosts.
The 2 TB Pooled CXL Memory system is a 2U rack-mountable machine built by H3 with eight 256 GB Samsung CXL memory modules connected using XConn's XC50256 CXL 2.0 switch supporting 256 PCIe Gen5 lanes and 32 ports. The firmware of the 2 TB Pooled CXL Memory system allows you to connect it to up to eight hosts that can dynamically use CXL memory when they need it, thanks to software by MemVerge.
The Pooled CXL Memory system was developed to overcome limitations in memory capacity and composability in today's system architecture, which involves tight coupling between CPU and DRAM. Such architecture leads to performance challenges in highly distributed AI/ML applications, such as spilling memory to slow storage, excessive memory copying, I/O to storage, serialization/deserialization, and Out-of-Memory errors that can crash applications.
Attaching 2 TB of fast, low-latency memory using a PCIe 5.0 interface with the CXL 2.0 protocol on top to eight host systems and using it dynamically between eight hosts saves a lot of money while providing loads of performance benefits. According to companies, the initiative represents a significant step towards creating a more robust and flexible memory-centric data infrastructure for modern AI applications.
"Modern AI applications require a new memory-centric data infrastructure that can meet the performance and cost requirements of its data pipeline," said Charles Fan, CEO and co-founder of MemVerge. "Hardware and software vendors in the CXL Community are co-engineering such memory-centric solutions that will deeply impact our future."
The jointly developed demonstration system can be pooled, tiered with main memory, and dynamically provisioned to applications with Memory Machine X software from MemVerge and its elastic memory service. Viewer service showcases the system's physical layout and provides a heat map indicating memory capacity and bandwidth consumption per application.
"The concept system unveiled at Flash Memory Summit is an example of how we are aggressively expanding its usage in next-generation memory architectures," said JS Choi, Vice President of New Business Planning Team at Samsung Electronics. "Samsung will continue to collaborate across the industry to develop and standardize CXL memory solutions, while fostering an increasingly solid ecosystem."
On Friday, SK Hynix said it had started mass production of 24 GB LPDDR5X memory stacks that can be used for ultra-high-end smartphones and PCs. The company's LPDDR5X-8500 devices combine ultra-high-performance with high density, thus enabling fast systems with sufficient memory capacity. SK Hynix says such modules could be used well beyond smartphones, PCs, and even servers.
SK Hynix's 24 GB LPDDR5X package features an 8500 MT/s data transfer rate and a wide 64-bit interface, thus offering a peak bandwidth of 68 GB/s in the ultra-low voltage range of 1.01 to 1.12V. From a typical PC perspective, this is comparable to bandwidth provided by a dual-channel DDR5-4800 memory subsystem (76.8 GB/s), but at considerably lower power and an orders of magnitude smaller footprint.
An interesting wrinkle of the SK Hynix's announcement is that the company started to supply these 24 GB LPDDR5X modules well before the announcement as the devices are already used in Oppo's Oneplus Ace 2 Pro smartphone launched on August 10, 2023.
Oppo is not the only maker of high-end Android smartphones out there, so I would expect more companies to follow suit in the coming months as they roll out handsets based on the Qualcomm Snapdragon 8 Gen 2 system-on-chip.
But SK Hynix envisions its LPDDR5X devices to be used beyond smartphones, so think PCs. Apple was the first company to use LPDDR for desktops and laptops. Still, now that PC SoCs from AMD and Intel support LPDDR5X expect other leading makers of notebooks to adopt LPDDR5X in general and SK Hynix's 24 GB packages in particular.
Meanwhile, 64-bit LPDDR5X-8500 devices look particularly compelling for the automotive industry, combining performance, capacity, and a very compact form factor. Given the fact that modern infotainment systems require high memory bandwidth, such memory devices will be pretty beneficial. SK Hynix says these memory stacks could be used for servers and even high-performance computing (HPC) applications.
"Along with a faster advancement in broader IT industry, our LPDDR products will be able to support a growing list of applications such as PC, server, high-performance computing (HPC) and automotive vehicles," said Myoungsoo Park, Vice President and Head of DRAM Marketing at SK Hynix. "The company will cement our leadership in the premium memory market by providing the highest performance products that meet customers' needs."
Source: SK Hynix
TrendForce projects a remarkable 105% increase in annual bit shipments of high-bandwidth memory (HBM) this year. This boost comes in response to soaring demands from AI and high-performance computing processor developers, notably Nvidia, and cloud service providers (CSPs). To fulfill demand, Micron, Samsung, and SK Hynix are reportedly increasing their HBM capacities, but new production lines will likely start operations only in Q2 2022.
Memory makers managed to more or less match the supply and demand of HBM in 2022, a rare occurrence in the market of DRAM. However, an unprecedented demand spike for AI servers in 2023 forced developers of appropriate processors (most notably Nvidia) and CSPs to place additional orders for HBM2E and HBM3 memory. This made DRAM makers use all of their available capacity and start placing orders for additional tools to expand their HBM production lines to meet the demand for HBM2E, HBM3, and HBM3E memory in the future.
However, meeting this HBM demand is not something straightforward. In addition to making more DRAM devices in their cleanrooms, DRAM manufacturers need to assemble these memory devices into intricate 8-Hi or 12-Hi stacks, and here they seem to have a bottleneck since they do not have enough TSV production tools, according to TrendForce. To produce enough HBM2, HBM2E, and HBM3 memory, leading DRAM producers have to procure new equipment, which takes 9 to 12 months to be made and installed into their fabs. As a result, a substantial hike in HBM production is anticipated around Q2 2024, the analysts claim.
A noteworthy trend pinpointed by TrendForce analysts is the shifting preference from HBM2e (Used by AMD's Instinct MI210/MI250/MI250X, Intel's Sapphire Rapids HBM and Ponte Vecchio, and Nvidia's H100/H800 cards) to HBM3 (incorporated in Nvidia's H100 SXM and GH200 supercomputer platform and AMD's forthcoming Instinct MI300-series APUs and GPUs). TrendForce believes that HBM3 will account for 50% of all HBM memory shipped in 2023, whereas HBM2E will account for 39%. In 2024, HBM3 is poised to account for 60% of all HBM shipments. This growing demand, when combined with its higher price point, promises to boost HBM revenue in the near future.
Just yesterday, Nvidia launched a new version of its GH200 Grace Hopper platform for AI and HPC that uses HBM3E memory instead of HBM3. The new platform consisting of a 72-core Grace CPU and GH100 compute GPU, boasts higher memory bandwidth for the GPU, and it carries 144 GB of HBM3E memory, up from 96 GB of HBM3 in the case of the original GH200. Considering the immense demand for Nvidia's offerings for AI, Micron — which will be the only supplier of HBM3E in 1H 2024 — stands a high chance to benefit significantly from the freshly released hardware that HBM3E powers.
TrendForce also noted a consistent decline in HBM product ASPs each year. To invigorate interest and offset decreasing demand for older HBM models, prices for HBM2e and HBM2 are set to drop in 2023, according to the market tracking firm. With 2024 pricing still undecided, further reductions for HBM2 and HBM2e are expected due to increased HBM production and manufacturers' growth aspirations.
In contrast, HBM3 prices are predicted to remain stable, perhaps because, at present, it is exclusively available from SK Hynix, and it will take some time for Samsung to catch up. Given its higher price compared to HBM2e and HBM2, HBM3 could push HBM revenue to an impressive $8.9 billion by 2024, marking a 127% YoY increase, according to TrendForce.
SK Hynix commanded 50% of the HBM memory market in 2022, followed by Samsung with 40% and Micron with a 10% share. Between 2023 and 2024, Samsung and SK Hynix will continue to dominate the market, holding nearly identical stakes that sum up to about 95%, TrendForce projects. On the other hand, Micron's market share is expected to hover between 3% and 6%.
Meanwhile, (for now) SK Hynix seems to have an edge over its rivals. SK Hynix is the primary producer of HBM3, the only company to supply memory for Nvidia's H100 and GH200 products. In comparison, Samsung predominantly manufactures HBM2E, catering to other chip makers and CSPs, and is gearing up to start making HBM3. Micron, which does not have HBM3 in the roadmap, produces HBM2E (which Intel reportedly uses for its Sapphire Rapids HBM CPU) and is getting ready to ramp up production of HBM3E in 1H 2024, which will give it a significant competitive advantage over its rivals that are expected to start making HBM3E only in 2H 2024.
This week, Micron announced the sample availability of its first CXL 2.0 memory expansion modules for servers that promise easy and cheap DRAM subsystem expansions.
Modern server platforms from AMD and Intel boast formidable 12- and 8-channel DDR5 memory subsystems offering bandwidth of up to 460.8 – 370.2 GB/s and capacities of up to 6 – 4 TB per socket. But some applications consume all DRAM they can get and demand more. To satisfy the needs of such applications, Micron has developed its CZ120 CXL 2.0 memory expansion modules that carry 128 GB and 256 GB of DRAM and connect to a CPU using a PCIe 5.0 x8 interface.
"Micron is advancing the adoption of CXL memory with this CZ120 sampling milestone to key customers," said Siva Makineni, vice president of the Micron Advanced Memory Systems Group.
Micron's CZ120 memory expansion modules use Microchip's SMC 2000-series smart memory controller that supports two 64-bit DDR4/DDR5 channels as well as Micron's DRAM chips made on the company's 1α (1-alpha) memory production node. Every CZ120 module delivers bandwidth up to 36 GB/s (measured by running an MLC workload with a 2:1 read/write ratio on a single module), putting it only slightly behind a DDR5-4800 RDIMM (38.4 GB/s) but orders of magnitude ahead of a NAND-based storage device.
Micron asserts that adding four of its 256 GB CZ120 CXL 2.0 Type 3 expansion modules to a server running 12 64GB DDR5 RDIMMs can increase memory bandwidth by 24%, which is significant. Perhaps more significant is that adding an extra 1 TB of memory enables such a server to handle nearly double the number of database queries daily.
Of course, such an expansion means using PCIe lanes and thus reducing the number of SSDs that can be installed into such a machine. But the reward seems quite noticeable, especially if Micron's CZ120 memory expansion modules are cheaper than actual RDIMMs or have comparable costs.
For now, Micron has announced sample availability, and it is unclear when the company will start to ship its CZ120 memory expansion modules commercially. Micron claims that it has already tested its modules with major server platform developers, so right now, its customers are probably validating and qualifying the modules with their machines and workloads, so it is reasonable to expect CZ120 to be deployed already in 2024.
"We have been developing and testing our CZ120 memory expansion modules utilizing both Intel and AMD platforms capable of supporting the CXL standard," added Makineni. "Our product innovation coupled with our collaborative efforts with the CXL ecosystem will enable faster acceptance of this new standard, as we work collectively to meet the ever-growing demands of data centers and their memory-intensive workloads."
While DDR5 memory has been out and in use for a couple of years now, so far we haven't seen the memory reach its full potential – at least, not for rank-and-file standards-compliant DIMMs. The specification allows for speeds as high as DDR5-6400, but to date we've only seen on-spec kits (and processors) as fast as DDR5-5600. But at last, it looks like things are about to change and DDR5 is set to live up to its full potential, going by a new memory kit announcement from TeamGroup.
The memory kit vendor on Monday introduced its new ElitePlus-series DDR5-6400 memory modules, the first DDR5-6400 kit to be announced as JEDEC specification compliant. This means their new kit not only hits 6400 MT/s with standards-compliant timings, but arguably more importantly, it does so at DDR5's standard voltage of 1.1V as well. And while there are no platforms on the market at this time that are validated for JEDEC DDR5-6400 speeds, TeamGroup's product page already lists compatibility with Intel's yet-to-be-announced "Z790 Refresh" platform – so suitable processors seem to be due soon.
TeamGroup's Elite and ElitePlus DDR5-6400 memory modules come in 16 GB and 32 GB capacities (32 GB and 64 GB dual-channel kits) and feature JEDEC-standard CL52 52-52-103 timings as well as 1.1V voltage, as specified by the organization overseeing DRAM specs. For the moment, at least, TeamGroup's DDR5-6400 modules are the industry's fastest UDIMMs that are fully compliant with the JEDEC specifications.
And while DDR5-6400 speeds (and far higher) are available today with factory overclocked XMP/EXPO, the announcement of a JEDEC standards-compliant kit is still significant for a few different reasons. Being able to hit DDR5-6400B speeds and timings at 1.1V means DDR5 memory has improved to the point to make higher speeds at low voltages more viable, which has potential payoffs for memory at every speed grade by allowing for improved speeds and reduced power consumption/heat. And for OEM and other warrantied systems that only use JEDEC-complaint RAM, this allows for a straightforward improvement in memory speeds and bandwidth. About the only downside to faster on-spec kits is that they lack XMP or EXPO serial presence detect (SPD) profiles, which makes their configuration slightly more complicated on existing platforms from AMD and Intel, as they don't officially support DDR5-6400.
Meanwhile, on their product pages TeamGroup notes that the new RAM is compatible with Intel's "Z790 Refresh" platform, a platform that has yet to be officially announced, but is rumored to go hand-in-hand with Intel "Raptor Lake Refresh" processors. Despite the lack of formal announcements from Intel there, TeamGroup seems to have let the cat out of the bag. So, prospective owners of Z790 Refresh systems can look forward to having access to specs-compliant 1.1V DDR5-6400 memory when that platform launches later this year.
As for the modules at hand, traditionally, TeamGroup's Elite and ElitePlus memory modules are minimalistic and are aimed both at system integrators and at enthusiasts who are not after fancy designs of heat spreaders, RGB lighting, and maximum performance. In fact, TeamGroup's Elite modules do not have heat spreaders at all, whereas ElitePlus modules have a minimalistic heat spreader that will not interfere with large CPU coolers.
TeamGroup says its Elite and ElitePlus DDR5-6400 memory modules will be available separately and in dual-channel kits starting from August in North America and Taiwan. And from that, we'd assume, Raptor Lake Refresh will not be far behind.
In addition to unveiling its first HBM3 memory products yesterday, Micron also published a fresh DRAM roadmap for its AI customers for the coming years. Being one of the world's largest memory manufacturers, Micron has a lot of interesting things planned, including high-capacity DDR5 memory devices and modules, GDDR7 chips for graphics cards and other bandwidth-hungry devices, as well as HBMNext for artificial intelligence and high-performance computing applications.
We all love inexpensive high-capacity memory modules, and it looks like Micron has us covered. Sometimes in the late first half of 2024, the company plans to roll-out its first 32 Gb DDR5 memory dies, which will be produced on the company's 1β (1-beta) manufacturing process. This is Micron's latest process node and which does not use extreme ultraviolet lithography, but rather relies on multipatterning.
32 Gb DRAM dies will enable Micron to build 32 GB DDR5 modules using just eight memory devices on one side of the module. Such modules can be made today with Micron's current 16 Gb dies, but this requires either placing 16 DRAM packages over both sides of a memory module – driving up production costs – or by placing two 16 Gb dies within a single DRAM package, which incurs its own costs due to the packaging required. 32 Gb ICs, by comparison, are easier to use, so 32 GB modules based on denser DRAM dies will eventually lead to lower costs compared to today's 32 GB memory sticks.
But desktop matters aside, Micron's initial focus with their higher density dies will be to build even higher capacity data center-class parts, including RDIMMs, MRDIMMs, and CXL modules. Current high performance AI models tend to be very large and memory constrained, so larger memory pools open the door both to even larger models, or in bringing down inference costs by being able to run additional instances on a single server.
For 2024, Micron is planning to release 128GB DDR5 modules based on these new dies. In addition, the company announced plans for 192+ GB and 256+ GB DDR5 modules for 2025, albeit without disclosing which chips these are set to use.
Meanwhile, Micron's capacity-focused roadmap doesn't have much to say about bandwidth. While it would be unusual for newer DRAM dies not to clock at least somewhat higher, memory manufacturers as a whole have not offered much guidance about future DDR5 memory speeds. Especially with MRDIMMs in the pipeline, the focus is more on gaining additional speed through parallelism, rather than running individual DRAM cells faster. Though with this roadmap in particular, it's clear that Micron is more focused on promoting DDR5 capacity than promoting DDR5 performance.
Micron was the first larger memory maker to announce plans to roll out its GDDR7 memory in the first half of 2024. And following up on that, the new roadmap has the the company prepping 16 Gb and 24 Gb GDDR7 chips for late Q2 2024.
As with Samsung, Micron's plans for their first generation GDDR7 modules do not have them reaching the spec's highest transfer rates right away (36 GT/sec), and instead Micron is aiming for a more modest and practical 32 GT/sec. Which is still good enough to enable upwards of 50% greater bandwidth for next-generation graphics processors from AMD, Intel, and NVIDIA. And perhaps especially NVIDIA, since this roadmap also implies that we won't be seeing a GDDR7X from Micron, meaning that for the first time since 2018, NVIDIA won't have access to a specialty GDDR DRAM from Micron.
In addition to GDDR7, which will be used by graphics cards, game consoles, and lower-end high-bandwidth applications like accelerators and networking equipment, Micron is also working on the forthcoming generations of its HBM memory for heavy-duty artificial intelligence (AI) and high-performance computing (HPC) applications.
Micron expects its HBMNext (HBM4?) to be available in 36 GB and 64 GB capacities, which points to a variety of configurations, such as 12-Hi 24 Gb stacks (36 GB) or 16-Hi 32 Gb stacks (64 GB), though these are pure speculations at this point. As for performance, Micron is touting 1.5 TB/s – 2+ TB/s of bandwidth per stack, which points to data transfer rates in excess of 11.5 GT/s/pin.
Micron today is introducing its first HBM3 memory products, becoming the latest of the major memory manufacturers to start building the high bandwidth memory that's widely used in server-grade GPUs and other high-end processors. Aiming to make up for lost time against its Korean rivals, Micron intends to essentially skip "vanilla" HBM3 and move straight on to even higher bandwidth versions of the memory they're dubbing "HBM3 Gen2", developing 24 GB stacks that run at over 9 GigaTransfers-per-second. These new HBM3 memory stacks from Micron will target primarily AI and HPC datacenter, with mass production kicking off for Micron in early 2024.
Micron's 24 GB HBM3 Gen2 modules are based on stacking eight 24Gbit memory dies made using the company's 1β (1-beta) fabrication process. Notably, Micron is the first of the memory vendors to announce plans to build HBM3 memory with these higher-density dies, while SK hynix offers their own 24 GB stacks, the company is using a 12-Hi configuration of 16Gbit dies. So Micron is on track to be the first vendor to offer 24 GB HBM3 modules in the more typical 8-Hi configuration. And Micron is not going to stop at 8-Hi 24Gbit-based HBM3 Gen2 modules, either, with the company saying that they plan to introduce even higher capacity class-leading 36 GB 12-Hi HBM3 Gen2 stacks next year.
Besides taking the lead in density, Micron is also looking to take a lead in speed. The company expects its HBM3 Gen2 parts to hit date rates as high as 9.2 GT/second, 44% higher than the top speed grade of the base HBM3 specification, and 15% faster than the 8 GT/second target for SK hynix's rival HBM3E memory. The increased data transfer rate enables each 24 GB memory module to offer peak bandwidth of 1.2 TB/sec per stack.
Micron says that 24GB HBM3 Gen2 stacks will enable 4096-bit HBM3 memory subsystems with a bandwidth of 4.8 TB/s and 6096-bit HBM3 memory subsystems with a bandwidth of 7.2 TB/s. To put the numbers into context, Nvidia's H100 SXM features a peak memory bandwidth of 3.35 TB/s.
| HBM Memory Comparison | ||||||
| "HBM3 Gen2" | HBM3 | HBM2E | HBM2 | |||
| Max Capacity | 24 GB | 24 GB | 16 GB | 8 GB | ||
| Max Bandwidth Per Pin | 9.2 GT/s | 6.4 GT/s | 3.6 GT/s | 2.0 GT/s | ||
| Number of DRAM ICs per Stack | 8 | 12 | 8 | 8 | ||
| Effective Bus Width | 1024-bit | |||||
| Voltage | 1.1 V? | 1.1 V | 1.2 V | 1.2 V | ||
| Bandwidth per Stack | 1.2 TB/s | 819.2 GB/s | 460.8 GB/s | 256 GB/s | ||
High frequencies aside, Micron's HBM3 Gen2 stacks are otherwise drop-in compatible with current HBM3-compliant applications (e.g., compute GPUs, CPUs, FPGAs, accelerators). So device manufacturers will finally have the option of tapping Micron as an HBM3 memory supplier as well, pending the usual qualification checks.
Under the hood, Micron's goal to jump into an immediate performance leadership position within the HBM3 market means that they need to one-up their competition from a technical level. Among other changes and innovations to accomplish that, the company increased the number of through-silicon vias (TSVs) by two times compared to shipping HBM3 products. In addition, Micron shrunk the distance between DRAM devices in its HBM3 Gen2 stacks. These two changes to packaging reduced thermal impendence of these memory modules and made it easier to cool them down. Yet, the increased number of TSVs can bring other advantages too.
Given that Micron uses 24 Gb memory devices (rather than 16 Gb memory devices) for its HBM3 Gen2 stacks, it is inevitable that it had to increase the number of TSVs to ensure proper connectivity. Yet, doubling the number of TSVs in an HBM stack can enhance overall bandwidth (and shrink latency), power efficiency, and scalability by facilitating more parallel data transfers. It also improves reliability by mitigating the impact of single TSV failures through data rerouting. However, these benefits come with challenges such as increased manufacturing complexity and increased potential for higher defect rates (already an ongoing concern for HBM), which can translate to higher costs.
Just like other HBM3 memory modules, Micron's HBM3 Gen2 stacks feature Reed-Solomon on-die ECC, soft repair of memory cells, hard-repair of memory cells as well as auto error check and scrub support.
Micron says it will mass produce its 24 GB HBM3 modules starting in Q1 2024, and will start sampling its 12-Hi 36GB HBM3 stacks around this time as well. The latter will enter high volume production in the second half of 2024.
To date, the JEDEC has yet to approve a post-6.4GT/second HBM3 standard. So Micron's HBM3 Gen2 memory, as well as SK hynix's rival HBM3E memory, are both off-roadmap standards for the moment. Given the interest in higher bandwidth HBM memory and the need for standardization, we'd be surprised if the group didn't eventually release an updated version of the HBM3 standard that Micron's devices will conform to. Though as the group tends to shy away from naming battles ("HBM2E" was never a canonical product name for faster HBM2, despite its wide use), it's anyone's guess how this latest kerfuffle over naming will play out.
Beyond their forthcoming HBM3 Gen2 products, Micron is also making it known that the company already working on HBMNext (HBM4?) memory. That iteration of HBM will provide 1.5 TB/s – 2+ TB/s of bandwidth per stack with capacities ranging from 36 GB to 64 GB.
In a surprising turn of events, Cadence and Rambus entered into a definitive agreement late last week for Cadence to buy memory physical interface IP and SerDes businesses from Rambus. As a result, Cadence will get a comprehensive portfolio of memory PHY IP and an established client base. Meanwhile, with the sale of its PHY and SerDes assets, Rambus will now solely focus on licensing digital IP.
Historically, Rambus developed memory technologies, including RDRAM and XDR DRAM. At some point, the company patented fundamental technologies enabling SDRAM, DDR SDRAM, and their successors. Doing this allowed them to effectively sue virtually all memory makers and designers of memory controllers (including AMD and Nvidia) and make them pay license fees.
Over time the company began to license memory controllers and PHY. It became a one-stop shop for chip developers needing a turnkey memory, PCIe, or MIPI solution for their designs. Nowadays, it is possible to come to Rambus and get one of the industry's best memory controllers and silicon-proven interfaces. But while Rambus plans to retain memory and interface controllers and everything logic-related, it intends to get rid of its PHY and SerDes IP assets and sell them to Cadence.
Getting silicon-proven PHY and SerDes IP assets and clients for Cadence makes perfect sense.
"The acquisition of the Rambus PHY IP broadens Cadence's well-established enterprise IP portfolio and expands its reach across geographies and vertical markets, such as the aerospace and defense market, providing complete subsystem solutions that meet the demands of our worldwide customers," said Boyd Phelps, senior vice president and general manager of the IP Group at Cadence.
But the rationale behind Rambus's decision to sell PHY and SerDes business is less obvious. On the one hand, memory PHY and SerDes businesses require Rambus to invest in expensive tape-outs on the latest nodes, and this requires capital and increases risks as Rambus has to compete against companies like Cadence and Synopsys that are larger and have more money. On the other hand, Rambus can be a one-stop shop for memory controllers and PHY, which has advantages (i.e., Rambus can charge a premium).
Meanwhile, without needing to keep its physical IP assets up to date, Rambus can now focus on licensing pure technologies and no longer invest in physical IP like PHY or SerDes.
"With this transaction, we will increase our focus on market-leading digital IP and chips and expand our roadmap of novel memory solutions to support the continued evolution of the data center and AI," said Sean Fan, senior vice president and chief operating officer at Rambus.
The transaction is projected to have a negligible impact on the revenue and earnings of each company for this year. The anticipated closing date is in the third quarter of 2023, subject to specific closing conditions.
Source: Cadence
Samsung has announced this evening that they have completed development on their first generation of GDDR7 memory. The next iteration of the high bandwidth memory technology, which has been under industry-wide development, is expected to hit the market in 2024, with Samsung in prime position to be one of the first memory vendors out of the gate. With their first generation of GDDR7 parts slated to hit up to 32Gbps/pin of bandwidth – 33% more than their best GDDR6 parts today – the company is looking to deliver a sizable increase in GDDR memory bandwidth on the back of the technology’s adoption of PAM-3 signaling.
Samsung’s announcement comes as we’ve been seeing an increase in disclosures and announcements around the next iteration of the widely-used memory technology. While a finished specification for the memory has yet to be released by JEDEC, Samsung rival Micron has previously announced that it plans to introduce its own GDDR7 memory in 2024 – a similar timeline as to Samsung’s current schedule. Meanwhile, EDA tool firm Cadence disclosed a significant amount of technical details earlier this year as part of announcing their GDDR7 verification tools, revealing that the memory would use PAM-3 signaling and reach data rates of up to 36Gbps/pin.
With today’s announcement, Samsung has become the first of the major memory manufacturers to publicly announce that they’ve completed development of their first generation of GDDR7. And while the company tends to make these sorts of memory announcements relatively early in the bring-up process – well before memory is ready for commercial mass product – it’s none the less an important milestone in the development of GDDR7, as it means that memory and device manufacturers can begin validation work against functional hardware. As for Samsung itself, the announcement gives the Korean conglomerate a very visible opportunity to reinforce their claim of leadership within the GDDR memory industry.
Besides offering an update on the development process for GDDR7, Samsung’s announcement also provides some high-level technical details about the company’s first generation of GDDR7 – though “high-level” is the operative word as this is not by any means a technical deep dive.
| GPU Memory Math | |||||
| GDDR7 | GDDR6X | GDDR6 | |||
| B/W Per Pin | 32 Gbps (Projected) | 24 Gbps (Shipping) | 24 Gbps (Sampling) | ||
| Chip Density | 2 GB (16 Gb) | 2 GB (16 Gb) | 2 GB (16 Gb) | ||
| Total B/W (256-bit bus) | 1024 GB/sec | 768 GB/ssec | 768 GB/ssec | ||
| DRAM Voltage | 1.2 V | 1.35 V | 1.35 V | ||
| Data Rate | QDR | QDR | QDR | ||
| Signaling | PAM-3 | PAM-4 | NRZ (Binary) | ||
| Packaging | 266 FBGA | 180 FBGA | 180 FBGA | ||
According to Samsung’s announcement, they’re expecting to reach data rates as high as 32Gbps/pin. That’s 33% higher than the 24Gbps data rate the company’s top GDDR6 products can hit today. Samsung and Cadence have both previously disclosed that they expect GDDR7 memory to eventually hit 36Gbps/pin, though as with the development of GDDR6 – a full 50% faster than GDDR6 – this is likely going to take multiple generations of products.
Interestingly, this is starting much closer to projected limits of GDDR7 than we’ve seen in past generations of the memory technology. Whereas GDDR6 launched at 14Gbps and eventually scaled up to 24Gbps, Samsung wants to start at 32Gbps. At the same time, however, GDDR7 is going to be a smaller generational leap than we saw for GDDR6 or GDDR5; rather than doubling the signaling bandwidth of the memory technology over its predecessor, GDDR7 is only a 50% increase, owing to the switch from NRZ (2 state) signaling to PAM-3 (3 state) signaling.
It’s also worth noting that, at present, the fastest GDDR6 memory we see used video cards is only running at 20Gbps.Samsung’s own 24Gbps GDDR6, though announced just over a year ago, is still only “sampling” at this time. So the multitude of other GDDR6-using products notwithstanding, the effective jump in bandwidth for video cards in 2024/2025 could be more significant, depending on just what speed grades are available at the time.
As for capacity, Samsung’s first GDDR7 chips are 16Gb, matching the existing density of today’s top GDDR6(X) chips. So memory capacities on final products will not be significantly different from today’s products, assuming identical memory bus widths. DRAM density growth as a whole has been slowing over the years due to scaling issues, and GDDR7 will not be immune to that.
Samsung is also claiming that their GDDR7 technology offers a “20%-improvement in power efficiency versus existing 24Gbps GDDR6 DRAM,” though this is a broad claim where the devil is in the details. As power efficiency for DRAM is normally measured on a per-bit basis (picojoules-per-bit/pJpb), then our interpretation is that this is the figure Samsung is referencing in that claim.
Update (7/19, 3pm ET): Following a round of Q&A, Samsung has provided some additional technical details on their first-generation GDDR7 memory, including voltages (1.2v), how they are measuring energy efficiency, and the process node being used (D1z). The rest of the article has been updated accordingly.
At a high level, the good news is that Samsung’s GDDR7 is slated to deliver a tangible increase in energy efficiency. With a nominal voltage of 1.2v, GDDR7 requires less voltage to drive it than its predecessor. But with only a 20% overall improvement in energy efficiency for a memory technology that is delivering up to 33% more bandwidth, this means that the absolute power consumption of the memory is going up versus the previous generation. As Samsung’s energy efficiency figures are for GDDR7@32Gbps versus GDDR6@24Gbps, we're looking at around a 7% increase in total energy consumption. Which, thankfully, is not a huge increase in power consumption. But it is an increase none the less.
Broadly speaking, this is the same outcome as we saw with the introduction of GDDR6(X), where despite the energy efficiency gains there, overall power consumption increased from one generation to the next, as energy efficiency gains are not keeping pace with bandwidth demands. Not to say that any of this is unexpected, but it means that good cooling will be even more critical for GDDR7 memory.
But for clients with strict power/cooling needs, Samsung is also announcing that they will be making a low-voltage version of their GDDR7 memory available. This will be a 1.1v version of their GDDR7 chips, and while clockspeeds haven't been disclosed, we'd expect something closer to their GDDR6 clockspeeds. With current-generation GDDR6, high-end laptops are typically paired with low-voltage memory running no faster than 18Gbps, so low-voltage GDDR7 running at around 24Gbps would still represent a significant step forward.
Samsung has also confirmed that they're going to be using their D1z memory fab process for their first generation GDDR7. This is the same EUV-based process the company is using for their most recent GDDR6 – but curiously, it's older than the 12nm process Samsung revealed they'll be using elsewhere in a recent DDR5 memory announcement. The fact that the company isn't using a newer node here is a bit surprising, but from an architectural perspective, it also means that Samsung has achieved a 20% increase in energy efficiency without a newer node. Architecturally-driven energy gains are few and far between in both the memory and logic spaces these days, all of which makes for a promising sign for the future of GDDR7.
Speaking of architecture and design, the company notes that their GDDR7 memory employs “IC architecture optimization” to keep power and heat generation in check. Though at this point we don't have any further details on just what that means.
Electronics production aside, the final major innovation with Samsung’s GDDR7 will be decidedly physical: epoxy. Clearly mindful of the already high heat loads generated by existing GDDR memory clocked at its highest speeds, Samsung’s press release notes that they’re using a new epoxy molding compound (EMC) for GDDR7, which is designed to better transfer heat. All told, Samsung is claiming a 70% reduction in thermal resistance versus their GDDR6 memory, which should help ensure that a good cooler can still pull enough heat from the memory chips, despite the overall increase in heat generation.
Wrapping things up, now that initial development on their GDDR7 memory has been completed, Samsung is moving on to verification testing with their partners. According to the company, they’ll be working with key customers on verification this year; though at this point, the company isn’t saying when they expect to kick off mass production of the new memory.
Given the timing of Samsung’s announcement (as well as Micron’s), the initial market for GDDR7 seems to be AI and networking accelerators, rather than the video cards that GDDR7 gets its name from. With both AMD and NVIDIA barely a quarter of the way through their current architectural release cycles, neither company is likely be in a position to use GDDR7 in 2024 when it’s ready. Instead, it’s going to be the other users of GDDR memory such as networking products and high-performance accelerators that are likely to be first to use the technology.
Micron late on Wednesday revealed plans to introduce its first GDDR7 memory devices in the first half of 2024. The memory is expected to be used by next generation of graphics cards, and deliver performance that will be considerably higher than that of GDDR6 and GDDR6X.
"We plan to introduce our next-generation G7 product on our industry-leading 1ß node in the first half of calendar year 2024," said Sanjay Mehrotra, chief executive of Micron, as part of the company's earnings call.
Micron did not provide any additional specifics about its GDDR7 SGRAM devices that are set to be introduced in the first half of calendar 2024, though some general things about the technology have already been revealed by Cadence and Samsung in the recent months.
Samsung expects next generation GDDR to hit data transfer rates of 36 GT/s, though it's unclear whether they're talking about initial speeds for the new memory, some time later on. In any case, any increase over current 22 ~ 23 GT/s offered by GDDR6X will make the new type of memory more preferable for bandwidth-hungry devices like high-end graphics cards.
Meanwhile, Cadence has previously disclosed that GDDR7 will use PAM3 signaling, a three-level pulse amplitude modulation (which includes -1, 0, and +1 signaling levels), which allows it to transfer three bits of data over a span of two cycles. PAM3 provides a more efficient data transmission rate per cycle compared to two-level NRZ used by GDDR6, thereby reducing the need to upgrade to higher memory interface clocks and the subsequent signal loss challenges that this might cause. GDDR6X currently does something similar with PAM4 (four states), so GDDR7 will be a bit different still. PAM3 ultimately transmits a bit less data per clock edge (1.5 bits vs. 2 bits), but it trades off with less stringent signal-to-noise ratio requirements.
While Micron plans to introduce its first GDDR7 product in the first half of 2024, an official launch of a new memory device indicates conclusion of its development and not its immediate use in commercial products. As GDDR7 employs a new encoding mechanism, it requires brand-new new memory controllers, and hence, graphics processors. While it is reasonable to anticipate AMD, Intel, and NVIDIA to introduce their next generation GPUs in the 2024 – 2025 timeframe, the exact timing of these graphics processor releases remains completely unclear.
For now, Cadence has GDDR7 verification solution for chip designers that need to ensure that their controllers and PHY are compliant with the upcoming specification while they finalize design of their GPUs and other processors.
G.Skill has quietly started selling its 24 GB DDR5 memory modules with AMD EXPO profiles for single-click overclocking. G.Skill's Trident Z5 Neo RGB modules are among the first EXPO-profiled DIMMs larger than 16GB to hit the market, with G.Skill offering kits as large as 48GB (2 x 24GB).
G.Skill's 24 GB Trident Z5 Neo RGB memory modules with AMD EXPO profiles support are designed for a 6000 MT/s data transfer rate, which is considered to be a sweet spot for AMD's Ryzen 7000-series processors based on the Zen 4 microarchitecture. As for timings, the manufacturer recommends CL40 48-48-96 settings at 1.35V, which is a rather significant (22%) overvoltage for DDR5 memory.
Traditional for G.Skill's Trident-series modules for PC hardware enthusiasts and overclockers, 24 GB Trident Z5 Neo RGB DIMMs come equipped with aluminum heat spreaders and, as their name suggests, addressable RGB bars. Keeping in mind that these memory sticks are overvolted and also carry a power management IC and voltage regulating module onboard, these heat spreaders promise to come handy.
The key selling point of G.Skill's 24GB Trident Z5 Neo RGB modules and associated 48GB dual-channel kits is support for AMD's EXPO memory technology, which enables single-click overclocking profiles for modules rated to operate at beyond standard settings.
24GB DIMMs have been available in the market for a bit, but the first products were aimed at Intel systems, and shipped with XMP 3.0 profiles. Initial media reports have demonstrated that there have been some compatibility issues with these new 24GB XMP DIMMs and AMD systems, so having DIMMs that are formally tested for AMD platforms is a welcome step forward. Though most of the heavy lifting is coming from UEFI BIOS updates to account for the relatively novel, non-power-of-two organization of these new DIMMs.
G.Skill's Trident Z5 Neo RGB 48GB dual-channel (2 x 24GB) DDR5-6000 kit F5-6000J4048F24GX2-TZ5NR is now available from Newegg for $159.99.
At Computex 2023, TeamGroup unveiled two additions to its extensive family of memory products. Available with ARGB heatsinks or subtle black non-RGB heatsinks, the TeamGroup T-Force Xtreem DDR5 memory promises high performance for memory overclockers and gamers.
Looking to rival the high-speed DDR5 kits from companies such as G.Skill, the T-Force Xtreem DDR5 memory is available up to DDR5-8266, with overclocked XMP 3.0 profiles designed to be used with Intel's 13th Gen Core series processors. Regarding the competition, the T-Force Xtreme DDR5-8266 will be one of, if not the fastest, fastest kit available at retail upon its launch, as G.Skill has a kit of their Trident Z5 DDR5-8000 at retail.
The most glamorous looking of the two kits is the T-Force Xtreem ARGB DDR5 memory, with dual light pipes that sit between black translucent acrylic panels. Underneath the acrylic is a 2 mm thick aluminum alloy heatsink which has been anodized black. Between the memory ICs and the heatsink, TeamGroup claims they use a 'highly thermally conductive silicone gel' to provide optimal heat dissipation.
Despite TeamGroup not disclosing the height of the DRAM modules, they look pretty tall, so they might not be compatible with all air coolers, especially those large dual/triple designs. TeamGroup claims that the heat spreaders themselves have a texture similar to black sea sand, although the feel of memory is less important than other things, such as style and actual performance.
The second kit has the same overall shape and design as the ARGB variant but without the actual ARGB LED lighting. This gives the T-Force Xtreem DDR5 modules a classier look with a subtle matte black color scheme throughout, while one side of the heat spreader has a T-Force logo badge which reminds me of something Star Trek characters have on their uniforms.
Due to the speed of these kits, the T-Force Xtreem kits are primarily for Intel's current 13th Gen Core series processors, as AMD's Ryzen 7000 series just cannot hit the memory speeds that Intel's 13th Gen chips can. The TeamGroup T-Force Xtreem ARGB DDR5 and T-Force Xtreem DDR5 kits will come in kits ranging between DDR5-7000 and a blazingly fast DDR5-8266 kit.
As it stands, TeamGroup hasn't unveiled its expected pricing for the T-Force Xtreem ARGB and non-ARGB DDR5 memory kits, nor has it provided what memory ICs they are using or the latency timings. These are expected to be unveiled closer to their launch, which at the moment is unknown.
Lateo.net - Flux RSS en pagaille (pour en ajouter : @ moi) 