Intel Launches Cooper Lake: 3rd Generation Xeon Scalable for 4P/8P Servers
by Dr. Ian Cutress on June 18, 2020 9:00 AM ESTSocket, Silicon, and SKUs
Cooper Lake Xeon Scalable ushers in a new socket, given that it is difficult to add in UPI links without adding additional pins. The new socket is known as LGA4189, for which there will be two variants: LGA4189-4 and LGA4189-5. When asked, Intel stated that Cooper Lake supports the LGA4189-5 socket, however when we asked an OEM about the difference between the sockets, we were told it comes down to the PCIe version.
LGA4189-5, for Cooper Lake, uses PCIe 3.0. LGA4189-4, which is for Ice Lake we were told, will be PCIe 4.0 Nonetheless, Intel obfuscates the difference by calling both of them ‘Socket P+’. It’s not clear if they will be interchangeable, given that technically PCIe 4.0 can work in PCIe 3.0 mode, and a PCIe 3.0 chip can work in a PCIe 4.0 board at PCIe 3.0 speeds, but it will come down to how the UPI links are distributed, and any other differences.
We've since been told that the design of the socket is meant to make sure that Ice Lake Xeon processors should not be placed in Cooper Lake systems, however Cooper Lake processors will be enabled in systems built for Ice Lake.
We’re unsure if that means that LGA4189 / Socket P+ will be a single generation socket or not. Sapphire Rapids, mean to be the next generation Xeon Scalable, is also set for 2nd gen Optane support, which could imply a DDR4 arrangement. If Sapphire Rapids supports CXL, then that’s a PCIe 5.0 technology. There’s going to be a flurry of change within Intel’s Xeon ecosystem it seems.
On the silicon side, Intel has decided to not disclose the die configurations for Cooper Lake. In previous generations of Xeon and Xeon Scalable, Intel would happily publish that it used three different die sizes at the silicon level to separate up the core count distribution. For Cooper Lake however, we were told that ‘we are not disclosing this information’.
I quipped that this is a new level of secrecy from Intel.
Given that Cooper Lake will be offered in variants from 16 to 28 cores, and is built on Intel’s 14nm class process (14+++?), we can at least conclude there is a ’28 core XCC’ variant. Usually on these things the L3 cache counts are a good indicator of something smaller is going to be part of the manufacturing regime, however each processor sticks to the 1.375 MB of L3 cache per core configuration.
This leads us onto the actual processors being launched. Intel is only launching Platinum 8300, Gold 6300, and Gold 5300 versions of Cooper Lake, given that its distribution is limited to four socket systems or greater, and to high scale OEMs only. TDPs start at 150-165 W for the 16-24 core parts, moving up to 205-250 W for the 18-28 core parts. The power increases come from a combination of slight frequency bumps, higher memory speed support, and double the UPI links.
Intel 3rd Gen Xeon Scalable Cooper Lake 4P/8P |
||||||||||
AnandTech | Cores | Base Freq |
1T Turbo |
DDR4 1DPC |
DDR4 2DPC |
DDR4 TiB |
TDP W |
4P 8P |
Intel SST |
Price |
Xeon Platinum 8300 | ||||||||||
8380HL | 28C | 2900 | 4300 | 3200 | 2933 | 4.5 | 250 | 8P | No | $13012 |
8380H | 28C | 2900 | 4300 | 3200 | 2933 | 1.125 | 250 | 8P | No | $10009 |
8376HL | 28C | 2600 | 4300 | 3200 | 2933 | 4.5 | 205 | 8P | No | $11722 |
8376H | 28C | 2600 | 4300 | 3200 | 2933 | 1.12 | 205 | 8P | No | $8719 |
8354H | 18C | 3100 | 4300 | 3200 | 2933 | 1.12 | 205 | 8P | No | $3500 |
8353H | 18C | 2500 | 3800 | 3200 | 2933 | 1.12 | 150 | 8P | No | $3003 |
Xeon Gold 6300 | ||||||||||
6348H | 24C | 2300 | 4200 | - | 2933 | 1.12 | 165 | 4P | No | $2700 |
6328HL | 16C | 2800 | 4300 | - | 2933 | 4.5 | 165 | 4P | Yes | $4779 |
6328H | 16C | 2800 | 4300 | - | 2933 | 1.12 | 165 | 4P | Yes | $1776 |
Xeon Gold 5300 | ||||||||||
5320H | 20C | 2400 | 4200 | - | 2933 | 1.12 | 150 | 4P | Yes | $1555 |
5318H | 18C | 2500 | 3800 | - | 2933 | 1.12 | 150 | 4P | No | $1273 |
All CPUs have Hyperthreading |
Quite honestly, Intel's naming scheme is getting more difficult to follow. Every generation of Xeon Scalable becomes a tangled mess of feature separation.
No prices are attached to any of the Cooper Lake processors from our briefings, but Intel did publish them in its price document. We can compare the top SKUs from the previous generations, as well as against AMD's best.
Intel Xeon 8x80 Compare | ||||
Xeon 8180M |
Xeon 8280L |
Xeon 8380HL |
AnandTech | EPYC 7H12 |
Skylake | Cascade | Cooper | Platform | Rome |
14nm | 14+ nm | 14++ nm? | Node | 7nm + 14nm |
$13011 | $13012 | $13012 | Price | ~$8500 |
28 C | 28 C | 28 C | Cores | 64 C |
2500 MHz | 2700 MHz | 2900 MHz | Base | 2600 MHz |
3800 MHz | 4000 MHz | 4300 MHz | 1T Turbo | 3300 MHz |
6 x 2666 | 6 x 2933 | 6 x 3200 | DDR4 | 8 x 3200 |
1.5 TiB DDR4 | 4.5 TiB Optane | 4.5 TiB Optane | Max Mem | 4 TiB DDR4 |
205 W | 205 W | 250 W | TDP | 280 W |
1P to 8P | 1P to 8P | 1P to 8P | Sockets | 1P, 2P |
3 x 10.4 GT/s | 3 x 10.4 GT/s | 6 x 10.4 GT/s | UPI/IF | 64 x PCIe 4.0 |
3.0 x48 | 3.0 x48 | 3.0 x48 | PCIe | 4.0 x128 |
AVX-512 F/CD/BW/DQ |
AVX-512 F/CD/BW/DQ + VNNI |
AVX-512 F/CD/BW/DQ + VNNI +BF16 |
AVX | AVX2 |
The new processor improves on base frequency by +200 MHz and turbo frequency by +300 MHz, but it does have that extra 45 W TDP.
Compared to AMD’s Rome processors, the most obvious advantages to Intel are in frequency socket support, the range of vector extensions supported, and also memory capacity if we bundle in Optane. AMD’s wins are in has core counts, price, interconnect, PCIe count, and memory bandwidth. However, the design of Intel’s Cooper Lake with BF16 support is ultimately for customers who weren’t looking at AMD for those workloads.
We should also point out that these SKUs are the only ones Intel is making public. As explained in previous presentations, more than 50% of Intel's Xeon sales are actually custom versions of these, with different frequency / L3 cache / TDP variations that the big customers are prepared to pay for. In Intel's briefing, some of the performance numbers given by its customers are based on that silicon, e.g. 'Alibaba Customized SKU'. We never tend to hear about these, unfortunately.
Platform
As hinted above, Intel is still supporting PCIe 3.0 with Cooper Lake, with 48 lanes per CPU. The topology will also reuse Intel’s C620 series chipsets, providing 20 more lanes of PCIe 3.0 as well as USB 3.0 and SATA.
Intel did not go into items such as VROC support or improvements for this generation, so we expect support for those to be similar to Cascade Lake.
99 Comments
View All Comments
azfacea - Thursday, June 18, 2020 - link
are u suggesting these will compete with IBM z platform or something else on reliability? clearly this is not a reliability play. its commodity x86. and if max core count and max memory and max IO of 8s server does not beat a 4s EPYC, not sure what the selling point is, never mind charging a premium.unless there is particular order from like facebook for BFloat16 its not going anywhere. with a 2x perf disadvantage even that wont be enof for long.
SarahKerrigan - Thursday, June 18, 2020 - link
Not on reliability, just on scalability. 4s/8s x86 is largely replacing RISC/UNIX (*not* z, which is a separate animal.)As for 4s Epyc... you realize that Epyc only goes to 2s, right? If you want a really big tightly-bound x86 system, whether to replace RISC/UNIX or just because you have an interconnect-sensitive app that eats a lot of RAM, Intel goes higher than AMD. That's not a value judgment, it's a statement of fact. That's also an incredibly niche market and always has been - but it's one with good margins, which presumably is why Intel still bothers.
kc77 - Thursday, June 18, 2020 - link
No Eypc can scale further than that. Second, these chips top out at 28 cores. AMD is at a double density advantage (Actually it's worse) . Hell you have to go to 8S on these parts just to beat out the 2S AMD counter parts. The power and density lost is crazy. These are super niche parts. Aside from FaceBook I don't see anyone else getting these.Deicidium369 - Thursday, June 18, 2020 - link
And it seems to the people making the decisions about what goes into the Datacenter - AMD supposed "advantages" are meaningless. The 4 and 8 socket Cooper Lake is destined for hyperscalers.Zibi - Thursday, June 18, 2020 - link
Like Facebook OCP Delta Lake Cooper Lake perhaps ?Too bad it's 2S xD
Deicidium369 - Friday, June 19, 2020 - link
Cooper Lake is 4 and 8 sockets - designed for AI / Hyper scalersIce Lake SP is single and dual socket 38C and 64 PCIe4 lanes per socket.
Deicidium369 - Friday, June 19, 2020 - link
Ice Lake SP has 76 cores and offers 128 lanes of PCIe4 in a dual socket system - this is the mainstream platform - most servers in traditional data centers are 2 socket... makes for an efficient VM farm - better to have 2 dual socket than a single 4 socket. And with the significant IPC increase the Sunny Cove brought (~20%) makes the 76 cores in a dual socket config equivalent to 90 or 91 cores when compared to Skylake derived Comet and Cooper and by extension Epyc. So Epyc may have 128 cores in a dual socket config - that really is not a huge advantage anymore - and with the same # of PCIe4 lanes.. Epyc shows little advantage here.You would be hard pressed to find any motherboard that supports more the 2 Epyc CPUs. there is a poster on Reddit Optilasgar that explains why more than 2 sockets on Epyc are basically not possible
https://www.reddit.com/r/Amd/comments/6jogw9/are_t...
Yeah the 4/8 socket will go mostly to the hyper scalers - Facebook was one of the driving factors for Cooper Lake at 4 or 8 sockets - but you can bet they won't be the only hyper scalers getting them.
Apparently what you see as AMDs advantage isn't what the large customers - hyper scalers or traditional data centers want - revenue show that to be true.
Spunjji - Friday, June 19, 2020 - link
@Deicidum"the significant IPC increase the Sunny Cove brought (~20%) makes the 76 cores in a dual socket config equivalent to 90 or 91 cores when compared to Skylake derived Comet and Cooper and by extension Epyc."
What's this "by extension Epyc" nonsense? Everybody knows Epyc has better IPC than Skylake.
We don't know the clock speeds for Ice Lake SP either, but if it ends up anything like the mobile variants then the IPC increase will be eaten by the clock speed decrease.
Deicidium369 - Saturday, June 20, 2020 - link
Yeah server variants with 270W are going to have the same clocks as the 15W mobile variant,,,you are really grasping at straws. AMD Epyc are roughly comparable to Skylake derived cores - so Comet Lake and Cooper Lake are Skylake derived cores, and Epyc is trying to compete with Skylake - therefore - by extension.. means that Sunny Cove has a 20% IPC advantage over Skylake - which is Comet Lake, Cooper Lake, and AMD Epyc.
mtfbwy - Thursday, June 18, 2020 - link
Then why are the 'rate' numbers for SPEC CPU 2017 Dominated by EPYC? spots #1, #2, and #3 are all EPYC, with socket count of 16, 24, and 32.While the "glue" in this case is software instead of a hardware node-controller, it still makes for a scale-up server; the same technology is also used with Xeons for customers running workloads like SAP HANA - it makes for a far cheaper and more flexible architecture to scale up your memory.