Introduction to ARM Servers

"Intel does not have any competition whatsoever in the midrange and high-end (x86) server market". We came to that rather boring conclusion in our review of the Xeon E5-2600 v2. That date was September 2013.

At the same time, the number of announcements and press releases about ARM server SoCs based on the new ARMv8 ISA were almost uncountable. AppliedMicro was announcing their 64-bit ARMv8 X-Gene back in late 2011. Calxeda sent us a real ARM-based server at the end of 2012. Texas Instruments, Cavium, AMD, Broadcom, and Qualcomm announced that they would be challenging Intel in the server market with ARM SoCs. Today, the first retail products have finally appeared in the HP Moonshot server.

There has been no lack of muscular statements about the ARM Server SoCs. For example, Andrew Feldman, the founder of micro server pioneer Seamicro and the former head of the server department at AMD stated: "In the history of computers, smaller, lower-cost, and higher-volume CPUs have always won. ARM cores, with their low-power heritage in devices, should enable power-efficient server chips." One of the most infamous silicon valley insiders even went so far as to say, "ARM servers are currently steamroller-ing Intel in key high margin areas but for some reason the company is pretending they don’t exist."

Rest assured, we will not stop at opinions and press releases. As people started talking specifications, we really got interested. Let's see how the Cavium Thunder-X, AppliedMicro X-Gene, Broadcom Vulcan, and AMD Opteron A1100 compare to the current and future Intel Server chips. We are working hard to get all these contenders in our lab, and we are having some success, but it is too soon for a full blown shoot out.

Micro Servers and Scale-out Servers

Micro servers were the first the target of the ARM licensees. Typically, a discussion about Micro servers quickly turns into a wimpy versus brawny core debate. One of the reasons for that is that Seamicro, the inventor of the micro server, first entered the market with Atom CPUs. The second reason is that Calxeda, the pioneer of ARM based servers, had to work with the fact that the Cortex-A9 core was a wimpy core that could not deal with most server workloads. Wikipedia also associates micro servers with very low power SoCs: “Very low power and small size server based on System-on-Chip, typically centered around ARM processor”.

Micro servers are typically associated with low end servers that serve static HTML, cache web objects, and/or function as slow storage servers. It's true that you will not find a 150W high-end Xeon inside a micro server, but that does not mean that micro servers are defined by low power SoCs. In fact, the most successful micro servers are based on 15-45W Xeon E3s. Seamicro, the pioneer of micro servers, clearly indicated that there was little interest in the low power Atom based systems, but that sales spiked once they integrated Xeon E3s.

Currently micro servers are still a niche market. But micro servers are definitely not hype; they are here to stay, although we don't think they will be as dominant as rack servers or even blade servers in the near future. To understand why we would make such a bold statement, it is important to understand the real reason why micro servers exist.

Let us go back to the past decade (2005-2010). Virtualization was (and is) embraced as the best way to make enterprises with many heterogeneous applications running on underutilized servers more efficient. RAM capacity and core counts shot up. Networking and storage lagged but caught up – more or less – as flash storage, 10 Gbit Ethernet, and SRIOV became available. But the trend to notice was that virtualization made servers more I/O feature rich: the number and speed of network NICs and PCI-e expansion slots for storage increased quickly. Servers based on the Xeon E5 and Opterons have become "software defined datacenters in a box" with virtual switching and storage. The main driver for buying complex servers with high processor counts and more I/O devices is simple: professionals want the benefits that highly integrated virtualization software brings. Faster provisioning, high availability (HA), live migration (vMotion), disaster recovery (DR), keeping old services alive (running on Windows 2000 for example): virtualization made everything so much easier.

But what if you did not need those features because your application is spread among many servers and can take a few hardware outages? What if you do not need the complex hardware sharing features such as SRIOV and VT-d? The prime example is an application like Facebook, but quite a few smaller web farms are in a similar situation. If you do not need the features that come with enterprise virtualization software, you are just adding complexity and (consultancy/training) costs to your infrastructure.

Unfortunately, as always, the industry analysts came with unrealistic high predictions for the new micro server market: in 2016, they would be 10% of the market, no less than "a 50 fold jump"! The simple truth is that there is a lot of demand for "non-virtualized" servers, but they do not all have to be as dense and low power as the micro servers inside the Boston Viridis. The "very low power", extremely dense micro servers with their very low power SoCs are not a good match for most workloads out there, with the exception of some storage and memcached machines. But there is a much larger market for servers denser than the current rack servers, but less complex and cheaper than the current blade servers, and there's a demand for systems with a relatively strong SoC, currently the SoCs with a TDP in the 20W-80W range.

Not convinced? ARM and the ARM licensees are. The first thing that Lakshmi Mandyam, the director of ARM servers systems at ARM, emphasized when we talked to her is that ARM servers will be targeting scale-out servers, not just micro servers. The big difference is that micro servers are using (very) low power CPUs, while scale-out servers are just servers that can run lots and lots of threads in parallel.

The Current Intel Offerings
Comments Locked

78 Comments

View All Comments

  • jjj - Tuesday, December 16, 2014 - link

    If you look at phones and tabs ,we might be getting some rather big custom cores in 2015 and 2016. Apple and Nvidia already have that, ofc much smaller than Intel's core when adjusting for process (actually that's an assumption when it comes to Denver since don't think we've seen any die shots).
    Intel at the same time in consumer is pushing for more non-CPU/GPU compute units and low power and they might face a tough question about core size and even process (if they target low clocks, low power , or the opposite).Got to wonder if at some point they'll have to go for a big core just for server.Would make things even more interesting.
    Might not matter but Apple kinda has the perf for an ARM Macbook Air if they go quad. Not something worth doing for such low volume but doable when they go quad on all ipads or sooner if they launch a bigger ipad. Could be a trigger for others pushing more ARM based Chromebooks and beyond. That would set the stage for even bigger ARM cores.
    Also got the feeling Nintendo will go ARM in 2016 and not many reasons for Sony and M$ not to go that way if they ever make a new gen- just another market for bigger ARM cores, any significant revenue helps with dev costs so it matters.
  • CajunArson - Tuesday, December 16, 2014 - link

    1. The Core-m is widely derided as not being fast enough for the MacBook Air.
    2. The Core-m is easily twice as fast as the A8X in benchmarks that count... even Anandtech's own benchmarks show that. Furthermore, when you step away from web browsers and get to use the advanced features of the Core-m like AVX, that advantage jumps to about 8x faster in compute-heavy benchmarks like Linpack.
    3. Even the mythical A9 coming in 2015 is expected to have roughly a 20% performance boost over the A8x.
    4. Any real computer using an ARM chip would have to have a translation layer just like the old Rosetta to run the huge library of x86 software out there. Rosetta sort of worked because the Core 2 chips from Intel were *massively* faster than the PowerPC parts they replaced. Now you expect to run the translation overhead on an A9 chip that is slower -- by a large margin -- than the Core-m parts you've already derided as not being good enough?

    Yeah, I'm not holding my breath.
  • fjdulles - Tuesday, December 16, 2014 - link

    You may be right, but remember that ARM chips using the same power budget as Intel core i* will no doubt be clocked higher and perform that much better. Not sure if that will be competitive but it would be interesting to see.
  • wallysb01 - Tuesday, December 16, 2014 - link

    Only if you want a glorified tablet as a laptop. The software most people use in real work on laptops/desktops is not going to be ported over to ARM at an speed, even if ARMs could do that work reasonably well.
  • Kevin G - Wednesday, December 17, 2014 - link

    I'm under the impression that a good chunk has already been ported. MS Office for example is native ARM on Windows RT. Various Linux distributions have ARM ports completed with ARM based office and desktop software. The main thing missing are some big commercial applications like Photoshop etc.

    The server side of thing is similar with Linux and open software ports. MS is weirdly absent but I suspect that an ARM based version of Windows 2012/2014 is waiting of major hardware to be released. Much of the Windows base is already ported over to ARM due to Windows RT.
  • Kevin G - Wednesday, December 17, 2014 - link

    Indeed. Performance of ARM platforms once power constraints have been removed is a very open question. So far all the core designs in products have been used in mobile where SoC power consumption is less than 5 W. What a 100 W product would look is an open and very interesting question.
  • Ratman6161 - Wednesday, December 17, 2014 - link

    If they "use the same power budget as an Intel core i*" then what would be the point?
  • jjj - Tuesday, December 16, 2014 - link

    Ok you are focusing on the wrong thing but lets do that anyway.
    I have never claimed that Apple's own SoC would beat Intel's current SoCs, just that the perf would be enough if they go quad and obviously higher clocks.
    When you talk Core M you should remember that the price at launch was $281 so it's not good enough for anything.
    Anyway how about you compare a possible Apple SoC with a MacBook Air from 2011, lets face it the Air is a crap machine anyway , not much perf and TN panel for w/e ridiculous price it costs now and it's users are certainly not doing any heavy lifting with it.
    At the same time Apple's own 15- 20$ SoC would allow them a much cheaper machine and a presence in a price segment they never competed in, adding at least 5B of revenue per year (including cannibalization) and a share gain in PC of 2-3%.
    But then again the point was that there are a bunch of trends that could favor bigger ARM cores.
  • Morawka - Wednesday, December 17, 2014 - link

    it might cost them $20 for the A8X in fab cost, but the R&D for that chip is in the 10's of millions. Factor that in, to however many they ship, and it adds at least another $20 per chip
  • jospoortvliet - Wednesday, December 17, 2014 - link

    Even more obvious then that this would save them money by spreading out the fixed costs over more devices...

Log in

Don't have an account? Sign up now