KLC AI Hardware Accelerators 2020-21 (Part 2): Data Centers and HPC, July 2020 | Kisaco Research

About the Author

Author:

Michael Azoff

Chief Analyst
Kisaco Research

With over 17 years analyst experience, most recently at Ovum/ Informa, Michael Azoff joined Kisaco Research, the company behind the AI Hardware and Edge AI Summit series, in 2020 as Chief Analyst. 

Eitan Michael Azoff, PhD, MSc, BEng.

HQ’d in Kisaco Research’s London office, Michael's current focus is launching Kisaco Research vendor product comparison reports with the new Kisaco Leadership Chart (KLC) analyst chart. The first KLC is also the first analyst chart in the AI chip industry, with 16 vendors having participated in the research.

In his career Michael worked at Rutherford Appleton Laboratory building simulators for electron and hole transport in semiconductors for UK national and European community research projects and published papers in learned journals. He then turned to building neural networks when KR Analysis and Michael Azoff introduction © Kisaco Research. All rights reserved. Unauthorized reproduction prohibited. 3 backpropagation was invented and created a startup selling his Prognostica Microsoft Excel add-in for time series forecasting and wrote a book on the topic for publisher John Wiley & Sons in 1994.

Since 2003 Michael has worked as an IT industry analyst covering software engineering topics, from agile and DevOps, to application lifecycle management and cloud native computing. He started covering machine learning when deep learning emerged as the most recent wave of interest in AI and left his position as Distinguished Analyst at Ovum/Informa to join Kisaco Research and help build an analyst capability within the company.

My analyst coverage areas at KR Analysis

My first research project at KR was to create the first analyst comparison chart for AI chips. We invited AI chip producers to participate and were fortunate to have 16 vendors participate from across the globe: USA, UK, France, and China, and a mix of established players (Nvidia, Imagination, Intel, and Xilinx, to startups.

Our analysis showed that the market naturally fell into three areas of hot activity:

▪ Data centers and high-performance computing environments (HPC): here large boxes are installed and the aim is to achieve maximum performance for training and inferencing AI systems. The buyers are cloud hyperscalars, national research labs and agencies, and some large enterprises with big investments in AI.

▪ Small edge: the opposite end of the spectrum, building the smallest useful chip possible to sell as cheap as possible and embed in edge devices. AI is inferencing here.

▪ Automotive: an active industry in AI but highly regulated creating hurdles and technology adoption cadences that can be challenging for suppliers. AI is mainly inferencing here (for systems installed in vehicles).

We produced four Kisaco Leadership Charts out of this research.

We are also researching the machine learning (ML) software tools space, and our first report here is ML Lifecycle Solutions. The biggest challenge for enterprises is taking the research AI systems developed by their data scientist and deploying these into production at scale. Using a host of open source tools to achieve this is possible but time consuming to build and maintain, as well as prone to breakdown. This is why the ML lifecycle solution space exists.

Finally, in our first batch of KR Analysis reports we produced the KLC on engineering application lifecycle management (ALM) solutions. While ALM has been in existence as a distinct practice since KR Analysis and Michael Azoff introduction © Kisaco Research. All rights reserved. Unauthorized reproduction prohibited. 4 around 2003, it continues to evolve. We found the engineering and highly regulated industries relying on engineering and compliance oriented ALM to help manage risk and complexity.

  • Motivation

    Artificial intelligence (AI) is out of the research laboratory and is in the realm of practical engineering applications. AI engineering today is largely about running machine learning (ML) models on digital computers, and these models are typically simulations of brain-inspired models such as neural networks, with deep learning (DL) being the most successful example today. With the plateauing out of CPU performance improvements and the end of Moore’s law, even with multi-core CPU machines, the community has turned to hardware accelerators to run their AI models.

    Following the success of general-purpose computing on graphical processing units (GPGPUs) in 2010, a market initiated by Nvidia and helping the company triple its size, the next decade has seen the market explode in investment funding into startups aiming to capture a share with innovative AI focused architectures. In our series of AI hardware accelerator vendor comparisons, we focus here on data centers (DCs) and the high-performance computing (HPC) community.

    In this report Kisaco Research provides two Kisaco Leadership Charts (KLCs) 2020-21: one for DC/HPC AI training and one for ED/HPC AI inferencing, with full profiles and assessments of all participating vendors.

  • What you will learn

    • The attributes of the DC & HPC markets and how they differ from the rest of the market.
    • Who are the key players in the DC & HPC market, with deep profiles of nine of our participating vendors competing in AI training and AI inferencing, including strengths and weaknesses?
    • Two analyst charts, the Kisaco Leadership Chart (KLC), on the participating vendors: one for AI training and one for AI inferencing. These are the first such charts ever produced in the AI chip market.
    • What are the attributes of AI chips in the DC/HPC space versus other use cases in the market? Variables such as energy consumption, cost, precision and more. 
    • The importance of software in overall AI system performance. The best model accuracy is not necessarily the model per chip combination with highest operations per second.
  • Contents

    Kisaco Research View. 2

    Motivation. 2

    Note: Definitions are important 2

    Key findings. 2

    Companion reports. 3

    Solution Analysis: Training and inferencing in the data center and for HPC applications. 3

    Market trends. 3

    Market size. 3

    Training in the DC. 4

    Inferencing in the DC. 4

    HPC community. 4

    AI accelerator characteristics. 5

    Precision. 5

    Operations (model complexity) 6

    AI accelerator memory. 6

    Impact of batch size. 7

    Power issues. 7

    Market segments. 8

    Solution analysis: vendor comparisons. 8

    Kisaco Leadership Chart on AI Hardware Accelerators 2020-21: Data Centers and HPC. 8

    Introduction. 8

    Data center and HPC: AI accelerator vendor comparisons. 10

    The KLC charts for AI hardware accelerators: DC and HPC. 11

    Edge and /automotive (a companion report) 14

    Vendor analysis. 15

    Cerebras, Kisaco evaluation: Innovator 15

    Kisaco Assessment 17

    Graphcore, Kisaco recommendation: Innovator 17

    Kisaco Assessment 20

    Habana Labs (an Intel company), Kisaco evaluation: Innovator 20

    Kisaco Assessment 22

    Imagination Technologies, Kisaco evaluation: Contender 23

    Kisaco Assessment 25

    LightOn, Kisaco recommendation: Emerging Player 26

    Kisaco Assessment 28

    Kalray, Kisaco recommendation: Innovator 29

    Kisaco Assessment 31

    Mythic, Kisaco evaluation: Innovator 32

    Kisaco Assessment 34

    Nvidia, Kisaco evaluation: Leader 35

    Kisaco Assessment 39

    Xilinx, Kisaco evaluation: Leader 40

    Kisaco Assessment 42

    Appendix. 43

    Vendor solution selection. 43

    Inclusion criteria. 43

    Exclusion criteria. 43

    Methodology. 43

    Definition of the KLC. 43

    Kisaco Research ratings. 44

    Further reading. 44

    Acknowledgements. 44

    Author 44

    Copyright notice and disclaimer 44

  • Figures

    Figure 1: Relating AI, hardware, and data science

    Figure 2: Nvidia GPU architecture history

    Figure 3: AI accelerator architecture runway

    Figure 4: AI application power characteristics and use cases

    Figure 5: AI applications and characteristics of AI accelerators

    Figure 6: Types of AI hardware accelerators

    Figure 7: Type A startup AI hardware accelerator players by geography

    Figure 8: Type B startup AI hardware accelerator players by geography

    Figure 9: Type A established AI hardware accelerator players by geography

    Figure 10: Type B established AI hardware accelerator players by geography

    Figure 11: Startups and established players by numbers and startup funding

    Appendix

    Figure 12: Deep Learning and how the research segments in AI relate to each other

    Figure 13: The current state-of-the-art in AI we call Machine Intelligence

    Figure 14: How AI researchers define AI

  • About the Author

    Author:

    Michael Azoff

    Chief Analyst
    Kisaco Research

    With over 17 years analyst experience, most recently at Ovum/ Informa, Michael Azoff joined Kisaco Research, the company behind the AI Hardware and Edge AI Summit series, in 2020 as Chief Analyst. 

    Eitan Michael Azoff, PhD, MSc, BEng.

    HQ’d in Kisaco Research’s London office, Michael's current focus is launching Kisaco Research vendor product comparison reports with the new Kisaco Leadership Chart (KLC) analyst chart. The first KLC is also the first analyst chart in the AI chip industry, with 16 vendors having participated in the research.

    In his career Michael worked at Rutherford Appleton Laboratory building simulators for electron and hole transport in semiconductors for UK national and European community research projects and published papers in learned journals. He then turned to building neural networks when KR Analysis and Michael Azoff introduction © Kisaco Research. All rights reserved. Unauthorized reproduction prohibited. 3 backpropagation was invented and created a startup selling his Prognostica Microsoft Excel add-in for time series forecasting and wrote a book on the topic for publisher John Wiley & Sons in 1994.

    Since 2003 Michael has worked as an IT industry analyst covering software engineering topics, from agile and DevOps, to application lifecycle management and cloud native computing. He started covering machine learning when deep learning emerged as the most recent wave of interest in AI and left his position as Distinguished Analyst at Ovum/Informa to join Kisaco Research and help build an analyst capability within the company.

    My analyst coverage areas at KR Analysis

    My first research project at KR was to create the first analyst comparison chart for AI chips. We invited AI chip producers to participate and were fortunate to have 16 vendors participate from across the globe: USA, UK, France, and China, and a mix of established players (Nvidia, Imagination, Intel, and Xilinx, to startups.

    Our analysis showed that the market naturally fell into three areas of hot activity:

    ▪ Data centers and high-performance computing environments (HPC): here large boxes are installed and the aim is to achieve maximum performance for training and inferencing AI systems. The buyers are cloud hyperscalars, national research labs and agencies, and some large enterprises with big investments in AI.

    ▪ Small edge: the opposite end of the spectrum, building the smallest useful chip possible to sell as cheap as possible and embed in edge devices. AI is inferencing here.

    ▪ Automotive: an active industry in AI but highly regulated creating hurdles and technology adoption cadences that can be challenging for suppliers. AI is mainly inferencing here (for systems installed in vehicles).

    We produced four Kisaco Leadership Charts out of this research.

    We are also researching the machine learning (ML) software tools space, and our first report here is ML Lifecycle Solutions. The biggest challenge for enterprises is taking the research AI systems developed by their data scientist and deploying these into production at scale. Using a host of open source tools to achieve this is possible but time consuming to build and maintain, as well as prone to breakdown. This is why the ML lifecycle solution space exists.

    Finally, in our first batch of KR Analysis reports we produced the KLC on engineering application lifecycle management (ALM) solutions. While ALM has been in existence as a distinct practice since KR Analysis and Michael Azoff introduction © Kisaco Research. All rights reserved. Unauthorized reproduction prohibited. 4 around 2003, it continues to evolve. We found the engineering and highly regulated industries relying on engineering and compliance oriented ALM to help manage risk and complexity.

Purchase the report

Single User | $4,999 | Buy Online

Company-wide Use (Private Company) |  $5,999 | Buy Online

Company-wide Use (Listed Company) | $9,999 | Buy Online

VAT will be added for companies based in the UK. For further information or support in purchasing a report, please email: [email protected]


More Reports