About the Author
Michael Azoff
With over 17 years analyst experience, most recently at Ovum/ Informa, Michael Azoff joined Kisaco Research, the company behind the AI Hardware and Edge AI Summit series, in 2020 as Chief Analyst.
Eitan Michael Azoff, PhD, MSc, BEng.
HQ’d in Kisaco Research’s London office, Michael's current focus is launching Kisaco Research vendor product comparison reports with the new Kisaco Leadership Chart (KLC) analyst chart. The first KLC is also the first analyst chart in the AI chip industry, with 16 vendors having participated in the research.
In his career Michael worked at Rutherford Appleton Laboratory building simulators for electron and hole transport in semiconductors for UK national and European community research projects and published papers in learned journals. He then turned to building neural networks and created a startup selling his Prognostica Microsoft Excel add-in for time series forecasting, and wrote a book on the topic for publisher John Wiley & Sons in 1994.
Since 2003 Michael has worked as an IT industry analyst covering software engineering topics, from agile and DevOps, to application lifecycle management and cloud native computing. He started covering machine learning when deep learning emerged as the most recent wave of interest in AI and left his position as Distinguished Analyst at Ovum/Informa to join Kisaco Research and help build an analyst capability within the company.
My analyst coverage areas at KR Analysis
My first research project at KR was to create the first analyst comparison chart for AI chips. We invited AI chip producers to participate and were fortunate to have 16 vendors participate from across the globe: USA, UK, France, and China, and a mix of established players (Nvidia, Imagination, Intel, and Xilinx, to startups.
Our analysis showed that the market naturally fell into three areas of hot activity:
▪ Data centers and high-performance computing environments (HPC): here large boxes are installed and the aim is to achieve maximum performance for training and inferencing AI systems. The buyers are cloud hyperscalars, national research labs and agencies, and some large enterprises with big investments in AI.
▪ Small edge: the opposite end of the spectrum, building the smallest useful chip possible to sell as cheap as possible and embed in edge devices. AI is inferencing here.
▪ Automotive: an active industry in AI but highly regulated creating hurdles and technology adoption cadences that can be challenging for suppliers. AI is mainly inferencing here (for systems installed in vehicles).
We produced four Kisaco Leadership Charts out of this research.
We are also researching the machine learning (ML) software tools space, and our first report here is ML Lifecycle Solutions. The biggest challenge for enterprises is taking the research AI systems developed by their data scientist and deploying these into production at scale. Using a host of open source tools to achieve this is possible but time consuming to build and maintain, as well as prone to breakdown. This is why the ML lifecycle solution space exists.
Finally, in our first batch of KR Analysis reports we produced the KLC on engineering application lifecycle management (ALM) solutions. While ALM has been in existence as a distinct practice since KR Analysis and Michael Azoff introduction © Kisaco Research. All rights reserved. Unauthorized reproduction prohibited. 4 around 2003, it continues to evolve. We found the engineering and highly regulated industries relying on engineering and compliance oriented ALM to help manage risk and complexity.
-
Motivation
The current revolution or dramatic evolution in artificial intelligence (AI) we are witnessing was sparked by the arrival of hardware accelerators onto which deep learning neural networks were ported: training times that took months ran in days or hours on Nvidia GPUs. This gave rise to the explosion in AI hardware accelerator chips competing to take a share of the large and still growing accelerator market. Now a new form of optimization, that encompasses a host of features beyond and inclusive of acceleration, has appeared in the AI market, purely software based: meaning that they operate at the software level in the machine learning (ML) technology stack. Many of the AI software optimization (AISO) products have emerged from relatively recent startups. These products can optimize ML models that run on just central processing units (CPUs) or enhance performance on standard AI accelerators: graphics processing units (GPUs), field-programmable gate arrays (FPGAs), and digital signal processors (DSPs). AISO products also compete with the newer breed of AI chips (which we label as domain specific architectures, DSAs), making the whole AI field a lot more nuanced and competitive. For both enterprise users and product manufacturers there are now wider options in choosing the best combination of software and hardware for their AI applications and products requirements.
In this report we feature the leading players in the AISO market, compared side by side in our Kisaco Leadership Chart (KLC). We explain what this technology has to offer, reveal our analysis of the top players, and profile each of these vendors.
-
What you will learn
- We define and explain the new AI software optimization solution market.
- We explain where in the ML technology stack AISO products operate.
- We delve into the different types of optimization techniques, from standard operations to the advanced techniques available from the AISO vendors.
- The relevance of ML lifecycle management and how AISO vendors offer lifecycle functionality.
- We compare seven leading players in the AISO space side by side in our KLC assessment.
- We provide heatmaps to visually see what key features the KLC vendor products have to offer.
- We provide a profile on each of the KLC participating vendors together with three strengths and three weaknesses.
-
Contents
Kisaco Research View..............................................................................3
Motivation...............................................................................................3
Key findings.............................................................................................4
AI software optimization: market and technology landscape.................4
Defining AISO..........................................................................................4
AISO products operate across multiple layers of the ML stack...............5
Hardware layer..........................................................................................5
Runtime layer............................................................................................6
Model inference.......................................................................................8
Model development and training............................................................9
AutoML and vertical specific applications...............................................9
Hybrid optimization.................................................................................9
The AISO solution market is an emerging one........................................9
Market view..............................................................................................9
The role for ML lifecycle management...................................................10
Solution analysis: vendor comparisons...................................................11
Kisaco Leadership Chart on AI software optimization solutions 2021...11
AI software optimization solution vendor comparisons.........................11
The KLC chart for AI software optimization solutions............................13
Vendor analysis.......................................................................................15
Applied Brain Research, Kisaco evaluation: Leader................................15
Kisaco Assessment..................................................................................18
Codeplay.................................................................................................19
Deci AI, Kisaco evaluation: Leader...........................................................21
Kisaco Assessment...................................................................................25
Deep-AI....................................................................................................25
Deeplite, Kisaco evaluation: Leader.........................................................28
Kisaco Assessment....................................................................................32
Eta Compute, Kisaco evaluation: Emerging Player...................................33
Kisaco Assessment.....................................................................................35
Mipsology, Kisaco evaluation: Contender..................................................35
Kisaco Assessment......................................................................................39
OctoML, Kisaco evaluation: Contender......................................................39
Kisaco Assessment......................................................................................43
SigOpt, an Intel Company, Kisaco evaluation: Leader................................43
Kisaco Assessment......................................................................................46
Appendix.....................................................................................................47
Vendor solution selection............................................................................47
Inclusion criteria..........................................................................................47
Exclusion criteria.........................................................................................47
Methodology..............................................................................................47
Definition of the KLC..................................................................................48
Kisaco Research ratings..............................................................................48
Further reading...........................................................................................48
Acknowledgements....................................................................................49
Author.........................................................................................................49
Kisaco Research Analysis Network.............................................................49
Copyright notice and disclaimer.................................................................49
-
Figures
Figure 1: The ML technology stack and the optimizations possible at each level.
Figure 2: Comparing different bit precision formats.
Figure 3: Where Sycl and OpenCL sit in the technology stack.
Figure 4: Comparing number of supported Nvidia GPU operations by different interfaces.
Figure 5: Apache TVM: working at intermediate representation (IR) level (TVM in grey).
Figure 6: ML lifecycle in the ML tool eco-system (covering DataOps, AutoML, MLOps).
Figure 7: AISO features available in the participating vendor products.
Figure 8: Heat map analysis of participating vendor solution technical features.
Figure 9: Kisaco Leadership Chart on AI software optimization solutions 2021.
Figure 10: Kisaco Leadership Chart on AI software optimization 2021: ranking of vendors
Figure 11: ABR’s temporal dithering.
Figure 12: ABR’s LMU characteristics versus the well-known LSTM neural network model.
Figure 13: ABR LMU models for keyword spotting. Note x-axis goes from large to small models left to right.
Figure 14: An AI horizontal AI hardware organization.
Figure 15: Working on top of open industry standards.
Figure 16: The Deci Platform.
Figure 17: The Deci Platform: dashboard.
Figure 18: The Deci Platform Insights screen.
Figure 19: Deep-AI training and inference flow.
Figure 20: Deep-AI software stack.
Figure 21: Deeplite Edge AI solution.
Figure 22: Deeplite automated model optimization.
Figure 23: Deeplite Neutrino profiler: results readout.
Figure 24: Eta Compute Tensai platform.
Figure 25: Mipsology Zebra software stack.
Figure 26: Mipsology Zebra performance metric efficiency: comparisons.
Figure 27: Where TVM (in blue) sits in the ML technology stack.
Figure 28: AutoTVM overview.
Figure 29: Where SigOpt sits in the ML technology stack
Figure 30: SigOpt features for management of the training process.
-
FAQs
1. What is the KLC?
The Kisaco Leadership Chart (KLC) is KR Analysis’s take on the classis industry analyst chart in which vendor products are assessed and their scores plotted on a chart comprising four quadrants: Leader, Contender, Innovator, and Emerging Player. The x-axis represents strength of technical features, the y-axis the strength of market execution and strategy, and the size of plotted circle represents market revenue normalized to the strongest participating player in the research.
In researching the KLC we receive privileged information from a vendor. As explained in question 3, participating vendors are actively engaged in our research. Confidential privileged vendor information is not disclosed in our report but helps us assess vendors in our analysis.
2. What is the vendor selection process for a KLC project?
KR Analysis creates a shortlist of vendors to invite to the research project. The aim is to include the leading players as well as innovative smaller players, across startup and established vendors. KLC research can at best be representative of the market and is not designed to be exhaustive – in some markets the sheer number of players would make an exhaustive KLC unmanageable, in smaller markets we are still dependent on vendors agreeing to participate.
We do create KR Analysis Technology and Market Landscape reports in which we typically list the players in the markets with thumbnail profiles providing information such as company leadership, location, funding status, and main product(s) details. While we cannot guarantee exhaustiveness, the landscape report does aim to list the most important vendors and does not require vendor participation.
3. In a KLC what does participating entail for a vendor?
First of all, we do not charge vendors to participate in a KLC. Participating vendors need to be actively engaged in a KLC research project, this involves completing a comprehensive questionnaire, which we score and use as the basis for positioning the vendor in the report’s KLC. We also hold a deep dive briefing and engage in plenty of Q&A. Finally, we research publicly available material on the vendor and its product(s) to complete our final view of the vendor.
4. Why are some notable vendors missing from the report?
As explained in question 2, we do invite the leaders in a market segment we are researching, however not all such players agree to participate. As explained in question 3, participating involves active engagement and example reasons vendors offer for declining our invitation are, often ending with “...but please consider us next cycle of the report.”:
- We are in the midst of an event in which our relevant staff do not have the time to engage in your process.
- We are going through a major change in strategy or product re-architecture and the timing is not right for us to participate.
- We are about to have our IPO and this is not the right time to participate.
- We are about to launch our flagship product and the report timing is not right for us.
-
About the Author
Michael Azoff
Chief AnalystKisaco ResearchWith over 17 years analyst experience, most recently at Ovum/ Informa, Michael Azoff joined Kisaco Research, the company behind the AI Hardware and Edge AI Summit series, in 2020 as Chief Analyst.
Eitan Michael Azoff, PhD, MSc, BEng.
HQ’d in Kisaco Research’s London office, Michael's current focus is launching Kisaco Research vendor product comparison reports with the new Kisaco Leadership Chart (KLC) analyst chart. The first KLC is also the first analyst chart in the AI chip industry, with 16 vendors having participated in the research.
In his career Michael worked at Rutherford Appleton Laboratory building simulators for electron and hole transport in semiconductors for UK national and European community research projects and published papers in learned journals. He then turned to building neural networks and created a startup selling his Prognostica Microsoft Excel add-in for time series forecasting, and wrote a book on the topic for publisher John Wiley & Sons in 1994.
Since 2003 Michael has worked as an IT industry analyst covering software engineering topics, from agile and DevOps, to application lifecycle management and cloud native computing. He started covering machine learning when deep learning emerged as the most recent wave of interest in AI and left his position as Distinguished Analyst at Ovum/Informa to join Kisaco Research and help build an analyst capability within the company.
My analyst coverage areas at KR Analysis
My first research project at KR was to create the first analyst comparison chart for AI chips. We invited AI chip producers to participate and were fortunate to have 16 vendors participate from across the globe: USA, UK, France, and China, and a mix of established players (Nvidia, Imagination, Intel, and Xilinx, to startups.
Our analysis showed that the market naturally fell into three areas of hot activity:
▪ Data centers and high-performance computing environments (HPC): here large boxes are installed and the aim is to achieve maximum performance for training and inferencing AI systems. The buyers are cloud hyperscalars, national research labs and agencies, and some large enterprises with big investments in AI.
▪ Small edge: the opposite end of the spectrum, building the smallest useful chip possible to sell as cheap as possible and embed in edge devices. AI is inferencing here.
▪ Automotive: an active industry in AI but highly regulated creating hurdles and technology adoption cadences that can be challenging for suppliers. AI is mainly inferencing here (for systems installed in vehicles).
We produced four Kisaco Leadership Charts out of this research.
We are also researching the machine learning (ML) software tools space, and our first report here is ML Lifecycle Solutions. The biggest challenge for enterprises is taking the research AI systems developed by their data scientist and deploying these into production at scale. Using a host of open source tools to achieve this is possible but time consuming to build and maintain, as well as prone to breakdown. This is why the ML lifecycle solution space exists.
Finally, in our first batch of KR Analysis reports we produced the KLC on engineering application lifecycle management (ALM) solutions. While ALM has been in existence as a distinct practice since KR Analysis and Michael Azoff introduction © Kisaco Research. All rights reserved. Unauthorized reproduction prohibited. 4 around 2003, it continues to evolve. We found the engineering and highly regulated industries relying on engineering and compliance oriented ALM to help manage risk and complexity.
Request a sample
Please complete the form below to receive a sample of this report.
Purchase the report
Company-wide Use | $3,999 | Buy Online
VAT will be added for companies based in the UK. For further information or support in purchasing a report, please contact email: [email protected]