Premise

This research investigates “Matrix Workloads,” a workload type characterized by being able to process very large amounts of data in real-time. Examples of Matrix workloads include real-time systems of intelligence, real-time analytics, AI inferencing, robotics, autonomous vehicles, and other data-driven real-time or near real-time use cases. In this research we put forth and defend three key themes:

  • Matrix workloads can help deploy more aggressive end-to-end data strategies, which together can greatly simplify and automate operational processes.
  • The value of well-designed Matrix workloads together with an end-to-end data strategy can deliver an order of magnitude greater value than the traditional applications they enhance.
  • New hardware and software architectures, techniques and tools are required to develop and deploy matrix-based workloads in the enterprise. These technologies are likely to derive from consumer technologies.

Wikibon projects that Enterprise Matrix Workloads will be 42% of Compute revenue by the end of this decade.

Executive Summary

Matrix Workload Technology & Capability

We believe real-time Matrix workloads will be a major contributor to the implementation of digital innovation and differentiation over the next decade. In addition, the following four points are noteworthy in terms of understanding this emerging type of workload:

  1. Real-time inference Matrix workloads are already being deployed in large numbers in the volume consumer space. Examples include facial recognition, digital photography, voice recognition and enhancement, and health monitoring.
  2. These technologies can enable breakthrough performance improvements relative to traditional enterprise workloads by factors of 10 to 1,000.
  3. These workloads are data-intensive and will require enterprises to establish an end-to-end data strategy.
  4. Disruptors can use expertise in Matrix workloads together with establishing an end-to-end data strategy to radically simplify operational processing and improve speed to deploy innovation.
  5. Wikibon projects that the growth of Enterprise Matrix workloads will be very strong over the second part of the decade, and will represent about 42% of enterprise compute spend by the end of the decade.

We believe artificial intelligence (AI) is an important framework to develop real-time inferencing, which is a Matrix workload. However, other technologies such as advanced analytics, Bayesian networks, Causal inferencing, and others will also be used either together with or separately to develop real-time inference code and other Matrix workloads. 

Continuing Wikibon Research into Matrix Workloads

This research is the first in a series of that will define Matrix workloads, the hardware, and software requirements to make these workloads possible to execute in real-time, the compute types that will be necessary to deliver Matrix workloads, and the platforms, architectures, and vendors likely to drive enterprise Matrix workloads.

* I chose the term “Matrix workloads” in memory of moving from scalar to matrix operations in statistics before the electronic calculator, and in the fear and joy of tensor ranks. And of course the iconic Matrix movies, with the latest The Matrix 4 now being filmed in San Francisco. I recently learned that David Moschella also uses the term in his book “Seeing Digital”. In it, he describes moving from a cloud of services somewhere out there to a ubiquitous matrix of intelligent capabilities, a compelling viewpoint.

Matrix Workloads 101

Matrix Workloads and Artificial Intelligence (AI)

“Matrix” workloads are primarily data-driven. The volume of data is very high and is often parallel in nature, i.e., sound, images, video, radar, ultrasound, and any number of IoT devices. AI is an important technology that can assist in developing Matrix workloads.

Figure 1 below shows AI development in blue on the left-hand side and AI execution in green on the right-hand side. AI development is often split into data engineering, and statistical modeling and training, as shown in Figure 1. Both are storage bound batch workloads where I/O is normally the bottleneck. NVMe over Fabric NAND flash storage can offload IO overheads and allows direct efficient connection to multiple large data sources at a compute location. 

Companies such as Pure, in particular, have developed innovative private cloud hardware platforms optimized for preparation and training, and Dell, HPE & IBM also have specialized integrated systems. As a development workload, demand for infrastructure resources is very intermittent and well suited to as-a-service cloud offerings.

Wikibon also points out that machine learning needs relevant quality data to be effective. Quantity without quality or relevance results in “garbage in, garbage out.”

Bottom Line: AI development workloads require access to quality and relevant data, and the end-to-end data architecture must ensure this is the case. However, AI development workloads are not real-time Matrix workloads.

Relationship between Matrix Workloads and AI
Figure 1: Relationship between AI and Matrix Workloads
Source: © Wikibon 2020

Real-Time AI Inferencing is a Matrix Workloads

Inference workloads are execution workloads, and the characteristics are shown on the right-hand side of Figure 1 above in green. Inferencing is usually compute-bound and needs to be close to the data sources to improve execution times. The inferencing bottleneck is usually memory bandwidth. The memory is often SRAM heavy to improve in-memory compute workflows. The compute hardware is heterogeneous (see “Heterogeneous Compute Architecture” section below). This allows a much higher degree of parallelism and speed to completion. Heterogeneous systems increasingly deploy neural networks and are optimized for inferencing.

The conclusions are far-reaching. Real-time inferencing is a matrix workload. Inference is by far the most compute-intensive component of AI. AWS points out that over 95% of compute for Alexa is used for inference compute. Wikibon projects that percentage will rise to 99% by the end of the decade. AI and ML are an important source of inferencing, but not the only source (see in the Premise section above for additional inference sources). One of the requirements of an end-to-end data strategy is to ensure the availability of good quality and relevant data for developing inferencing.

Matrix Workloads Moves to the Edge

Real-time also means that compute has to be placed very close to where the data is created. There is no time to transfer data even a few miles. The cost of moving data and the reduction of data quality from loss of context are both factors that will result in a significant portion of compute moving to the Edge over the next decade. Earlier research from Wikibon shows the cost case for moving compute to the Edge is overwhelming.

Heterogeneous Compute Architecture for Matrix Workloads

Matrix workloads have large amounts of matrix data to compute. Traditional processors cannot compute this in real-time. However, deploying a number of parallel accelerators with different limited instruction sets can enable completion of the workload in real-time. 

This means that specialized architectures are required for real-time Matrix workloads. These are heterogeneous compute architectures

Heterogeneous compute architectures are composed of many different compute elements (processors or cores), with different instruction sets. These elements can include general-purpose processors, GPUs, NPUs, ASICs, FPGAs, and others. These elements are usually combined into an SoC (System on a Chip), such as the consumer Arm-based Apple iPhone SoCs. 

The major advantages are radically improved performance and lower electrical power requirements. The major design challenges for heterogeneous systems include managing access to memory and data and programming complexity. The CXL industry standard is emerging as a possible standard high-speed CPU interconnect that will accelerate solutions. 

In conclusion, heterogeneous compute allows a much higher degree of parallelism and speed to completion and is very well suited for Matrix workloads. Wikibon expects to see a rapid increase in enterprise heterogeneous compute over the next decade. Wikibon’s planned research into heterogeneous compute is discussed in the section titled “Upcoming Matrix Workload Research” below.

Innovation Happens First in Volume Consumer Technologies

New technologies are usually first adopted in the volume consumer space and are then adopted by enterprises a few years later. A good example is the x86 technology, which was used in the consumer PC duopoly dominated by Intel x86 hardware and Microsoft Windows software. The x86 volumes, lower costs, and advanced technology allowed Intel to expand into enterprise servers, and replace the majority of RISC servers over a decade. x86 now dominates the compute market and a now shrinking PC market.

The introduction of the Apple smartphone in 2007 started to move expenditure to mobile devices. The sheer volume of and investment in consumer devices now drives the majority of hardware and software innovation. Almost all mobile technology is implemented in Arm-based systems, and the number of Arm-based wafers created is now 10 times greater than any other platform. This innovation is coming from companies such as Apple, Arm, Google, Nvidia, Qualcomm, Samsung, and many other consumer hardware and software vendors.

Consumer Matrix workloads are in full swing on consumer mobile platforms. Apple uses facial recognition and neural cores for securing financial transactions. Google deploys a neural chip to improve image and voice processing in the Pixel 4 smartphone. Both are using neural networks to enable real-time photography and video enhancement, improved voice recognition, health monitoring, and many other areas. Both emphasize the importance of neural networks and place much less emphasis on GPUs. 

This has rapidly spread to mobile app developers, especially gaming apps. Large numbers of consumer developers are training themselves on the techniques and technologies required. This source of developers will be an important source of talent as enterprises start to adopt real-time Matrix workloads. 

Bottom line: the technologies and software used for the large and rapidly expanding Matrix workloads in the consumer space are very likely to be deployed in the enterprise space, as well as the experts.

Case Study I – Tesla Autonomous Edge Real-time Matrix Workloads

Wikibon believes that real-time Matrix workloads will be the foundation for the majority of high-value and effective enterprise digital initiatives. The business Edge will be an important early implementation point for IoT initiatives. Because of MEMS**, sensors have plummeted in price, and a huge amount of IoT data is available at the Edge. Real-time Matrix workloads will enable major simplification and automation of workflows at the Edge.

A good example of the successful deployment of an enterprise Matrix workload is Tesla, which is developing an Automated Driving System (ADS). The business objective of ASD is to improve the function and margins of their electric vehicles and to open up new business opportunities. This example illustrates the software, hardware, and end-to-end data implications for a real enterprise running real-time Matrix workloads at the Edge.

** MEMS (Micro-Electro-Mechanical Systems) are miniaturized mechanical and electro-mechanical devices made in silicon that use many of the silicon chip technologies to reduce cost dramatically over time. These microsensors can convert the mechanical signal into a digital signal and are now usually more accurate than the original macro-sensors they replace. A mobile phone includes an ever-increasing number of sensors measuring acceleration, compass bearing, barometric pressure, color, GPS position, proximity, and others. The BOM price for all these sensors is about $1.50.

Tesla – The Dream

The reaction time, reliability and accuracy of ASD are potentially far better than any human driver. Primary life-death decisions are made from data and Matrix technologies. In the long-run ASD will radically reduce the 35,000 people killed in vehicle accidents every year in the US, as well as the many more accident-related severe injuries. The NHTSA study assesses the cost of road accidents is about $600 billion every year. They state “94% of serious crashes are due to human error. Automated vehicles have the potential to remove human error from the crash equation, which will help protect drivers and passengers, as well as bicyclists and pedestrians.” The NHTSA also states that the age of automated driving will start in 2025.

Tesla is on a journey to ASD and hopes to be first across the line. Tesla electric vehicles have independent electric engines. The brakes are independent and regenerative. There is feedback from all the wheels in relation to the ground as a result of actions from the electric engines and brakes. The low-latency electronic components are integrated with precise knowledge of the position of the vehicle, together with knowledge of the vehicle’s internal data and capabilities. This is an internal product end-to-end data architecture and creates the potential for a faster-acting and much safer drive. 

What is also needed for ASD is a way to integrate all the internal sensor information with external sensor data from the vehicle and create an action plan of where and how to drive the vehicle. The final piece of the puzzle is how to assess the quality of the plans, and improve them over time. Key challenges are to understand the data requirements, and how to set up an end-to-end data strategy that will create an improvement feedback loop at an affordable cost.

Tesla Real-time Matrix Workload Problem

The first problem to solve for the Tesla Matrix workload is how to process 1 billion pixels/second coming from the 8 cameras running at 60 full-frames/second in real-time. In addition, there is input from radar, 12 ultrasonic sensors, high-precision GPS, maps, and all the internal vehicle sensors such as wheel ticks, steering angle, acceleration, etc. A system needs to process these inputs in real-time, and produce a plan of action. Together, all this data drives all the driver-assist functions such as autopilot, all updates to autopilot and other functions, and in the future ASD.

Tesla refers to this system as the Tesla HW3 (Hardware-level 3). Some of the constraints (such as electrical power) will not be important for many enterprises. However, we believe the end-to-end architectures and data management decisions made by Tesla will reflect the type of design that will be important to most enterprises.

The journey to Tesla HW3

The Tesla software philosophy is that all the data should be processed by a single system, which theoretically provides the best quality of outcome and fastest response. The original Tesla compute hardware uses MobileEye EyeQ technology, but this is too slow. The Tesla HW2 (Hardware-level 2) uses Nvidia Drive hardware and uses GPUs to process the matrix data. Tesla engineers, like the aforementioned Apple & Google engineers, understand that neural networks are much faster than GPUs to process higher levels of data abstraction.

At the time (2016) there were no other suitable off-the-shelf hardware or software solutions available. As a result, Tesla invested in developing its own hardware solution, the HW3, to drive its own software. Figure 2 below shows the HW3 board. There are two heterogeneous Arm-based SoCs (System on a Chip) in the middle of this board (green and blue squares) for full redundancy. Inside the SoCs, the CPUs and GPU are Arm-based components, together with a native Tesla-designed NPU. The compute power of both NPUs is a total of 72 TOPS (Trillion Operations Per Second) @ 2 GHz. The inputs from the sensors are on the right of the board, up to 2.5 billion pixels/second. The power supplies are on the left. 

This board is installed at the back of the Tesla glove box, as a replacement for earlier boards and for all new vehicles. The total power requirement is only 75 Watts. A key performance metric for the HW3 is the number of HD frames/second that can be processed. The HW3 delivers 2,300 frames/second with NPU hardware running the Tesla software. The same software on HW2 hardware delivers 110 frames/second, less than 5% of the HW3 throughput. This is insufficient for ASD. 

For Tesla, this enterprise Matrix workload is core to its mission to produce driving and safety enhancements, and eventually ASD. Tesla believes that ASD can be achieved on the HW3. Over time, it is likely that the amount of data from the sensors will increase, and the technologies used should allow for additional increases in performance. Tesla is planning to update its technology with 2-3 times more performance by 2022.

Data Strategy for Tesla HW3

The second problem to solve is how to set up an end-to-end data strategy that will create an improvement feedback loop at an affordable cost. There are some who believe that all the data from every car should and will be kept. Some have argued that 5G will whisk away all this data for free. 

Of course, this data is important at the very early stages of development. However, some very simple math by the reader on the billion pixels/second problem will show the fallacy in this thinking when applied to millions of cars and billions of miles over the whole world. In addition, that billion pixels/second will grow into many more billions/second over time.

Wikibon research has pointed out that the vast majority (>99%) of data created at the Edge will be deleted at the Edge because the value has already been extracted from it. Tesla’s approach underlines this projection. The HW3 keeps the data in SRAM for just 10 minutes. During that time, a very small proportion of the data is selected to be sent over the network. 

Tesla HW3 Board
Figure 2: Tesla HW3 Board
Source: © Tesla 2019 Tesla Autonomy Day 2019

The data selected is to complete the feedback loop from events such as accidents, near misses, pavement anomalies (e.g., pot-holes), and internal and external compliance data. In addition, data can be requested from the entire vehicle fleet by engineers working on solutions for specific and rare environments. Examples are how human drivers deal with driving conditions such as snow, sunsets or large animals in the road. This is an innovative and cost-effective solution to the serious “long-tail” problem, discussed in the next section.

“Long-tail” Problem

There is a very “long-tail” of very low probability events in the overall learning process for ASD. Access to this data, the precise data about the vehicle components, plus the reaction of human drivers not using or overriding automation from across a fleet of vehicles is essential for developing recognition of rare events and training the ability for the vehicle to react correctly. 

The Tesla “long-tail” solution allows rare events to be identified, and the cliplogs data about these events back to the development system. If Tesla can increase the number of cars on the road from 1 million to many millions, and the miles tracked over years to billions, the end-to-end data capture and improvement system should deliver exponentially quicker improvements to the overall quality of the ASD system. The end-to-end data architects had to understand and build-in this end-to-end capability at the initial design stage.

This elegant end-to-end data architecture helps solve the long-tail problem and significantly reduces the time and cost to develop full ASD. 

MobileEye and Waymo are two other companies that are developing different Matrix technologies and are building up a fleet of vehicles.

Tesla Conclusions 

The Tesla approach has many positive attributes. These include clear ownership and access to all the data from all Tesla cars, a clear end-to-end software and data architecture, and an ability to focus data capture on events that matter. Tesla has designed low-latency electric vehicles with independent electrical engines/brakes integrated and with precise knowledge of the vehicle’s design and internal data. These can react much faster, much more effectively and much more safely than traditional ICE vehicles. If Tesla can link this internal data to a reliable plan generator, the result is probably an order of magnitude safer vehicles, with much better road utilization. If Tesla can execute and increase its fleet of cars on the road to millions and the miles driven to billions, Tesla could create an end-to-end data feedback and improvement loop unique in the transport industry and potentially save millions of lives worldwide.

If successful, Tesla can also use this data architecture for many adjacent business opportunities, including high-level feedback on individual driver safety as an input into the cost of a Tesla-branded insurance policy. Other opportunities include maintenance, leasing, reporting pot-holes and equipment malfunctions to towns and cities, infotainment, shipping, and journey end-points. 

On the negative side, the Tesla case illustrates some of the challenges inherent in developing an early Matrix workload solution, including the level of investment required to develop and maintain unique software and hardware. If competitors such as Mobileye and Waymo become volume suppliers, Tesla’s technical overheads may lead to slower adoption of innovation from others in the industry. Other volume suppliers could make it harder for Tesla to convince compliance regulators that the Tesla “long-tail” data solution will work as well as alternatives.

Bottom Line: there are a lot of “big ifs” for Tesla and incumbents – and very high stakes for all. 

Other Examples of Edge Disruption

Whatever the outcome of Tesla’s attempt to disrupt the car industry, Wikibon believes that this type of end-to-end architectural thinking about Matrix applications and data will drive digital innovation and differentiation in many other industries.

There are many real-time use cases at the business Edge, including robotics, optimization of supply chains, monitoring of wind-farms, automation for civilian and military aircraft, automated warehouses, automated stores, and numerous other examples from every industry. In addition, many cross-industry applications such as advanced analytics will profit from Matrix application techniques.

Many of these workloads will include at least some primary life-death decision components. Automation of a warehouse can bring potentially fatal interaction between large machines and warehouse staff or external deliverers. Automation of manufacturing with robots brings fast-moving machines in close proximity with operators with potentially fatal consequences. Automated safety systems on oil-rigs must include the safety of rig workers who cannot escape easily. This makes internal and external compliance particularly important for developers.

Amazon, for example, is developing this type of technology to completely automate retail in Amazon Go stores. Every enterprise should evaluate its current processes, determine how much more efficient an end-to-end Matrix workload architecture could be and determine how to create end-to-end data architecture to support these workloads.

Bottom Line: Wikibon believes that mastery of real-time Matrix workloads and end-to-end data architectures are absolutely essential in creating digital differentiation. Self-disruption is the best form of protection for enterprises against external disruption threats.

Case Study II – Systems of Intelligence & Real-time Pricing

Introduction

So, how should enterprises think about incorporating Matrix technologies into their workload portfolio? Enterprises who successfully implement Matrix workloads with an end-to-end data architecture can address three strategically very important opportunities.

  1. They can expect to generate an order of magnitude more additional application value, compared to organizations who continue with the current serialized transaction systems. 
  2. They can create protection against external disruption from low-cost alternatives, and expand the options available to identify and protect against external disruption.
  3. They are better placed to expand into adjacent industry areas and be a potential source of disruption.

The core of the change is to migrate from the many current systems of record and data warehouse systems to “Systems of Intelligence”. Wikibon has discussed and defined Systems of Intelligence in previous research. In a nutshell, Systems of Intelligence are systems of record integrated with Matrix workloads such as real-time advanced business analytics and/or AI.

We will take one narrow example to estimate potential benefits. Currently, most enterprises address pricing changes with a series of steps by professionals. They eventually agree on the new pricing over weeks (or months), and then IT implements those changes at a convenient quiet time. A section below entitled “The Business Case for Systems of Intelligence and Real-time Pricing” discusses the potential benefits of a real-time Matrix workload which enables automation of price adjustments compared to the current slower and more labor-intensive manual system.

Traditional Transaction Systems of Record vs. Matrix Workloads

Currently, the systems of record can rarely perform all the workflows in a single transaction. There are multiple serialized transactions, and complex workflows to manage them. 

Table 1 below shows the differences between traditional enterprise transactional workloads compared with updated systems of record integrated with enterprise Matrix workloads. This can create a unique solution that can radically simplify workflows and can provide very high levels of automation.

In these Matrix workloads, Table 1 shows the users are likely to include machines and other applications, the control is through the work-flow and APIs, and the data is likely to be distributed, highly parallel and in very high volume. Matrix workloads are most likely to be executed where the data is created, which will often be at a business Edge. There will also be Matrix workloads that are best executed centrally. 

The focus of Matrix workloads is to perform a number of transactions in parallel in real-time or near real-time. This will mean much larger and distributed databases, and a compute model that will be cloud-first, hyperconverged, with (in the future) heterogeneous compute architectures (see discussion above entitled “Heterogeneous Compute Architecture”).

Table 1 indicates that enterprise Matrix workloads will drive significant changes in workload hardware and software architecture, as well as data performance and data management improvements. 

Matrix vs. Traditional Workloads
Table 1: Enterprise Matrix Workload vs. Traditional Transactional Workload Characteristics
Source: © Wikibon 2020

Bottom Line: the benefit of Matrix workloads is a simplification of processes and much higher levels of automation.

Avoiding Conversions of Systems of Record

Some vendors and professionals argue that the current Systems of Record and analytic systems should all be converted to a “modern” platform. This may be possible for small companies or very small systems. However, converting large scale systems of record and analytic systems will bring years of delay and a very high risk of failure. Wikibon strongly believes that building on the application value and data value of existing systems of record is quicker, less expensive, and far less risky. Wikibon recommends that CXOs reject any digital transformation & differentiation strategy that requires the conversion of existing systems of record.

The Business Case for Systems of Intelligence and Real-time Re-pricing

Industries such as airlines and hotels use frequent repricing to optimize revenue. Some airlines like Norwegian use real-time dynamic pricing. 

A good example of this real-time pricing in another industry is the dynamic pricing of ride-sharing, introduced by Uber. If there is a low supply of drivers, and prices remain fixed, customer wait-time for rides increases. Customer satisfaction decreases significantly. When Uber raises prices in real-time if demand is higher than driver supply, rising prices reduce demand and supply is increased. Service levels remained reasonable, and customer satisfaction is significantly improved because the time to pick up and reliability of arrival at the destination is much more important for those who choose to pay more. An analysis of the total Uber system shows that the overall benefit of real-time pricing is about 3.5% of gross revenue.

Another extension of automated pricing might be to individualize prices according to the ease of doing business with a customer. If a customer has a higher probability of returning goods or of making unfavorable social media comments, it might make sense to reduce or eliminate discounts given to these customers. 

This real-time pricing example requires access to large amounts of information on all aspects of supply and demand, sophisticated models that analyze this data in real-time, and the ability to apply those changes to the existing systems of record. The potential business benefits are improved revenues, customer satisfaction, and lower costs. 

A starting point for calculating the enterprise benefit of real-time pricing would be an increase of 3.5% in gross revenue (from the Uber research above). If all the data is available and the pricing can be automated, a reasonable estimate might be a factor of 5 reductions in the manpower required to operate pricing. 

Bottom line: This case study does not conclude that all enterprises should develop real-time pricing as a priority. This case study is just one possible Matrix Inference workload project. However, just starting the process of looking at where Matrix workloads that could increase revenue by a few percents, might improve customer satisfaction, and could reduce operational costs by a factor of 5 will drive innovation. At best this provides sustainable differentiation and at least ensures competitiveness.

Enterprise Matrix Workload Implementation Strategies

Enterprises have a wide variety of software and data assets deployed to assist operational and planning processes, with significant investments across the enterprise in multiple locations and platforms. Wikibon believes that a strategy that attempts to put all the data on one platform in one place will fail for almost all enterprises – most especially, large enterprises. As discussed above, the cost of conversion is crippling and will take many years to achieve. 

Instead, Wikibon recommends that enterprises focus on establishing an effective end-to-end data strategy, starting with where data is created today and where it will reside in the future. Enterprises should then implement efficient multi, hybrid and adjacent cloud strategies to ensure:

  • The right data is in the right place at the right time.
  • The movement of data is minimized by pushing compute to data where possible.
  • The responsibility for deployment and maintenance of infrastructure services is moved to software and hardware vendors and cloud providers, and not by enterprise IT.
  • The advantage is taken of RPA (Robotic Process Automation) and other techniques to improve existing processes.

At the same time, enterprises should determine the key parts of the business where real-time Matrix workloads can make a difference, and be experimenting earnestly to design and develop solutions that will enable radical simplification and automation. Executive management should monitor these projects closely to improve their understanding of the risk of being disrupted, and their ability to detect potential disruptors earlier.

Bottom Line: Enterprise should identify potential Matrix workloads, and develop business cases. Enterprises should focus speeding up systems of record, and plan the migration path to Systems of Intelligence. Other methods, like robotic process automation (RPA), which can also be used to streamline systems. Systems of Intelligence can become a foundation for implementing sustainable strategic digital differentiation, with the ability to dramatically improve business innovation cycle-times. The potential benefits for enterprises will range from ensuring survival to overwhelming long-term business value.

Matrix Workload Projections

Key Drivers & Constraints for Matrix Workloads

Figure 3 below is the Wikibon compute-only revenue projections for the emerging Matrix enterprise workloads, shown in grey. The growth is constrained in the first four years by significant constraints in software tools, hardware availability, and skills availability. In particular, it is constrained by the lack of enterprise heterogeneous compute systems. 

The traditional enterprise workloads compute-only revenue projection is shown in Figure 3 in orange. Over the second part of the decade, there is significant movement of spend from traditional workloads to Matrix workloads.

More details on the CAGRs (Compound Annual Growth Rates) in Figure 3 can be found in Table 3 in Footnotes2 below.

Bottom Line: Wikibon projects that enterprise Matrix workloads compute spend will grow rapidly, and become 42% of enterprise compute spend by the end of the decade.

Matrix vs. Traditional Workload Projections
Figure 3: Enterprise Matrix Workload vs. Traditional Workload Projections
Source: © Wikibon 2020

Conclusions & Recommendations

Case study conclusions

As a disruptor, Tesla has to do things very differently than the current manufacturers. Tesla also has to successfully deal with other very important factors, including manufacturing expertise, the cost of batteries, the management of batteries, the availability of chargers, and charging times. These are well beyond the scope of this research. 

Wikibon would emphasize two key components of their data processing strategy – the investment in Matrix processing technologies, and the integrated end-to-end data strategy. These are difficult to implement for traditional auto vendors who assemble vehicles from component suppliers and are experts in global supply chains. We suspect that many brands will fail over the next decade.

The Systems of Intelligence case study shows that significant progress can be made to improving the performance of existing systems of record, and start on the process of using existing and new data to provide the real-time advanced analytics to help simplify and automate processes. Wikibon discusses the potential benefit of a specific Matrix workload, automated pricing. Wikibon concludes that the potential estimate of the benefit of implementing real-time pricing updates could be an increase of 3.5% in gross revenue, and a reduction by a factor of five in the operational costs to perform pricing.

Getting Started on Matrix workloads

Real-time Matrix workloads will be a major contributor to digital innovation and differentiation over the next decade. Wikibon believes that a strategy that attempts to put all the data on one platform in one place will fail, as the cost of conversion is crippling and will take many, many years to achieve. Techniques such as RPA (Robotic Process Automation) can automate serialized transaction environments, and make the implementation of real-time Matrix workloads quicker to develop and implement. However, current RPA techniques are not sufficient on their own. Real-time Matrix workloads will make a much bigger impact on systemic simplicity and automation.

Wikibon recommends that the enterprises should focus on establishing an effective end-to-end data strategy, starting with where data is created today and where it will be in the future. Enterprises should implement an efficient multi-cloud, hybrid cloud and adjacent cloud strategies to ensure the right data is in the right place, to minimize the movement of data by moving compute to the data, and to move the responsibility for deployment and maintenance of infrastructure services to software and hardware vendors and cloud providers.

Wikibon also recommends that current systems of record are upgraded and some components possibly outsourced to improve performance and reduce operational costs. The tier-1 mission-critical databases are often a bottleneck. The only tier 1 large-scale mission-critical databases are IBM DB2, Microsoft SQL Server, and Oracle. All provide cloud options, either on-premises or in their own clouds. The vendor is responsible for performance and reliability and operational upgrades for the complete database stack, from microcode on servers to database upgrades. 

It is now possible to run the tier-1 database separate from the operational systems that run the applications that call the tier-1 databases. Both components are in adjacent clouds running in the same datacenter, and connected by low-latency direct communications between the two adjacent clouds. 

For example, Microsoft Azure and VMware offer the ability to run the applications on their cloud, with a very low-latency connection to Oracle adjacent databases also running in the same cloud datacenter. Microsoft and Oracle engineers have worked to provide the connectivity between the two clouds. This allows the flexibility of the database vendor being responsible for using Matrix technologies to speed up the database, and the other clouds to use heterogeneous architectures to accelerate the Matrix inference workloads. 

The importance of making such changes is to speed up the current systems of record and allow time for the Matrix workloads to run in parallel. At the same time, enterprises should determine the key parts of the business where real-time Matrix workloads can make a difference, and be experimenting in earnest to design and develop solutions that will enable radical business process simplification and automation. Senior executives should be keeping a wary eye for potential disruptors to their enterprise. 

Action Items

The Tesla case study shows the power of an end-to-end data strategy, the investments and commitments necessary to achieve it, and the threat to incumbent auto companies. The case study also shows the power of executing matrix workloads to develop ASD and build continuous and potentially exponential improvement processes.

Wikibon strongly recommends that senior enterprise executives develop an end-to-end data strategy, and develop a deep understanding of Matrix workloads. It is important that the strategy focuses on real-time data flows, and not on historical data, which is likely to be of much lower value.

This exercise will allow senior management to understand where external innovation threats are most likely to come from, and identify early where Matrix workloads could be used by external disruptors. This allows senior management to understand the level and nature of any disruptive threat, and to either acquire, partner, or bring forward internal projects to ensure the enterprise remains competitive. 

In addition, the development of an end-to-end data strategy can help to avoid losing control of access to key data. This can happen if part of an organization does not understand the loss to the company as a whole if it gives away data to a supplier or potential competitor.

Upcoming Matrix Workload Research 

Scope

The next Wikibon research report on Matrix workloads will be entitled “Heterogeneous Compute Accelerates Matrix Workloads”. The research will look at the changes in compute architecture required to enable Matrix workloads to operate in real-time, with 10 to 1,000 times more data than today’s transactional systems.

Premise

The simple premise of this research note will be that heterogeneous compute architecture can increase the performance of Matrix workloads by a factor a 50, the price-performance by a factor of 200, with 200 times better power & cooling costs. These improvements dramatically increase the range of matrix application inferences that can be real-time and the quantity and quality of the dataflows.

Publication Date

The publication date is scheduled for the middle of March.

Feedback

Wikibon encourages feedback on this research on the subject of Matrix workloads. You can contact the author by email at David.Floyer@Wikibon.org, by Twitter at @dfloyer, and also on LinkedIn.

Footnotes:

Footnote 1: Wikibon Matrix Workload Definition

Table 2 below gives a more formal Wikibon definition of Matrix workloads compared with traditional workloads.

Matrix Workloads Definition
Table 2: Wikibon Matrix Workloads Definition
Source: © Wikibon 2020

Footnote 2: Matrix Workload vs. Traditional Workload Compute Revenue Projections and CAGR Table

Table 3 below gives the worldwide compute revenue for Matrix workloads and traditional workloads 2019 – 2030. The CAGRs 2019 – 2030, CAGR 2019-2025, and CAGR 2025-2030 are also shown in the table. As expected, the CAGR growth of Matrix workloads is the fastest 2019-2025 ( CAGR 42%), but the year-on-year actual revenue growth continues to increase all the way through to 2030 ($11B growth 2029-2030). 

The main business driver for this growth is the very high business returns on Matrix workloads. The main inhibitors to growth are a development learning curve and the availability of hardware and software platforms.

Matrix vs. Traditional Workload Table
Table 3: Matrix Workload vs. Traditional Workload Compute Revenue Projections and CAGR Table
Source: © Wikibon 2020