Software Startup Aims at Cluster Computing Barriers

Poorly written code often undermines hardware resource utilization. Hardware manufacturers are actively working to solve this problem, but software manufacturers are also providing tools to take advantage of the best computing resources available in geographically dispersed cloud environments.

With its roots in high-performance and quantum computing, Agnostiq upgraded a tool called Covalent as a simple way to partially offload code execution from the cloud to the best hardware resources. Coders only need to write a few lines of code. This frees up computing resources from Amazon Web Services for scientific computing and simulation.

This tool helps IT organizations, researchers, and scientists expand the reach of hybrid clouds deployed on-premises and in public clouds. This tool is designed to view on-premises and AWS infrastructure as a single system, and Covalent scales hardware availability according to the size of the problem being solved.

beyond the CPU

Agnostiq’s tools are well suited for shifting computing more broadly beyond CPUs to specialized accelerators such as graphics and artificial intelligence processors that do heavy work for machine learning and scientific tasks. In many cases, coders still enter the specific hardware that the code needs to run, but toolkits from Nvidia and Intel companies are now automating that process.

Also Read :  NEC Launches New Model of the 'SX-Aurora TSUBASA' Vector Supercomputer

Will Cunningham, head of quantum software at Agnostiq, said Covalent is a layer on top of parallel programming toolkits like Nvidia’s CUDA.

“Users can manage all of the various software dependencies in their environment for specific tasks,” Cunningham said.

For example, a software stack can be deployed across multiple clouds, and Covalent coordinates operations across multiple clusters, manages software revisions, submits scripts, and manages data.

“Without tools that can span multiple on-premises clusters like this, it becomes very difficult to understand. Where is the latest version of the code? If the data is fragmented, where is the data and where is my computing easiest to use?” said Cunningham.

Also Read :  7 Common Uses of Cloud Computing

tool

Covalent is a Python-based tool that adds AWS’s Lambda, ECS, and EC2 compute resources to your toolkit with just a few lines of code. Current implementations focus on entering a specific set of code to run on an Amazon cluster and waiting for an output to arrive that can be put on the stack.

The tool includes a software development kit component to construct a workflow, and a server component with a backend dispatch service and user interface.

“If a particular node is available for more common workflow tools these days, that server itself can run on an HPC cluster. Or it could be running in the cloud, which may be more appropriate if users are using a hybrid cloud configuration,” said Cunningham.

On-premises servers may not have the massive hardware resources like Nvidia’s GPUs needed for applications like machine learning. Machine learning typically requires a lot of memory and storage close to the GPUs and CPUs inside the servers that are easily available on AWS.

Also Read :  Zain KSA signs a MoU with StarLink

A typical offload to the cloud involves sending input to a remote cloud server and waiting for output to be received, which can then be integrated into the software stack. Every second counts for code execution, and real-time simulation of hybrid clouds with on-premises and public cloud infrastructure is considered one of the biggest challenges in cloud computing. Barriers are suboptimal interconnections for real-time data exchange.

This toolkit does not include streaming technology that adds elements of real-time processing and analysis of data running on hardware.

“That’s on the roadmap for future releases,” said Cunningham.

group I made a sketch.

Source

Leave a Reply

Your email address will not be published.