|
|
Introducing a pioneering managed service designed to facilitate seamless simulation execution across various scales on Amazon Web Services (AWS). By leveraging a robust scheduler, researchers will collaborate seamlessly within a renowned High-Performance Computing (HPC) environment, thereby streamlining their project timelines and minimizing infrastructure concerns.
In November 2018, we introduced Slurm, a cloud-agnostic, open-source cluster management tool backed by Amazon Web Services (AWS), empowering users to efficiently deploy and manage high-performance computing (HPC) clusters within the secure and scalable AWS Cloud environment. With AWS ParallelCluster, clients can quickly build and deploy proof-of-concept and production-ready high-performance computing (HPC) environments. Developers will leverage various open-source libraries, including and , to build a user-friendly consumer interface. They are responsible for ensuring timely updates, which might involve decommissioning and reconfiguring existing cluster architectures as needed. We have received numerous requests from customers seeking a fully managed Amazon Web Services (AWS) solution that eliminates the need for manual operational tasks when setting up and managing high-performance computing (HPC) environments.
AWS PCS simplifies management of high-performance computing (HPC) environments on AWS, offering access through the AWS Software Development Kit (SDK), as well as. Your system administrators can create customised Slurm clusters that leverage their existing compute and storage settings, ID specifications, and job scheduling priorities. AWS Parallel Cluster Service (PCS) leverages Slurm, a highly scalable and fault-tolerant job scheduler widely utilized across various high-performance computing (HPC) clients, to efficiently schedule and orchestrate complex simulations. Finishing customers corresponding to scientists, researchers, and engineers can log into AWS PCS clusters to execute and manage high-performance computing (HPC) workloads, utilize interactive software applications on virtual desktops, and access data. You’ll be able to deliver your workloads to Amazon Web Services (AWS) Professional Cloud Services rapidly, without requiring significant effort to re-architect or port your code.
By leveraging absolute control over remote desktops for seamless visualization, combined with job telemetry or utility logs, IT professionals can efficiently manage complex high-performance computing (HPC) workflows from a centralized location.
AWS Parallel Computing (PCS) is optimized for a diverse range of conventional and emerging compute- or data-intensive workloads across domains such as computational fluid dynamics, climate modeling, finite element analysis, digital design automation, and reservoir simulations, utilizing established methods for preparing, executing, and analyzing complex simulations and computations.
To explore AWS Professional and Certified Services (PCS), refer to the relevant section within the comprehensive AWS documentation. Create a private digital cloud (VPC) using an AWS CloudFormation template within your account to host your AWS Professional Certification Study (PCS) environment, featuring shared storage resources. To learn more about AWS services and features, visit the tutorials and guides available within the AWS documentation?
Within the organization, select a persistent utility that provides for efficient management of assets and effective workload allocation.
Enter your cluster title and specify the controller dimension for your Slurm scheduler. You’ll have the flexibility to choose from three tiers: select up to 32 nodes and 256 jobs, scale to a maximum of 512 nodes and 8,192 jobs, or opt for 2,048 nodes and 16,384 jobs to define the boundaries of your cluster workloads. Select your existing VPC and subnet for launching the cluster, as well as specify the associated security group used by your cluster.
You can optionally set the Slurm configuration corresponding to an idle time earlier than compute nodes will scale down; this is achieved by specifying the `Timeout` parameter in the Slurm configuration file. Additionally, you may list Prolog and Epilog scripts that will be executed on launched compute nodes, allowing for custom initialization or cleanup tasks. Furthermore, a useful resource choice algorithm parameter used by Slurm can be set to optimize job scheduling and allocation of resources.
Select . The provisioning process for the cluster typically requires some time to complete.
After setting up your cluster, you can establish compute node teams, a virtual collection of nodes that Amazon Web Services Parallel Computing (PCS) uses to provide interactive access to a cluster or execute jobs within it. When outlining a compute node group, you define key characteristics tied to EC2 instance types, including minimum and maximum instance requirements, target VPC subnets, purchasing options, and custom launch configurations. To deploy compute node teams, you need a compute environment profile to associate with an EC2 instance and an EC2 launch template used by AWS ParallelCluster Services (PCS) to configure EC2 instances it launches for. To learn more, visit the AWS documentation at and .
To create a compute node group in the console, navigate to your cluster by selecting the corresponding tab, followed by clicking on the “Create Node Group” button.
Customers will be able to access two compute node teams: a login node group designed for end-users, and a separate job node group dedicated to running high-performance computing (HPC) jobs.
To establish a functional high-performance computing (HPC) job on compute node groups, specify a descriptive title for the group, select an existing Amazon Elastic Compute Cloud (EC2) launch template, choose an applicable Identity and Access Management (IAM) instance profile, and designate suitable subnets within your cluster’s Virtual Private Cloud (VPC) to deploy the nodes.
When launching compute nodes, select your preferred EC2 instance types to ensure optimal performance, considering factors such as processing power, memory, and storage needs. Additionally, specify the minimum and maximum instance counts for scaling to effectively manage workloads and allocate resources efficiently. I selected the hpc6a.48xlarge Occurrences of this kind are typically limited to a small scale, rarely exceeding eight instances. A user can choose a more limited scope for their login node, equivalent to a single instance. c6i.xlarge occasion. You can also select both the ‘EC2’ and ‘buy’ options if the occasion demands it. You may optionally select an existing Amazon Machine Image (AMI).
Select . Provisioning of the compute node group requires a considerable amount of time. To learn more, visit the AWS documentation at [links] within the relevant sections.
Once you have created and organized your compute nodes into teams, you can then submit a job to the desired queue to initiate its execution. The job remains in the queue until AWS Parallel Computing (PCS) schedules it to execute on a compute node group, primarily driven by the availability of provisioned capacity. Each queue is linked to multiple compute node teams that provide necessary EC2 instances for processing tasks.
In order to establish a queue in the console, navigate to your designated cluster and click on the designated tab and then the corresponding button.
Select the queue title and designate the computer node teams responsible for processing jobs in that queue.
While awaiting the creation of the queue.
When a login compute node group is active, you must connect to the corresponding EC2 instance that was launched. Visit the instance and select your preferred EC2 instance from the login compute node group. To learn more, explore the tutorials on AWS Lambda and Amazon Rekognition within the official AWS documentation.
To successfully run a job using Slurm, one must craft a thoughtful submission script that clearly outlines job requirements and submit it to an available queue. sbatch command. Occasionally, a shared listing is used to complete that task, resulting in similar areas for accessing data on both login and compute nodes.
You can also run a message-passing interface (MPI) job in Amazon Web Services Parallel Cluster Service (AWS PCS) using Slurm. For additional information on being taught extra, refer to [AWS Documentation](https://docs.aws.amazon.com/) or [AWS Guides](https://docs.aws.amazon.com/guides/).
You’ll be able to access a fully managed and supported NICE DCV remote desktop for visualization purposes. To initiate the process, utilize the provided CloudFormation template.
When calculating the aerodynamic drag around a bike and rider using computational fluid dynamics (CFD), I applied the necessary equations and parameters to accurately simulate the flow of air and predict the resulting forces. The simulation was executed utilizing a total of 864 processing cores, distributed across three identical HPC6A configurations. The output can be visualized within the session after logging in to the network interface of the DC/OS occasion.
Upon completing HPC jobs with the clusters and nodes you’ve established, it’s essential to purge the corresponding assets to avoid unnecessary expenditures. To learn more, refer to the comprehensive AWS documentation within.
Here are some key points to grasp regarding this function:
- AWS PCS initially assists Slurm 2.3.11, providing mechanisms that enable clients to swiftly adapt their primary Slurm variants whenever new variations become available. AWS PCS is engineered to seamlessly replace the Slurm controller with patch variations through robotic automation. For additional learning, refer to the comprehensive AWS documentation.
- You can reserve EC2 capacity in a chosen Availability Zone, guaranteeing access to necessary compute resources on demand by reserving the desired capabilities for a specific time frame using On-Demand Capacity Reservations. For additional learning opportunities, refer to the comprehensive resources provided within the Amazon Web Services (AWS) documentation.
- You’ll be able to connect community storage volumes where data, knowledge, and records can be written and accessed alongside CSV, JSON, and XML files. You can also utilize self-managed volumes, equivalent to NFS servers. To learn more, refer to the comprehensive AWS documentation within.
Are now accessible within the United States East (Northern) region. Across the globe, Amazon Web Services (AWS) maintains a vast network of regions in Virginia), AWS US East (Ohio), US West (Oregon), Asia Pacific (Singapore), Asia Pacific (Sydney), Asia Pacific (Tokyo), Europe (Frankfurt), and Europe (Stockholm), providing unparalleled infrastructure capabilities to its customers.
AWS Professional Services (PCS) deploys all relevant resources within your existing Amazon Web Services account. You will be billed accordingly for these assets. To obtain additional information, please refer to the provided documentation.
Explore convenient shipping options for technical assistance or troubleshooting with your regular AWS Support contacts via phone or online chat.
—
