Google Cloud Next 2024: AI networking gets a boost

Google announced new cloud networking capabilities that aim to help enterprises securely connect AI and multicloud workloads. The new features expand on the company’s Cross-Cloud Network service and are focused on high-speed networking for AI/ML workloads, any-to-any cloud connectivity, AI-specific load balancing options, security enhancements, and AI-powered network management capabilities.

Key to Google’s Cross-Cloud Network service is its Cross-Cloud Interconnect product, introduced last year, which helps establish high-bandwidth connectivity between Google Cloud and other service providers’ clouds. Cross-Cloud Interconnect offers 10 Gbps or 100 Gbps managed, encrypted links and supports security options such as IPsec VPN or MACsec, according to Muninder Sambi, vice president of cloud networking, who wrote a blog about the cloud networking news announced at Google’s Cloud Next 2024 conference.

(Google also showcased faster processors, bigger virtual machines, and storage upgrades at its Cloud Next conference, along with new features for its distributed cloud offering.)

The idea is to use Google Cloud to deliver high-performance, low-latency network infrastructure across zones and regions to enable gen AI training at scale, Sambi stated.

“Gen AI workloads have unique traffic patterns, with large requests and responses. This can lead to variable processing times, resulting in suboptimal user response times,” Sambi wrote. “To address this, an intelligent network can distribute foundation model queries based on usage and availability of resources.”

Google Cloud’s latest networking innovations are built to address these challenges and optimize performance for AI workloads, he stated.

One example is its new Model as Service Endpoint solution, which is available now. This feature lets model creators designate and own the service endpoint that contains the specific AI model under development. Application developers then connect to that model, Sambi stated.

The Model as Service Endpoint solution includes Google Cloud’s Private Service Connect for AI model producer and consumer connectivity, which provides links to managed service networking without leaving the Google Cloud network. According to Google, this feature works by allowing a network connection from a Private Service Connect (PSC) interface to Google Cloud, which then allocates an IP address to the interface. The IP address is linked from the AI model or consumer subnet that’s specified by the network connection. The AI or consumer and producer networks are then connected and can communicate by using internal IP addresses, the vendor stated.

Model as Service Endpoint also includes Google’s Cloud Load Balancing for optimal traffic distribution and access to its App Hub for service discoverability.

Looking ahead, Google Cloud is planning a number of enhancements to its Cloud Load Balancing capabilities for inference workloads, according to Sambi.

One example is the ability to include custom metrics for cloud load balancing; this “provides queue depth as a metric for load balancing AI workloads to deliver faster user response time to prompts while optimizing TPU and GPU utilization,” Sambi wrote.

Another example is cloud load balancing for streaming inference, which “uses metrics based on number of streams, bytes-in, and bytes-out, versus requests per second and CPU utilization to optimize performance,” Sambi wrote. 

Lastly, Google Cloud will enable cloud load balancing with traffic management for AI models; this feature “monitors the health of individual model service endpoints and routes requests to healthy endpoints, initiates cross-region failover when an outage is detected, and splits traffic across different models and model versions, helping organizations manage rollouts,” Sambi stated.

Security upgrades and AI-powered network management help

For multicloud connectivity, Google is introducing Private Service Connect (PCS) transitivity over Network Connectivity Center (NCC), available in preview this quarter. Network Connectivity Center lets enterprises connect their on-premises networks, Google Cloud resources, and other cloud environments as spokes through a centralized logical hub on Google Cloud, according to Sambi. 

The new feature will allow services in a spoke-configured virtual private clouds (VPC) to be accessible from other spoke VPCs. Customers will be able to set up a service VPC to create multiple PSC consumer endpoints that are accessible to other VPCs. “In combination with VPC spokes for NCC, this simplifies cloud network designs,” Sambi wrote.

In the security realm, Sambi highlighted three new security innovations in Cross-Cloud Network:

• Cloud NGFW Enterprise (formerly Cloud Firewall Plus), generally available now, provides network threat protection powered by Palo Alto Networks’ technology, plus network security posture controls for organization-wide perimeter and zero-trust microsegmentation, Sambi stated.

• Identity-based authorization with mTLS “integrates the Identity-Aware Proxy with our internal application Load Balancer to support Zero Trust network access, including client-side and soon back-end mutual [Transport Layer Security],” Sambi wrote.

• In-line network data-loss prevention (DLP), which will be in preview soon, “integrates Symantec DLP into our Load Balancers and Secure Web Proxy using Service Extensions. This will help safeguard sensitive data-in-transit from accidental and malicious exposure,” Sambi wrote.

On the management front, Google Cloud added an AI-based assistant called Gemini Cloud Assist, which offers intelligent help and insights related to network design, operations, and optimization, Sambi stated.  

“Cloud administrators can ask Gemini Cloud Assist to solve a variety of tasks and recommendations such as generate configurations, recommend capacity, correlate changes with issues, identify vulnerabilities, and optimize performance,” Sambi wrote. “In preview, Gemini Cloud Assist expedites network provisioning and management, enabling organizations to deliver business results faster and more reliably.”

IDC weighs in on Cross-Cloud Network

Google Cloud’s Cross-Cloud Network provides a secure, resilient, and high-performance solution for any company looking to deploy hybrid or multicloud resources, according to a recent IDC report, Accelerating the Enterprise AI Journey with Cross-Cloud Network, written by Vijay Bhagavath, research vice president, cloud and datacenter networks, at IDC.

“Cross-cloud networking introduces the concept of leveraging services from multiple clouds and building an application stack that may have different layers hosted by different cloud providers. Google Cloud’s Cross-Cloud Network effectively provides the experience of a single cloud across multiple clouds and simplifies the network to help enterprises accelerate agility,” according to the IDC report.

Cross-Cloud Network supports traditional networking models and service-centric architectures, which makes it easier to connect and secure applications across clouds while reducing the total cost of ownership by up to 40%, according to Bhagavath.

“With service-centric Cross-Cloud Network, enterprises can set up Google Cloud or partner-managed services across hybrid and multicloud environments with Private Service Connect. Google Cloud services include Cloud SQL, Looker, Spanner, and Vertex AI, and partner-managed services include MongoDB, Databricks, Datastax, and Redis. Private Service Connect empowers developers and data scientists to connect their applications securely within minutes, bridging the gap between DevOps, NetOps and SecOps,” Bhagavath stated.