What Actually Happened

A flaw in the Google Vertex AI SDK for Python allowed attackers to hijack machine learning model uploads by exploiting predictable Cloud Storage bucket names. Discovered by Palo Alto Networks’ Unit 42, this vulnerability, dubbed “Pickle in the Middle,” enabled unauthorized code execution within Google’s serving infrastructure without any credentials or phishing attempts. The issue stemmed from the SDK failing to verify bucket ownership, allowing attackers to create a bucket with a name derived from the victim’s project ID and region, thus intercepting model uploads.

The Implementation Reality

This vulnerability primarily impacts teams using Google Vertex AI for developing and deploying machine learning models. If developers rely on the SDK’s default behaviors, they inadvertently expose themselves to risks associated with bucket squatting. Affected users who do not specify a staging bucket will find that their model uploads could be redirected to an attacker-controlled bucket, leading to potential model poisoning and execution of malicious code during model serving. The blast radius extends to all environments where the vulnerable SDK is used, including CI/CD pipelines, notebooks, and production workloads. Teams must ensure they are using SDK version 1.148.0 or later, which includes ownership verification and other security enhancements. Failure to do so not only leaves the door open for model theft but also compromises sensitive information, such as OAuth tokens and access to other model artifacts.

What to Do About It

• Update the Google Vertex AI SDK to version 1.148.0 or later to enable bucket ownership verification.
• Explicitly set the staging_bucket parameter when uploading models to a Cloud Storage location you control.
• Audit all instances of the SDK in your environment, including CI jobs, training pipelines, and notebooks, to ensure they are running the latest secure version.
• Monitor your Cloud Storage buckets for unusual activity and implement alerts for unauthorized access attempts.
• Review your overall security posture for Google Cloud resources, especially focusing on project IDs and permissions related to Vertex AI.

Source: The Hacker News

At q52, we specialize in AI-augmented security operations and SIEM implementation. Let us help you operationalize threat detection with LLM enrichment — faster triage, fewer false positives, and security intelligence your team can actually act on. Learn about Noogenesis, our AI-powered SIEM platform and connect with us on LinkedIn.