The $760.86 Google API Bill That Taught Me Cloud Cost Security From Scratch

TL;DR

A Google API key bundled in my Expo mobile app got scraped and burned $644.80 of Places Autocomplete and Place Details requests over two days while I was not looking. After 18% Senegal VAT and currency-conversion fees, that worked out to roughly $760.86 out of my bank account for two days of someone else's traffic.

I had API restrictions on the key. I did not have application restrictions, a billing budget, or a kill switch.

What follows is the full course: the attack, the audit, the four-layer defense, and a working provisioning script that recreates the whole thing on a new project in under five minutes.

The Notification That Started It

It did not start with a billing email from Google. It started with an AMEX push notification on my phone: a charge from a vendor I had not authorized that month. My first thought was fraud on the card itself.

The vendor was Google.

I opened my Cloud billing dashboard and the shape of the spend was the giveaway: $0 every day for three weeks, then a sharp spike on April 7 ($150) and April 8 ($495). Two days, then back to zero.

That pattern is not human usage. Human usage has texture: small variations day to day, weekend dips, spikes that taper. Two flat days at the top, then nothing, is automation.

I knew exactly which API too. The "Places API (legacy)" line item dwarfed everything else. Places Autocomplete + Place Details, called server-to-server with no caching, no rate limit, no anything.

Someone had my key.

The Smoking Gun

I had three Google API keys at the time:

1. A backend key — used by my Laravel server for Places, Geocoding, Directions
2. A second backend key — used by a CRM-related scraping job
3. An internal-tool key — used by a side service for Gemini

I ran gcloud services api-keys describe on each. The first one's restrictions read like this:

displayName: Android key (auto created by Firebase)
restrictions:
  apiTargets:
  - service: directions-backend.googleapis.com
  - service: firebaseinstallations.googleapis.com
  - service: geocoding-backend.googleapis.com
  - service: maps-android-backend.googleapis.com
  - service: places-backend.googleapis.com
  # ...

apiTargets is the API allowlist. That part was fine: this key could only call Maps and Places APIs, not Cloud Storage or BigQuery.

What is missing from the YAML above is the smoking gun. There is no androidKeyRestrictions block. No iosKeyRestrictions. No serverKeyRestrictions. No browserKeyRestrictions. The key's display name says "Android key" because Firebase named it that, but Firebase did not actually restrict it to Android apps. It was open to anyone who held the value.

And the value was bundled inside my Expo mobile app, in app.config.js:

expo: {
  extra: {
    googleMapsApiKey: process.env.GOOGLE_MAPS_API_KEY,
  },
  ios: {
    config: {
      googleMaps: {
        apiKey: process.env.GOOGLE_MAPS_API_KEY,
      },
    },
  },
}

Anyone who downloads my IPA or APK can extract that string in seconds with strings or any reverse-engineering tool. Then they can call Places Autocomplete from any IP, with no Android header, no certificate, nothing. Google's only check is "is this key valid and is the API in the allowlist?". Both yes. They serve the request and bill it.

That is the full attack. There is no zero-day, no exotic vulnerability. It is the default behavior when you create a Firebase Android app and forget to add the package name + SHA-1 fingerprint to the auto-generated key.

What "Restricted" Actually Means

Google has two kinds of restriction on an API key, and you need both:

A key with only API restrictions is a public key. Anyone who reads it can use it to call those APIs from anywhere. Limiting which APIs they can call only limits the kind of bill they can run up, not whether they can run one up.

A key with only application restrictions but no API restrictions is also wrong: if someone discovers a way to forge the application context (cert spoofing, header injection, network-level abuse), they get access to every API in your project including the expensive ones.

You need both layers. Always.

The Four-Layer Defense

Here is the model I now run on every project. Each layer is independent. They escalate in blast radius — Layer 1 only kills the leaked key, Layer 4 disables every Google API for the whole project until manually re-enabled.

I will walk through each.

Layer 1: Lock Every Key Down

This is the cheapest, highest-leverage step. Every key in your project should have both API restrictions and application restrictions, with as narrow a scope as you can tolerate.

Server keys (Laravel, Django, Node services calling Google APIs from a known IP):

gcloud services api-keys create \
  --project=YOUR_PROJECT \
  --display-name="myproject-server" \
  --allowed-ips=YOUR_PROD_SERVER_IP \
  --api-target=service=places-backend.googleapis.com \
  --api-target=service=places.googleapis.com \
  --api-target=service=geocoding-backend.googleapis.com \
  --api-target=service=directions-backend.googleapis.com

If you have multiple servers, pass --allowed-ips multiple times. If you run on a NAT or load balancer, allowlist its egress IP, not the internal one.

Mobile keys (iOS Maps SDK):

gcloud services api-keys create \
  --project=YOUR_PROJECT \
  --display-name="myproject-mobile-ios" \
  --allowed-bundle-ids=com.yourapp \
  --api-target=service=maps-ios-backend.googleapis.com

Mobile keys (Android Maps SDK + Firebase):

gcloud services api-keys update YOUR_KEY_UID \
  --project=YOUR_PROJECT \
  --allowed-application=sha1_fingerprint=AA:BB:CC:...,package_name=com.yourapp \
  --allowed-application=sha1_fingerprint=DD:EE:FF:...,package_name=com.yourapp \
  --api-target=service=maps-android-backend.googleapis.com \
  --api-target=service=maps-backend.googleapis.com \
  --api-target=service=firebaseinstallations.googleapis.com

Critical for Android: you need both SHA-1 fingerprints — your upload key (used by EAS/Gradle to sign the APK before upload) and Google Play's app-signing key (which Google re-signs with before delivering to users). The upload key SHA is in your build credentials. The Play app-signing SHA is in Google Play Console → Test and Release → App integrity → App signing key certificate.

If you only add the upload key SHA, Maps will work in your internal builds and break for real users on the Play Store. Ask me how I know.

Audit existing keys:

gcloud services api-keys list --project=YOUR_PROJECT --format=json

Any key without an iosKeyRestrictions, androidKeyRestrictions, serverKeyRestrictions, or browserKeyRestrictions block in the output is unrestricted. Lock it down or delete it.

Delete keys you do not use. Firebase auto-creates a "Browser key" with permissions to call BigQuery, Cloud SQL admin, Cloud Storage, and 50+ other APIs, with browserKeyRestrictions: {} (empty allowlist). If you are not using Firebase web SDK, delete that key. If you are, lock it to your own domain.

Layer 2: Per-API Quota Caps

Even if a key is properly restricted, you can add a second guarantee: hard daily caps on the APIs that drive cost.

This is a one-time Cloud Console step (no clean gcloud equivalent for some metrics):

1. Console → APIs & Services → <API> → Quotas & System Limits
2. Find the relevant per-day metric (e.g. "Place Autocomplete requests per day")
3. Edit → set to a number well above your normal usage but well below pain (e.g. 10,000/day)

Once a quota is exhausted, Google starts returning 429 for that specific API for the rest of the day. Other APIs keep working. It is the most surgical defense layer.

For my own project I cap (or plan to cap, this is the layer I am still tightening) Places Autocomplete at 10k/day, Place Details at 5k/day, and Geocoding at 5k/day. Actual daily usage is in the dozens, so the cap leaves three orders of magnitude of headroom for legitimate growth and zero room for a leaked-key flood.

Layer 3: Budget With Email Alerts

This is the slow-burn detector. A leaked key getting hit at low frequency might not trip a per-API quota for weeks. A monthly budget will tell you when total spend crosses meaningful thresholds.

gcloud billing budgets create \
  --billing-account=YOUR_BILLING_ACCOUNT \
  --display-name="myproject monthly hard cap" \
  --budget-amount=60USD \
  --threshold-rule=percent=0.5 \
  --threshold-rule=percent=0.75 \
  --threshold-rule=percent=0.9 \
  --threshold-rule=percent=1.0 \
  --notifications-rule-pubsub-topic=projects/YOUR_PROJECT/topics/your-budget-topic \
  --filter-projects=projects/YOUR_PROJECT_NUMBER

A few things worth knowing:

• Budgets are billing-account-scoped, not project-scoped. Without --filter-projects, a budget on a billing account that funds five projects will trigger when any of them collectively exceed the threshold. Almost always not what you want.
• --filter-projects takes the project number, not the project ID. (Find it with gcloud projects describe YOUR_PROJECT --format='value(projectNumber)'.)
• The default email recipients are anyone with roles/billing.admin or roles/billing.user on the billing account. If you are the project owner, that is already you. If you want a different email, you have to wire up a Cloud Monitoring notification channel — usually overkill for one address.

The budget alone does not stop spend. It just publishes a Pub/Sub message and sends an email. If you sleep through the email, the spend keeps going. Which brings us to the last layer.

Layer 4: The Hard Kill Switch

This is Google's documented "cap costs" pattern: when a Pub/Sub notification crosses your hard threshold, a Cloud Function detaches billing from the project, which causes every Google API to start returning BILLING_DISABLED until you manually re-attach.

It sounds extreme. It is. That is the point — it is the only mechanism that actually stops spend. You set the trigger to 100% (or 200% if you want a buffer above the alert thresholds), and you let it fire if everything else fails.

The Function

# devops/gcp/killswitch/main.py
import base64
import json
import os

from googleapiclient import discovery


PROJECT_ID = os.environ["PROJECT_ID"]
DISABLE_AT_PERCENT = float(os.environ.get("DISABLE_AT_PERCENT", "1.0"))


def stop_billing(event, context=None):
    payload = base64.b64decode(event["data"]).decode("utf-8")
    data = json.loads(payload)

    cost = float(data.get("costAmount", 0) or 0)
    budget = float(data.get("budgetAmount", 0) or 0)

    if budget <= 0:
        print(f"Skip: budget is zero/missing in payload: {data}")
        return

    ratio = cost / budget
    if ratio < DISABLE_AT_PERCENT:
        print(f"Under threshold: cost={cost} budget={budget} ratio={ratio:.2f}")
        return

    billing = discovery.build("cloudbilling", "v1", cache_discovery=False)
    project_name = f"projects/{PROJECT_ID}"

    info = billing.projects().getBillingInfo(name=project_name).execute()
    if not info.get("billingEnabled", False):
        print(f"Already disabled: {PROJECT_ID}")
        return

    response = billing.projects().updateBillingInfo(
        name=project_name,
        body={"billingAccountName": ""},
    ).execute()
    print(f"DISABLED billing for {PROJECT_ID}: {response}")

# devops/gcp/killswitch/requirements.txt
google-api-python-client==2.155.0
google-auth==2.36.0

The function is idempotent: if billing is already detached when the next message fires, it logs and returns. No infinite loop, no retries doing damage.

The IAM dance

The function needs to call cloudbilling.projects.updateBillingInfo. That requires:

• On the project: roles/billing.projectManager (contains cloudbilling.projects.updateBillingInfo)
• On the billing account: roles/billing.user (lets it know it can detach itself)

Plus, since Cloud Functions Gen 2 actually run on Cloud Run under the hood, the Eventarc trigger needs roles/run.invoker on the underlying Run service. Without this last one, every Pub/Sub message returns 401 and your kill switch silently does nothing. Ask me how I know that one too.

The provisioning script

I packaged everything into a single idempotent script. Save as devops/scripts/gcp-bootstrap.sh, edit the config section at the top, and run.

#!/usr/bin/env bash
set -euo pipefail

# ─── Config ─────────────────────────────────────────────────────────────────
PROJECT_ID="myproject"
PROJECT_NUMBER="123456789"
BILLING_ACCOUNT="ABCDEF-123456-7890AB"
REGION="europe-west1"

BUDGET_AMOUNT_USD="60"
BUDGET_TOPIC="myproject-budget-alerts"
THRESHOLD_PERCENTAGES=(0.5 0.75 0.9 1.0)

KILLSWITCH_FN_NAME="myproject-billing-killswitch"
KILLSWITCH_SA_NAME="myproject-killswitch"
KILLSWITCH_SA_EMAIL="${KILLSWITCH_SA_NAME}@${PROJECT_ID}.iam.gserviceaccount.com"
DISABLE_AT_PERCENT="1.0"

REPO_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
KILLSWITCH_SOURCE_DIR="${REPO_ROOT}/devops/gcp/killswitch"

log() { printf "▶ %s\n" "$*"; }

# ─── 1. Required APIs ───────────────────────────────────────────────────────
log "Enabling APIs"
gcloud services enable \
    billingbudgets.googleapis.com \
    cloudbilling.googleapis.com \
    pubsub.googleapis.com \
    cloudfunctions.googleapis.com \
    cloudbuild.googleapis.com \
    run.googleapis.com \
    eventarc.googleapis.com \
    artifactregistry.googleapis.com \
    --project="${PROJECT_ID}"

# ─── 2. Pub/Sub topic ───────────────────────────────────────────────────────
log "Ensuring Pub/Sub topic"
gcloud pubsub topics describe "${BUDGET_TOPIC}" --project="${PROJECT_ID}" >/dev/null 2>&1 \
    || gcloud pubsub topics create "${BUDGET_TOPIC}" --project="${PROJECT_ID}"

# ─── 3. Service account + IAM ───────────────────────────────────────────────
log "Ensuring killswitch service account"
gcloud iam service-accounts describe "${KILLSWITCH_SA_EMAIL}" --project="${PROJECT_ID}" >/dev/null 2>&1 \
    || gcloud iam service-accounts create "${KILLSWITCH_SA_NAME}" \
         --display-name="Billing Killswitch" --project="${PROJECT_ID}"

log "Granting IAM"
gcloud projects add-iam-policy-binding "${PROJECT_ID}" \
    --member="serviceAccount:${KILLSWITCH_SA_EMAIL}" \
    --role="roles/billing.projectManager" >/dev/null
gcloud billing accounts add-iam-policy-binding "${BILLING_ACCOUNT}" \
    --member="serviceAccount:${KILLSWITCH_SA_EMAIL}" \
    --role="roles/billing.user" >/dev/null

# ─── 4. Cloud Function ──────────────────────────────────────────────────────
log "Deploying Cloud Function"
gcloud functions deploy "${KILLSWITCH_FN_NAME}" \
    --gen2 --runtime=python312 --region="${REGION}" \
    --source="${KILLSWITCH_SOURCE_DIR}" \
    --entry-point=stop_billing \
    --trigger-topic="${BUDGET_TOPIC}" \
    --service-account="${KILLSWITCH_SA_EMAIL}" \
    --set-env-vars="PROJECT_ID=${PROJECT_ID},DISABLE_AT_PERCENT=${DISABLE_AT_PERCENT}" \
    --max-instances=1 --memory=256Mi --timeout=60s \
    --project="${PROJECT_ID}"

log "Granting run.invoker on the Cloud Run service to the killswitch SA"
gcloud run services add-iam-policy-binding "${KILLSWITCH_FN_NAME}" \
    --region="${REGION}" --project="${PROJECT_ID}" \
    --member="serviceAccount:${KILLSWITCH_SA_EMAIL}" \
    --role="roles/run.invoker" >/dev/null

# ─── 5. Budget ──────────────────────────────────────────────────────────────
log "Ensuring budget"
THRESHOLD_FLAGS=()
for pct in "${THRESHOLD_PERCENTAGES[@]}"; do
    THRESHOLD_FLAGS+=(--threshold-rule="percent=${pct}")
done

EXISTING_BUDGET="$(gcloud billing budgets list \
    --billing-account="${BILLING_ACCOUNT}" \
    --filter="displayName='myproject monthly hard cap'" \
    --format='value(name)' 2>/dev/null | head -n1)"

if [[ -z "${EXISTING_BUDGET}" ]]; then
    gcloud billing budgets create \
        --billing-account="${BILLING_ACCOUNT}" \
        --display-name="myproject monthly hard cap" \
        --budget-amount="${BUDGET_AMOUNT_USD}USD" \
        "${THRESHOLD_FLAGS[@]}" \
        --notifications-rule-pubsub-topic="projects/${PROJECT_ID}/topics/${BUDGET_TOPIC}" \
        --filter-projects="projects/${PROJECT_NUMBER}"
else
    gcloud billing budgets update "${EXISTING_BUDGET##*/}" \
        --billing-account="${BILLING_ACCOUNT}" \
        --display-name="myproject monthly hard cap" \
        --budget-amount="${BUDGET_AMOUNT_USD}USD" \
        "${THRESHOLD_FLAGS[@]}" \
        --notifications-rule-pubsub-topic="projects/${PROJECT_ID}/topics/${BUDGET_TOPIC}" \
        --filter-projects="projects/${PROJECT_NUMBER}"
fi

echo "Done."

The script is idempotent. Re-running it after editing the budget amount, threshold percentages, or topic name will update the live resources in place.

Testing the kill switch without nuking your billing

Publish a synthetic Pub/Sub message that intentionally stays below the threshold and check the function logs:

gcloud pubsub topics publish myproject-budget-alerts \
  --project=myproject \
  --message='{"costAmount":15,"budgetAmount":60}'

gcloud functions logs read myproject-billing-killswitch \
  --region=europe-west1 --project=myproject --limit=5

You should see Under threshold: cost=15.0 budget=60.0 ratio=0.25 within 30 seconds. If you instead see The request was not authenticated warnings, the Eventarc trigger does not have run.invoker on the Cloud Run service. Add it.

When the kill switch fires for real

1. The function logs DISABLED billing for myproject and returns.
2. Within seconds, every Google API call from your services starts returning BILLING_DISABLED errors. Maps stops working. Storage buckets become read-only. Cloud SQL connections fail.
3. You investigate which API or which key drove the spend, and fix it.
4. You re-attach billing:

gcloud billing projects link myproject --billing-account=ABCDEF-123456-7890AB

5. The kill switch automatically re-arms for the next month.

Application-Level Belt-and-Suspenders

Layers 1–4 are infrastructure-side. None of them help if the abuser is going through your own backend (e.g., your /api/place proxy with valid auth tokens stolen from a real user).

For the cases where the abuse comes through your own app, you want application-level rate limits with sane per-IP and global ceilings. In Laravel:

RateLimiter::for('place', function (Request $request) {
    return [
        Limit::perMinute(30)->by($request->user()?->getKey() ?: $request->ip()),
        Limit::perHour(200)->by($request->user()?->getKey() ?: $request->ip()),
        Limit::perDay(500)->by($request->user()?->getKey() ?: $request->ip()),
        Limit::perDay(5000),  // global ceiling, no `by` clause
    ];
});

The global ceiling is the important one. Per-user limits do not help if the attacker rotates user accounts or scrapes your own auth flow.

You can also keep a Redis daily counter that increments on every Place lookup and hard-blocks the route when the counter exceeds a number you choose. Belt-and-suspenders against keys you have not yet identified as leaked.

$key = "places:daily:" . now()->format("Y-m-d");
$count = Redis::incr($key);
Redis::expire($key, 60 * 60 * 26);

if ($count > config('services.places.daily_hard_cap', 5000)) {
    abort(503, 'Daily Places cap reached');
}

What I Should Have Done From Day One

Three things, in order of priority:

1. Application restrictions on every key from creation. If a key gets created without a package + SHA, IP, or bundle, treat that as a P0. The default Firebase auto-created "Android key" is a trap. Rename it, lock it down, narrow its API targets, before you ship anything that uses it.

2. A monthly budget on every project before the first paying customer. A budget with email alerts is free. The Pub/Sub kill switch is free up to 2 million invocations a month. There is no reason not to have these from day one. Set the budget low — $5 or $10 if you have no idea yet what your usage will look like. You will get an email at 50% within hours of any abnormal usage and you will know exactly when to investigate.

3. Treat every embedded credential as public. Anything in a mobile bundle, a JS web bundle, a public GitHub repo, or a Docker image layer is public. Restrict it as if you have already published it on a billboard. If a credential cannot tolerate being public, it cannot be embedded — proxy it through your backend instead.

The Takeaway

The Apr 7-8 incident was a rite of passage. Every founder running Google Cloud, AWS, Azure or any pay-as-you-go cloud account will eventually have a moment like this — a number in the dashboard that should not be there, with no obvious cause, and no signal to act sooner.

The cheapest version of this lesson is the version where you read someone else's postmortem and ship the four-layer defense before you ever get the bill. That is what this post is for.

The provisioning script and Cloud Function code are above. Edit the config block, run the script, verify the wiring with a test Pub/Sub message, and move on. Total time investment: 30 minutes the first time, 2 minutes for every project after.

Whatever you spend setting this up will be the cheapest insurance you ever buy.