Guest Software not Installed by an OSConfig Guest Policy onto an Eligible Google Compute Engine VM

Question

A Google Compute Engine (GCE) instance ($GCE_INSTANCE_NAME) was just created within a Google Cloud Platform (GCP) project $GCP_PROJECT_ID. There is an OSConfig guest policy ($GUEST_POLICY_NAME) that is supposed to install guest software packages onto $GCE_INSTANCE_NAME; however, when the Cloud SDK (gcloud) is used to lookup the guest policies applied to $GCE_INSTANCE_NAME:

gcloud beta compute os-config guest-policies lookup \
$GCE_INSTANCE_NAME \
--zone=$GCE_INSTANCE_ZONE

$=>

No effective guest policy found for [projects/$GCP_PROJECT_NAME/zones/$GCE_INSTANCE_ZONE/instances/$GCE_INSTANCE_NAME].

$GUEST_POLICY_NAME is not listed.

When the lookup command is used for another GCE instance ($GCE_ANOTHER_INSTANCE) with identical OS version, GCE metadata and GCE labels:

gcloud beta compute os-config guest-policies lookup \
$GCE_ANOTHER_INSTANCE \
--zone=$GCE_ANOTHER_ZONE

#=>

┌──────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│                                           SOFTWARE RECIPES                                               │
├───────────────────────────────────────────────────────────┬────────────────────┬─────────┬───────────────┤
│                          SOURCE                           │        NAME        │ VERSION │ DESIRED_STATE │
├───────────────────────────────────────────────────────────┼────────────────────┼─────────┼───────────────┤
│ projects/$GCP_PROJECT_ID/guestPolicies/. . .              │        . . .       │ . . .   │   . . .       │
│ projects/$GCP_PROJECT_ID/guestPolicies/$GUEST_POLICY_NAME │ $GUEST_POLICY_NAME │ 1.0     │ INSTALLED     │
│ projects/$GCP_PROJECT_ID/guestPolicies/. . .              │        . . .       │ . . .   │   . . .       │
└───────────────────────────────────────────────────────────┴────────────────────┴─────────┴───────────────┘

$GUEST_POLICY_NAME is listed.

Why?

Answer 1

There could be a few reasons why $GUEST_POLICY_NAME isn't showing up in the response from the lookup command on $GCE_INSTANCE_NAME:

• latency: it may take some time for OSConfig to propagate $GUEST_POLICY_NAME when $GCE_INSTANCE_NAME was just created

• while you might have enabled project-wide GCE metadata, as suggested here, it may help to add:

  • enable-guest-attributes: TRUE
  • enable-osconfig: TRUE

  to $GCE_INSTANCE_NAME with the add-metadata command:

  gcloud compute instances add-metadata \
  $GCE_INSTANCE_NAME \
  --metadata="enable-guest-attributes=true,enable-osconfig=TRUE" \ 
  --zone=$GCE_INSTANCE_ZONE
  
  #=>
  
  Updated [https://www.googleapis.com/compute/v1/projects/$GCP_PROJECT_NAME/zones/$GCE_INSTANCE_ZONE/instances/$GCE_INSTANCE_NAME].
  

• if $GUEST_POLICY_NAME uses a Google Cloud Storage (GCS) Bucket to store packages or executables, check to see if the GCE default service account ($GCE_SERVICE_ACCOUNT) has at least one curated role with the storage.objects.get permission (e.g., storage.objectViewer) with the GCS CLI (gsutil):

  gsutil iam get "gs://$GCS_BUCKET_NAME"
  
  #=>
  
  {
    "bindings": [
    . . .
    {
      "members": [
        "serviceAccount:$GCE_SERVICE_ACCOUNT"
      ],
      "role": "roles/storage.objectViewer"
    }
    . . .
    ]
  }
  

  if $GCE_SERVICE_ACCOUNT does not have a role with the storage.objects.get permission, you can use the ch command for the iam group to grant the storage.objectViewer curated role:

  gsutil iam ch \
  "serviceAccount:$GCE_SERVICE_ACCOUNT:roles/storage.objectViewer" \
  "gs://GCS_BUCKET_NAME"
  

• Make sure that Private Google Access is turned on for the subnet $GCE_INSTANCE_NAME is running in:

  1. Easily discover which subnet $GCE_INSTANCE_NAME is using with both the --flatten and --format flags for the describe command:

     gcloud compute instances describe $GCE_INSTANCE_NAME \
     --flatten="networkInterfaces" \
     --format="value(networkInterfaces.subnetwork)" \
     --zone=$GCE_INSTANCE_ZONE
     
     #=>
     
     https://www.googleapis.com/compute/v1/projects/$GCP_PROJECT_NAME/regions/$GCE_INSTANCE_REGION/subnetworks/$GCE_INSTANCE_SUBNETWORK
     

  2. Find out if $GCE_INSTANCE_SUBNETWORK has Google Private Access turned on:

     gcloud compute networks subnets describe $GCE_INSTANCE_SUBNETWORK\
     --format="value(privateIpGoogleAccess)" \
     --region=$GCE_INSTANCE_REGION
     
     #=>
     
     True
     

     if the above is False, then enable Private Google Access with the update subcommand for the same subnets subgroup:

     gcloud compute networks subnets update $GCE_INSTANCE_SUBNET \
     --enable-private-ip-google-access \
     --region=$GCE_INSTANCE_REGION
     
     #=>
     
     Updated [https://www.googleapis.com/compute/v1/projects/$GCP_PROJECT_NAME/regions/$GCE_INSTANCE_REGION/subnetworks/$GCE_INSTANCE_SUBNETWORK].
     

And if all of the above fail, make sure that $GCE_INSTANCE_NAME aligns with all of the criteria from $GUEST_POLICY_NAME:

gcloud beta compute os-config guest-policies describe \
$GUEST_POLICY_NAME \
--format="yaml(assignment)"

#=>

assignment:
  groupLabels:
  - labels: . . .
  instances: . . .
  instanceNamePrefixes: . . .
  osTypes:
  - osArchitecture: . . .
    osShortName: . . .
    osVersion: . . .
  zones: . . .