update-terraform-fields

KCC Update Terraform Fields (Agentic-Friendly Guide)

This skill provides step-by-step instructions for an AI agent to add or update fields or resources managed by the Terraform/legacy controller (distinguished by the label cnrm.cloud.google.com/tf2crd: "true" or the static mappings). It also covers patching the local/vendored copy of the Terraform Google Beta Provider (TPG Beta).

---

1. Pre-requisites & Codebase Discovery

Before making any changes, the agent must inspect the repository to locate the resource files:

1. Identify Kind and API Group:
   • Determine the Kubernetes resource Kind (e.g., SpannerDatabase) and the api group (e.g., spanner.cnrm.cloud.google.com).
2. Find the Terraform Resource Name:
   • Locate the service mapping file in config/servicemappings/<service>.yaml (e.g., config/servicemappings/spanner.yaml).
   • Find the mapping block matching the Kind to extract the underlying Terraform resource name (e.g., google_spanner_database).
3. Verify the Controller Mode:
   • Check if the resource is managed by the Terraform controller by verifying that config/servicemappings/<service>.yaml contains the kind.
   • Also, search for the resource's Group and Kind in pkg/controller/resourceconfig/static_config.go to ensure DefaultController is set to k8s.ReconcilerTypeTerraform (or similar for legacy) and is NOT set to k8s.ReconcilerTypeDirect. Note that static_config.go is auto-generated by dev/tasks/generate_static_config.py and should not be edited manually.
4. Identify Go Types Existence:
   • Run a search (e.g., find_by_name) for apis/<service>/<version>/<kind>_types.go.
   • Agentic Pitfall: Some files omit the service prefix in their filename (e.g., healthcheck_types.go instead of computehealthcheck_types.go). If find_by_name returns 0 results, verify by running grep_search across apis/<service>/ for type <Kind> struct or WithKind("<Kind>").
   • If it exists: Go to Section 3 (Case B).
   • If it does NOT exist: Go to Section 3 (Case A).

---

2. Modifying the Vendored Terraform Provider (TPG Beta)

If the field or bug fix is not yet present in the vendored copy of TPG Beta under third_party/github.com/hashicorp/terraform-provider-google-beta/, patch the provider locally. Refer to .gemini/skills/update-tf-provider-for-resource/SKILL.md for detailed guidelines on comparing, aligning, and backporting upstream changes while preserving local patches.

Steps to Patch TPG Beta:

1. Locate the Provider Code:
   • Locate the schema and flattener/expander files under third_party/github.com/hashicorp/terraform-provider-google-beta/google-beta/services/<service>/.
   • Common files are named resource_<terraform_resource_name>.go or node_config.go.
2. Implement Changes:
   • Add/update the field in the schema map (Schema: map[string]*schema.Schema).
   • Ensure the field type (TypeSchema), description, optional/required attributes, and ForceNew properties match the GCP API behavior.
   • Update the corresponding expander (which maps KRM/Terraform values to GCP SDK structures) and flattener (which maps GCP responses back to Terraform schema maps).
3. Handle Optional/ForceNew Block Flattening (Critical Pitfall):
   • For optional blocks/structs that are immutable (ForceNew: true), the flattener MUST explicitly return an empty slice or nil if all attributes inside the block are empty. Otherwise, Terraform calculates a diff (from 0 blocks to 1 block containing default values) and fails the update.
   • Example:

     func flattenMyConfig(c *service.MyConfig) []map[string]interface{} {
         if c == nil || c.SomeAttribute == "" {
             return nil // Or return []map[string]interface{}{}
         }
         ...
     }
     

4. Verify Compilation:
   • Run the compiler check locally:

     go vet ./third_party/github.com/hashicorp/terraform-provider-google-beta/...
     

5. Stage Provider Changes:
   • Stage the modified third-party files. You can commit them together with KCC changes as the repository no longer enforces subtree change isolation:

     git add third_party/
     

---

3. Updating KCC CRD Definitions

Case A: The resource is NOT yet migrated to Go types in apis/

KCC dynamically generates the CRD directly from the Terraform schema:

1. Modify Service Mapping:
   • Open config/servicemappings/<service>.yaml.
   • Locate the resource definition block.
   • Ensure the field is not listed in ignoredFields or ignoredOutputOnlySpecFields.
   • Resource References: If the field points to another KCC resource, define it under resourceReferences:

     resourceReferences:
       - tfField: encryption.default_kms_key_name
         key: kmsKeyRef
         gvk:
           kind: KMSCryptoKey
           version: v1beta1
           group: kms.cnrm.cloud.google.com
         targetField: self_link
     

   • Observed Fields: If the field is output-only and should be updated in the resource status, specify it in observedFields.
2. Regenerate CRD Manifests:
   • Run:

     make manifests
     

   • Run git diff config/crds/resources/ to verify that the new field is properly generated in the CRD schema.
3. Regenerate Resource Reference Docs (CRITICAL):
   • Run:

     make resource-docs
     

   • Verify if any documentation or generated files under scripts/generate-google3-docs/resource-reference/generated/ are modified using git status or git diff. If there are modifications, stage and commit them.

Case B: The resource HAS Go types in apis/

If a <kind>_types.go file exists under apis/<service>/<version>/:

1. Update Go Types:
   • Open apis/<service>/<version>/<kind>_types.go.
   • Locate the <Kind>Spec struct (or the nested block struct).
   • Add the field using the correct Go type and tag. Use camelCase for the json tag:

     // +optional
     MyNewField *string `json:"myNewField,omitempty"`
     

   • Ensure you use appropriate kubebuilder annotations (e.g. // +optional, // +kubebuilder:validation:Enum).
   • Resource References: If the new field is a resource reference (e.g., kmsKeyRef of type v1alpha1.ResourceRef), ensure the struct field is defined correctly.
2. Update Service Mapping (For Resource References):
   • CRITICAL: The TF controller relies on service mappings to translate reference fields and map them to the underlying Terraform schema.
   • Even though the resource has Go types, if you are adding a resource reference, you MUST also configure the reference under resourceReferences in config/servicemappings/<service>.yaml:

     resourceReferences:
       - tfField: encryption.default_kms_key_name
         key: kmsKeyRef
         gvk:
           kind: KMSCryptoKey
           version: v1beta1
           group: kms.cnrm.cloud.google.com
         targetField: self_link
     

3. Regenerate CRD & Mappers:
   • Run:

     dev/tasks/generate-types-and-mappers
     

   • Verify the generated files (_types.go tags, auto-generated mappers, and config/crds/resources/) via git diff.
4. Regenerate Resource Reference Docs (CRITICAL):
   • Run:

     make resource-docs
     

   • Verify if any documentation or generated files under scripts/generate-google3-docs/resource-reference/generated/ are modified using git status or git diff. If there are modifications, stage and commit them.

---

4. Verification & E2E Testing

Once definitions are updated, the agent must verify correctness of the field using the fixture testing framework.

1. Check Copyright Headers:

   • If you created any new files, make sure they have the copyright header:

     // Copyright 2026 Google LLC
     //
     // Licensed under the Apache License, Version 2.0 (the "License");
     // you may not use this file except in compliance with the License.
     ...
     

2. Proceed to Test Skill (CRITICAL):

   • Refer to .gemini/skills/test-terraform-fields/SKILL.md to set up E2E tests, record golden files, check against MockGCP, and run linters.

   • [!IMPORTANT] When running E2E verification, do NOT just test your newly created test case. Modifying a resource schema will often affect other existing tests of that resource type (e.g. by adding new fields to their observedState or status). You MUST run the full group presubmit test suite under dev/ci/presubmits/ (e.g., dev/ci/presubmits/tests-e2e-fixtures-<service_name>) to capture and commit these golden file updates.

3. Field Ownership Conflicts with oneOf:

   • Gotcha: Some legacy resources are exempted from Server-Side Apply (SSA) for object creation in ratcheting.go. During fixture tests, the object is created using a legacy Create call, but updates are always applied via Apply (SSA). If your update switches between choices in a oneOf field (e.g., from rrdatas to rrdatasRefs), the legacy field is not owned by the SSA field manager and remains in the live object. This triggers a validation failure: "spec" must validate one and only one schema (oneOf). Found 2 valid alternatives.
   • Solution: Enable Server-Side Apply for the resource by removing it from the exempted list in ratcheting.go. This ensures both creation and updates use SSA, and unreferenced fields under a oneOf choice are correctly cleaned up.

4. Local CI/CD Presubmit Verification (CRITICAL):

   • To ensure that generated PRs do not fail CI/CD validation pipelines (validate-generated-files and validations), you MUST run the resource docs generation and the local presubmit verifications before submitting:

     make resource-docs
     dev/ci/presubmits/validate-generated-files
     scripts/validate-prereqs.sh
     

   • If these scripts or commands generate any updates to auto-generated mappers, CRDs, static configs, documentation, or GitHub Actions workflows, verify with git status and stage and commit them:

     git add -A
     git commit -m "chore: ensure clean generated state for CI/CD presubmits"