Fixes#1702.
@chrisst I'm putting you as a reviewer, but no rush. Feel free to ask as many questions as you have! Also feel free to offer suggestions 😃 (or just say it's perfect as-is, that works too)
In testing an upcoming `google_compute_region_disk` resource, I had to make these changes. Checking them in separately so that when the magician runs, these changes will already be a part of TF.
Make it clear that regional backend services are only for internal load
balancing, and fix the default for protocol. It's not HTTP, as the API
docs claim, but is TCP instead.
This was done as its own resource as suggested in slack, since we don't have the option of making all fields Computed in google_compute_instance. There's precedent in the aws provider for this sort of thing (see ami_copy, ami_from_instance).
When I started working on this I assumed I could do it in the compute_instance resource and so I went ahead and reordered the schema to make it easier to work with in the future. Now it's not quite relevant, but I left it in as its own commit that can be looked at separately from the other changes.
Fixes#1582.
An instance is an abstract container of clusters, it's the cluster that
has the nodes and holds the data, so the number of nodes and location
apply to the cluster.
Added node config 'disk_type' which can either be 'pd-standard' or
'pd-ssd', if left blank 'pd-standard' will be the default used by google
cloud.
Closes: #1656
## What
As well as https://github.com/terraform-providers/terraform-provider-google/pull/1282 , make `resource_container_node_pool` importer accept `{project}/{zone}/{cluster}/{name}` format to specify the project where the node pool belongs to actually.
## Why
Sometimes I want to import container pool in different project from default SA's. However, currently there is no way to specify project the target node pool belongs to, Terraform tries to retrieve node pool from SA's project, then it fails due to `You cannot import non-existent resources using Terraform import.` error.
As discussed in #1326, we're not going to remove name_prefix for
compute_ssl_certificate, because it makes the common use case more
ergonomic by a good amount, and the only cost is it's harder to maintain
the autogenerated code, and we've decided the benefits outweigh the
costs in this circumstance.
* Added a link to the console page where you can download a file
* Removed instructions on how to get to that page, since now you can just click on the link
* Added caveat for application default credentials
@sethvargo @theacodes @kimcam let me know if this seems reasonable / you have any suggestions!
If you try and specify them together you will get this error:
google_compute_global_forwarding_rule.your-rule: Error creating Global Forwarding Rule: googleapi: Error 400: Invalid value for field 'resource.ipVersion': 'IPV4'. Both IP Version and IP Address cannot be specified., invalid
* Allow using in repo configuration for cloudbuild trigger
Cloudbuild triggers have a complex configuration that can be defined
from the API. When using the console, the more typical way of doing this
is to defined the configuration within the repository and point the
configuration to the file that defines the config.
This can be supported by sending the filename parameter instead of the
build parameter, however only one can be sent.
* Acceptance testing for cloudbuild trigger with filename
Ensure that when a cloudbuild repo trigger is created with a filename,
that filename is what actually ends up in the cloud.
* Don't specify "by default" in cloudbuild-trigger.
The docs shouldn't say that "cloudbuild.yaml" is used by default. There
is no default from the APIs, but the console suggest using this value.
Just say it's the typical value in documentation.
* Clarify format of GCP machineType property
This should not be in a URL style formatting, which the previous language seemed to be implying
* machineType docs for compute_instance_template
* Remove some accidental spaces
* Update from feedback
Closes GH-1323
* Update container_cluster.html.markdown
Just a more clear explanation of what happens when this field is not provided, since there was already an issue with this topic.
* Update container_cluster.html.markdown
Cleared extra comma.
* Add "server_ca_cert" Attribute reference
Documents the addition of the `server_ca_cert` attribute for use in configuring SSL with GCP SQL Instances - added in PR #1020
* Updated formatting
Updates formatting to match the above fields.
* vendor service usage api
* use serviceusage api instead of servicemanagement for project services
* add bigquery-json to test
* add import for project service
* add serviceusage_operation.go
The docs currently don't use the datasource version of
'google_client_config'. This PR just prefixes 'data.'
to 'google_client_config' in the datasource docs.
IAP has no reasonable support policy, because PATCH is broken, and IAP
must be configured with an OAuth2 client ID and secret that belongs to
the project the app is associated with. There's no programmatic way to
create Clients. But we create the project and the app at the same time,
and we can't update because PATCH is broken. So this just drops IAP. It
also forces all our updates to ForceNew, because we can't update.
Also, adds more test coverage and docs, and fixes import by not relying
on the config for setting app engine info in state.
* Revert "Merge pull request #1434 from terraform-providers/paddy_revert_beta"
This reverts commit 118cd71201, reversing
changes made to d59fcbbc59.
* add ConvertSelfLinkToV1 calls to places where beta links are stored
This PR also switched us to using the beta API in all cases, and that had a side effect which is worth noting, note included here for posterity.
=====
The problem is, we add a GPU, and as per the docs, GKE adds a taint to
the node pool saying "don't schedule here unless you tolerate GPUs",
which is pretty sensible.
Terraform doesn't know about that, because it didn't ask for the taint
to be added. So after apply, on refresh, it sees the state of the world
(1 taint) and the state of the config (0 taints) and wants to set the
world equal to the config. This introduces a diff, which makes the test
fail - tests fail if there's a diff after they run.
Taints are a beta feature, though. :) And since the config doesn't
contain any taints, terraform didn't see any beta features in that node
pool ... so it used to send the request to the v1 API. And since the v1
API didn't return anything about taints (since they're a beta feature),
terraform happily checked the state of the world (0 taints I know about)
vs the config (0 taints), and all was well.
This PR makes every node pool refresh request hit the beta API. So now
terraform finds out about the taints (which were always there) and the
test fails (which it always should have done).
The solution is probably to write a little bit of code which suppresses
the report of the diff of any taint with value 'nvidia.com/gpu', but
only if GPUs are enabled. I think that's something that can be done.
* adding google folder data source with get by id, search by fields and lookup organization functionality
* removing search functionality
* creating folders for each test and updating documentation with default values
* Add support for regional GKE clusters in google_container_cluster:
* implement operation wait for v1beta1 api
* implement container clusters get for regional clusters
* implement container clusters delete for regional cluster
* implement container clusters update for regional cluster
* simplify logic by using generic 'location' instead of 'zone' and 'region'
* implement a method to generate the update function and refactor
* rebase and fix
* reorder container_operation fns
* cleanup
* add import support and docs
* additional locations cleanup
* Updates the default GKE legacy ABAC setting to false
* Updates docs for container_cluster
* Update test comments
* Format fix
* Adds ImportState test step to default legacy ABAC test
* Add time partitioning field to google_bigquery_table resource
* Fix flatten time partitioning field to google_bigquery_table resource
* Add resource bigquery table time partitioning field test
* Move resource bigquery table time partitioning field test to basic
* Add step to check that all the fields match
* Mark resource bigquery table time partitioning field as ForceNew
* Add time partitioning field test to testAccBigQueryTable config
* Updated google.golang.org/api/container/v1beta1
* Added support for private_cluster and master_ipv4_cidr
This is to implement #1174. See
https://groups.google.com/forum/#!topic/google-cloud-sdk-announce/GGW3SQSANIc
* Added simple test for private_cluster and master_ipv4_cidr
* Review replies
* Added some documentation for private_cluster
* move setid calls back
* add support for pod security policy
* pod security policy docs
* Revert "move setid calls back"
This reverts commit 0c7b2dbf92aff33dac8c5beb95568c2bc86dd7de.
* cleanup
* remove comments about disabling update
* add extra wait for storage bucket object deletion
* make timeout for object deletion 5 minutes, make it succeed 3 times
* delete the cluster before deleting the bucket
* deprecate delete_autogen_bucket
* improve deprecation message
Exposes existing `google_compute_backend_service` as data sources.
This addresses #149 .
This allows, for instance, to collect a backend service's self_link and
use it from an other workspace/tfstate, sharing most of the
loadbalancers definition.
* add import helpers for generated code
* Updates to backend bucket and transport.go from MM
* add generated http(s)_health_check resources
* name is required; transport import style
* update docs with new fields/timeouts
* fixes
* Support `distributionPolicy` when creating regional instance group managers.
* Better match the API structure of distributionPolicy.
* Switch to "distribution_policy_zones".
This approach lets us more simply allow a list of zones to use, while
providing a deprecation path for implementing the distribution policy
field more holistically, avoiding backwards-incompatible changes.
* fix typo
* use slice instead of Set for flattenDP
* Storage Default Object ACL resource
* Fixed the doc
* Renamed the resource id. Log change
* Complying with go vet
* Changes for review
* link to default object acl docs in sidebar
* Support for GCS notifications
* docs for storage notification
* docs for storage notification
* Clarified the doc
* Doc modifications
* Addressing requested changes from review
* Addressing requested changes from review
* Using ImportStatePassthrough
* Storage Default Object ACL resource
* Fixed the doc
* Renamed the resource id. Log change
* Complying with go vet
* Changes for review
* link to default object acl docs in sidebar
* Import google_compute_shared_vpc_host_project/google_compute_shared_vpc_service_project resources.
* Incorporate testing of resource import into main acceptance tests.
* Add update support for compute instance fields that require the machine to be stopped
* add warnings in docs about stopping the instance before updating
* add allow_stopping_for_update field
* Update sqladmin api
Pull in updates to the generated sqladmin api and update callers for
the change in the StorageAutoResize setting
* Add support for availability_type setting
Allow specifying ZONAL or REGIONAL to allow for PostgreSQL HA
setup.
* vendor: update sqladmin/v1beta4
* Test setting AvailabilityType for PostgreSQL
Add tests that cover the creation of a Postgres database with
AvailabilityType set to REGIONAL, and correct some small issues that
were preventing compilation.
* Fix breaking change w/ disk_autoresize in cloudsql
95e5582766
The cloudsql admin client changed the way it handles StorageAutoResize
as a parameter, in order to be more explicit about when the server has
ommitted the field. This changed the type from being bool to *bool, and
we need to modify provider code so that we supply the right value to the
api client.
* read boot disk initialization param from API
* make fmt
* Mark the initialize_params list as computed to support boot source
* Ensure private family test follow naming pattern
* Improve docs
* Add import support to google_dns_record_set
* Add import test to NS record
* Minimize diff change
* Improve docs
* Make error message more helpful
* Add note about trailing dot at the end of the record name
Add support for Google Dataflow jobs
Note: A dataflow job exists when it is in a nonterminal state, and does not exist if it
is in a terminal state (or a non-running state which can only transition into terminal
states). See doc for more detail.
* Add internalIpOnly support for Dataproc clusters
* Add internal_ip_only to dataproc cluster docs
* Add default/basic dataproc internal ip test case
* Add test for dataproc internal_ip_only=true
* fixup cluster_config.gce_cluster_config to include .0.
* Remove redundant depends_on
* Add %s rnd to network and subnetwork
* Use variable for subnet CIDR and reference via source_ranges
* Add depends_on back to dataproc cluster test
* Fix cluster attribute refs (.0. again)
* Add 'google_organization' data source.
* Use 'GetResourceNameFromSelfLink'.
* Remove 'resourcemanager_helpers'.
* Use 'ConflictsWith' in schema.
* Add 'organization' argument and make 'name' an output-only attribute.
* Add 'google_billing_account' data source.
* Use 'GetResourceNameFromSelfLink'.
* Use 'ConflictsWith' in schema.
* Use pagination for List() API call.
* Add ability to filter by 'open' attribute.
* Don't use 'ForceNew' for data sources.
* Add 'billing_account' argument and make 'name' an output-only attribute.
* Correct error message.
* Add google_kubernetes_cluster datasource
Add documentation for google_kubernetes_cluster datasource
Rename datasource to google_container_cluster
To be consistent with the equivalent resource.
Rename datasource in docs.
google_kubernetes_cluster -> google_container_cluster.
Also add reference in google.erb file.
WIP
Datasource read needs to set an ID, then call resource read func
Add additional cluster attributes to datasource schema
* Generate datasource schema from resource
Datasource documentation also updated.
* add test for datasourceSchemaFromResourceSchema
* Code review changes
* Add IAM support for pubsub topic
* Fix resource name
* Add update test for iam_policy resource
* Standardize policy conversion function
* Standardize policy conversion function all resources
* Create google_kms_secret datasource
* Create google_kms_secret datasource documentation
* Remove duplicated code
* Create acceptance test
* Fix indentation
* Add documentation to sidebar
* Update Cloud SDK link in docs
* Oxford comma
* Rename variable to make it clear which resource is under test
* Update test to use utils from provider_test
* Add new data source: compute region instance group manager's groups.
* Add documentation for wait_for_instances and for the timeout mechanism in resourceComputeRegionInstanceGroupManagerCreate.
Add consistency for for IAM imports.
- Adds imports for projects, folders, crypto keys, organizations, and key rings.
- Anything else with IAM can implement a simple method and begin working immediately.
- Add tests for all the IAM imports.
- Import documentation for IAM resources.
We removed ipv4_range, but the API still exists, it's just deprecated.
This breaks configs for users that haven't migrated off yet. I added it
back, added some tests to use it, included it in the docs, and basically
tried to put things back the way they were. The main difference now is
that the auto_create_subnetworks field defaults to true, and we want to
keep that behaviour to avoid a breaking change. So now if users want to
use the lagacy API, they need to set auto_create_subnetworks to false
explicitly.
* Move AliasIpRange helpers into utils
To reflect the fact they'll be used by multiple resources.
* Pass Config to build helpers, not meta
It's the only thing meta is used for.
* Refactor getNetwork util methods to return early for the happy path.
* Update compute APIs
compute.Instance.MinCpuPlatform is now GA.
* Fix panic in TestComputeInstanceMigrateState
This seemed to be a pre-existing issue, i.e. I could repro it in master.
--- FAIL: TestComputeInstanceMigrateState (0.00s)
panic: interface conversion: interface {} is nil, not *google.Config [recovered]
panic: interface conversion: interface {} is nil, not *google.Config
goroutine 85 [running]:
testing.tRunner.func1(0xc4205d60f0)
/usr/local/Cellar/go/1.9.1/libexec/src/testing/testing.go:711 +0x2d2
panic(0x203acc0, 0xc4205d2080)
/usr/local/Cellar/go/1.9.1/libexec/src/runtime/panic.go:491 +0x283
github.com/terraform-providers/terraform-provider-google/google.migrateStateV3toV4(0xc4205f2000, 0x0, 0x0, 0x0, 0x48, 0xc4205f2000)
/Users/negz/control/go/src/github.com/terraform-providers/terraform-provider-google/google/resource_compute_instance_migrate.go:182 +0x2405
github.com/terraform-providers/terraform-provider-google/google.resourceComputeInstanceMigrateState(0x2, 0xc4205f2000, 0x0, 0x0, 0x0, 0x0, 0xe0000000000)
/Users/negz/control/go/src/github.com/terraform-providers/terraform-provider-google/google/resource_compute_instance_migrate.go:48 +0x21a
github.com/terraform-providers/terraform-provider-google/google.runInstanceMigrateTest(0xc4205d60f0, 0x2260816, 0x8, 0x227d23a, 0x20, 0x2, 0xc4205ec0f0, 0xc4205ec120, 0x0,
0x0)
/Users/negz/control/go/src/github.com/terraform-providers/terraform-provider-google/google/resource_compute_instance_migrate_test.go:803 +0xc1
github.com/terraform-providers/terraform-provider-google/google.TestComputeInstanceMigrateState(0xc4205d60f0)
/Users/negz/control/go/src/github.com/terraform-providers/terraform-provider-google/google/resource_compute_instance_migrate_test.go:71 +0xc84
testing.tRunner(0xc4205d60f0, 0x22d81c0)
/usr/local/Cellar/go/1.9.1/libexec/src/testing/testing.go:746 +0xd0
created by testing.(*T).Run
/usr/local/Cellar/go/1.9.1/libexec/src/testing/testing.go:789 +0x2de
FAIL github.com/terraform-providers/terraform-provider-google/google 0.035s
* Use only the v1 API for resource_compute_instance
Alias IP ranges, Accelerators, and min CPU platform are now GA.
* Move common instance code into utils.go
Methods used by both resource_compute_instance and
resource_compute_instance_template are currently spread between their respective
files, and utils.go.
This commit moves them all into utils.go for the sake of consistency. It may be
worth considering an instance_common.go file or similar.
* Unify compute_instance and compute_instance_template network_interface and service_account code
This has the side effect of enabling Alias IP range support for
compute_instance_templates.
* Add tests for compute instance template Alias IP ranges
* Mark instance template region as computed
We compute it from the subnet its network interfaces are in. Note this
is not new behaviour - I believe it was erroneously missing the computed
flag.
* Support guest accelerators for instance templates
Since most of the code is already there.
* Add a test for using 'address' rather than 'network_ip' for instance templates
* Don't mark assigned_nat_ip as deprecated
* Remove network_interface schema fields that don't make sense for a compute instance template
* Add newline after count in instance template docs
* Don't try to dedupe guest accelerator expansion code
The API calls to Google to create guest accelerators take different values
for instances and instance templates. Instance templates don't have a zone
and can thus *only* be passed a guest accelerator name.
* Use ParseNetworkFieldValue instead of getNetworkLink
* Add support for parsing regional fields, and subnetworks specifically
Currently unused because subnetworks may have a separate project from that
of the instance using them, which complicates looking up the project field.
* Fall back to provider region when parsing regional field values
Also slightly refactors getXFromSchema field helper functions for readability.
* Revert to assigned_nat_ip in compute instance docs
* Add beta scaffolding to compute instance and compute instance template
Note these resources don't currently use beta features - this is futureproofing.
* Fix indentation in comment about instance template alias IP ranges
* Consolidate metadata helper functions in metadata.go
* Move compute instance (and template) related helpers into their own file
* add support for ip aliasing in `google_container_cluster`
* [review] cleanup galore, infer feature enablement from `ip_allocation_policy`
* [review] cleanup, round 2
* add nil check back (when reading ip allocation policy from API)
* Add IAM bindings and member resources for KMS KeyRings
* Add IAM bindings and member resources for KMS CryptoKeys
* Docs for key ring and crypto key IAM resources
* Exctract KMS policy conversions to helper functions
* Split iam_binding and iam_member tests for KMS
* Docs for kms IAM member resources
* Run KMS IAM tests in own project
* add support for `google_container_node_pool` management (sans-tests)
* [review] add tests, docs, general cleanup
* add docs
* [review] amend test to check updates and terraform fmt
* test updates, make nested management fields non-computed
* Add the "compute_global_address" datasource
* Add a basic test for the "compute_global_address" datasource
* Include the "compute_global_address" in the provider
* Add docs for the "compute_global_address" datasource
* Remove import support for org policy
* Add ForceNew to org_id field
* Revert "Remove import support for org policy"
This reverts commit f65d9fb347d6b0998be55ef823c5726e876c53ae.
* Add import documentation for google_organization_policy
* Add the "compute_address" datasource
* Add a basic test for the "compute_address" datasource
* Include the "compute_address" datasource in the provider
* Add the documentation for the "compute_address" datasource
* Adds support for creating KMS CryptoKeys resources
* Destroy extant CryptoKeyVersions on CryptoKey destroy
* Inherit project, location etc from KeyRing in CryptoKey
* Add function to calculate next rotation
* Implement RotationPeriod parameter on CryptoKey
* Import CryptoKey state
* Uncommit my local acceptance test hacks
* Docs for google_kms_crypto_key
* Clear id at the end of CryptoKey deletion
Also add more detail to warning message.
* Fix parseCryptoKeyId error messages
* Use correct naming in CryptoKeyIdParsing test
* Check RotationPeriod is present in acceptance test
* Rename variable in test function for consistency
* Fix wrong resource name in cryptokey docs
* Add KeyRing to CryptoKey doc example
* Run test CryptoKey configs through terraform fmt
* Don't set CryptoKey purpose in terraform state on import
* Fix indentation in CryptoKey test
* Parallelise CryptoKey tests
* Set rotation_key on CryptoKey read
* Move RotationPeriod validation to planning phase
* Use import state passthrough for CryptoKey
* Correct casing issues in test case names
* Remove redundant CheckDestroy calls in CryptoKey tests
* Add explanatory comment about extra test steps
* More explicit error handling in CryptoKey tests
* Explicit dependency on project services in test keyring configs
* Clean up comments in cryptokey resource
* Do not repeat in cryptokey id regexes