Cloud SQL Proxy Dial Error

[james926]

Hi all,

I've been connected to a Cloud SQL Postgres DB with `cloud-sql-proxy` and earlier this week it was working without issue. Earlier today attempts to connect this way failed, with the error `failed to connect to instance: Dial error: failed to dial` appearing when I tried to connect to the database. Note that the proxy happily authorises with application default credentials (my creds that have global powers for my GCP account), starts listening and accepts connections - but any connection attempt fails. Similar errors occurred using `

cloud-sql-python-connector` but whitelisting my IP and connecting directly worked without issue. 

I since created a second db2, and have been able to connect to this database using all of the above methods - no issues present despite the 2 databases being configured identically. 

I have tried multiple different networks, disabling router firewalls, restarting both my computer and the cloud SQL instances and have had no luck being able to connect to db1 through the proxy.

Its worth noting that if I try to authenticate with an invalid user (or incorrect password), my connection to db2 gives me a `password authentication failed` error while db1 has the same connection error - indicating that I cannot reach db1 for some reason?

A comparison of the settings of the 2 databases is below, I can't see any reason why I can only connect to one of them.

db	db1	db2
backendType	SECOND_GEN	SECOND_GEN
connectionName	[REDACTED]	[REDACTED]
createTime	'2023-07-30T20:05:16.136Z'	'2023-08-05T22:54:52.032Z'
databaseInstalledVersion	POSTGRES_15_2	POSTGRES_15_2
databaseVersion	POSTGRES_15	POSTGRES_15
etag	[REDACTED]	[REDACTED]
gceZone	europe-west1-d	europe-west1-d
instanceType	CLOUD_SQL_INSTANCE	CLOUD_SQL_INSTANCE
ipAddresses	[REDACTED]	[REDACTED]
kind	sql#instance	sql#instance
maintenanceVersion	POSTGRES_15_2.R20230530.01_08	POSTGRES_15_2.R20230530.01_08
name	[DB1-NAME]	[DB2-NAME]
project	[SAME]	[SAME]
region	[SAME]	[SAME]
selfLink	[REDACTED]	[REDACTED]
serverCaCert	[CERT DETAILS]	[CERT DETAILS]
serviceAccountEmailAddress	[REDACTED]	[REDACTED]
settings.activationPolicy	ALWAYS	ALWAYS
settings.availabilityType	ZONAL	ZONAL
settings.backupConfiguration	[DETAILS]	[DETAILS]
settings.connectorEnforcement	NOT_REQUIRED	NOT_REQUIRED
settings.dataDiskSizeGb	'10'	'10'
settings.dataDiskType	PD_SSD	PD_SSD
settings.deletionProtectionEnabled	false	false
settings.edition	ENTERPRISE	ENTERPRISE
settings.insightsConfig	[DETAILS]	{}
settings.ipConfiguration	ipv4Enabled: true, requireSsl: false	ipv4Enabled: true, requireSsl: false
settings.locationPreference	zone: europe-west1-d	zone: europe-west1-d
settings.maintenanceWindow	day: 0, hour: 0, kind: sql#maintenanceWindow	day: 0, hour: 0, kind: sql#maintenanceWindow
settings.pricingPlan	PER_USE	PER_USE
settings.replicationType	SYNCHRONOUS	SYNCHRONOUS
settings.settingsVersion	'32'	'2'
settings.storageAutoResize	true	true
settings.storageAutoResizeLimit	'0'	'0'
settings.tier	db-f1-micro	db-f1-micro
state	RUNNABLE	RUNNABLE

For reference, here is a dump of the logs from cloud-sql-proxy:

./cloud-sql-proxy project:region:db1
2023/08/06 00:08:31 Authorizing with Application Default Credentials
2023/08/06 00:08:32 [project:region:db1] Listening on 127.0.0.1:5432
2023/08/06 00:08:32 The proxy has started successfully and is ready for new connections!
2023/08/06 01:45:40 [project:region:db1] accepted connection from 127.0.0.1:53655
2023/08/06 01:46:10 [project:region:db1] failed to connect to instance: Dial error: failed to dial (connection name = "project:region:db1"): dial tcp [db1_ip_address]:3307: i/o timeout

Does anyone have any ideas or pointers on what to try next?

Thank you!

[ms4446]

Given the details you've provided, it appears that the cloud-sql-proxy is struggling to establish a connection to your db1 instance. This could be attributed to several factors. Let's break them down and provide a systematic approach to troubleshooting:

Potential Causes:

• Firewall Restrictions: A firewall rule might be blocking traffic from your proxy to the db1 instance.
• Network Issues: There could be a network-related problem preventing the proxy from accessing the db1 instance.
• Proxy Configuration: There might be a misconfiguration in the cloud-sql-proxy settings.

Troubleshooting Steps:

1. Verify Firewall Rules: Check the firewall settings in your Google Cloud Console to ensure that traffic from your proxy's IP is allowed to reach the db1 instance.
2. Try a Direct Connection: Try connecting directly to the db1 instance from your machine. If this works, it indicates that the network path is clear, and the issue likely lies with the cloud-sql-proxy configuration.
3. Check Proxy Logs: Examine the cloud-sql-proxy logs for any specific error messages or indications that might point to the root cause. The logs can provide valuable insights into connection failures.
4. Check Service Account Permissions: If you're using a service account with the proxy, ensure it has the necessary permissions (Cloud SQL Client role at a minimum) to connect to the db1 instance.

Additional Considerations:

• Private IP Address: The cloud-sql-proxy typically uses the database instance's private IP for connections. Ensure you're on the same network or VPC as the db1 instance.
• VPN Configurations: If you're connecting via a VPN, ensure it's configured to allow traffic to the db1 instance's private IP.
• Proxy Server: If there's an intermediary proxy server, ensure it's set up to forward traffic correctly to the db1 instance's private IP.

Conclusion:

Given that you can connect directly to the db1 instance and considering the provided logs, it's highly probable that the issue lies within the cloud-sql-proxy configuration or its interaction with the network.

Here are some specific things you can try:

• Check the firewall rules for your network to make sure that traffic to the db1 instance is allowed.
• Try connecting to the db1 instance using a different computer or network.
• Reconfigure the cloud-sql-proxy settings according to the documentation.

[ramiroz]

@james926 I am having the same issue, trying to connect from a VM to a database using cloud sql auth proxy and private ip. Any luck?

[llmiell]

Hi,

@james926 @ramiroz 

I faced on this issue , Have any luck on this ?

Thanks

[H1berto]

Também estou tendo o mesmo problema!

[Diogo_Reis]

I'm facing a similar issue!

I'll discribe what I have done so far. I was trying to connect with gcp-cloud-proxy and it was giving me 403 ERROR, like bellow:

"boss::NOT_AUTHORIZED: Not authorized to access resource. Possibly missing permission cloudsql.instances.get on resource instances/[[MY_INSTANCE_HERE]]., forbidden"

After a lot of research, then I tryied to connect directly by the public IP and it works. So I tryied to connect again with gcp-cloud-proxy, but now adding da token genereted by the google api and again it works.

But since I was aiming to connect throught a private IP, I changed the configuration on my instance to connect only with private, then I tryied one more time to connect using the proxy and I used a similar console command like this one:

.\cloud-sql-proxy.exe --port 5432 --private-ip --token [[GIVEN_TOKEN_ON_GOOGLE_API]] [[INSTANCE_CONNECTION_STRING]]

 After that I received the same Dial Error reported by @james926

failed to connect to instance: Dial error: failed to dial (connection name = "[[MY_INSTANCE_NAME_HERE]]"): dial tcp [[IP_HERE]]:3307: connectex: A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.

Other things that I tryied with no success:

• Remove and add again 'Cloud SQL Client' and 'Cloud SQL Admin'
• Created my own role with "cloudsql.instances.get" and "cloudsql.instances.login", adding this role to both my account and api account 
•  Created another project under the same organization and creating a new postgres sql instance

I hope someone could help us!