-
Notifications
You must be signed in to change notification settings - Fork 668
test: run the bigtable system tests in google cloud build #8245
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Changes from all commits
f14703f
248638f
cdb2ad1
372a03b
7caf8a0
55b0dec
7a3fadd
5f930f9
f3772ea
83f73dd
d7782b2
6be7893
1f71bd1
e6cc411
c836303
1ab42bc
8e6ff1d
051f019
20b11db
a74e61e
0aae31a
af443d1
57bbdad
f6d43fe
8dd7d70
fd18c1a
bfe5155
e51a2e8
11f861b
88a19c7
c6a65a8
83539cc
9f6587c
9916905
3ce9c3f
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,98 @@ | ||
| steps: | ||
| # 0. Check for changes in handwritten/bigtable | ||
| - name: 'gcr.io/cloud-builders/git' | ||
| id: 'check-changes' | ||
| entrypoint: 'bash' | ||
| args: | ||
| - '-c' | ||
| - | | ||
| git fetch origin main --quiet | ||
| # Compare with origin/main to detect changes. | ||
| if git diff --quiet origin/main... -- . ; then | ||
| echo "No changes in handwritten/bigtable. Skipping tests." | ||
| echo "skip" > /workspace/skip_tests | ||
| else | ||
| echo "Changes detected in handwritten/bigtable. Proceeding with tests." | ||
| fi | ||
| dir: 'handwritten/bigtable' | ||
|
|
||
| # 1. Set up Node.js environment | ||
| - name: 'gcr.io/cloud-builders/npm' | ||
| entrypoint: 'bash' | ||
| args: | ||
| - '-c' | ||
| - | | ||
| if [ -f /workspace/skip_tests ]; then exit 0; fi | ||
| npm install -g npm@latest | ||
| npm config set prefix /usr/local | ||
| npm install | ||
| npm run presystem-test | ||
| dir: 'handwritten/bigtable' | ||
| id: 'install-dependencies' | ||
| waitFor: ['check-changes'] | ||
|
|
||
| # 2. Configure environment variables for the tests and run system tests | ||
| # - GOOGLE_APPLICATION_CREDENTIALS: GCB steps run as a service account | ||
| # that is typically granted permissions directly. Explicitly setting | ||
| # GOOGLE_APPLICATION_CREDENTIALS might not be needed if the GCB service | ||
| # account has the right roles (e.g., Bigtable Admin, Bigtable User). | ||
| # If you need to use specific service account key JSON, you'd store it | ||
| # in Secret Manager and mount it here. For simplicity, we'll rely on | ||
| # the GCB service account's inherent permissions. | ||
| # - GCLOUD_PROJECT: Can be passed as a build variable. | ||
| - name: 'gcr.io/cloud-builders/npm' | ||
| entrypoint: 'bash' | ||
| args: | ||
| - '-c' | ||
| - | | ||
| if [ -f /workspace/skip_tests ]; then exit 0; fi | ||
| npm run system-test | ||
| dir: 'handwritten/bigtable' | ||
| env: | ||
| - 'GCLOUD_PROJECT=${_GCP_PROJECT_ID}' # Pass project ID via build variable | ||
| # If you need specific credentials from Secret Manager, uncomment these: | ||
| # - 'GOOGLE_APPLICATION_CREDENTIALS=/secrets/sa-key.json' | ||
| id: 'run-system-tests' | ||
| waitFor: ['install-dependencies'] | ||
| # For Secret Manager, uncomment these (adjust secret name and volume path as needed): | ||
| # secretEnv: ['SA_KEY'] | ||
| # volumes: | ||
| # - name: 'sa-keys' | ||
| # path: '/secrets' | ||
|
|
||
| # 3. (Optional) Code Coverage Reporting | ||
| - name: 'gcr.io/cloud-builders/npm' | ||
| entrypoint: 'bash' | ||
| args: | ||
| - '-c' | ||
| - | | ||
| if [ -f /workspace/skip_tests ]; then exit 0; fi | ||
| # Check if nyc is installed and run report | ||
| if [ -f ./node_modules/nyc/bin/nyc.js ]; then | ||
| ./node_modules/nyc/bin/nyc.js report || true # `|| true` prevents build failure if nyc report itself exits non-zero | ||
|
Comment on lines
+71
to
+72
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Reaching into if [ -f ./node_modules/.bin/nyc ]; then
./node_modules/.bin/nyc report || true # `|| true` prevents build failure if nyc report itself exits non-zero
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It system tests for other libraries, nyc already reaches into the node_modules to do the code coverage report so I don't think we should risk changing this. |
||
| else | ||
| echo "nyc not found, skipping coverage report." | ||
| fi | ||
| # The original codecov.sh script from Kokoro needs to be made available to GCB. | ||
| # Options: | ||
| # a) Commit codecov.sh into your repo (e.g., .kokoro/codecov.sh) and call it: | ||
| # if [ -f .kokoro/codecov.sh ]; then . ./.kokoro/codecov.sh; fi | ||
| # b) Replicate its functionality directly in this step. | ||
| # c) Store it in a GCS bucket and fetch it. | ||
| echo "Codecov reporting (if desired) would be integrated here." | ||
| dir: 'handwritten/bigtable' | ||
| id: 'coverage-report' | ||
| waitFor: ['run-system-tests'] | ||
|
|
||
| # If you use Secret Manager for credentials, uncomment and configure: | ||
| # availableSecrets: | ||
| # secretManager: | ||
| # - versionName: projects/${PROJECT_ID}/secrets/YOUR_SERVICE_ACCOUNT_KEY_SECRET_NAME/versions/latest | ||
| # env: 'SA_KEY' # This env var will hold the secret value. Use it as GOOGLE_APPLICATION_CREDENTIALS in step 3 if needed. | ||
|
|
||
| # Define a substitution variable for your project ID | ||
| # Replace 'long-door-651' with the actual GCP Project ID your system tests should run against. | ||
| substitutions: | ||
| _GCP_PROJECT_ID: 'long-door-651' | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The project ID _GCP_PROJECT_ID: '${PROJECT_ID}'
Contributor
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. We used to hardcode the project so this change is likely okay. |
||
|
|
||
| timeout: '10800s' | ||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Updating
npmglobally and changing its prefix during the build process increases execution time and is generally unnecessary in thegcr.io/cloud-builders/npmenvironment. The builder image already provides a stable version ofnpm. Removing these steps will streamline the CI pipeline and improve build speed.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I already made this change in #8192.