API Linter v2.0.0

[quirogas]

API Linter v2.0.0

API Linter v2.0.0 is a work in progress. This issue is intended as both an announcement and a single point of communication for the development of the new major version.
The new major version modernizes the linter's core foundation and aligns it with the future of Protobuf Go. It moves the entire codebase from the github.com/jhump/protoreflect module to the google.golang.org/protobuf module.

✨ What's New in 2.0.0?

• Modern protoreflect Backend: The linter now uses the official Go Protobuf reflection library, ensuring better stability and long-term compatibility with the Go ecosystem.
• Future-Proof Architecture: By aligning with google.golang.org/protobuf, we ensure the API Linter remains robust and compatible with future Protobuf advancements.

Release Plan

This new version is a new Go module github.com/googleapis/api-linter/v2. As part of our release process, we are going to make several pre-releases, to ensure the stability of the final version. Release candidates will be available for testing in our releases section, and will be tagged as release candidates using semantic versioning v2.0.0-beta.x.

⚠️ Breaking Changes

This release introduces a breaking change that only affects users who are using the lint and the locations packages. If you only use the api-linter CLI, these changes won’t affect you.

The core change is that descriptor types used in function signatures have changed to their corresponding google.golang.org/protobuf counterparts.

What does this look like?

- import "github.com/jhump/protoreflect/desc"
+ import "google.golang.org/protobuf/reflect/protoreflect"

  var myRule = &lint.MessageRule{
-   LintMessage: func(d *desc.MessageDescriptor) []lint.Problem {
-     if len(d.GetFields()) == 0 { /* ... */ }
+   LintMessage: func(d protoreflect.MessageDescriptor) []lint.Problem {
+     if d.Fields().Len() == 0 { /* ... */ }
      return nil
    },
  }

The migration is straightforward and consists of updating imports, type signatures, and method calls.

How to Try it out

To try out the new api-linter, use one of the following installation methods:

1. Use go install

go install github.com/googleapis/api-linter/v2/cmd/api-linter@v2.0.0-beta.2

2. Download the binary from our releases page

3. Build the binary from source

git clone git@github.com:googleapis/api-linter.git \ 
git fetch origin ref/tags/v2.0.0-beta.2 && git checkout v2.0.0-beta.2 \
go build cmd/api_linter -o api-linterv2.0.0-beta.2

How to report a Bug?

To file a bug report create a new bug, indicate the version in use is a release candidate v2.0.0-beta.2 (we will add a v2 label), and reference this issue as the parent issue to help us keep track of things.

[noahdietz]

Tagging a few people who have been vocal users of this repo as FYI: @loeffel-io @julioz @toumorokoshi (👋🏻 ) @seanlafferty-ibm @fredrikaverpil

Please unsubscribe from the issue if you don't want to follow updates. Thanks for being great users!

[quirogas]

v2.0.0-beta.1

[julioz]

Thank you and congratulations! Looking forward to what the future of the linter will look like.

With the move out of jhump/protoreflect, would this task be the best place for us to spend a moment discussing how the API surface for defining custom rules would look like in v2?

[loeffel-io]

Amazing, congratulations 🎉

[loeffel-io]

We are getting tons of bugs and false negative warnings with beta 1:

    - message: Missing comment over "SetOrganizationIamPolicyResponse". <-- this message does not exists but maybe comes from google.iam.v1.iam_policy.proto somehow
      location:
        start_position:
            line_number: 1
            column_number: 1
        end_position:
            line_number: 1
            column_number: 0
        path: mindful/earth/user/v1/organization.proto
      rule_id: core::0192::has-comments
      rule_doc_uri: https://linter.aip.dev/192/has-comments
    - message: Missing comment over "BATCH_GET_USERS_REQUEST_READ_MASK_ACCOUNT".
      location:
        start_position:
            line_number: 1
            column_number: 1
        end_position:
            line_number: 1
            column_number: 0
        path: mindful/earth/user/v1/user.proto
      rule_id: core::0192::has-comments
      rule_doc_uri: https://linter.aip.dev/192/has-comments

for all kind of protos

//mindful/earth/user/membership/v1:membership_v1_proto_api_linter        FAILED in 0.1s
  /private/var/tmp/_bazel_loeffel/2b8e6c5a281d05be113e22f1346b0da3/execroot/_main/bazel-out/darwin_arm64-fastbuild/testlogs/mindful/earth/user/membership/v1/membership_v1_proto_api_linter/test.log
//mindful/earth/user/profile/v1:profile_v1_proto_api_linter              FAILED in 0.1s
  /private/var/tmp/_bazel_loeffel/2b8e6c5a281d05be113e22f1346b0da3/execroot/_main/bazel-out/darwin_arm64-fastbuild/testlogs/mindful/earth/user/profile/v1/profile_v1_proto_api_linter/test.log
//mindful/earth/user/v1:user_v1_proto_api_linter                         FAILED in 0.1s
  /private/var/tmp/_bazel_loeffel/2b8e6c5a281d05be113e22f1346b0da3/execroot/_main/bazel-out/darwin_arm64-fastbuild/testlogs/mindful/earth/user/v1/user_v1_proto_api_linter/test.log

sh_test(
    name = "user_v1_proto_api_linter",
    srcs = ["@rules//:api_linter"],
    args = [
        "mindful/earth/user/v1/user.proto",
        "mindful/earth/user/v1/organization.proto",
        "--set-exit-status",
        "--descriptor-set-in $(location :user_v1_proto_descriptor_set)",
        "--disable-rule core::0191::java-package",
        "--disable-rule core::0191::java-multiple-files",
        "--disable-rule core::0191::java-outer-classname",
        "--disable-rule core::0157::request-read-mask-field",
        "--disable-rule core::0134::request-mask-field",
        "--disable-rule core::0132::request-field-types",
    ],
    data = [
        ":user_v1_proto",
        ":user_v1_proto_descriptor_set",
    ],
    visibility = ["//visibility:private"],
)

no issues with v1.70.0

[noahdietz]

would this task be the best place for us to spend a moment discussing how the API surface for #1485 would look like in v2?

@julioz not on this bug, no, we'd like to keep this to bug reports and updates for this refactoring specifically. We can reframe the discussion on #1485 to account for this refactoring (as I anticipated there previously 😄 ).

@loeffel-io thank you very much for trying it out. SetOrganizationIamPolicyResponse doesn't exist in googleapis/googleapis so this is weird. The finding path indicates mindful/earth/user/v1/organization.proto - mind just double checking this 😅 . That said, the locations look very off.

@quirogas can we double check the locations in the v1<>v2 googleapis comparison output, please?

[noahdietz]

I've moved this debugging to #1530 and copied comments there

[quirogas]

v2.0.0-beta.2

@loeffel-io Thank you for sharing your findings! Here is a new release addressing your issue.

[loeffel-io]

v2.0.0-beta.2

@loeffel-io Thank you for sharing your findings! Here is a new release addressing your issue.

//mindful/earth/user/membership/v1:membership_v1_proto_api_linter        PASSED in 0.4s
//mindful/earth/user/profile/v1:profile_v1_proto_api_linter              PASSED in 0.4s
//mindful/earth/user/v1:user_v1_proto_api_linter                         PASSED in 0.4s

beautiful @noahdietz @quirogas 🙏 great piece of software

[CuriousStork]

Hello,

I've tested api-linter 2.0.0-beta.2 and #1519 is unfortunately reproduced.

As I'm using files like google/api/field_behavior.proto in my .proto files, this issue prevents me from using v2 as well.

[quirogas]

v2.0.0-beta.3

[quirogas]

v2.0.0-beta.4

@CuriousStork Thank you again for sharing your findings and patiently wait for a fix 😄.