[BUG] $ref to common parameter not being resolved

[john3300]

Description

Validation is failing to resolve a $ref in the parameter list to a common parameter defined outside the operation. Neither refs local to the same file nor refs to external files work. Both should. I've tried it with both Swagger 2.0 and OAS 3.0 specs, both fail the same way.

Exception in thread "main" org.openapitools.codegen.SpecValidationException: There were issues with the specification. The option can be disabled via validateSpec (Maven/Gradle) or --skip-validate-spec (CLI).
 | Error count: 1, Warning count: 0
Errors: 
	-attribute paths.'/employees/{id}'. Declared path parameter id needs to be defined as a path parameter in path or operation level

	at org.openapitools.codegen.config.CodegenConfigurator.toClientOptInput(CodegenConfigurator.java:569)
	at org.openapitools.codegen.cmd.Generate.run(Generate.java:346)
	at org.openapitools.codegen.OpenAPIGenerator.main(OpenAPIGenerator.java:62)

openapi-generator version

4.0.0-SNAPSHOT

OpenAPI declaration file content or url

openapi: 3.0.0
info:
  title: Test
  description: Test YAML
  version: '1.0'
servers:
  - url: http://localhost
paths:
  /employees/{id}:
    get:
      operationId: getEmployeeById
      parameters:
        - $ref: '#/components/parameters/id'
      responses:
        200:
          description: Success
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/employee'
components:
  schemas:
    employee:
      type: object
      properties:
        name:
          type: string
      required:
        - name
  parameters:
    id:
      name: id
      in: path
      required: true
      schema:
        type: string

Command line used for generation

java -jar openapi-generator-cli-4.0.0-20181217.102953-101.jar generate -g html -i test.yaml

Related issues/PRs

#455 seems to be related. But that one was resolved. So that makes this a regression.

Suggest a fix

This could be another issue with the swagger-parser like in #455.

[auto-labeler]

👍 Thanks for opening this issue!
🏷 I have applied any labels matching special text in your issue.

Please review the labels and make any necessary changes.

[john3300]

I can't see how to add a label to this issue, but I think it should also have Issue: Regression

[jimschubert]

@john3300 Did this work with another tool or a previous version of the generator? It seems to me this is an issue with swagger parser.

If it previously worked, can you provide the previous version of openapi-generator so we can look at the regression?

[john3300]

It works fine with 3.3.4.

[ericraymond]

FYI if you expand the ref the issue goes away (not that it helps much):

openapi: 3.0.0
info:
  version: 1.0.0
  title: |
    This fails to generate code when the path paramter is defined via schema $ref
     - attribute paths.'/bar/{id}'. Declared path parameter id needs to be defined as a path parameter in path or operation level

paths:
  '/foo/{id}':
    get:
      summary: Returns the foo of a prior request
      operationId: fooGet
      parameters:
        -
          name: id
          in: path
          required: true
          description: ID returned from a prior POST operation of the same name
          schema:
            type: string
      responses:
        '200':
          description: A list of predicted text boxes
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/FooBarResponse'
  '/bar/{id}':
    get:
      summary: Returns the bar of a prior request
      operationId: barGet
      parameters:
        - $ref: '#/components/parameters/ID'
      responses:
        '200':
          description: A list of predicted text boxes
          content:
            application/json:
              schema:
                $ref: '#/components/schemas/FooBarResponse'

components:
  parameters:
    ID:
      name: id
      in: path
      required: true
      description: ID returned from a prior POST operation of the same name
      schema:
        type: string
  schemas:
    FooBarResponse:
      type: object
      properties:
        foo:
          type: string
        bar:
          type: string

[wing328]

@john3300 can you try with --skip-validate-spec? I tried to repeat the issue with the spec you provided but couldn't after using the --skip-validate-spec option.

[ericraymond]

Above test file works via the gradle plugin with validateSpec = false. It complains about the error, but seems to continue.

Larger test case is failing on another issue. This test case would further test the generated code. Will investigate and comment/log bugs as necessary. (Bug might be same as #1657)

[john3300]

As requested by @wing328 I tested with --skip-validate-spec and got the same results as @ericraymond. It lists the error but completes the work.

Obviously that's not a valid long term solution for something that worked in the previous release.

[advance512]

I can confirm this happens with 4.0.0 nightly and JavaScript generator.

I like the validation, so we'll wait until it is working before migrating to 4.0.0.

[jjlin]

@john3300 Did this work with another tool or a previous version of the generator? It seems to me this is an issue with swagger parser.

If it previously worked, can you provide the previous version of openapi-generator so we can look at the regression?

This issue was carried over from swagger-api/swagger-parser#911.

Upstream has an open PR: swagger-api/swagger-parser#977

[philcluff]

I assume this happens in all client generators - maybe we can remove the client:HTML tag?

[wing328]

Obviously that's not a valid long term solution for something that worked in the previous release.

@john3300 please use --skip-validate-spec for the time being until the issue (false alarm) is addressed in the swagger parser.

[wing328]

@geneticgenesis done

[zslajosfalvi]

I tried this with the new 4.0.0 beta2 version and I could still reproduce this problem.
I get error messages like:
-attribute paths.'/accounts/{account-id}/transactions'. Declared path parameter account-id needs to be defined as a path parameter in path or operation level

[fantavlik]

I believe this could be the fix? Hoping it gets merged soon. swagger-api/swagger-parser#977

My team is in the same boat as @advance512, we are blocked from upgrading to 4.0.0 beta2 until this is addressed.

[tcjcodes]

This seems to fail when in: path but in: query params will work.

The workaround for gradle works: openApiGenerate { .... validateSpec = false }

[philcluff]

swagger-api/swagger-parser#977 has now been merged into master of swagger-parser.

Can we pull in the latest swagger-parser and re-test?

Thanks

[ntfrnzn]

https://github.com/swagger-api/swagger-parser/tree/v2.0.9 is released & should be incorporated.

[wing328]

Please try the latest master with the upgrade parser to see if it's still an issue.

[philcluff]

Thanks @wing328

This seems to have fixed this specific issue, but sadly ref'd parameters have now been broken by swagger-api/swagger-parser#1063

[advance512]

@geneticgenesis

What do you mean when you say "now been broken"? You are referencing a different project.

Maybe you mean to say "this fix has exposed an older problem that was hidden" or something like that?

[philcluff]

Sorry I should be more clear @advance512:

Using parameters which involves refs was previously broken by swagger-api/swagger-parser#911 which was fixed by pulling in the upstream change in swagger-parser.

Using parameters which involves refs is now broken because of another issue in swagger-parser - swagger-api/swagger-parser#1063 which is not yet fixed upstream, so if you're using refs under path parameters, you still have to disable verification on master (and V4 betas) of openapi-generator.

[lacksfish]

Having a similar issue:

/random/api_endpoint:
    get:
        parameters:
            - $ref: '#/parameters/some_param'

When automatically generating the code for it, the parameters get omitted and the produced functions take no arguments... Troublesome. :parameters is defined in the root beginning of the same spec document. None of the $ref params make it into the autogenerated code, all methods just have no params if using $ref for the params.

I get a lot of those here when skipping validation as well (may be a separate issue):

[main] WARN  o.o.codegen.utils.ModelUtils - #/components/schemas/types/something_name is not defined

Same thing, :components is defined in the root and beginning of the same file..

[philcluff]

I believe the new issue is now fixed upstream in swagger-api/swagger-parser#1063

It looks like they cut release 2.0.12 of swagger-parser 18 hours ago with these fixes, so hopefully this can get bumped here soon.

[lacksfish]

Definitely sounds like my issue.

So I should update <swagger-parser-version>2.0.11-OpenAPITools.org-1</swagger-parser-version> in pom.xml to 2.0.12 version once available? Or can I check out a specific version right now?

[lacksfish]

@wing328 Since this issue is closed, is there another ticket to refer to? Waiting for the updated swagger parser.

[lacksfish]

"I got my eyes on you~" 🎶

https://github.com/OpenAPITools/swagger-parser/tags

[fantavlik]

For those following this issue or #2631, swagger-parser was updated: #2775 and I've confirmed that the validate function is no longer returning erroneous There are duplicate parameter values messages. So swagger-api/swagger-parser#1063 has now been fixed in openapi-generator. 🎉

[lacksfish]

Awesome. Thanks. I'm onto a new issue now: #2580

[lacksfish]

This might be only a symptom of a greater issue, so for the impatient:

To fix my problem, I've used https://github.com/APIDevTools/json-schema-ref-parser (after turning my main yaml spec into json) and de-referenced all the $ref's automatically with the aforementioned tool.

Then, I was able to generate the client code :)

PSA: This is a hotfix and any code generated should be well tested.

[philcluff]

@lacksfish Interesting... I'm using swagger-cli bundle to do the bundling and dereferencing today, but still run into issues.

I'll take a look at the tool you linked, thanks!

[stefano-xy]

I can still reproduce this problem with 4.0.3. @wing328 , can we please reopen it and release 4.0.4 with the swagger parser that contains the fix?

[ghacupha]

Experienced this issue once when I was running an application with openjdk 8 update 41 RI. I then changed the environment to use oracle jdk 8 update 202 ( 1.8.0_202), and that fixed the problem. My guess: missing schemas in your jdk?

[sumanthakannantha]

This issue is still visible with the latest version.

/v1/api_endpoint:
get:
parameters:
- $ref: '#/parameters/param'
-

Mostly visible for python target
java -jar openapi-generator-cli-6.1.0.jar generate -i swagger.yaml -g python -o python-client

Exception: Cannot invoke "io.swagger.v3.oas.models.media.Schema.getType()" because "schema" is null
at org.openapitools.codegen.DefaultGenerator.processOperation(DefaultGenerator.java:1187)
at org.openapitools.codegen.DefaultGenerator.processPaths(DefaultGenerator.java:1081)
at org.openapitools.codegen.DefaultGenerator.generateApis(DefaultGenerator.java:580)
at org.openapitools.codegen.DefaultGenerator.generate(DefaultGenerator.java:915)
at org.openapitools.codegen.cmd.Generate.execute(Generate.java:465)
at org.openapitools.codegen.cmd.OpenApiGeneratorCommand.run(OpenApiGeneratorCommand.java:32)
at org.openapitools.codegen.OpenAPIGenerator.main(OpenAPIGenerator.java:66)
Caused by: java.lang.NullPointerException: Cannot invoke "io.swagger.v3.oas.models.media.Schema.getType()" because "schema" is null
at org.openapitools.codegen.utils.ModelUtils.isObjectSchema(ModelUtils.java:453)
at org.openapitools.codegen.languages.PythonClientCodegen.getObjectExample(PythonClientCodegen.java:976)
at org.openapitools.codegen.languages.PythonClientCodegen.setParameterExampleValue(PythonClientCodegen.java:1428)
at org.openapitools.codegen.DefaultCodegen.finishUpdatingParameter(DefaultCodegen.java:4712)
at org.openapitools.codegen.DefaultCodegen.fromParameter(DefaultCodegen.java:5042)
at org.openapitools.codegen.DefaultCodegen.fromOperation(DefaultCodegen.java:4346)
at org.openapitools.codegen.DefaultGenerator.processOperation(DefaultGenerator.java:1155)
... 6 more