[Azure Logs] Handle event.original field from upstream forwarders

[zmoog]

What does this PR do?

With this PR, the Azure Logs pipelines will use the event.original field over the message field when it exists in the actual event.

Upstream event forwarders like Logstash can add their event.original field, depending on the circumstances (specific
configurations or tool versions), causing issues like #3636.

Checklist

• I have reviewed tips for building integrations and this pull request is aligned with them.
• I have verified that all data streams collect metrics or logs.
• I have added an entry to my package's changelog.yml file.
• I have verified that Kibana version constraints are current according to guidelines.

Related issues

• Closes [Azure Logs] Ingestion fails when event.original is present #3636

[zmoog]

Note for reviewers: I'm not a pipeline expert, and I'd love to hear from you if there are better ways to achieve the same goal.

[elasticmachine]

💚 Build Succeeded

the below badges are clickable and redirect to their specific view in the CI or DOCS

Expand to view the summary

Build stats

• Start Time: 2022-07-01T16:10:33.969+0000

• Duration: 15 min 54 sec

Test stats 🧪

Test	Results
Failed	0
Passed	89
Skipped	0
Total	89

🤖 GitHub comments

To re-run your PR in the CI, just comment with:

• /test : Re-trigger the build.

[elasticmachine]

🌐 Coverage report

Name	Metrics % (covered/total)	Diff
Packages	100.0% (6/6)	💚
Files	78.571% (11/14)	👎 -17.956
Classes	78.571% (11/14)	👎 -17.956
Methods	77.876% (88/113)	👎 -11.489
Lines	81.855% (1606/1962)	👎 -9.088
Conditionals	100.0% (0/0)	💚

[zmoog]

Another note for the reviewers: I'd also love to add a test for this change, but the test infrastructure focuses on the message, and AFAIK it does not offer options to customize event fields. Any advice about how to archive this?

[legoguy1000]

This seems like this would be an issue for every integration of logstash adds an event.original field

[kaiyan-sheng]

Agree with @legoguy1000 and I think the fix should be on logstash side.

[tommyers-elastic]

@kaiyan-sheng i agree this fix should be made on the logstash side, but I think it's reasonable to add this mitigation here as well. especially since we can release the integration independently and ahead of logstash. the only consideration is if this change has any performance impact on the pipeline (it looks like it would be pretty minimal to me)?

the logstash issue to track the fix over there is here

[tommyers-elastic]

these changes look good to me - thanks @zmoog. just one question regarding performance, any idea if the addition of these conditionals come with any measurable perf hit?

[tommyers-elastic]

after following up on the perf element, we do not have anything to worry about here.

[zmoog]

after following up on the perf element, we do not have anything to worry about here.

This sounds good; I'll compare this change's performance impact with the previous version to ensure it doesn't move the needle in the wrong direction.