Skip to content

box_events: fingerprint the box.event_id field for doc deduplication - #9498

Merged
efd6 merged 3 commits into
elastic:mainfrom
efd6:box_fingerprinting
Apr 10, 2024
Merged

box_events: fingerprint the box.event_id field for doc deduplication#9498
efd6 merged 3 commits into
elastic:mainfrom
efd6:box_fingerprinting

Conversation

@efd6

@efd6 efd6 commented Apr 3, 2024

Copy link
Copy Markdown
Contributor

Proposed commit message

See https://developer.box.com/reference/resources/event/#param-event_id for field semantics.

Checklist

  • I have reviewed tips for building integrations and this pull request is aligned with them.
  • I have verified that all data streams collect metrics or logs.
  • I have added an entry to my package's changelog.yml file.
  • I have verified that Kibana version constraints are current according to guidelines.

Author's Checklist

  • [ ]

How to test this PR locally

Related issues

Screenshots

@efd6 efd6 added enhancement New feature or request Integration:box_events Box Events Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations] labels Apr 3, 2024
@efd6 efd6 self-assigned this Apr 3, 2024
@efd6
efd6 force-pushed the box_fingerprinting branch from 4def614 to 3ef4773 Compare April 3, 2024 04:52
@elasticmachine

Copy link
Copy Markdown

🚀 Benchmarks report

To see the full report comment with /test benchmark fullreport

@efd6
efd6 marked this pull request as ready for review April 3, 2024 05:20
@efd6
efd6 requested a review from a team as a code owner April 3, 2024 05:20
@elasticmachine

Copy link
Copy Markdown

Pinging @elastic/security-service-integrations (Team:Security-Service Integrations)

@andrewkroh andrewkroh left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. I assume this ID is unique across Box accounts (in case users are running multiple instances of the integration).

@efd6

efd6 commented Apr 3, 2024

Copy link
Copy Markdown
Contributor Author

The text in the docs could be interpreted that way, "The ID of the event object. You can use this to detect duplicate events". The language is not quite strong enough for me to be completely happy. We could inject the client ID into the ingest pipeline for inclusion in the fingerprint set and then remove it. I think this would make me happier, though this value is per application, not per owner, so it may move.

The Client ID of the application that is requesting to authenticate the user. To get the Client ID for your application, log in to your Box developer console and click the Edit Application link for the application you're working with. In the OAuth 2.0 Parameters section of the configuration page, find the item labelled client_id. The text of that item is your application's Client ID.

I am unable to find a definitively documented identifier that we could use, so this is probably the best we have.

@efd6
efd6 requested a review from andrewkroh April 4, 2024 04:31

@kcreddy kcreddy left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this would make me happier, though this value is per application, not per owner, so it may move.

LGTM after adding client_id 👍🏼 although a test could be nice to have.

Comment on lines +21 to +23
- remove:
field: _conf.client_id
ignore_missing: true

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this required since there is a remove on _conf field later in the pipeline?

@efd6
efd6 enabled auto-merge (squash) April 10, 2024 20:44
@elastic-sonarqube

Copy link
Copy Markdown

@efd6
efd6 merged commit 6146d87 into elastic:main Apr 10, 2024
@elasticmachine

Copy link
Copy Markdown

💚 Build Succeeded

History

cc @efd6

@elasticmachine

Copy link
Copy Markdown

Package box_events - 2.8.0 containing this change is available at https://epr.elastic.co/search?package=box_events

@efd6
efd6 deleted the box_fingerprinting branch February 5, 2025 22:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request Integration:box_events Box Events Team:Security-Service Integrations Security Service Integrations team [elastic/security-service-integrations]

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants