Skip to content

feat(graphql,ui): Add Runs tab for DataFlow entities #15775

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
askumar27 merged 4 commits into from
Jan 5, 2026
Merged

feat(graphql,ui): Add Runs tab for DataFlow entities #15775

askumar27 merged 4 commits into from
Jan 5, 2026

Conversation

@askumar27
Copy link
Contributor

@askumar27 askumar27 commented Dec 29, 2025 •
edited
Loading

Summary

This PR adds support for displaying pipeline execution runs (history) in the DataFlow entity profile page. This enables users to track and monitor Azure Data Factory pipeline run history directly within DataHub.

Changes

GraphQL Schema (by @asikowitz)

  • Created runs.graphql schema with HasExecutionRuns interface
  • Extended DataFlow and DataJob types to implement HasExecutionRuns
  • Defines the runs(start: Int, count: Int) field returning paginated DataProcessInstanceResult

Backend GraphQL Resolver

  • Renamed DataJobRunsResolver to ExecutionRunsResolver for reuse across entity types
  • Resolver fetches DataProcessInstance entities by:
    • Filtering on parentTemplate field (links runs to their parent DataFlow/DataJob)
    • Filtering for entities with hasRunEvents to ensure only instances with actual run events are returned
    • Sorting by created timestamp in descending order (most recent first)
  • Registered resolver for both DataFlow and DataJob types

Backend Schema Registration Fix

  • Added RUNS_SCHEMA_FILE constant to Constants.java
  • Registered runs.graphql in GmsGraphQLEngine builder
  • Added HasExecutionRuns interface type resolver for proper GraphQL type resolution

Frontend UI

  • Added Runs tab to DataFlowEntity.tsx (entityV2)
  • Updated RunsTab.tsx to use the new generic getExecutionRuns query
  • Tab displays a table with: Time, Run ID, Status, Inputs, Outputs, and external link to Azure Data Factory

Tests

  • ExecutionRunsResolverTest.java - 7 unit tests covering:
    • Success scenarios for DataFlow and DataJob parents
    • Filter and sort criteria validation
    • Pagination parameter handling
    • Empty results handling
    • Exception handling
    • Null entity filtering
  • RunsTab.test.tsx - 3 unit tests covering:
    • Loading state rendering
    • Table headers rendering
    • Empty table rendering

Screenshots

DataFlow Runs Tab

image

The Runs tab displays:

  • Time: When the pipeline run was created
  • Run ID: Unique identifier for the run (e.g., ADF run ID)
  • Status: Failed/Succeeded with color-coded indicators
  • Inputs/Outputs: Data assets consumed/produced by the run
  • External Link: Direct link to Azure Data Factory monitoring page for the run

Testing

  1. Backend tests: ./gradlew :datahub-graphql-core:test --tests "com.linkedin.datahub.graphql.resolvers.jobs.ExecutionRunsResolverTest"
  2. Frontend tests: ./gradlew :datahub-web-react:yarnTest -Pfile=src/app/entityV2/dataJob/tabs/__tests__/RunsTab.test.tsx

Checklist

  • GraphQL schema for HasExecutionRuns interface
  • Backend ExecutionRunsResolver with proper filtering
  • Schema registration in GmsGraphQLEngine
  • Type resolver for HasExecutionRuns interface
  • Frontend Runs tab UI
  • Backend unit tests
  • Frontend unit tests
  • Linting passed

asikowitz and others added 3 commits December 11, 2025 16:47
@asikowitz
feat(graphql,ui): Support data flow runs tab
bda49c4
@askumar27
Merge remote-tracking branch 'oss/master' into dataflow-runs
ddfe6bc
@askumar27
fix(graphql): Register runs.graphql schema and add HasExecutionRuns t…
9f15067 
…ype resolver

This commit completes the DataFlow runs tab feature by adding missing schema
registration and type resolver configuration:

1. Added RUNS_SCHEMA_FILE constant to Constants.java
2. Registered runs.graphql schema file in GmsGraphQLEngine builder
3. Added HasExecutionRuns interface type resolver for DataFlow and DataJob

Without these changes, the runs.graphql schema was never loaded by the GraphQL
engine, causing 'Field runs in type DataFlow is undefined' errors.
@github-actions github-actions bot added the product PR or Issue related to the DataHub UI/UX label Dec 29, 2025
@askumar27 askumar27 requested a review from asikowitz December 29, 2025 22:50
@askumar27 askumar27 changed the title feat(graphql,ui): Add Runs tab for DataFlow entities (Azure Data Factory pipeline runs) feat(graphql,ui): Add Runs tab for DataFlow entities Dec 29, 2025
@datahub-cyborg datahub-cyborg bot added the needs-review Label for PRs that need review from a maintainer. label Dec 29, 2025
@alwaysmeticulous
Copy link

alwaysmeticulous bot commented Dec 29, 2025 •
edited
Loading

✅ Meticulous spotted 0 visual differences across 967 screens tested: view results.

Meticulous evaluated ~8 hours of user flows against your PR.

Expected differences? Click here. Last updated for commit 95baed6. This comment will update as new commits are pushed.

@codecov
Copy link

codecov bot commented Dec 29, 2025 •
edited
Loading

Codecov Report

❌ Patch coverage is 16.66667% with 10 lines in your changes missing coverage. Please review.
✅ All tests successful. No failed tests found.

Files with missing lines Patch % Lines
...com/linkedin/datahub/graphql/GmsGraphQLEngine.java 0.00% 10 Missing ⚠️

❌ Your patch status has failed because the patch coverage (16.66%) is below the target coverage (75.00%). You can increase the patch coverage or adjust the target coverage.

📢 Thoughts on this report? Let us know!

@askumar27
test(graphql,ui): Add tests for ExecutionRunsResolver and RunsTab
95baed6 
Backend tests (ExecutionRunsResolverTest.java):
- testGetRunsForDataFlowSuccess: End-to-end success for DataFlow parent
- testGetRunsForDataJobSuccess: End-to-end success for DataJob parent
- testFilterAndSortCriteriaForDataFlow: Verifies parentTemplate + hasRunEvents filter
- testPaginationParameters: Verifies pagination params are correctly passed
- testEmptyResults: Handles zero runs gracefully
- testExceptionHandling: Verifies RemoteInvocationException handling
- testHandlesNullEntityInBatchGet: Filters out null/deleted entities

Frontend tests (RunsTab.test.tsx):
- renders loading state initially
- renders table headers after loading
- renders empty table when no runs
@codecov
Copy link

codecov bot commented Dec 30, 2025 •
edited
Loading

Bundle Report

Changes will increase total bundle size by 311 bytes (0.0%) ⬆️. This is within the configured threshold ✅

Detailed changes
Bundle name Size Change
datahub-react-web-esm 31.29MB 311 bytes (0.0%) ⬆️

Affected Assets, Files, and Routes:

view changes for bundle: datahub-react-web-esm

Assets Changed:

Asset Name Size Change Total Size Change (%)
assets/index-*.js 311 bytes 19.25MB 0.0%

Files in assets/index-*.js:

  • ./src/app/entityV2/dataJob/tabs/RunsTab.tsx → Total Size: 4.99kB

  • ./src/app/entity/dataJob/tabs/RunsTab.tsx → Total Size: 4.29kB

  • ./src/app/entityV2/dataFlow/DataFlowEntity.tsx → Total Size: 5.98kB

@datahub-cyborg datahub-cyborg bot added pending-submitter-merge and removed needs-review Label for PRs that need review from a maintainer. labels Jan 5, 2026
@askumar27 askumar27 merged commit 17b74d6 into master Jan 5, 2026
48 of 49 checks passed
@askumar27 askumar27 deleted the dataflow-runs branch January 5, 2026 21:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@asikowitz asikowitz asikowitz approved these changes

Assignees

No one assigned

Labels

pending-submitter-merge product PR or Issue related to the DataHub UI/UX

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

2 participants

@askumar27 @asikowitz