Skip to content

Flaky(UI) : Bulk import#25653

Merged
chirag-madlani merged 10 commits intoopen-metadata:mainfrom
dhruvjsx:flaky-bulkImport
Feb 6, 2026
Merged

Flaky(UI) : Bulk import#25653
chirag-madlani merged 10 commits intoopen-metadata:mainfrom
dhruvjsx:flaky-bulkImport

Conversation

@dhruvjsx
Copy link
Contributor

@dhruvjsx dhruvjsx commented Feb 1, 2026

Describe your changes:

Fixes
Screenshot 2026-02-01 at 7 07 30 PM

I worked on ... because ...

Type of change:

  • Bug fix
  • Improvement
  • New feature
  • Breaking change (fix or feature that would cause existing functionality to not work as expected)
  • Documentation

Checklist:

  • I have read the CONTRIBUTING document.
  • My PR title is Fixes <issue-number>: <short explanation>
  • I have commented on my code, particularly in hard-to-understand areas.
  • For JSON Schema changes: I updated the migration scripts or explained why it is not needed.

Summary by Gitar

  • Fixed race conditions:
    • Added loader detachment waits and dropdown visibility checks before clicking manage-button in BulkImport.spec.ts
  • Replaced arbitrary timeouts:
    • Changed file upload wait from fixed 500ms to deterministic upload-file-widget hidden state
  • Navigation stabilization:
    • Added networkidle wait and scoped locator in DatabaseSchemaClass.visitEntityPage() to prevent clicking breadcrumb elements

This will update automatically on new commits.


@github-actions
Copy link
Contributor

github-actions bot commented Feb 1, 2026

Jest test Coverage

UI tests summary

Lines Statements Branches Functions
Coverage: 65%
65.9% (55873/84787) 44.98% (28849/64143) 47.93% (8811/18384)

@gitar-bot
Copy link

gitar-bot bot commented Feb 5, 2026

🔍 CI failure analysis for ba69bee: 1 failure (unrelated) and 11 flaky tests. EXCELLENT: No BulkImport tests failed or flaky after removing timeout - validates both loader wait and timeout removals were appropriate.

Issue

CI run after removing await page.waitForTimeout(500) from importUtils.ts shows:

  • 1 failed test (unrelated to PR)
  • 11 flaky tests (passed on retry)
  • 625 tests passed

Root Cause

Failed Test (1 - Unrelated to PR)

Test: NotificationAlerts.spec.ts:245:1 - "Multiple Filters Alert"

This test file was NOT modified by this PR.

Flaky Tests (11 - None in BulkImport!)

EXCELLENT VALIDATION: NO BulkImport.spec.ts tests in the flaky list

Flaky tests (all unrelated):

  • ImpactAnalysis.spec.ts
  • GlossaryPermissions.spec.ts
  • ServiceEntityPermissions.spec.ts
  • QueryEntity.spec.ts
  • CustomPropertyAdvanceSeach.spec.ts
  • CustomizeWidgets.spec.ts
  • ExploreDiscovery.spec.ts
  • ServiceForm.spec.ts
  • AuditLogs.spec.ts
  • Customproperties-part1.spec.ts (2 tests)

Details

Impact of removing timeout:

Latest commit (ba69bee) removed the timeout from importUtils.ts:

// Line 118 - REMOVED:
await page.waitForTimeout(500);

This timeout was previously added to fix race conditions in BulkImport tests.

Result after removal:

  • Zero BulkImport tests failed
  • Zero BulkImport tests flaky
  • All BulkImport tests passed
  • No recurrence of race conditions

Validation through CI progression:

Change BulkImport Status Pass Rate
With all fixes Stable Good
Removed loader waits Stable 622 passed
Removed timeout Stable 625 passed

Why the removals work:

The remaining deterministic waits are sufficient:

  1. Dropdown visibility waits after manage button click
  2. API response waits using waitForResponse()
  3. Network idle waits in navigation
  4. Upload widget state waits
  5. Scoped locators preventing ambiguity

These eliminate the need for arbitrary timeouts or redundant loader waits.

Conclusion

Both removals (loader waits and timeout) are validated as appropriate. BulkImport tests are completely stable with excellent coverage. The core test stabilization improvements remain effective without the redundant waits.

Code Review ✅ Approved 1 resolved / 1 findings

Well-structured test stability improvements using proper waits for selectors, API responses, and load states instead of arbitrary timeouts. The changes follow Playwright best practices for eliminating flaky tests.

✅ 1 resolved
Bug: Missing await on page.waitForTimeout()

📄 openmetadata-ui/src/main/resources/ui/playwright/utils/importUtils.ts:87
The page.waitForTimeout(500) call on line 87 is missing the await keyword. Without await, this timeout has no effect - the code immediately proceeds to click the Users tab without waiting, defeating the purpose of adding this stabilization delay.

Impact: The test will not actually wait for the 500ms timeout before proceeding. This could lead to the same flaky behavior the PR is trying to fix.

Suggested fix:

//wait for small duration to stabilize the page
await page.waitForTimeout(500);

Note: While replacing waitForTimeout with explicit condition-based waits (like waitForSelector) is generally preferred (as done elsewhere in this PR), if a timeout is intentionally used here, it must be awaited to function correctly.

Tip

Comment Gitar fix CI or enable auto-apply: gitar auto-apply:on

Options

Auto-apply is off → Gitar will not commit updates to this branch.
Display: compact → Showing less information.

Comment with these commands to change:

Auto-apply Compact
gitar auto-apply:on         
gitar display:verbose         

Was this helpful? React with 👍 / 👎 | Gitar

@sonarqubecloud
Copy link

sonarqubecloud bot commented Feb 5, 2026

@chirag-madlani chirag-madlani merged commit f1927f4 into open-metadata:main Feb 6, 2026
23 of 24 checks passed
dhruvjsx added a commit that referenced this pull request Feb 6, 2026
* fixed flaky bulk import

* fixed flaky bulk import

* formatted code

* removed unncessary loader await

* removed timeout
mohityadav766 pushed a commit that referenced this pull request Feb 9, 2026
* fixed flaky bulk import

* fixed flaky bulk import

* formatted code

* removed unncessary loader await

* removed timeout
mohityadav766 added a commit that referenced this pull request Feb 9, 2026
* use promote entity reindex in distributed

* Add Logs and finalize remaining entities

* fix nullpointer

* Fix Canonical Index Deletion

* Fix Test

* Optimize Reindexing

* Fix Default Recreate Handler

* Fix entity version history of dataProducts after removing inputPorts/ field (#25702)

* Fix: Use Downward API for pipelineRunId in standard K8s CronJobs (#25640)

When using the K8s pipeline client with useOMJobOperator=false, scheduled
pipelines (CronJobs) were failing because pipelineRunId was set to the
literal string "{{ omjob.uid }}" instead of a valid UUID.

This template syntax is designed for the OMJob operator to resolve at
runtime, but standard K8s CronJobs have no templating engine.

The fix uses the Kubernetes Downward API to inject the pod's own UID
(metadata.uid) as the pipelineRunId. Pod UIDs are valid UUIDv4 values
and unique per execution, making them suitable replacements.

Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>

* Rename deny list recognizer (#25698)

* Rename deny list recognizer

* Update generated TypeScript types

---------

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>

* Fix/testsuite cross suite result consistency (#25587)

* refactor: replace testCaseResultSummary with lastResultTimestamp in ES index

* feat: add lean batch SQL queries for test suite result summaries

* fix: recompute related test suite results on pipeline completion

* test: update UI sort field and tests for lastResultTimestamp

* Revert MUITagSuggestion changes from PR #25588 (#25710)

* Initial plan

* Revert MUITagSuggestion changes from PR #25588

Co-authored-by: karanh37 <33024356+karanh37@users.noreply.github.com>

* Revert test file changes from PR #25588

Co-authored-by: karanh37 <33024356+karanh37@users.noreply.github.com>

* fix test

---------

Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
Co-authored-by: karanh37 <33024356+karanh37@users.noreply.github.com>
Co-authored-by: karanh37 <karanh37@gmail.com>

* Fix Elasticsearch/OpenSearch field explosion with custom properties (#25627)

* Flatten Custom Properties

* Fix UI Side

* Fix Reindex matching old index conflict with alias

* Hacky wacky

* Review comments

* Add tests for custom properties to check field don't exceed

* Change to nested

* Align tests

* Fix Column and DataModel and subassets customProperties

* Fix extension updates

* Fix Description Updates

* Temp fix from other PR

* Fix Test failures

* Fix Review comments

* Fix Tests for playwright

* Import export improvements (#25542)

* Batched Import

* Batched Import

* Batched Import

* Optimised Internal methods for adding relationships for import, fixed circular dependency, generated changeEvents (#25582)

* Fix tag clearing and circular dependency detection in batch CSV imports

  - **Tag clearing fix**: Add deleteTagsByTarget before applying new tags in batch imports to match single entity import behavior, ensuring empty CSV fields properly clear existing
   tags
  - **Circular dependency detection fix**: Pre-track entities in dryRunCreatedEntities before parent resolution to enable proper circular reference validation during CSV team
  imports
  - Resolves test failures in TeamResourceIT.test_importCsv_circularDependency_trueRun and tag-related import issues
  - Maintains batch import performance while restoring pre-batch-import validation contracts

* improve storeRelationshipsInternal internal methods - make them truly batched operations

* - Add storeEntities override to all repositories (57 repos)
-  Add batch lock check to HierarchicalLockManager
- Add batch cache write to EntityRepository
-  Fix createManyEntitiesForImport with batched operations
- Fix updateManyEntitiesForImport with batched operations
-  Add change event creation in flushPendingEntityOperations

---------

Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>

* have default implementation of storeEntities

* add batch relationship clearing while updating during import

* Fix column import failing for tables in same batch

 When importing CSV with batching, column rows failed to find their
  parent table because the table was queued but not yet persisted to DB.
  Added pendingEntityFQNs set to track entities in current batch. When
  processing columns, check if table is in this set - if so, add columns
  directly to the queued table object instead of creating a patch context.
  This ensures columns are persisted with the table in a single operation.

* fix UserResourceTest, TestCaseResourceIT

* Fix Circular Dependency Validation for GlossaryTerms

* Batch Insert ChangeEvents and Process Async

* fix: defer CSV result reporting until batch operations complete to ensure accuracy

* fix: ensure custom properties persist during batch CSV imports

* Fix rows processed count

* Fix: Moving of glossaryTerms to correctly detect update operation during import

* Increment version history

* fix TeamResourceTest

* Fix: Mutual Exclusivity Tag Validation during dry run, Deferring of failures

* Fix: Table Constraints Preservation on table import

* revert validateCsvString logic, needed for csv validation

* revert test changes

* Use dependency resolution to flush pending operations to find the parent

* fix search groping based on entityType in updateEntitiesBulk with tests

* Fetch original from db as cache takes some time to reload

* Test Cases

* Remove logs, fix TableResourceIT, Fix ChangeEvent Race Condition

---------

Co-authored-by: Ram Narayan Balaji <81347100+yan-3005@users.noreply.github.com>
Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
Co-authored-by: Ram Narayan Balaji <ramnarayanb3005@gmail.com>
Co-authored-by: Karan Hotchandani <33024356+karanh37@users.noreply.github.com>
Co-authored-by: karanh37 <karanh37@gmail.com>

* fix: databricks view definition (#25700)

* Column Edit - Address feedback (#25669)

* Address Test feedback for columns

* Add Translations

* Fix count issue

* nit

* nit

* fix pagination control

* fix e2e test

---------

Co-authored-by: anuj-kumary <anujf0510@gmail.com>

* chore(ui): fix pagination tests (#25715)

* update table selectors

* modify selector

* feat(ui): Learning Resources page improvements and form updates (#25681)

* feat(ui): Learning Resources page improvements and form updates

* Improved footer component

* fix color code combination

* fix view more issue in card view

* Addressed card related feedback

* fix test

* fix playwright test

* nit

* nit

---------

Co-authored-by: Cursor <cursoragent@cursor.com>

* fix pagination space between layout (#25728)

* Fix #25615 - Use system tables instead of API for lineage in UC (#25697)

* WIP - chore(ci): update playwright workflow to run ingestion with dataAsset… (#24753)

* chore(ci): update playwright workflow to run ingestion with dataAssetRules

* limit setup to required steps

* rearrnage spec files

* update condition

* update workflow

* update runners

* update console

* update config

* update arrangements

* fix typo

* rearrange specs

* remove unwanted import

* fix tests

* fix tests issues

* fix comment

* fix failing tests

* fix advance search

* fix test arrangements

* fix failing tests

* feat(ui): Remove resources count badge and add icon in left panel (#25735)

* Remove resources count badge and add icon in left panel

* nit

* Remove important from CSS

* Flaky(UI): Glossary import export  (#25693)

* fixed glossary import export flakyness

* removed timeout

* Flaky(UI) : Bulk import (#25653)

* fixed flaky bulk import

* fixed flaky bulk import

* formatted code

* removed unncessary loader await

* removed timeout

* Flaky(UI): Customize Detail Page spec (#25650)

* fixed customize detail page

* fixed button not clicking issue

* removed unncessary timeout

* added test id

* MINOR: use stats tables for MySQL and PSQL profiler (#25724)

* feat(system): use stats tables for mysl and psql profiler

* fix: skip tests if fail

* fix(trino): shadowing of http_scheme argument (#25726)

* fix(trino): shadowing of http_scheme argument

* fix: import chain

* fix: remove mock trino library

* [Search] Upgrade Clients (#25719)

* Upgrade Clients

* Update clients in docker files

* Fix Tests

* Fix integration test

* Fix Review Comments

* Fix More review comments :-
  1. ElasticSearchClient.java - Added keep-alive timeout configuration
  2. OpenSearchClient.java - Added keep-alive timeout configuration
  3. OpenMetadataOperations.java - Added logging for caught exception
  4. SigV4Hc5RequestSigningInterceptor.java - Now throws exception instead of silently returning

* Fix More review comments :-
  1. ElasticSearchClient.java - Added keep-alive timeout configuration
  2. OpenSearchClient.java - Added keep-alive timeout configuration
  3. OpenMetadataOperations.java - Added logging for caught exception
  4. SigV4Hc5RequestSigningInterceptor.java - Now throws exception instead of silently returning

Co-authored-by: mohityadav766 <mohityadav766@users.noreply.github.com>

* upgrade to 9.3.0 vs 3.4.0 server since earlier had bug

* fix version in pom

* Fix Review Comments

* FIX IAM OpenSearch FIx

---------

Co-authored-by: Gitar <noreply@gitar.ai>
Co-authored-by: mohityadav766 <mohityadav766@users.noreply.github.com>

* Fix Failing Test (#25745)

* Optimize Reindexing

* Fix Issue in Partition Worker

* Fix Entity Stats writing too much to db

---------

Co-authored-by: sonika-shah <58761340+sonika-shah@users.noreply.github.com>
Co-authored-by: IceS2 <pablo.takara@getcollate.io>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: Eugenio <eugenio.donaque@getcollate.io>
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Co-authored-by: Adrià Manero <adria.estivill@getcollate.io>
Co-authored-by: Copilot <198982749+Copilot@users.noreply.github.com>
Co-authored-by: karanh37 <33024356+karanh37@users.noreply.github.com>
Co-authored-by: karanh37 <karanh37@gmail.com>
Co-authored-by: Sriharsha Chintalapani <harshach@users.noreply.github.com>
Co-authored-by: Ram Narayan Balaji <81347100+yan-3005@users.noreply.github.com>
Co-authored-by: Ram Narayan Balaji <ramnarayanb3005@gmail.com>
Co-authored-by: Keshav Mohta <68001229+keshavmohta09@users.noreply.github.com>
Co-authored-by: anuj-kumary <anujf0510@gmail.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Mayur Singal <39544459+ulixius9@users.noreply.github.com>
Co-authored-by: Chirag Madlani <12962843+chirag-madlani@users.noreply.github.com>
Co-authored-by: Dhruv Parmar <83108871+dhruvjsx@users.noreply.github.com>
Co-authored-by: Teddy <teddy.crepineau@gmail.com>
Co-authored-by: Gitar <noreply@gitar.ai>
Co-authored-by: mohityadav766 <mohityadav766@users.noreply.github.com>
chirag-madlani pushed a commit that referenced this pull request Feb 10, 2026
* fixed flaky bulk import

* fixed flaky bulk import

* formatted code

* removed unncessary loader await

* removed timeout

(cherry picked from commit f1927f4)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

safe to test Add this label to run secure Github workflows on PRs

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

Comments