ETL/ Datawarehouse QA

Responsibilities
1. QA Strategy and Planning

• Define QA strategy for a hybrid ETL stack involving Informatica, and dbt for transformation/modeling.

• Analyze STTMs (Source-to-Target Mapping documents), business rules, and dbt model definitions to derive test scope.

• Plan and manage test phases across Program Increments (PIs) or releases: unit testing support, system testing, integration testing, and UAT support.
2. Informatica-Specific QA Responsibilities

• Review and validate Informatica mappings, workflows, sessions, and parameter files.

• Validate:
o Source data extraction (flat files, RDBMS, mainframes, etc.)
o Data transformations via mappings (joins, lookups, aggregations, filters, etc.)
o Load strategies to staging/ODS layers (insert/update strategies, truncate/load, CDC).

• Use SQL queries and data comparison tools to perform reconciliation and transformation validation.

• Validate job dependencies and control table mechanisms (if used).

• Monitor job runs via Informatica Monitor/Workflow logs, and ensure correct execution paths.
3. dbt-Specific QA Responsibilities

• Validate dbt models, tests, and documentation in the analytics layer.

• Review and test:
o Model SQL logic for business rule accuracy and performance
o YAML files for correct metadata and testing configurations
o dbt tests (unique, not null, accepted values, relationships, custom tests)

• Execute dbt test suites using dbt test and investigate failures.

• Collaborate with analytics engineers to validate , materialization strategies (table, view, incremental), and source freshness logic.

• Review generated SQL in target/compiled to ensure correct transformations.
4. Data Validation & Reconciliation

• Perform full and sampled data validation across source staging ODS analytics data layers.

• Write complex SQL queries to compare record counts, aggregates, and field-level data across layers.

• Ensure data consistency and integrity using both manual validation and dbt tests.

• Validate PII masking and data obfuscation where applicable.
5. Automation and DevOps Integration

• Build and maintain reusable SQL-based test automation scripts for recurring validations.

• Integrate QA checks in CI/CD pipelines using git Actions.

• Monitor and validate Informatica batch schedules and dbt Cloud jobs (or CLI triggers).

• Contribute to QA dashboards and test reporting automation.
6. Defect Management & Reporting

• Log and track defects in JIRA, categorize by layer (Informatica/dbt), and severity.

• Facilitate daily defect triage calls with data engineering teams.

• Provide detailed test execution status reports and test coverage metrics by data domain.