ECS is seeking a
Data Engineer to work
remotely.
We are seeking a mid-level Data Engineer to support the development of an AI-driven solution aimed at reducing alert fatigue in Security Operations Centers (SOC). This individual will be responsible for building and maintaining scalable data pipelines, enabling robust data ingestion and transformation processes in the cloud, and collaborating with data scientists and security engineers to operationalize machine learning models.
Responsibilities: - Design, build, and maintain scalable and secure data pipelines to support real-time and batch processing of security event data.
- Develop cloud-based data ingestion workflows that aggregate data from SIEMs, EDRs, log sources, and ticketing systems.
- Clean, normalize, and structure raw security data to support machine learning model training and inference.
- Collaborate with security analysts, detection engineers, and data scientists to understand data requirements and ensure pipeline reliability.
- Ensure data lineage, quality, and observability for all ingest processes.
- Develop and maintain Python scripts and utilities for data transformation, enrichment, and export.
- Work within a CI/CD environment to test, version, and deploy data workflows efficiently.
- Participate in technical design sessions and sprint planning to align with the product roadmap.
Salary Range: $150,000 - $170,000
General Description of Benefits
- 6+ years of experience in data engineering or software development focused on data pipelines or ETL/ELT.
- Strong proficiency in Python for data manipulation and automation tasks.
- Experience with cloud-native data services (e.g., AWS Glue, SageMaker, Lambda, S3, Kinesis, Athena; or Azure equivalents).
- Experience designing and operating data ingestion and transformation pipelines using tools like Apache Airflow, dbt, or equivalent.
- Strong understanding of data formats (JSON, Parquet, Avro) and data modeling concepts.
- Working knowledge of SQL and experience with data warehouse technologies (e.g., Redshift, Snowflake, BigQuery).
- Experience using Git and modern CI/CD practices (e.g., GitHub Actions, Jenkins).
- Familiarity with REST APIs and data integration from external services.