Platform and HPC DATA Engineer

Job Number: 182 Job Category: GovTech Job Title: PLATFORM and HPC DATA ENGINEER - VIRGINIA -URGENT Job Type: Full-time Clearance Level: TS/SCI CI Poly Work Arrangement: On-site Job Location: Herndon VA Background Design and implement data management systems and architectures for HPC platforms, focusing on optimizing data flow, storage, and access in large-scale computing environments
Oversee the configuration, maintenance, and optimization of distributed file systems (e.g., Lustre, IBM Spectrum Scale, NFS, GPFS) and storage solutions used in HPC environments to ensure efficient performance, scalability, and reliability
Implement and manage metadata-driven systems for data labeling/tagging. This includes the development of strategies for classifying, indexing, and organizing datasets to enhance data discoverability, access control, and auditing
Configure and maintain various storage appliances (e.g., NetApp, Dell EMC, HPE) and integrated storage solutions. Ensure that storage
Implement security best practices for data access, protection, and management, ensuring compliance with government regulations and internal data governance policies. Configure encryption, access control, and secure data sharing method devices are optimized for performance, capacity, and availability within the HPC ecosystem
Develop and maintain automation scripts (e.g., using Python, Bash, or Perl) to streamline storage configurations, data labeling/tagging, and system monitoring tasks. Automate processes related to data integration and HPC platform management Requirements Bachelor's degree in computer science, information technology, engineering, or a related field. A Master's degree or higher
7+ years of experience in managing data infrastructure in HPC environments, with expertise in file systems, storage appliances, and data workflows
Hands-on experience with distributed file systems, including Lustre, IBM Spectrum Scale (GPFS), NFS, and others commonly used in HPC setting
Proven experience with storage appliance configuration (e.g., NetApp, Dell EMC, HPE, or similar systems), including performance tuning, capacity management, and reliability
Familiarity with data access protocols like GridFTP, rsync, and NFS for large-scale data transfer
nowledge of high-performance networking protocols (e.g., InfiniBand, RDMA) and their role in data transfer and storage optimization Preferred Experience with containerization (Docker, Singularity) in an HPC context for data processing and application deployment
Familiarity with high-performance computing (HPC) schedulers (e.g., SLURM, PBS, Torque) and their interaction with data storage systems
Experience with cloud storage integration or hybrid cloud environments, with knowledge of cloud-native storage solutions (e.g., AWS S3, Ceph, OpenShift)