Job Description J
ob Summary: We are seeking a skilled Data Engineer to join our team, focusing on building and optimizing data solutions to support our growing business. This role requires expertise in SQL, data warehousing, and Python scripting, with a strong background in a .NET environment. As a Data Engineer, you will work on designing, implementing, and maintaining data pipelines, ensuring efficient data flow and accessibility across the organization.
Key Responsibilities: - Design, develop, and maintain data pipelines to collect, process, and store data from various sources.
- Implement and optimize SQL-based data warehouses to ensure scalability and performance.
- Use Python scripting for data transformation, automation, and integration tasks.
- Collaborate with cross-functional teams to support data analysis, reporting, and business intelligence needs.
- Work within a .NET environment to build and integrate data solutions with existing systems.
- Manage and optimize relational (MySQL, PostgreSQL) and NoSQL databases (e.g., MongoDB, Cassandra).
- Leverage cloud platforms (AWS, Azure, or Google Cloud) for data storage, processing, and analytics.
- Ensure data quality and consistency through version control (Git) and adherence to best practices.
- Troubleshoot and resolve complex data-related issues with a focus on performance and reliability.
- Implement machine learning models using frameworks such as TensorFlow, Keras, or PyTorch (preferred but not required).
Required Skills and Experience: - 3+ years of experience as a Data Engineer or in a similar role.
- Strong proficiency in SQL and Python for data manipulation and automation.
- Experience working in a .NET environment and integrating data systems.
- Hands-on experience with data warehousing solutions and ETL pipelines.
- Familiarity with big data technologies (Hadoop, Spark, Kafka) is a plus.
- Knowledge of data visualization tools (e.g., Tableau, Power BI) is advantageous.
- Proficiency in using Git for version control and collaborative development.
- Strong problem-solving skills and the ability to work in an Agile/Scrum environment.
- Experience with cloud platforms (AWS, Azure, or Google Cloud) is a plus.
- Experience with machine learning frameworks and model deployment.
- Familiarity with R or Scala for advanced data analysis.
- Understanding of distributed systems and parallel computing principles.