Senior Big Data Engineer | Remote Position - Building Enterprise Data Infrastructure with Hadoop, Spark, Python & Cloud Technologies
--- About arenaflex Welcome to arenaflex, where innovation meets opportunity and data drives the future of financial services. At arenaflex, we believe in empowering individuals and businesses through cutting-edge technology and unparalleled expertise. As a global leader in the financial industry, we are committed to transforming the way people interact with money, investments, and financial services. Our mission is simple yet profound: to deliver exceptional value to our customers while fostering an environment where talented professionals can thrive, grow, and make meaningful contributions. We pride ourselves on maintaining a culture of innovation, collaboration, and continuous learning. Our team members are our greatest asset, and we invest heavily in their development, well-being, and career advancement. At arenaflex, we embrace the power of data to make smarter decisions, create better products, and deliver outstanding customer experiences. Our technology infrastructure supports millions of transactions daily, and we are constantly pushing the boundaries of what's possible with big data, machine learning, and cloud computing. Join us on our journey to revolutionize the financial industry through technology excellence.
About the Role
We are seeking a talented and passionate Senior Big Data Engineer to join our dynamic technology team at arenaflex. This is a remote position that offers the flexibility to work from anywhere while contributing to meaningful projects that impact millions of customers worldwide. As a Senior Big Data Engineer at arenaflex, you will be an integral part of our fast-paced, innovative engineering team. You will work on challenging problems involving massive-scale data processing, real-time analytics, and advanced machine learning applications. Your expertise will help shape the future of our data infrastructure and enable data-driven decision-making across the organization. This role offers an exciting opportunity to work with cutting-edge technologies including Hadoop, Spark, Python, SQL, and various cloud platforms. You will collaborate with talented engineers, data scientists, and product managers to deliver innovative solutions that power arenaflex's products and services.
Key Responsibilities
As a Senior Big Data Engineer, you will be responsible for designing, developing, and maintaining our enterprise-level data infrastructure. Your day-to-day responsibilities will include:
- Designing and Building Data Pipelines: Create robust, scalable data pipelines for both batch and real-time processing using technologies like Hadoop, Spark, Kafka, and cloud-native services. Ensure data flows seamlessly between various systems and platforms while maintaining data quality and integrity.
- Developing Big Data Solutions: Write efficient, maintainable code in Python, Scala, or Java to process large datasets. Implement data transformation, aggregation, and enrichment processes that support business intelligence and analytical requirements.
- Architecting Data Systems: Design and implement scalable data architectures that can handle petabytes of data while maintaining performance and reliability. Work with stakeholders to understand data requirements and translate them into technical solutions.
- Code Development and Review: Write high-quality code, including ongoing refactoring to improve code structure and quality. Conduct code reviews, provide technical guidance to junior team members, and ensure adherence to best practices and coding standards.
- API Development and Testing: Design and implement APIs for data access and integration. Develop comprehensive test plans, perform unit testing, integration testing, and ensure robust error handling and data validation.
- Performance Optimization: Optimize data processing jobs, queries, and pipelines for performance and efficiency. Identify bottlenecks and implement solutions to improve processing speeds and reduce resource consumption.
- Agile Collaboration: Act as a core member of Agile/Scrum teams, participating in sprint planning, daily stand-ups, and retrospectives. Contribute to user story analysis, elaboration, and estimation.
- Documentation and Knowledge Sharing: Create and maintain technical documentation, including design specifications, API documentation, and operational runbooks. Share knowledge with team members through mentoring and technical sessions.
- Innovation and Improvement: Identify opportunities to adopt new technologies and approaches. Embrace a mindset of continuous improvement by exploring innovative solutions beyond conventional approaches.
- Cross-Functional Partnership: Work closely wit
Apply tot his job Apply To this Job