Data Engineer

About Us

BGTS is a software and technology solutions company with over 1,800 professionals and 25+ years of experience. Through engineering expertise and industry insight , our international offices deliver tailored solutions, enabling clients worldwide to achieve their business goals with flexibility, speed, and impactful results .

What will you be doing

As a Data Engineer for Marketing Tech you will be working on a project to help us onboard and integrate some additional marketing-related technologies into our existing data stack, ensuring that data can be transmitted and received in a timely and reliable manner between these systems and our existing Datalake and Operational Data services. This role will work with other software and backend engineers, as well as our existing team of data engineers to plan out, design, and implement the solution to allow these solutions to function, connecting and exchanging data with our existing systems.

Requirements

We expect candidates to be familiar with common Data technologies from both open-source and cloud vendors such as Apache, Confluent, AWS, and GCP. Additionally, they should be familiar with general Data engineering practices, such as batch and streaming ETLs, schema management, data reconciliations / validations, and PII data handling.

Ideally, the candidate would have existing knowledge and working experience with marketing-related services such as Google Analytics, MoEngage, MixPanel, etc. and / or be capable of quickly undestanding documentation and handling REST APIs from such services and setting up new microservice endpoints to serve related data in a reliable and scalable manner.

Lastly, the candidate should be comfortable with managing responsibilities with a reasonable degree of independence, using their problem-solving skills to handle routine issues as well as their communication skills to raise larger concerns as appropriate. As such, having a clear logical framework for problem-solving and fluency in Business English is a must.

Relevant Technologies / Skillsets

• General computing concepts and expertise : Unix environments, networking, REST APIs
• Cloud computing data technologies : AWS S3, Athena, DynamoDB, GCP BigQuery, Pub / Sub
• Python frameworks and tooling : pip, pytest, boto3, google-cloud-python, pandas, polars, fastapi, requests
• Workflow scheduling tools : Airflow, dbt, Dagster
• Container management and orchestration : Docker, Kubernetes
• CI / CD tools : CircleCI, Jenkins, Spinnaker, ArgoCD, Github CI
• Distributed messaging and event streaming systems : Kafka
• Enterprise BI tools : Tableau, Looker, Superset, PowerBI, Quicksight
• Streaming data processing frameworks : Spark Streaming, Apache Beam, Apache Flink
• Infrastructure as Code : Terraform, OpenTofu, Terragrunt
• Log ingestion and monitoring : ELK stack (Elasticsearch, Logstash, Kibana), Datadog, Prometheus, Grafana
• Version control : git commands, branching strategies, collaboration etiquette, documentation best practices
• Agile / Lean project methodologies and rituals : Scrum, Kanban
• Data privacy and security tools and concepts : Tokenization, Hashing and encryption algorithms, Apache Ranger