Job Description

This is an exciting opportunity to build your data engineering expertise and join the Data, Technology and Analytics (DaTA) team at the Competition and Markets Authority (CMA) to play a key role at the forefront of technology, investigating and regulating digital firms and markets to make a positive impact on society.

The CMA is a world-leading competition authority at the forefront of tackling pressing public policy questions, including the regulation of the biggest digital markets. Our mission is to make markets work well in the interests of consumers, businesses, and the economy.

The Data, Technology and Analytics (DaTA) unit is an established multidisciplinary team that brings together data engineers, data scientists, technologists, behavioural scientists and digital forensics and eDiscovery experts.

The data science and engineering team

As a Lead Data Engineer in the DaTA unit, you will join an established team of around 15 data engineers and data scientists who are critical in designing and managing the CMA\'s analytical infrastructure and supporting the delivery of our most technically challenging projects.

Some of our recent work in Data Engineering includes:

• Designing pipelines featuring parallelization, workflow orchestration, data version control and MLOps to process large, complex data sources
• Securely translating large corpora of documents submitted by third parties using a containerised architecture on our cloud infrastructure
• Building end-to-end services, defined as Infrastructure as Code, to run jobs, including web scraping at scale and regular ML prediction on large datasets

The role

Your responsibilities will span the entire tech stack, with an emphasis on backend and infrastructure development, data orchestration and handling. This will include:

• Providing leadership and engineering expertise on projects
• Creating and implementing new services, pipelines and data driven tools within our cloud platform
• Driving the development and maintenance of our cloud-hosted analytics platform (JupyterHub, AWS EC2s) and existing services through Infrastructure as Code (AWS CDK)
• Promoting and producing high quality code in Python (and other languages, as needed)
• Building the CMA\'s coding and data management standards and helping others to improve their coding and data handling capabilities
• Identifying opportunities to support CMA cases with data engineering expertise

You will be expected to make architectural decisions and lead projects, along with managing internal stakeholders to make sure we are building the most impactful tools possible for the CMA.

You will be given time and support for study and development and will be expected to spend most of your time working on hands-on technical work.

What we\'re looking for

You will be an experienced Data Engineer, Developer, or DevOps specialist, with experience in leading and delivering data projects and a good understanding of service architecture and design best practices.

Significant experience of writing, documenting, testing and managing code, including Python, to work with large and messy datasets is essential.

A proven interest in learning about and understanding the latest methods and technologies within machine learning and data engineering is equally important.

You will also need experience of overseeing and writing both architectural, design and technical documentation, and presentations and reports for non-technical audiences.

What we can offer you

The CMA is a hugely diverse, rewarding and stimulating place to work - an opportunity to work for the public good and to stretch and develop yourself in doing so. We offer generous benefits, including an excellent pension scheme with an average employer contribution of 27%, at least 25 days leave (increasing to 30 over five years), and generous maternity/ paternity leave provision. You can also take advantage of interest-free season ticket loans, a cycle to work scheme, our employee assistance programme, and our extensive flexible working options.

To find out more about this opportunity and about working for the CMA, please click on the APPLY button.

The CMA, like many other organisations, operate a hybrid working model with a mix of office and home working. We believe this supports us to deliver our best work while allowing the flexibility to maintain balanced and healthy lives. Our staff work at least 50% of their time from their office location.

The CMA are an accredited Disability Confident Employer, and we are an equal opportunities employer, welcoming suitably qualified applicants from all backgrounds. We recruit by merit on the basis of fair and open competition, as outlined in the Civil Service Commission\'s recruitment principles.

Competition and Markets Authority