Job ID : AINC17464
We have partnered with a large biotechnology company in the South San Francisco, CA area to provide them with a Data Engineer, DevOps.
The Data Engineering DevOps team ensures the infrastructure which powers our biological data factory’s robots, instruments, and machine learning platform is reliable, scalable, and manageable. You will work closely with a cross-functional team of scientists, bioengineers, and data scientists to identify areas where data engineering can make a difference, by developing data architectures and systems on cutting edge, high throughput platforms that enable our scientists to be maximally productive. You will design, implement, and deploy cloud infrastructure, including managed databases, application servers, data warehouses, and interactive/batch computing environments, and work as part of a team to rigorously design our data platform, identify key architectural performance improvements, and join an on-call rotation to ensure that our platform runs at maximum productivity.
- 2-3 years of experience with provisioning AWS cloud services (Experience with GCP and Azure is also relevant).
- Experience with cloud configuration and resource management tools such as Terraform
- Experience architecting reliable infrastructure platforms including monitoring and alerting, load balancing, scalable services, multi-region
- Experience with at least one high-end distributed data processing environment (Hadoop, Spark, etc)
- Experience with batch computing systems such as AWS Batch, SLURM
- Experience with container build and deployment systems like Docker, Kubernetes, or ECS
- Ability to communicate effectively and collaborate with people of diverse backgrounds and job functions
- Proficiency in Linux environment (including shell scripting and Python programming), experience with database languages (e.g., SQL, No-SQL) and experience with version control practices and tools (Git, Mercurial, etc.)
- Passion for making a difference in the world
Requirements
- Experience with biological data
- Experience with managing medium-sized data sets (100TB+) in object storage systems like S3
- Experience with defining infrastructure following compliance (GDPR, HIPAA, etc).
- Experience with data processing pipelines
- Experience with deploying and monitoring machine learning models in a production environment
Benefits
- Medical Insurance
- Dental Insurance
- Vision Insurance