Cloud Data Engineer

in San Francisco, CA

Cloud Data Engineer Job

Job Description Job Attributes+

  • Job ID

    7469768

  • Req #

    23077

  • Job Location

    San Francisco, CA 94101, US

  • Job Category

    Information Technology

  • Job Type

    9+ Month Contract


Blackstone Talent Group, an award-winning technology consulting and talent agency is seeking a Cloud Data Engineer to join our team remotely for our client.



ResponsibilitiesData Pipeline Building, Optimization, and Operationalization

  • Create and maintain optimal data pipeline architecture,
  • Assemble large, complex data sets that meet functional / non-functional business requirements.
  • Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability,
  • Operationalizing, Improving the quality and reliability of data pipelines (monitoring, retry, failure detection)

Data Platform and Self-serving Data Infrastructure

  • Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using various data tools and cloud services.
  • Design and build abstractions that hide the complexity of the underlying big data processing and execution stack and that allow data consumers/partners to focus on their strengths: product, data modeling, data analysis, search, information retrieval, and machine learning.
  • Build processes and infrastructure to manage the lifecycle of datasets:  data structures, metadata, dependency, and workload management.
  • Building internal services and tools to help in-house partners implement, deploy and analyze datasets with a high level of autonomy and limited friction.

Qualifications

  • MS or BS in CS, Engineering, Math, Statistics, or a related field or equivalent practical experience in data engineering.
  • 3-5 years of experience working in data engineering using Python on Cloud platforms like AWS. Proficient in programming in Python. With the experience of packages like pandas, numpy, boto, s3fs, flask, fastapi, pyarrorw, pyspark etc.
  • Expert in SQL programming and experience working with relational databases, as well as working familiarity with a variety of databases (Postgres/Redshift/MySQL).
  • Experience building and optimizing ‘big data’ data pipelines, architectures, and data sets.
  • Strong analytic skills related to working with unstructured datasets. A successful history of manipulating, processing, and extracting value from large, disconnected datasets.
  • Strong problem-solving skills to understand complex structured and/or unstructured datasets and their relationships to find issues and provide programmatic solutions.
  • Working experience in data analytics (data wrangling, integration, visualization), and reporting using BI (Business Intelligence) tools
  • Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
  • Strong project management and organizational skills. Strong sense of ownership.
  • Collaborative personality with experience supporting and working with cross-functional teams in a dynamic environment.

Extra

  • Proficiency in other high-level programming languages (Go, Java, Scala, or equivalent)
  • Experience with Technologies like Kubeflow, EKS, Docker
  • Experience with stream-processing systems: Kafka, Storm, Spark-Streaming, etc.
  • Statistical analysis and modeling experience
  • Experience with machine learning algorithms 
  • A data-driven approach to problem-solving
  • The ability to visualize and communicate complex concepts


Blackstone Talent Group is a division of Blackstone Technology Group, a global IT services and solutions firm that implements technological solutions across commercial industry verticals and the US Federal Government. Blackstone’s global talent augmentation practice was founded in 1998. Blackstone Talent Group has offices in San Francisco, Denver, Houston, Colorado Springs, and Washington, DC. We specialize in providing clients with the best talent across a variety of industries and sectors.




EOE of Minorities/Females/Veterans/Disabilities

Additional Information

Who We Are

ABOUT US
Founded in 1998, Blackstone Talent Group is one of three divisions of Blackstone Technology Group. The other two divisions are Blackstone Federal, a premier engineering + transformation agency that serves the federal government, and Trellis Energy, the largest transaction management software firm fully dedicated to serving the natural gas industry. Blackstone Technology Group is a privately-held global IT services, solutions and resourcing firm. Headquartered in San Francisco, Blackstone now has more than 300+ employees across five offices nationwide. Blackstone Technology Group’s mission is to help clients adapt to the demands of a digital world, and has garnered an impressive track record of delivering successful results to a client list that includes Fortune 100 businesses and US federal, state and local government.

WHAT WE DO
We provide contract, permanent and managed resource services to our clients. We source for commercial businesses and non-profits, as well as government clients requiring security clearances from public trust through multiple levels of top secret. Some of our skill specialties are IT, IT Operations, Engineering, Accounting and Finance, and Executive Search.

HOW WE DO IT
Our team never stops building connections with talent. And our network of fully-vetted, ready-to-work professionals allows us to provide exceptional speed and accuracy in our resource delivery that is unmatched in the industry. With a relationship-first engagement approach, we most often become a trusted partner with our clients and operate as an extension of their organizations.

Saved Jobs