Data Engineer Job
Job Description Job Attributes+
Herndon, VA 20170, US
Blackstone Talent Group, an award-winning technology consulting and talent agency is seeking a Data Engineer to join our team at our client’s site in Herndon, VA.
The candidate must have a successful track record in ETL job design, development and automation activities with minimal supervision. The candidate will be expected to support a variety of structured, semi-structured and unstructured data in streaming and batch frameworks. The candidate must possess AWS working knowledge and proven skills on AWS data tools and services such as AWS Glue, Step Functions, Lambda and DMS. Troubleshoot, monitor and coordinate defect resolution related to data processing & preparation; Responsible for the creation and support of all data pipeline processes across various data assets within the current scope of the system.
· The successful candidate will be able to rapidly develop technical solutions working closely with the integrated product teams and developers with minimal direction from senior or lead resources
· Understand data needs and be able to construct data pipelines for automating event driven bi-directional selective data replication, along with micro-batch and batch based data pipelines
· Standardization of data processing modules to deliver modularity and enhance reusability
· Utilize identified tools and services such as AWS Glue, Python, Streamsets, Step functions and Lambda, Kinesis Data Streams and Data Firehose
· Create and maintain standards and best practices for data and pipeline standards
· Bachelor' s degree in Computer Science or related discipline
· 3+ years of hands on experience in with ETL tools such as SAP DS and Pentaho PDI
· 4+ years of hands on experience in with python and various python toolkits and libraries for data processing and pipelines
· 4+ year of experience and ability to create complex SQL queries and functions
· 2+ years of hands on experience in AWS Glue, Python, Step Functions, Kinesis Data Streams and Data Firehose
· 1+ year of experience in java and linux scripting is essential
· 1+ year experience working with CI/CD tools including Git for etl and scripts repository
· Experience of working in AWS environments and any AWS certifications is advantageous
· Experience working with SQL Server and tools like SSRS is advantageous
· Experience with big data tools such as EMR/Spark, Databricks/PySpark is an advantage
· Experience working with database versioning tools such as Flyway is an advantage
· Experience in Ansible and Jenkins scripting is an advantage
· The ability to successfully obtain and maintain a U.S. Suitability/Public Trust Background Clearance
Blackstone Talent Group is a division of Blackstone Technology Group, a global IT services and solutions firm that implements technological solutions across commercial industry verticals and the US Federal Government. Blackstone’s global talent augmentation practice was founded in 1998. Blackstone Talent Group has offices in San Francisco, Denver, Houston, Colorado Springs, and Washington, DC. We specialize in providing clients the best talent across a variety of industries and sectors.
EOE of Minorities/Females/Veterans/Disabilities
Who We Are
Founded in 1998, Blackstone Talent Group is one of three divisions of Blackstone Technology Group. The other two divisions are Blackstone Federal, a premier engineering + transformation agency that serves the federal government, and Trellis Energy, the largest transaction management software firm fully dedicated to serving the natural gas industry. Blackstone Technology Group is a privately-held global IT services, solutions and resourcing firm. Headquartered in San Francisco, Blackstone now has more than 300+ employees across five offices nationwide. Blackstone Technology Group’s mission is to help clients adapt to the demands of a digital world, and has garnered an impressive track record of delivering successful results to a client list that includes Fortune 100 businesses and US federal, state and local government.
WHAT WE DO
We provide contract, permanent and managed resource services to our clients. We source for commercial businesses and non-profits, as well as government clients requiring security clearances from public trust through multiple levels of top secret. Some of our skill specialties are IT, IT Operations, Engineering, Accounting and Finance, and Executive Search.
HOW WE DO IT
Our team never stops building connections with talent. And our network of fully-vetted, ready-to-work professionals allows us to provide exceptional speed and accuracy in our resource delivery that is unmatched in the industry. With a relationship-first engagement approach, we most often become a trusted partner with our clients and operate as an extension of their organizations.