Big Data Engineer
- Location HOBOKEN, NJ
- Department Technology
- Team Engineering
- Employment Type -
- Position -
- Requisition GH844291
What you'll do at
We need super smart engineers from all levels to help us build one of the best engineered e-commerce platform in the world (big talk we know, but that is our goal!). Our engineers combine creativity, curiosity, and drive to continuously perfect and revolutionize Jet from the inside out. We are looking to bring more intellectually curious engineers who are passionate about technology in general (Jet is a technology first company and prides itself on its culture of learning and knowledge sharing and we want all our engineers to be as passionate as we are!)
Our platform is largely an event driven platform implemented via a microservice architecture. The platform runs on Microsoft Azure and uses a large number of technologies and middle ware. The bulk of backend code is written in a functional style (F#) and our system of record is an event sourced system (essentially a log of all actions on the platform). We use a mix of Kafka, Redis, Elastic Search, Azure SQL, Event Store and Azure Storage for our data stores. About 50% of the platform runs on Linux and the rest Windows (although we move into more containerization we will see a shift towards Linux).
About the Job
We are looking for an exceptional Data Engineer to help build a world class analytical platform to collect, store and expose both structured and un-structured data generated by a vastly growing system landscape at Jet.com.
You can expect a freewheeling, informal work environment, populated by a combination of folks from top companies that have produced many successful products, as well as some PhD’s that have escaped the ivory tower.
Because we work on cutting edge technologies, we need someone who is a creative problem solver, resourceful in getting things done, and productive working independently or collaboratively.
What you will do:
- Design, implement and manage a near real-time ingestion pipeline into a data warehouse and Hadoop data lake.
- Gather and process raw data at scale - collect data across all business domains (our functional-first, event sourced, micro services backend) and expose mechanisms for large scale parallel processing
- Process unstructured data into a form suitable for analysis and then empower state-of-the-art analysis for analysts, scientists, and APIs
- Support business decisions with ad hoc analysis as needed
- Evangelize an extremely high standard of code quality, system reliability, and performance.
- Influence cross functional architecture in sprint planning
What you will have:
- Minimun four years of experience in running, using and trouble shooting the Apache Big Data stack i.e. Hadoop FS, Hive, HBase, Kafka, Pig, Oozie, Yarn
- Programming experience, ideally in Scala or F# but we are open to other experience if you’re willing to learn the languages we use
- Proficient scripting skills i.e. unix shell and/or powershell
- Experience processing large amounts of structured and unstructured data with MapReduce
- We use Azure extensively, so experience with cloud infrastructure will help you hit the ground running
Jet is reshaping ecommerce as we know it. Based in Hoboken NJ (just 10 min from Manhattan), we are a shopping site on the relentless pursuit to build the greatest shopping experience in the world. At Jet, we believe in bold. That means taking risks, asking “why not”, looking where no ones looked before and bringing it! Our engineers are utilizing world class technologies, to optimize the supply chain, remove unnecessary costs, sprinkle in some surprise and delight all while saving customers every penny possible and we’re just getting started. At Jet, we have worked hard to build a culture that stresses the importance of learning and sharing knowledge. If you want to be part of the team that is changing the shopping norm and learning a lot along the way, we have a hunch you’d look good in purple.
See what we’ve been up to
"A New & Improved Saving Experience" // by Jessica Anerella, Product Design at Jet.com