Description
The Role
LeafLink is seeking a Principal Data Engineer to join our remote-friendly team, headquartered in NYC, who is passionate about working with teams that solve interesting, large-scale problems rapidly. This impactful position enables LeafLink to coordinate and integrate with 3rd party data sets and proprietary data to produce valuable insights into business and customer needs. As a member of our engineering team, you will be in a position to have a direct and lasting impact everywhere in the company. Your contribution will be immediate and have positive ripple effects across not just our business, but also the business of each of our customers.
LeafLink is currently tackling a large-scale platform overhaul that will strengthen our position as a technical leader within the industry. As such, this role has the opportunity to help lead, shape, and grow the data and machine learning architecture within our platform, as well as work with new and growing technologies. It’s a very exciting time to join our engineering team!
Ideal candidates for this position should possess a keen mind for solving tough problems with the ideal solution, partnering effectively with various team members along the way. They should be deeply passionate about organizing and managing data at scale for various use cases. They should be personable, efficient, flexible, and communicative, have a strong desire to implement change, grow, mature, and have a passion and love for their work. This role comes with the opportunity to be a high performer within a fast-paced, dynamic, and quickly growing department in all areas.
What You’ll Be Doing
- Audit, design, and maintain a high-performing, modular, and optimal data pipeline architecture for structured and unstructured use cases around machine learning, reporting, and analytics
- Design and co-build with Cloud and DevOps the infrastructure and operations required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL, python, and AWS cloud technologies
- Keep up to date on modern technologies and trends and advocate for their inclusion within products when it makes sense
- Analyze and evaluate existing solutions and make decisions on whether to extend or refactor as needed with a major focus on improving our pipeline and reporting performance
- Work with the CTO and department stakeholders to properly plan short and long-term goals, and define and execute a technical roadmap that continues to evolve LeafLink’s data capabilities and functionality to meet the needs of our Business and Product Vision.
- Work collaboratively with multiple cross-functional agile teams to help deliver end-to-end products and features enabled by our data pipeline, seeing them through from conception to delivery
- Help define, document, evolve, and evangelize high engineering standards, best practices, tenants, and data management & governance across data and analytics engineering
- Move quickly and intelligently - seeing technical debt as your nemesis and eliminating risk
- Effectively communicate the complexity of your work to technical and non-technical audiences through non-written and written mediums
- Design, develop, and test data models in our data warehouse that enable data and analytics processes
- Help define and build our enterprise data catalog and dictionary
- Troubleshoot, diagnose and address data quality issues quickly and effectively while implementing solutions to combat this at scale, including improved quality controls and observability and monitoring
- Provide mentorship and growth to our BE and Data engineers while creating repeatable and scalable solutions and patterns
What You’ll Bring to the Team
- Minimum of 10 years experience in a professional working environment on a data or engineering team
- Advanced working SQL knowledge and experience working with relational and non-relational databases, query authoring (SQL) as well as working familiarity with a variety of data stores
- Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
- Expertise writing Python processing jobs to ingest a variety of structured and unstructured data received from various sources & formats such as Rest APIs, Flat Files, and Logs with the ability to support and scale to both smaller and larger dataset ingestions
- They should also have experience using the following software/tools:
- Experience with object-oriented/object function scripting in Python and data processing libraries such as requests, pandas, sqlalchemy
- Experience with relational SQL and NoSQL databases, such as Redshift or comparable cloud-based OLAP databases such as Snowflake
- Experience with data pipeline and workflow management tools: Airflow
- Experience with AWS cloud services
- Hands-on experience with technologies such as Dynamo, Terraform, Kubernetes, Fivetran, and dbt is a strong plus
- Experience with designing and implementing machine learning enablement tools and infrastructure
- Experience leveraging API-based LLM models, dynamic prompt generation, fine-tuning
- Comfortable working in a fast-paced growth business with many collaborators and quickly evolving business needs
- Individual contributor leadership to our data and analytics engineers and specialization on our current Platform Engineering team around data enterprise architecture and best practices
- Consistency and standards to how we visualize and use our enterprise data at LeafLink through helping us define our first Data Dictionary and Catalog
LeafLink Perks & Benefits
- Flexible PTO - you’re going to be working hard so enjoy time off with no cap!
- A robust stock option plan to give our employees a direct stake in LeafLink’s success
- 5 Days of Volunteer Time Off (VTO) - giving back is important to us and we want our employees to prioritize cultivating a better community
- Competitive compensation and 401k match
- Comprehensive health coverage (medical, dental, vision)
- Commuter Benefits through our Flexible Spending Account
LeafLink’s employee-centric culture has earned us a coveted spot on BuiltInNYC’s Best Places to Work for in 2021 list. Learn more about LeafLink’s history and the path to our First Billion in Wholesale Cannabis Orders here.
Please mention the word **AFFIRM** and tag RMzQuMTUwLjIyMy4yOA== when applying to show you read the job post completely (#RMzQuMTUwLjIyMy4yOA==). This is a beta feature to avoid spam applicants. Companies can search these words to find applicants that read this and see they're human.