Note: The job is a remote job and is open to candidates in USA. Scalence L.L.C. is seeking a Principal Data Platform Engineer to establish a greenfield data and AI platform for federal background investigations. The role involves designing and building the data platform architecture, partnering with cloud engineering, and ensuring compliance with federal requirements.

Responsibilities

• Design and stand up the organization's data and AI platform from the ground up — architecture, compute, storage, and the lakehouse foundation
• Codify the platform as infrastructure-as-code (Terraform) and build the CI/CD pipelines that promote work from development through to the accredited production environment
• Establish data governance, cataloging, lineage, and fine-grained access control as foundational, not bolted on later
• Build and own the ingestion, transformation, and pipeline layer that turns raw and synthetic data into governed, analysis-ready data products
• Design the platform to operate within FedRAMP Moderate, NIST 800-171, and CUI constraints, treating compliance as a first-class architectural requirement
• Define the artifact promotion process so only signed, validated artifacts cross into the accredited environment
• Partner with cloud engineering across the infrastructure/security boundary, with clear ownership of the in-platform layer
• Enable the data science and ML team with the platform capabilities, governed data, and tooling they need to ship models and AI features into the product
• Own platform reliability, performance, and cost discipline as usage scales
• Set the engineering standards, patterns, and documentation a growing data team will build on

Skills

• U.S. citizenship required
• Must be able to obtain and maintain a T5/SSBI federally adjudicated clearance; active clearance preferred
• [8]+ years in data engineering / data platform engineering, with demonstrated principal-level ownership
• Has stood up a data platform or lakehouse from scratch — owning the architecture and build end to end, not operating an inherited one
• Design of batch (and, where needed, streaming) data pipelines and SQL-based transformations on a lakehouse/Delta foundation, with sound analytical data modeling
• Infrastructure-as-code (Terraform) and CI/CD for data workloads, including environment promotion from development to production
• Platform-level data governance: cataloging, lineage, and fine-grained access control
• Hands-on cloud experience with a major provider (Azure preferred; AWS or GCP considered)
• Strong proficiency in Python and SQL
• Track record partnering across an infrastructure/security boundary and setting technical standards for other engineers
• Excellent analytical, troubleshooting, and communication skills
• Bachelor's in a technical field or equivalent experience
• Hands-on Databricks: Unity Catalog, Databricks Asset Bundles, MLflow
• Experience in regulated or accredited environments: FedRAMP, NIST 800-171, CMMC, CUI handling, or the ATO/RMF process
• Active security clearance (T5/SSBI or higher)
• Government or defense contracting experience
• Familiarity with MLOps patterns (model registry, model serving) to support a data science team
• Cost governance / FinOps discipline for cloud data platforms
• Spark / PySpark — relevant since the platform is Databricks, though the data volume here does not demand distributed-scale expertise

Company Overview