Senior Data Platform Engineer at DEFCON AI

Remote, USAFull-timeRemoteDefcon - EngineeringPosted 12 days ago

About the Role

<div class="content-intro">ABOUT DEFCON AI RESILIENCE IN THE FACE OF DISRUPTION. DEFCON AI is an insights company that leverages artificial intelligence, mathematical optimization, data analytics, and software engineering for resilient optimization of complex systems. In today’s dynamically changing world, DEFCON AI’s technology aligns outcomes with operational goals, better decision making, and empowers customers to anticipate assess, and mitigate the impacts of disruptions.</div>If your answer to “the two records disagree” is never overwrite, this role was built for you.  About the Role  As a Senior Data Platform Engineer, you’ll own the durable view of every resolved entity on the platform: the storage layer that holds what we know, where each fact came from, how confident we are in it, and how that picture has changed over time. When two sources disagree, both are kept with their evidence. When someone asks what we believed on a given date, and why, the platform can answer.  You’ll join the analytics and AI engineering team behind a system that ingests records from dozens of disparate sources, resolves them to the right entity, highlights what analysts should review first, and provides transparent, explainable recommendations that users can trust. Operating within a secure government cloud environment, the platform depends on a storage layer that never quietly erases a disagreement between sources.  Upstream matching is probabilistic. A match arrives with a confidence score rather than a yes, and the storage layer carries that uncertainty forward rather than collapsing it into one clean record. Sources change shape without warning and there is no shared key to join on, so this is a current data-engineering problem rather than a classic warehousing one.  The seniority this role calls for is about judgment, not tooling breadth: knowing why you never overwrite, and being able to reconstruct a past decision with its evidence a year later. If your instinct when records disagree is to preserve both and let a human decide, you’ll be at home here.  This is a fully remote role with occasional travel to DEFCON AI headquarters, customer sites, and partner facilities as needed.   Key Responsibilities  <ul> <li data-leveltext="" data-font="Symbol" data-listid="3" data-list-defn-props="{"335551671":0,"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="0" data-aria-level="1">Design and build the storage layer that preserves source disagreement and history rather than resolving them away at write time </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="3" data-list-defn-props="{"335551671":0,"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="1" data-aria-level="1">Build the first release’s result and evidence store: saved per-person results linked to the originating record, prior results and human feedback retained separately, and configuration versions on every result </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="3" data-list-defn-props="{"335551671":0,"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="2" data-aria-level="1">Implement provenance and lineage across every node and edge against the platform’s evidence-record contract </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="3" data-list-defn-props="{"335551671":0,"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="3" data-aria-level="1">Store the versioned rule library and configuration, and record on every result the exact rule versions and ordered context supplied to any model-assisted step </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="3" data-list-defn-props="{"335551671":0,"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="4" data-aria-level="1">Ensure every write is traceable to its origin and every decision is reproducible </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="3" data-list-defn-props="{"335551671":0,"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="5" data-aria-level="1">Carry match confidence and other uncertainty forward through the platform rather than collapsing it into a single value </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="3" data-list-defn-props="{"335551671":0,"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="6" data-aria-level="1">Support point-in-time questions: what did we believe about this entity on this date, and on what evidence </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="3" data-list-defn-props="{"335551671":0,"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="7" data-aria-level="1">Define, with entity resolution and ML teammates, what the storage layer needs to receive and what it must serve downstream </li> </ul>   Required Qualifications  <ul> <li data-leveltext="" data-font="Symbol" data-listid="4" data-list-defn-props="{"335551671":0,"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="8" data-aria-level="1">6+ years in data engineering or data platform engineering, including production experience with versioned, temporal, or historized data models </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="4" data-list-defn-props="{"335551671":0,"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="9" data-aria-level="1">Experience designing storage where history, provenance, and source disagreement are first-class rather than resolved away </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="4" data-list-defn-props="{"335551671":0,"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="10" data-aria-level="1">Comfort building on probabilistic upstream output rather than a clean shared key </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="4" data-list-defn-props="{"335551671":0,"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="11" data-aria-level="1">Strong Python and SQL, with production experience on PostgreSQL or comparable </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="4" data-list-defn-props="{"335551671":0,"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="12" data-aria-level="1">US Citizenship Required </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="4" data-list-defn-props="{"335551671":0,"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="13" data-aria-level="1">Active US Secret clearance </li> </ul>   Preferred Qualifications  <ul> <li data-leveltext="" data-font="Symbol" data-listid="5" data-list-defn-props="{"335551671":0,"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="14" data-aria-level="1">Experience building the storage layer downstream of an entity-resolution or record-linkage system </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="5" data-list-defn-props="{"335551671":0,"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="15" data-aria-level="1">Bi-temporal, event-sourced, graph, or temporal data modeling in production </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="5" data-list-defn-props="{"335551671":0,"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="16" data-aria-level="1">Federal, government, or regulated-industry experience where a historical decision had to be reproducible on demand </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="5" data-list-defn-props="{"335551671":0,"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="17" data-aria-level="1">Active Top Secret clearance </li> </ul>   What Success Looks Like  <ul> <li data-leveltext="" data-font="Symbol" data-listid="6" data-list-defn-props="{"335551671":0,"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="18" data-aria-level="1">A durable view that never silently overwrites a disagreement between sources </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="6" data-list-defn-props="{"335551671":0,"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="19" data-aria-level="1">“What did we believe on this date, and on what evidence” is answerable a year later </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="6" data-list-defn-props="{"335551671":0,"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="20" data-aria-level="1">The storage layer holds up as probabilistic match output flows in, rather than assuming a clean join </li> </ul>   What We Offer  <ul> <li data-leveltext="" data-font="Symbol" data-listid="7" data-list-defn-props="{"335551671":0,"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="21" data-aria-level="1">A fully remote, results-based environment </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="7" data-list-defn-props="{"335551671":0,"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="22" data-aria-level="1">Competitive salary, bonus, and equity package </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="7" data-list-defn-props="{"335551671":0,"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="23" data-aria-level="1">100% employer paid, comprehensive health insurance including medical, dental, and vision for you and your family </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="7" data-list-defn-props="{"335551671":0,"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="24" data-aria-level="1">Unlimited PTO, with your manager’s approval </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="7" data-list-defn-props="{"335551671":0,"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="25" data-aria-level="1">Flexible work environment where you manage your work day </li> </ul> <ul> <li data-leveltext="" data-font="Symbol" data-listid="7" data-list-defn-props="{"335551671":0,"335552541":1,"335559685":720,"335559991":360,"469769226":"Symbol","469769242":[8226],"469777803":"left","469777804":"","469777815":"multilevel"}" data-aria-posinset="26" data-aria-level="1">14 weeks of fully-paid parental leave </li> </ul>   Salary Range: $165,000—$200,000. This represents the typical salary range for this position based on experience, skills, and other factors.   <div class="content-conclusion"><div>We’re an Equal Opportunity Employer: You’ll receive consideration for employment without regard to race, sex, color, religion, sexual orientation, gender identity, national origin, protected veteran status, or on the basis of disability. </div> <hr> <div>Applicant Data Disclosure   </div> <div>By submitting an application, you acknowledge that Defcon AI uses third-party service providers to facilitate its recruitment and hiring processes. These providers include applicant tracking systems, candidate verification platforms, and fraud detection tools (collectively, "Hiring Platforms"). Your application materials, including your résumé, cover letter, work samples, responses to application questions, and any other information you submit, may be transmitted to and processed by these Hiring Platforms for the following purposes:  </div> <ul> <li>Managing and administering your application throughout the hiring process; </li> <li>Verifying the accuracy and authenticity of application materials, including by cross-referencing information you provide against publicly available sources and proprietary databases; </li> <li>Identifying indicators of potentially fraudulent, fabricated, or materially misleading application content, including but not limited to discrepancies between submitted materials and publicly available professional profiles, geographic anomalies, and fabricated work histories. </li> </ul> <div class="elementToProof">Applications that are flagged through this process as containing indicators of fraud or material misrepresentation may be declined from further consideration. If you have questions about the status of your application or the evaluation process, please contact <a href="mailto:[email protected]">[email protected]</a>.  </div> <div class="elementToProof"> </div> <div>Defcon AI requires its Hiring Platform providers to process your information solely for the purposes described above and in accordance with applicable law. Your information will be retained only for as long as necessary to fulfill these purposes and any applicable legal obligations, after which it will be deleted in accordance with Defcon AI's data retention policies.</div> <div>For more information about how your data is used, please refer to our Privacy Policy and <a id="OWAa5fbf800-7cff-a47d-90dd-97168e270e0a" class="x_OWAAutoLink" href="https://nam09.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.redcellpartners.com%2Fapplicant-privacy-policy%2F&data=05%7C02%7Ckat.creamer%40redcellpartners.com%7Cc0f94f3daed94dc7503108de8b61955e%7Cf861de501a2a42359dbf28afd57d1d97%7C0%7C0%7C639101448841578570%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=WoPiBMZqQyPPK5N4E1sCagkbMY2S8P3aSRSoy7T4los%3D&reserved=0" data-auth="NotApplicable" data-outlook-id="7bba607b-d838-4280-a6ca-ed63ecc87c50">Applicant Privacy Notice</a>.  </div>  </div>

About the Role

Related Roles