Full Description

What You Will Do

Perform independent end-to-end validation of fraud detection ML models, including conceptual soundness, data integrity, feature engineering, model development, deployment design, and monitoring frameworks. Develop challenger models. Review and challenge first-line fraud model methodologies, assumptions, and implementation choices (e.g., scikit-learn, LightGBM, graph models, anomaly detection techniques, GenAI components). Build and deploy agentic AI tools to support model validation workflows — automating review of model documentation and code, surfacing risks and inconsistencies. Assess model performance using appropriate fraud metrics (e.g., precision/recall, ROC-AUC, PR-AUC, cost-sensitive metrics, fraud rate capture, business impact trade-offs). Evaluate model stability, drift detection, retraining strategies, and production monitoring practices. Independently replicate model results where necessary and conduct challenger analyses to assess model robustness and limitations. Review large-scale transaction datasets and feature pipelines (e.g., >100M transactions, hundreds of features) to assess data representativeness, leakage risks, and bias. Evaluate model governance documentation, explainability approaches, and transparency — including regulatory compliance related to model risk, fairness, and data privacy. Validate new technologies applied in fraud detection, such as Graph Networks, Behavioral Biometrics, Anomaly Detection, and GenAI-based systems. Assess controls around CI/CD pipelines, deployment processes (e.g., Docker, Jenkins), and cloud environments (e.g., AWS SageMaker, S3, Athena, Lambda). Develop and maintain validation frameworks, testing standards, and model performance monitoring tools (e.g., SQL, PySpark, Python-based validation libraries). Collaborate closely with first-line fraud data scientists, ML engineers, product, and business stakeholders to ensure transparent communication of model risks and validation findings. Provide actionable recommendations and formally document validation outcomes in line with internal model governance standards and external regulatory expectations. Stay up to date with evolving fraud typologies, emerging ML/AI techniques, and regulatory developments in model risk management.

Who you are

Advanced degree (Master’s or PhD) in a quantitative field such as Data Science, Statistics, Mathematics, Computer Science, Physics, or Engineering. 3+ years of hands-on experience in fraud-related modeling (e.g., transaction fraud, account takeover, identity fraud, payments fraud etc). Strong expertise in machine learning methods used in fraud detection, including tree-based models (e.g., LightGBM), anomaly detection, graph/network models, and advanced ML techniques. Deep understanding of the end-to-end ML lifecycle — from conceptual design and feature engineering to production deployment and monitoring — with the ability to critically challenge each stage. Strong programming skills in Python and SQL; experience with PySpark/Spark and large-scale data processing. Experience building agentic AI workflows. Familiarity with cloud-based ML platforms (e.g., AWS SageMaker, Lambda, S3, Athena) and production deployment workflows. Strong knowledge of model validation principles, model risk governance frameworks, and regulatory expectations. Experience assessing model bias, fairness, explainability, and privacy risks. Excellent analytical thinking and structured problem-solving skills, with the ability to assess complex models and clearly articulate risks and limitations. Strong communication skills, capable of translating technical findings into clear, actionable insights for senior stakeholders and non-technical audiences. Ability to work independently while constructively challenging first-line teams in a collaborative manner.

Awesome to have

Experience in BNPL, credit cards, payments, or other transaction-heavy financial products. Experience validating models in highly regulated environments. Experience mentoring junior validators or leading validation reviews. Exposure to inference of rejected transactions and understanding of fraud/credit overlap. Familiarity with AI governance frameworks and emerging AI regulatory requirements.