Book My Growth Assessment
insights

Building Bias Mitigation Into AI Products

Bias in AI systems is not a fringe problem — it is the default outcome when mitigation is not deliberately engineered in. Here is how to do it.

Ravve Jay Prevendido
Ravve Jay Prevendido·Aug 18, 2024·4 min read
17+ industry awards · Brand architect behind OWWA, Nuvia & 100+ brands · ravvejay.com
Share
Building Bias Mitigation Into AI Products

Bias in AI systems does not arrive through malice. It arrives through data. Every AI model learns from historical information, and historical information encodes historical inequities. A hiring model trained on a decade of successful hires will learn to prefer candidates who look like past hires. A credit model trained on repayment histories will encode every structural barrier that affected access to credit in the past. The model is not bigoted. The model is accurate about a history that was.

This is why bias mitigation cannot be handled by good intentions or diversity statements. It requires specific technical interventions at specific stages of the model development lifecycle. This piece covers those interventions in practical terms — what they are, when they apply, and how to evaluate whether they are working.

The baseline for this work is understanding that AI without mitigation is not neutral — it is a replication engine for whatever inequities the training data contains. The businesses that understand this build better products. The businesses that don't create liability exposure that is entirely avoidable.

Stage one: Training data audit

The audit starts before training. Before a model learns anything, the team responsible for the data needs to answer: Who is represented in this dataset, and who is missing? A medical AI trained primarily on data from academic medical centers in the United States will underperform for populations that seek care through community health centers, rural hospitals, or international providers. A facial recognition system trained predominantly on lighter-skinned faces will perform worse on darker-skinned faces. These gaps need to be identified and documented before training, not discovered after deployment.

The audit should also look for proxy variables — features that are not protected characteristics themselves but that correlate strongly with them. Zip code, for example, correlates with race in many U.S. cities. Name-based features correlate with ethnicity. Browsing history can correlate with gender and age. A model that excludes race but retains proxies for race will produce racially correlated outputs. Identifying and addressing proxy variables requires domain expertise, not just statistical analysis.

Document which demographic groups are represented and which are underrepresented in the training data.

Identify proxy variables that may carry protected characteristics through the model.

Decide explicitly what to do about gaps: supplement data, weight correction, scope limitation, or documented acceptance.

Stage two: Fairness metric selection

"Fair" is not a single number. There are multiple mathematical definitions of algorithmic fairness — demographic parity, equalized odds, calibration — and they are provably incompatible with each other in most real-world situations. A system that is fair by one definition will be unfair by another. The right metric depends on the context: in a hiring system, you might prioritize equal opportunity (same true positive rate across groups). In a recidivism prediction system, you might prioritize calibration (equal accuracy across groups). These are different choices with different tradeoffs, and they should be made explicitly by people who understand the consequences, not implicitly by whoever runs the training loop.

TTGC's approach, developed through Ravve's engineering work on production AI systems, is to define fairness metrics before model development based on what the system is actually being used to decide, who it affects, and what the consequences of each type of error are. That definition then becomes a test suite that the model must pass before deployment — not a post-hoc evaluation of whether it "looks okay."

Stage three: Pre-deployment bias testing

Testing the model against the target population before deployment is non-negotiable for any AI system making consequential decisions. This means running the model against a held-out test set that is representative of the actual users the system will affect — not just a random slice of the training data. Performance metrics should be broken out by demographic group, not just reported as an aggregate. An aggregate accuracy of 92% that hides 78% accuracy for a minority subgroup is not a 92% accurate model. It is a discriminatory one.

The pre-deployment questions every business leader should be asking include these testing requirements. A vendor who cannot show you disaggregated performance metrics by relevant demographic groups should be treated as a vendor who has not done this work.

Stage four: Post-deployment monitoring

Bias mitigation is not a one-time pre-deployment audit. The world changes, the user population changes, and the distribution of inputs to the model changes — often in ways that reintroduce bias patterns that were addressed in training. Production monitoring should include disaggregated performance tracking, anomaly detection on output distributions by demographic group, and a defined process for triggering model retraining or intervention when metrics drift beyond defined thresholds.

This is particularly important in the sectors where bias has the most consequential effects. For fintech lending and credit, the specific fairness requirements and monitoring frameworks are shaped by the Equal Credit Opportunity Act and Fair Housing Act. For hiring, the EEOC's guidance on AI and the discrimination risks in AI-assisted recruitment define what monitoring looks like in that context.

Bias mitigation is not a feature you add to a finished AI product. It is an engineering discipline that runs through the entire development lifecycle — from data selection through post-deployment monitoring.

Building an AI system that makes decisions about people? Talk to TTGC about bias auditing, fairness testing, and accountability frameworks built in from day one.

Book a free Brand and Growth Assessment and see exactly how Through The Glass Creatives would approach it.

Get Your Free AssessmentGet Your Free Assessment

Sources

  1. NIST — "Towards a Standard for Identifying and Managing Bias in Artificial Intelligence" (2022). Technical framework for AI bias identification and mitigation.
  2. MIT Media Lab — Buolamwini, J. & Gebru, T., "Gender Shades: Intersectional Accuracy Disparities in Commercial Gender Classification" (2018). Foundational research on demographic disparities in AI systems.
  3. ACM Conference on Fairness, Accountability, and Transparency — "Fairness and Abstraction in Sociotechnical Systems" (2019). Framework for understanding incompatible fairness criteria.
  4. U.S. Federal Trade Commission — "Using Artificial Intelligence and Algorithms" (2020). FTC framework on algorithmic bias and consumer protection.
  5. Google Research — "A Survey on Bias and Fairness in Machine Learning" (2021). Comprehensive technical survey of bias types and mitigation techniques.

Results shared by Through The Glass Creatives Global and its founders are not typical and are not a guarantee of your success. Ravve Jay Prevendido and Mherie Vic Palomo Prevendido are experienced business owners, and your results will vary depending on your industry, effort, application, experience, and market conditions. We do not guarantee that you will achieve specific outcomes by using our services. Consequently, your results may significantly vary. We do not give investment, tax, or other financial advice. Case studies and client experiences are mentioned for informational purposes only. The information contained within this website is the property of Through The Glass Creatives Global - FZCO. Any use of the images, content, or ideas expressed herein without the express written consent of Through The Glass Creatives Global FZCO is prohibited. Copyright © 2026 Through The Glass Creatives Global FZCO. All Rights Reserved.