Bias is always present in data.

Document the limitations and scope of your data as best as possible.

Problems to watch for:

• Out of domain data: data used for training are not relevant to the model application
• Domain shift: model becoming inadapted as conditions evolve
• Feedback loop: initial bias exacerbated over the time
  
  

The last one is particularly problematic whenever the model outputs the next round of data based on interactions of the current round of data with the real world.

Solution: ensure there are human circuit breakers and oversight.

e.g. YouTube recommendation algorithm