Document the limitations and scope of your data as best as possible.
Problems to watch for:
Out of domain data: data used for training are not relevant to the model application
Domain shift: model becoming inadapted as conditions evolve
Feedback loop: initial bias exacerbated over the time
The last one is particularly problematic whenever the model outputs the next round of data based on interactions of the current round of data with the real world.
Solution: ensure there are human circuit breakers and oversight.
Transformation of subjects
Algorithms are supposed to help us, not transform us
