According to the federal Second Circuit Court of Appeals, it can mean discrimination.
From Wait A Second blog:
The primary issue on appeal is whether Abrams was denied the Van assignment for discriminatory reasons. The Court of Appeals says this is a close case, but that the jury can find in plaintiff's favor. The "fit in" comment can have racial connotations. The Fifth Circuit in 2004 said that a "fit in" comment is "at least as consistent with discriminatory intent as it is with nondiscriminatory intent: The employer just might have found the candidate 'not sufficiently suited' because of a protected trait such as age, race, or engaging in a protected activity." So, while some cases hold the plaintiff to a higher standard in promotion denial cases when management says that someone else was simply better qualified for the position (see Byrnie v. Town of Cromwell, 243 F.3d 93 (2d Cir. 2001)), this case is different because the "fit in" comments suggest that something else -- race -- as opposed to mere qualification was going on here.