Here is the barn which all the animals live. Crook also lives in the barn because he can't live with the others because he is black.
Crook wasn't treated like the other men. All the other men got to live with each other while Crook was leaving under these conditions. Any time the men where doing something like playing horseshoe Crook couldn't join them because of his race.
When Crook was just sitting in his bunk all alone Lennie had come in because he saw a light. Crook was being rude to Lennie by telling Lennie that he wasn't allowed in his bunk because Crook wasn't allowed in his bunk. Crook was being rude because he thought Lennie was like everyone else hating on him because of his race.