Help! my neighbor's cat won't leave us alone?

A family who lives down the street from my home,has several cats. One of the cats named Morris, always hangs around my house. It's like he wants to live with me and my family because one time my mom put some food out for the poor thing and showed him some affection. Problem is my cat does not like him and got into a bad fight with him one time. Another problem is I am already trying to take care of 2 cats who belonged to a lady who lived down the street from me. She just moved out and left her cats. Me and my mom took pity on them and took them in. We are trying to find them homes, but one of the cats is still too afraid to come into our home, and what's worse is Morris constantly chases him off. This makes me very angry,at least Morris has a home the other poor cat doesn't and looks scared and sad. I don't want to get into with my neighbors. However, I want them to take care of their own cat, and keep it off our property. How do I approach them, without their being any hard feelings?