I don't think you are crazy at all for wanting to lose weight. I'm the same height as you, and approaching your weight (I'm around 53 or 54 kg now, and gaining - I was 48.5 kg about 18 months ago). I completely understand how you feel. Myself, I get so uncomfortable in my body at times that I can't sit still or get to sleep at night!
But yeah, I understand that you cop attitude from people when you tell them you want/need to lose weight when you are still at a healthy BMI
It's not much fun! You have a right to feel happy and healthy, and if losing weight is what you need to do, then go for it!!! Don't worry about what other's say