I cut dairy products out of my diet. I had chronic asthma growing up and someone told me that eliminating dairy would help 'cure' it so I gave up dairy when I was about 21. My asthma reduced significantly after that, however it could be coincidence. Many people outgrow chronic asthma at that age and also I moved from a high asthma area (Albury-Wodonga) to a low asthma area (Melbourne). I continued to follow a vegan diet though, but mainly for ethical reasons so I'm not going to preach veganism as the be-all and end-all of dieting to you!
Diet-wise low fat dairy foods aren't that evil. Just keep an eye on the calorie and fat content. Full fat dairy is not good for losing weight because let's face it cow's milk is designed in its natural state to make a wee little baby calf grow to about half a tonne of weight in a year! Low fat dairy is full of calcium but you need to check that the fat isn't replaced by sugar to make it taste good.
You can get calcium from non-dairy sources. For people who eat fish, fish bones are chock full of calcium. So sardines are particularly good. Dark green leafy vegetables are full of calcium too. So is tofu, soybeans, fortified soy milk, bok choy, broccoli and chinese cabbage. Some health experts argue that vegetable derived calcium is easier to absorb than animal derived calcium (link
). And some doctors and nutritionists claim there is no greater incidence of osteoporosis among vegans than those who consume dairy food, but the latest medical literature seems to suggest that vegans do have lower bone density than other people. However this could be because of a lack of vitamin D, not because of a lack of calcium (which is really splitting hairs because the greatest source of vitamin D is... you guessed it... dairy food.)
If you like fish and tofu then you're laughing. Osteoporosis is lowest among people following a Japanese style diet. Low in dairy and meat but high in vegetables, tofu and fish. (link
If you don't like dairy and you are worried about your calcium intake you can always take a supplement. There are calcium tablets with vitamin D added which aids absorption (not safe for vegans as the vitamin D is milk derived).Link to the Victorian government Better Health page on calcium for more information.