low man on the what?!?

So here's the deal. I am work today and was talking to my boss. We are getting a new apprentice because my coworker is taking paternity leave and we need someone to fill in while he's gone. The girl is a student and needs to constant supervision. My work is considering trading me to the other "on call team" so that there will be someone there to supervise her on the weekends. The thing is, we wouldn't need to switch if my other boss would just show up on those weekends that he is on call. So in order to alleviate him comming to work on those weekends they are going to trade me to the other shift since I can work unsupervised. The thing is, I don't want to go to the other side. It screws up all the long weekends that I am supposed to be off this calender year. Anyways I stated that I didn't want to go to the other team. My boss replied that it sucks when you are the low man on the totem pole. Pardon Me?!! I have been there seven years. I have put my career before everything else. That includes my husband who wants to have another baby, I didn't want to because I knew it wouldn't be good for my career. So what's the deal here. I give it 100% commitment and that's what I get back. Sometimes I am just a f*ckin idiot. argh!