<human>: Give me the gist of the text in a nutshell

I'm a college student who lives at home for about another 18-24 months. My mom's boyfriend comes by frequently, and is quite the asshole. I'm rarely around but when I am home I pretty much keep to myself, but this guy feels the need to be highly critical of anything I do. I never initiate any confrontation with him. Most recently I came home to him complaining to my mother that I'm a bum and disrespectful, so I stepped in and it became quite the altercation- but nothing physical. However, he was threatening to fight me, but I told him that I'd just call the cops and get his ass kicked out. He laughed in my face, and said that it would never happen because I'm a bitch. So I'm curious, if my license shows I live here with my mom, and his does not...doesn't that mean I have some say in whether or not a guest is welcome? I live in Pennsylvania, 23 years old, and a male if that matters. I'm not looking to prevent him from ever visiting, but next time he gets threatening I would rather him just be escorted out by cops. 
Oh and I'm a "disrespectful, bum" because I spend most of my time working or studying for the MCAT. Yet this guy is unemployed, gets high all day, and thinks he is the paragon of excellence.
<bot>: Can I kick my mom's boyfriend out if he doesn't live with us? Live in PA, I'm 23M