<human>: Could you distill the essence of the text into 1-3 sentences?

Ok, so a little background:
My son (D) is going to be 7 in September. 
When D was 4 months old his father (M) went to prison for a little over 2 years. (Drug offenses). 
When M was released he attempted to see D about once a month, more if his mother wanted to see him or he had a girlfriend. I would say he saw him maybe a total of 20 times over 2 years, never bought him anything, and did not pay much child support.
This went on until he met his current girlfriend of 3 years (K). 
He then took me to court for visitation. I agreed to it and he saw him once a week for 3 months.
M and I have not had a good relationship since I left him.
I moved 3 hours from our hometown on June 5, 2011. M did not visit or call for a year. When I would attempt to arrange visits it would not go well. I went to my hometown for a month visit in June 2012 and M saw D 3 times. I then left again and I have not heard from M except when I filed for a change of venue. That caused a great deal of harassment. My question is this: Can I terminate his rights? How do I do that? I live in PA.
<bot>: 7 yrs old son's father has only seen him 3 times in 2 years. Can I terminate his rights? He does pay support but only 50 dollars a month.