Should I break up with this guy?

I know this is long, but please bear with me. I'm almost 17 and am admittedly socially awkward... or rather just really shy (whichever way you want to put it). This guy (let's just call him John) is the first person to ever really flirt with me. He's also the first to ask me out. No, I'm not an ugly person, I just don't talk to alot of people, and tend to bring up conversations that some people find horrifying or odd. We traded phone numbers and had been talking for perhaps 3 or so days before I agreed to go out with him... I guess I just feel curious or compelled to do it. Not really sure if I like him THAT way. Anyway, for the next two weeks after that, we only met at school for perhaps 15-30 minutes a day. I can't tell my strict (crazy) mother about this yet, so we can only talk for limited amount of time on the phone. Case and point: I didn't know him that well. Until today. I sweet talked someone into bringing us to a restaurant and movies (since they were going there anyway). Despite warning him of our arival beforehand, he was playing with his friends when we showed up, then made us wait while he got ready. Took almost 30 minutes. John's grades are horrible, he doesn't know what to do with his life, and he's... not necessarily the biggest gentleman on the planet. The date was rather awkward for the most part, I actually talked more than him (THATS a feat). We paid for our own stuff (he's poor too, so I paid for his snack). During the movie, he held my hand, kept kissing my cheek, put his arm around my shoulder (kept 'accidently' touching my [appologized later on]), and he kept calling me beautiful; that if he ever hit me, he'd kill himself; that he was lucky to be dating an upper-clman... and that his friends were jealous. I didn't say anything, but tried playing along (again, I'm VERY shy). After the movie, he saw me talking to my sister (she's the one who brought us). John couldn't hear us, but my sister really dislikes him- says that I could do much better. I'm really not sure about the whole thing, after all I KNOW that it isn't going to last forever. Anyway, the ride back was silent. He didn't say a word when we dropped him off (even after I said I'd call him tomorrow). John seemed to be brooding on the way back to his home- he probably guessed what my sis and I were talking about. He left me a voicemail a little while ago saying it was alright if I broke up with him... sounded depressed. I've got a soft heart, and I feel sorry for him since he's probably got a ruff home-life... I kinda want to try again and see what happens. Still not sure if I really care for him that way, though. There's a catch next time though (if there's one): my sister refused to be the one to give us a ride. My mom will have to be the one, and I know she's going to hate him (my sister was the easy one to please, and he failed at that). What should I do?