General use, portraits, sports, landscapes, etc

I'm a beginning photographer and I'm having a hard time figuring out which camera to get.  What I really would like doesn't seem to exist,  a small(like a point and shoot size)  but a dslr that i can take with me everywhere.  The Nikon 1 is the perfect size, but i don't like the way it auto focuses over the manual focus.  Mostly I've been looking at the canon rebel t3 and the nikon d3100.  They both have their pros and cons, and it's making it more confusing on which one to get.  It's a big investment for me so I'm looking for the best camera for the money.  If anyone has any recommendations, even if it's not a t3 or a d3100 it would be appreciated immensely!