But if you find you don't like their site after a week or two, then simply unfollow them.
Sometimes it is a bit tricky to work out which blog your new follower is from. The Google names they follow on are often not blog names. The first thing to do is to open your dashboard and click on followers. Then click on the new follower's picture - this will take you to their Blogger profile. Often under Links, their blog will be listed. If you have tried to figure out which blog it is and failed, obviously you have done your best.
If you are going to follow a blog and want them to follow back, I suggest you put in a comment saying you are now a follower and include your blog's name. This makes it easier for them to follow back. And there is absolutely nothing wrong with asking for a follow back.
I guess you all know how to unfollow blogs using the Blogger Old Interface. I have actually found that an easier way is to go to the blog, sign in as a member using your Google password and then Google will put your icon up at the top and you can click Options and then Site Settings and then Stop Following This Site - it will then ask you to confirm again that you really mean it. Click and you are no longer following that blog.