Can you take a picture of what you are talking about and post it? I'm guessing it may be something other than caulk (maybe grout) but without a picture, it's hard to tell. You can post a picture by using imgur.com and adding a link to your post/comment.