I thought that my castle was set up pretty good, didn't think that I would have too much to worry about. Turns out that wasn't really the case, after spending a while digging down in my mine, I had that Creeper hanging out right outside my door. Seriously that's the sort of thing that can really ruin your day, had I gone out there.
I guess that I need to make a large wall to surround my entire estate, that way I won't have to worry about it again. I knew the water out front was a weak spot, but I kind of liked how it looked, oh well.