There is no fear in love, but perfect love drives out fear, because fear involves punishment. The one who fears has not been perfected in love.
It's possible to want to get closer to God while a quiet part of you braces for what you might find there. The picture in the back of your mind may be of someone stern who keeps his distance and seems pleased when you flinch, because flinching shows you take him seriously.