I try NOT to... but sometimes you have no choice as I have to fish when I can - not when it's most convenient with regards to wind. I have installed one of those "anchor pully locks" on all my boats.... makes dropping and lifting the anchor much easier - most notably if you fish alone.
Only issue is when you pull it up and leave it "hanging" you can bang it against the boat as you row. I do two things... one if I'm making a big move - pull it up and into the boat. But if just re- positioning or moving a few hundred yards--- let it hang down a little just out of the water - on a V-hull there will be a good distance here and you'd need to really rock the boat to bang it. On a jon - this is a little harder - but doable.
Other notable thing - without using a big heavy anchor - you will get pushed around even with one..... as I try to use lighter anchors or even concrete "anchors" I've made/poured so I can just leave them at each boat without care/fear if they disappear.