This issue has been driving me nuts for hours.
I am running PHP on IIS, I had the wincache module installed, when running a recursive delete a certain folder would get "stuck" and throw permissions errors. I was not able to delete them with PHP or in windows itself. The only way to delete the folder was to wait 5 min and run the script again, or stop the IIS server and the folder would delete on its own. Disabling the wincachce module resolved the issue.
Hope this helps.