It turns out these are caused by the new Module::Metadata release. It is leaving some build relics behind that 'git status' doesn't show due to the commit just after this one in blead. If you remove those directories, the test passes. That means that in a smoke, the first one out of the box will pass, and all others will fail. Karen Etheridge is looking at it, but in the meantime, you can get things to pass by reverting 94945d95701937a050ccc231eafa0087c9a74ee3, doing a git status, and removing the directories under Module-MetadataThread Next