Was triggering an error if ivy and smex layer were both used at the same time and smex layer stole ownership of smex package. Fixes #6281