When adding a local Swift package in Xcode, if we added the package using the Add Local button then we cannot specify package version. To add specific package version enter the full path to the local package in the search box. This will show the package where we can select the version. Make sure to tag version for the git repo.

Local Swift package with version