Running different python environments within python script in linux

63 Views Asked by At

For my Uni thesis project, I need firstly to setup automatically multiple github repositories within python script. Problems I am facing is that:

  • each repository has its own requirements and some of them conflict with other repository's requirements. The most prominent is 'diffusers', for one method I am using huggingface's diffusers for other method I am using ShivamShrirao's one. but when importing, it it is only import diffusers, so I think when I 'pip install' one of them it is overwritting the other one.
  • I need to call functions from these repositories inside the code, so I think I need to add the cloned repositories to PATH.

so for that I created a folder structure as follows:

main
├── method_1_folder
│   ├── method_1_github_repository_cloned_folder
│   └── venv
├── method_2_folder
│   ├── method_2_github_repository_cloned_folder
│   └── venv
└── running_python_file.py

and tried to create a separate environment for each method so I would call a prep_work function that would clone the github, create an environment, activate it, then add its path to python path then install all requirements. makes sense right? but when I print the path of the active environment after all the actions above, I find out that I am still working on the running_python_file.py environment. would you please help to correct whatever I am doing wrong?

def prep_work(filename, repo_file_name, repo_url, characters, trainingstep=None):
    environment_not_created = True

    # cloning respository
    path = os.path.join(os.getcwd(), filename, repo_file_name)
    if not os.path.exists(path):
        Repo.clone_from(repo_url, path)
        subprocess.run(["python", "-m", "venv", f"{os.path.join(os.getcwd(), filename, 'venv')}"], check=True)
        environment_not_created = False

    os.chdir(os.path.join(os.getcwd(), filename))
    os.system(f'source {os.path.join(os.getcwd(), "venv", "bin", "activate")}')
    sys.path.append(path)
    print("Active environment:", sys.prefix)
0

There are 0 best solutions below