QBoard » Advanced Visualizations » Viz - Python » Python's subprocess does not interpret “~” as expected on cygwin
User Dashboard

Python's subprocess does not interpret “~” as expected on cygwin

Back To Topics
Tags : None
  • 346 3
    If shell is True, the specified command will be executed through the shell. This can be useful if you are using Python primarily for the enhanced control flow it offers over most system shells and still want convenient access to other shell features such as shell pipes, filename wildcards, environment variable expansion, and expansion of ~ to a user’s home directory.

    However, on cygwin, output from bash isn't the same as from Python's subprocess, viz:

    Bash:

    $ ls -ls ~rbarakx
total 0
0 drwxr-xr-x 1 Administrator None 0 Aug 21 17:54 bash
0 drwxr-xr-x 1 Administrator None 0 Jul 11 09:11 python​

    Python:

    >>> subprocess.call(["ls","-ls","~rbarakx"],shell=True)
RCS  mecha.py  print_unicode.py  req.py  requests_results.html  selen.py
0

    It looks as if subprocess.call is executing just ls.

    Can you suggest why?


    My Environment:
    Python: Python 2.7.3 (default, Dec 18 2012, 13:50:09) [GCC 4.5.3] on cygwin
    cygwin: CYGWIN_NT-6.1-WOW64 ... 1.7.22(0.268/5/3) ... i686 Cygwin
    windows: Windows 7 Ultimate

      August 21, 2021 12:11 PM IST
    0
  • 461
    On Windows, the shell (cmd.exe) doesn't support the expansion of ~ to a user's home directory. Nor wildcard expansion, for that matter. The Python doc is very UNIX/Linux-oriented on this point.

    "But wait!" I hear you cry. "I've installed cygwin, I have bash!" Sure you have, but you can hardly expect Python to know that. You're on Windows, so it's going to use the Windows shell. If it didn't, Python scripts that expected the shell to be cmd.exe would be mighty confused.
      September 6, 2021 1:23 PM IST
    0
  • 317

    Because on Unix (cygwin) your call is equivalent to:

    # WRONG: DON'T DO IT
rc = subprocess.call(["/bin/sh", "-c"] + ["ls","-ls","~rbarakx"])


    In other words the argument list is passed to the shell itself i.e., ls doesn't see -ls or ~rbarakx they are considerred /bin/sh arguments.

    What you want is:

    rc = subprocess.call(["/bin/sh", "-c"] + ["ls -ls ~rbarakx"])


    Or using shell=True:

    rc = subprocess.call("ls -ls ~rbarakx", shell=True)

    Here's the quote from subprocess docs for reference:

    On Unix with shell=True, the shell defaults to /bin/sh. If args is a string, the string specifies the command to execute through the shell. This means that the string must be formatted exactly as it would be when typed at the shell prompt. This includes, for example, quoting or backslash escaping filenames with spaces in them. If args is a sequence, the first item specifies the command string, and any additional items will be treated as additional arguments to the shell itself. That is to say, Popen does the equivalent of:

    Popen(['/bin/sh', '-c', args[0], args[1], ...])

    You could also execute the command without any shell:

    import os
import subprocess

rc = subprocess.call(["ls", "-ls", os.path.expanduser("~rbarakx")])



      November 17, 2021 1:01 PM IST
    0
  • 259
    Interestingly when passing the command as a string instead of a tuple, it seems to work as if you were executing it in bash.

    subprocess.call(["ls -ls ~"], shell=True)
​

    If you insist on using tuple, consider the following workaround:

    cmd = subprocess.list2cmdline(["ls","-ls","~rbarakx"])
subprocess.call(cmd, shell=True)


    or this:

    from os.path import expanduser
subprocess.call(["ls","-ls", expanduser("~rbarakx")])
      August 28, 2021 1:49 PM IST
    0