Python tooling is fragile

One thing I run into way too often with Python tooling is stuff like the following:

Traceback (most recent call last):                                                                                     
  File "/opt/google-cloud-sdk/bin/bootstrapping/gsutil.py", line 16, in <module>                                       
    import bootstrapping                                                                                               
  File "/opt/google-cloud-sdk/bin/bootstrapping/bootstrapping.py", line 32, in <module>                                
    import setup  # pylint:disable=g-import-not-at-top                                                                 
    ^^^^^^^^^^^^                                                                                                       
  File "/opt/google-cloud-sdk/bin/bootstrapping/setup.py", line 57, in <module>                                        
    from googlecloudsdk.core.util import platforms                                                                     
  File "/opt/google-cloud-sdk/lib/googlecloudsdk/__init__.py", line 23, in <module>                                    
    from googlecloudsdk.core.util import importing                                                                     
  File "/opt/google-cloud-sdk/lib/googlecloudsdk/core/util/importing.py", line 23, in <module>                         
    import imp                                                                                                         
ModuleNotFoundError: No module named 'imp'      

If something changes on your host system (new Python version, wrong mix of package manager and pip modules, etc), things break. Python virtual environments help some, but it is a pain to remember what all virtual envs you have installed for various projects.

Perhaps a better solution is statically linked tooling in Go or Rust …

package management is hard, and entrusting the end-user to be able to handle it more often results in such errors and bad taste, one can argue that its empowering the end user by providing the choice what to install but it does have these kind of fallouts which is little fire everywhere and wears you out. So I think yes simpler setups always get adopted even though they might do such encapsulations with somewhat complexity, its worth it at times.

1 Like

The error “ModuleNotFoundError: No module named ‘imp’” typically occurs in Python 3.4 and later versions because the imp module has been deprecated and replaced by importlib.

Updated google-cloud-sdk to latest, and how getting:

/opt/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/__init__.py:1105: SyntaxWarning: invalid escape sequence '\c'
  """
/opt/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/pyami/config.py:98: SyntaxWarning: invalid escape sequence '\s'
  match = re.match("^#import[\s\t]*([^\s^\t]*)[\s\t]*$", line)
/opt/google-cloud-sdk/platform/gsutil/third_party/pyasn1/pyasn1/type/constraint.py:100: SyntaxWarning: invalid escape sequence '\*'
  """Create a SingleValueConstraint object.
/opt/google-cloud-sdk/platform/gsutil/third_party/pyasn1/pyasn1/type/constraint.py:141: SyntaxWarning: invalid escape sequence '\*'
  """Create a ContainedSubtypeConstraint object.
/opt/google-cloud-sdk/platform/gsutil/third_party/pyasn1/pyasn1/type/constraint.py:301: SyntaxWarning: invalid escape sequence '\*'
  """Create a PermittedAlphabetConstraint object.
/opt/google-cloud-sdk/platform/gsutil/third_party/pyasn1/pyasn1/type/constraint.py:457: SyntaxWarning: invalid escape sequence '\*'
  """Create a ConstraintsIntersection logic operator object.
/opt/google-cloud-sdk/platform/gsutil/third_party/pyasn1/pyasn1/type/constraint.py:501: SyntaxWarning: invalid escape sequence '\*'
  """Create a ConstraintsUnion logic operator object.
/opt/google-cloud-sdk/platform/gsutil/third_party/pyasn1/pyasn1/type/namedval.py:15: SyntaxWarning: invalid escape sequence '\*'
  """Create named values object.
/opt/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/connection.py:672: SyntaxWarning: invalidescape sequence '\w'
  '(?:(?P<user>[\w\-\.]+):(?P<pass>.*)@)?'
/opt/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/connection.py:673: SyntaxWarning: invalidescape sequence '\w'
  '(?P<host>[\w\-\.]+)'
/opt/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/connection.py:674: SyntaxWarning: invalidescape sequence '\d'
  '(?::(?P<port>\d+))?'
/opt/google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/connection.py:1121: SyntaxWarning: invalid escape sequence '\.'
  'specific endpoint this request was sent to\.'

but it still seems to work …

Tried fixing the issue by:

gcloud components update

But, it said all components are up to date.

Perhaps Arch Python is too new?

In other projects that need GCP, I have switched to Go based tooling – time to do that here …

This must be my week for Python pain, Ansible is now complaining:

This server has Python3 3.6.9, and a working server has Python 3.9.2. Perplexity suggests:

Ansible Core 2.12 and later versions require Python 3.8 or newer on the control node (the machine where Ansible is run) .

Python 3.6 reached its end-of-life on December 23, 2021, and is no longer supported by the Python core team. Consequently, newer versions of Ansible do not support Python 3.6

So, I really need to keep my servers updated better … :frowning_face:

Installed Python 3.8:

sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt update
sudo apt install python3.8

And then added the following line in the Ansible playbook:

  vars:
    python_interpreter: /usr/bin/python3.8

Voila!

I’ve nearly the same invalidescape sequence warning in our google/cloud-sdk:alpine docker container:

gsutil-cp /google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/__init__.py:1105: SyntaxWarning: invalid escape sequence '\c'
gsutil-cp /google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/pyami/config.py:98: SyntaxWarning: invalid escape sequence '\s'
gsutil-cp match = re.match("^#import[\s\t]*([^\s^\t]*)[\s\t]*$", line)
gsutil-cp /google-cloud-sdk/platform/gsutil/third_party/pyasn1/pyasn1/type/constraint.py:100: SyntaxWarning: invalid escape sequence '\*'
gsutil-cp """Create a SingleValueConstraint object.
gsutil-cp /google-cloud-sdk/platform/gsutil/third_party/pyasn1/pyasn1/type/constraint.py:141: SyntaxWarning: invalid escape sequence '\*'
gsutil-cp """Create a ContainedSubtypeConstraint object.
gsutil-cp /google-cloud-sdk/platform/gsutil/third_party/pyasn1/pyasn1/type/constraint.py:301: SyntaxWarning: invalid escape sequence '\*'
gsutil-cp """Create a PermittedAlphabetConstraint object.
gsutil-cp /google-cloud-sdk/platform/gsutil/third_party/pyasn1/pyasn1/type/constraint.py:457: SyntaxWarning: invalid escape sequence '\*'
gsutil-cp """Create a ConstraintsIntersection logic operator object.
gsutil-cp /google-cloud-sdk/platform/gsutil/third_party/pyasn1/pyasn1/type/constraint.py:501: SyntaxWarning: invalid escape sequence '\*'
gsutil-cp """Create a ConstraintsUnion logic operator object.
gsutil-cp /google-cloud-sdk/platform/gsutil/third_party/pyasn1/pyasn1/type/namedval.py:15: SyntaxWarning: invalid escape sequence '\*'
gsutil-cp """Create named values object.
gsutil-cp /google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/connection.py:672: SyntaxWarning: invalid escape sequence '\w'
gsutil-cp '(?:(?P<user>[\w\-\.]+):(?P<pass>.*)@)?'
gsutil-cp /google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/connection.py:673: SyntaxWarning: invalid escape sequence '\w'
gsutil-cp '(?P<host>[\w\-\.]+)'
gsutil-cp /google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/connection.py:674: SyntaxWarning: invalid escape sequence '\d'
gsutil-cp  '(?::(?P<port>\d+))?'
gsutil-cp /google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/connection.py:1121: SyntaxWarning: invalid escape sequence '\.'
gsutil-cp  'specific endpoint this request was sent to\.'
gsutil-cp /google-cloud-sdk/platform/gsutil/gslib/vendored/boto/boto/https_connection.py:80: SyntaxWarning: invalid escape sequence '\.'
gsutil-cp  host_re = host.replace('.', '\.').replace('*', '[^.]*')
gsutil-cp Copying file:///tmp/backup/XXXXXXXX [Content-Type=application/octet-stream]...
gsutil-cp
/ [0/1 files][    0.0 B/ 22.7 KiB]   0% Done                                    
/ [1/1 files][ 22.7 KiB/ 22.7 KiB] 100% Done                                    
gsutil-cp
Operation completed over 1 objects/22.7 KiB

The GCP-SDK is an actual one:

Google Cloud SDK 486.0.0 
bq 2.1.7 
bundled-python3-unix 3.11.8   
core 2024.07.26
gcloud-crc32c 1.0.0
gsutil 5.30