Description
Summary
I had good luck with the aws_ssm plugin until attempting to use it against the latest Amazon Linux AMI. Simple commands that work well with a CentOS 7 host fails when trying to run them against AMI. It appears to be something to do with interpreting shell output.
A 'raw' command like this works fine:
ansible -i inventory.aws_ec2.yaml -m 'raw' -a 'whoami' tag_role_FAKEMX
However when running the equivalent 'command' module it fails for Amazon Linux, but works on CentOS 7.
❯ ansible -i inventory.aws_ec2.yaml -m 'command' -a 'whoami' tag_role_FAKEMX -l ec2-13-58-203-89.us-east-2.compute.amazonaws.com
[WARNING]: Unhandled error in Python interpreter discovery for host ec2-13-58-203-89.us-east-2.compute.amazonaws.com: unexpected output from Python interpreter discovery
ec2-13-58-203-89.us-east-2.compute.amazonaws.com | FAILED | rc=-1 >>
failed to transfer file to /Users/username/.ansible/tmp/ansible-local-78807y4966nip/tmpqn_vbdbk /AnsiballZ_command.py:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0Warning: Failed to open the file 2004h2004l/AnsiballZ_command.py: No such file
Warning: or directory
1 129k 1 1531 0 0 19698 0 0:00:06 --:--:-- 0:00:06 19883
curl: (23) Failure writing output to destination
Versus CentOS 7:
❯ ansible -i inventory.aws_ec2.yaml -m 'command' -a 'whoami' tag_role_FAKEMX -l ip-10-240-172-59.us-east-2.compute.internal
[WARNING]: Platform linux on host ip-10-240-172-59.us-east-2.compute.internal is using the discovered Python interpreter at /usr/libexec/platform-python, but future installation of another Python interpreter could change the meaning of that path. See https://docs.ansible.com/ansible-
core/2.14/reference_appendices/interpreter_discovery.html for more information.
ip-10-240-172-59.us-east-2.compute.internal | CHANGED | rc=0 >>
root
From '-vvvv' output I see things like this:
<i-0cc859c89f4aaf5f4> ssm_retry: (success) (0, '\x1b[?2004h\x1b[?2004l\r\r\r\nPLATFORM\r\r\nLinux\r\r\nFOUND\r\r\n/usr/bin/python3.9\r\r\n/usr/bin/python3\r\r\nENDFOUND\r\r\n\x1b[?2004h\x1b[?2004l\r\r\r', '')
[WARNING]: Unhandled error in Python interpreter discovery for host ec2-13-58-203-89.us-east-2.compute.amazonaws.com: unexpected output from Python interpreter discovery
Using module file /Users/username/.asdf/installs/python/3.11.2/lib/python3.11/site-packages/ansible/modules/command.py
That's a failure to find the python version. Versus:
<i-03c9cbe64572b3eb0> ssm_retry: (success) (0, 'PLATFORM\r\r\nLinux\r\r\nFOUND\r\r\n/usr/libexec/platform-python\r\r\n/usr/bin/python2.7\r\r\n/usr/bin/python\r\r\n/usr/bin/python\r\r\nENDFOUND\r\r', '')
<ip-10-240-172-59.us-east-2.compute.internal> Python interpreter discovery fallback (pipelining support required for extended interpreter discovery)`
Where it didn't complain and seems to be successful. Notice the additional "x1b[?2004h\x1b[?2004" strings in the output.
I see the similar string in other problems, like "Warning: Failed to open the file 2004h2004l/AnsiballZ_command.py"
Note that if I set the python interpreter it will remove the warning, but it will still error out on the "Failed to open the file 2004h2004l/AnsiballZ_command.py: No such file Warning: or directory"
Tried this with community.aws collection versions 4.5.0, 5.2.0, and 5.3.0 and the errors is the same every time. Also tried a few different amazon.aws collection versions and had the same error.
These examples are taken from a Mac running python 3.11.2 and ansible 2.14.2, but the same errors occured in a Linux-based Execution Environment for AWX running in EKS.
Issue Type
Bug Report
Component Name
aws_ssm connection
Ansible Version
ansible [core 2.14.2]
config file = None
configured module search path = ['/Users/username/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules']
ansible python module location = /Users/username/.asdf/installs/python/3.11.2/lib/python3.11/site-packages/ansible
ansible collection location = /Users/username/.ansible/collections:/usr/share/ansible/collections
executable location = /Users/username/.asdf/installs/python/3.11.2/bin/ansible
python version = 3.11.2 (main, Feb 21 2023, 11:07:56) [Clang 13.1.6 (clang-1316.0.21.2.5)] (/Users/username/.asdf/installs/python/3.11.2/bin/python3.11)
jinja version = 3.1.2
libyaml = True
Collection Versions
Collection Version
----------------------------- -------
ansible.netcommon 4.1.0
ansible.posix 1.5.1
ansible.utils 2.9.0
ansible.windows 1.13.0
arista.eos 6.0.0
awx.awx 21.11.0
azure.azcollection 1.14.0
check_point.mgmt 4.0.0
chocolatey.chocolatey 1.4.0
cisco.aci 2.3.0
cisco.asa 4.0.0
cisco.dnac 6.6.3
cisco.intersight 1.0.23
cisco.ios 4.3.1
cisco.iosxr 4.1.0
cisco.ise 2.5.12
cisco.meraki 2.15.0
cisco.mso 2.2.1
cisco.nso 1.0.3
cisco.nxos 4.0.1
cisco.ucs 1.8.0
cloud.common 2.1.2
cloudscale_ch.cloud 2.2.4
community.azure 2.0.0
community.ciscosmb 1.0.5
community.crypto 2.10.0
community.digitalocean 1.23.0
community.dns 2.5.0
community.docker 3.4.0
community.fortios 1.0.0
community.general 6.3.0
community.google 1.0.0
community.grafana 1.5.3
community.hashi_vault 4.1.0
community.hrobot 1.7.0
community.libvirt 1.2.0
community.mongodb 1.4.2
community.mysql 3.5.1
community.network 5.0.0
community.okd 2.2.0
community.postgresql 2.3.2
community.proxysql 1.5.1
community.rabbitmq 1.2.3
community.routeros 2.7.0
community.sap 1.0.0
community.sap_libs 1.4.0
community.skydive 1.0.0
community.sops 1.6.0
community.vmware 3.3.0
community.windows 1.12.0
community.zabbix 1.9.1
containers.podman 1.10.1
cyberark.conjur 1.2.0
cyberark.pas 1.0.17
dellemc.enterprise_sonic 2.0.0
dellemc.openmanage 6.3.0
dellemc.os10 1.1.1
dellemc.os6 1.0.7
dellemc.os9 1.0.4
dellemc.powerflex 1.5.0
dellemc.unity 1.5.0
f5networks.f5_modules 1.22.0
fortinet.fortimanager 2.1.7
fortinet.fortios 2.2.2
frr.frr 2.0.0
gluster.gluster 1.0.2
google.cloud 1.1.2
grafana.grafana 1.1.0
hetzner.hcloud 1.9.1
hpe.nimble 1.1.4
ibm.qradar 2.1.0
ibm.spectrum_virtualize 1.11.0
infinidat.infinibox 1.3.12
infoblox.nios_modules 1.4.1
inspur.ispim 1.2.0
inspur.sm 2.3.0
junipernetworks.junos 4.1.0
kubernetes.core 2.3.2
lowlydba.sqlserver 1.3.1
mellanox.onyx 1.0.0
netapp.aws 21.7.0
netapp.azure 21.10.0
netapp.cloudmanager 21.22.0
netapp.elementsw 21.7.0
netapp.ontap 22.2.0
netapp.storagegrid 21.11.1
netapp.um_info 21.8.0
netapp_eseries.santricity 1.4.0
netbox.netbox 3.10.0
ngine_io.cloudstack 2.3.0
ngine_io.exoscale 1.0.0
ngine_io.vultr 1.1.3
openstack.cloud 1.10.0
openvswitch.openvswitch 2.1.0
ovirt.ovirt 2.4.1
purestorage.flasharray 1.16.2
purestorage.flashblade 1.10.0
purestorage.fusion 1.3.0
sensu.sensu_go 1.13.2
splunk.es 2.1.0
t_systems_mms.icinga_director 1.32.0
theforeman.foreman 3.8.0
vmware.vmware_rest 2.2.0
vultr.cloud 1.7.0
vyos.vyos 4.0.0
wti.remote 1.0.4
# /Users/username/.ansible/collections/ansible_collections
Collection Version
------------- -------
amazon.aws 4.3.0
community.aws 5.2.0
AWS SDK versions
❯ pip show boto boto3 botocore
WARNING: Package(s) not found: boto
Name: boto3
Version: 1.26.76
Summary: The AWS SDK for Python
Home-page: https://github.com/boto/boto3
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /Users/username/.asdf/installs/python/3.11.2/lib/python3.11/site-packages
Requires: botocore, jmespath, s3transfer
Required-by:
---
Name: botocore
Version: 1.29.76
Summary: Low-level, data-driven core of boto 3.
Home-page: https://github.com/boto/botocore
Author: Amazon Web Services
Author-email:
License: Apache License 2.0
Location: /Users/username/.asdf/installs/python/3.11.2/lib/python3.11/site-packages
Requires: jmespath, python-dateutil, urllib3
Required-by: boto3, s3transfer
Configuration
❯ ansible-config dump --only-changed
CONFIG_FILE() = None
OS / Environment
Darwin ENG-NMOSEMAN-MB 22.3.0 Darwin Kernel Version 22.3.0: Mon Jan 30 20:42:11 PST 2023; root:xnu-8792.81.3~2/RELEASE_X86_64 x86_64 i386 Darwin
and
modified version of quay.io/ansible/awx-ee:latest to include community.aws, and ssm stuff.
Steps to Reproduce
❯ ansible -i inventory.aws_ec2.yaml -m 'command' -a 'whoami' tag_role_FAKEMX -l ec2-13-58-203-89.us-east-2.compute.amazonaws.com
[WARNING]: Unhandled error in Python interpreter discovery for host ec2-13-58-203-89.us-east-2.compute.amazonaws.com: unexpected output from Python interpreter discovery
ec2-13-58-203-89.us-east-2.compute.amazonaws.com | FAILED | rc=-1 >>
failed to transfer file to /Users/username/.ansible/tmp/ansible-local-78807y4966nip/tmpqn_vbdbk /AnsiballZ_command.py:
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0Warning: Failed to open the file 2004h2004l/AnsiballZ_command.py: No such file
Warning: or directory
1 129k 1 1531 0 0 19698 0 0:00:06 --:--:-- 0:00:06 19883
curl: (23) Failure writing output to destination
Expected Results
❯ ansible -i inventory.aws_ec2.yaml -m 'command' -a 'whoami' tag_role_FAKEMX -l ip-10-240-172-59.us-east-2.compute.internal
ip-10-240-172-59.us-east-2.compute.internal | CHANGED | rc=0 >>
root
Actual Results
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0Warning: Failed to open the file 2004h2004l/AnsiballZ_command.py: No such file
Warning: or directory
1 129k 1 1531 0 0 19698 0 0:00:06 --:--:-- 0:00:06 19883
curl: (23) Failure writing output to destination
Code of Conduct
- I agree to follow the Ansible Code of Conduct