Investigate your dependencies with Deptective

Have you ever tried compiling a piece of open-source software, only to discover that you neglected to install one of its native dependencies? Or maybe a binary “fell off the back of a truck” and you want to try running it but have no idea what shared libraries it needs. Or maybe you need to use a poorly packaged piece of software whose maintainers neglected to list a native dependency.

Deptective, our new open-source tool, solves these problems. You can give it any program, script, or command, and it will find a set of packages sufficient to run the software successfully.

Here’s Deptective automatically finding all of jq’s build-time dependencies:

Wait, isn’t there already a tool for that?

There are many existing tools that automatically find software dependencies. For example, Trail of Bits created and maintains it-depends, which uses package specifications to enumerate dependencies and their vulnerabilities. But that’s not the problem Deptective is intended to solve. Deptective detects dependencies not based upon the software’s self-reported requirements, but instead by observing what the software needs at runtime.

Deptective can work on any Linux process: native binaries, shell scripts, or even build systems. For example, simply run deptective ./configure or deptective cmake .. in an open-source repo, and it will automatically determine the native packages necessary to install to get the software to build!

Deptective is spiritually similar to nix-autobahn, but Deptective is not tied to Nix and can also enumerate arbitrary runtime dependencies.

How does it work?

There are more details below, but in short, Deptective traces the software to record files that the software tried to read but are missing from the environment; finds a package that provides the missing files; installs the package; then repeats the process for further missing packages, backtracking as necessary.

Tracing

At their core, packages are groups of files; installing a package puts its constituent files onto your local system. Programs attempt to access their dependencies’ files, failing when they don’t exist. Deptective runs the target program while tracing its system calls using strace. Deptective analyzes the resulting system call trace to record all failed file accesses. If the program fails to execute (i.e., returns a nonzero exit code), Deptective proceeds to find the packages that contain the missing files.

Finding the right packages to install

Once we know the missing files the program failed to load, how do we determine the packages that provide them? Luckily, most Linux distributions provide an index that maps files to their corresponding packages. Deptective searches the selected distribution’s index to find packages that contain the desired files. Once it selects a candidate package, it creates a new container snapshot, installs the package, and re-traces the target program in the environment with the package installed. We employ a simple heuristic to determine if the installed package was correct: if the trace is identical to the previous trace, the package is irrelevant and can be removed from consideration. If the presence of the new package produces a unique trace from the target program, the package is relevant. Deptective proceeds to install candidate packages until either there are no more to try or the software completes with exit code zero.

Installing potential dependencies

Sometimes there are multiple packages in the index that can satisfy a dependency. In that case, Deptective tries every candidate until it finds one that produces a distinct program trace. It traces the program in a Docker container that matches the system’s distribution and version. Deptective installs each candidate in a separate container and deletes the ones that don’t pass our heuristic. Once Deptective determines that a package is relevant, it snapshots the Docker container, using it as a base for future installations. Using Docker provides a “clean” starting environment and does not pollute the host operating system’s packages. It also means that Deptective can run not only on Linux, but also macOS and Windows.

Try it out

As with all of our open-source tools, you can find Deptective on our GitHub. Follow the instructions written in the README to get it up and running.

Deptective is just one of many custom tools that Trail of Bits has developed to gain insight into software supply chains. Please drop us a line if this interests you or your organization!