Videos from Ethereum-focused Empire Hacking

On December 12, over 150 attendees learned how to write and hack secure smart contracts at the final Empire Hacking meetup of 2017. Thank you to everyone who came, to our superb speakers, and to Datadog for hosting this meetup at their office.

Watch the presentations again

We believe strongly that the community should share what knowledge it can. That’s why we’re posting these recordings from the event. We hope you find them useful.

A brief history of smart contract security

Jon Maurelian of Consensys Diligence reviewed the past, present, and future of Ethereum with an eye for security at each stage.


  • Ethereum was envisioned as a distributed shared computer for the world. High level languages such as Solidity enable developers to write smart contracts.
  • This shared computer where anyone can execute code comes with a number of inherent security issues. Delegate calls, reentrancy, and other idiosyncrasies of Ethereum have been exploited on the public chain for spectacular thefts.
  • Among the most exciting upcoming developments include safer languages like Viper, the promise of on-chain privacy with zk-SNARKs, and security tooling like Manticore and KEVM.

A CTF Field Guide for smart contracts

Sophia D’Antoine of Trail of Bits discussed recent Capture the Flag (CTF) competitions that featured Solidity and Ethereum challenges, and the tools required to exploit them.


  • CTFs have started to include Ethereum challenges. If you want to set up your own Ethereum CTF, reference Sophia’s scripts from CSAW 2017.
  • Become familiar projects from Trail of Bits, like Manticore, Ethersplay, and Not So Smart Contracts to learn about Ethereum security and compete in CTFs.
  • Integer overflows and reentrancy are common flaws to include in challenges. Review how to discover and exploit these flaws in write-ups from past competitions.

Automatic bug finding for the blockchain

Mark Mossberg of Trail of Bits explained practical symbolic execution of EVM bytecode with Manticore.


  • Symbolic execution is a program analysis technique that can achieve high code coverage, and has been used to create effective automated bug finding systems.
  • When applied to Ethereum, symbolic execution can automatically discover functions in a contract, generate transactions to trigger contract states, and check for failure states.
  • Manticore, an open source program analysis tool, uses symbolic execution to analyze EVM smart contracts.

Addressing infosec needs with blockchain technology

Paul Makowski introduced PolySwarm, an upcoming cybersecurity-focused Ethereum token, and explained how it aligns incentives and addresses deficiencies in the threat intelligence industry.


  • The economics of today’s threat intelligence market produce solutions with largely overlapping detection capabilities which result in limited coverage and expose enterprises to innovative threats.
  • Ethereum smart contracts provide a distributed platform for intelligent, programmed market design. They fix the incentives in the threat intelligence space without becoming a middleman.
  • PolySwarm unlocks latent security expertise by removing barriers to participate in tomorrow’s threat-intelligence community. PolySwarm directs this expertise toward the greater good, getting more security experts to create a better collective defense for all.

Learn more about Empire Hacking

Let’s secure your smart contracts

We’ve become one of the industry’s most trusted providers of audits, tools, and best practices for securing smart contracts and their adjacent technologies. We’ve secured token launches, decentralized apps, and entire blockchain platforms.

Contact us for help.

What are the current pain points of osquery?

You’re reading the second post in our four-part series about osquery. Read post number one for a snapshot of the tool’s current use, the reasons for its growing popularity among enterprise security teams, and how it stacks up against commercial alternatives.

osquery shows considerable potential to revolutionize the endpoint monitoring market. (For example, it greatly simplifies the detection of signed malware with Windows executable code signature verification.) However, the teams at five major tech firms and osquery developers whom we interviewed for this series say that the open source tool has room for improvement.

Some of these qualms relate to true limitations. However, we’ve also heard some recent grumbling among prospective users and industry competitors that doesn’t align with osquery’s actual shortcomings.

As with many rapidly improving open source projects, documentation updates lagging behind a blistering development schedule may be to blame for these misconceptions. So, before diving into what true pain points the osquery community should tackle next, let’s shed light on some current myths about osquery’s limitations.

Mythical limitations of osquery

“osquery has no support for containers”

Oh, but it does! Teams like Uptycs have poured significant development support into this much-requested feature within the past year. Currently, osquery can perform container introspection at the management host layer (more efficient) or it can operate in each container (more granularity) without dominating CPU.

“osquery cannot operate in real time”

osquery handles file integrity and process auditing for MacOS and Linux. It can also monitor user access, hardware events, and socket events in select operating systems. It performs these tasks through interaction with the audit kernel API. Unlike its other pull-based queries, these monitoring services create event-based logs in real time and ensure that osquery doesn’t miss important events between queries. These features are essential to osquery’s power in incident detection.

“osquery is high overhead”

osquery is a lightweight solution when correctly deployed and managed. As we’ll touch on later in the post, the leading causes of performance issues come from misconfiguration: scheduling runaway queries, performing event-based queries on high-traffic file paths, or running osquery in a resource-constrained environment without implementing CPU controls. In fact, respondents who implemented safeguards such as Cgroups were so confident in osquery’s performance that they deployed the tool on every endpoint, including production servers.

Current limitations of osquery; the facts

Demand for user support outstrips supply

For an open-source project, osquery offers a lot of user support; a website, an active slack channel, extensive documentation, and lots of blogs (like this one!). That said, the community can do better.

  • Documentation updates have not matched pace with the growing list of project feature updates, leaving users unaware of new functionality or confused about how to use it.
  • Confused users flocking to the osquery slack channel ask similar questions. Experts try to help these individuals instead of creating FAQs, writing comprehensive query packs, and making tutorials.

Users test feature integrity

Facebook has done an excellent job at thoroughly reviewing new code and hosting productive debates about how best to build new features. However, there has been a lack of oversight into the efficacy of new features. So far, this has been the job of users who report edge cases and unexpected behavior to the slack channel or github repo.

A recent example of this issue: Developers and users saw false negatives in file integrity monitoring for osquery. The audit backend contained multiple bugs that caused inaccurate logs. This has persisted unchecked since FIM was first enabled in 2015. Thankfully, our own Alessandro Gario is implementing a fix.

Issues with extensions

This crucial component of osquery has been causing problems for users. The issues stem from insufficient support during their development. osquery’s current SDK only provides the bare minimum needed to integrate with osquery. The documentation and APIs are also limited. Because of these factors, many extensions aren’t well-built or well-maintained, and therefore, introduce unreliable results.

Fortunately, the community has started to resolve the issues. Kolide’s osquery-go provides a rich SDK for developers to create osquery extensions with Go. Last week, we explained how to write an extension for osquery. We also released a repository of our own well-maintained osquery extensions that users can pull from (there’s only one in there right now but more to come, soon!). We intend to help the community navigate the extension-building process and to create a reliable source of updated extensions.

Limited platform support beyond Linux and macOS

Users are eager for osquery to support more platforms and provide better introspection on all endpoints. osquery’s current limit to just a subset of endpoints leaves holes in users’ monitoring capacity. Further, they noted that some supported platforms lacked important features.

For example, macOS and Linux platforms can collect usable real-time data about a variety of event types. Real-time data in osquery for Windows is limited to the Windows Event Log, which only exposes a stream of difficult-to-parse system data. No users whom we interviewed had successfully implemented or parsed these logs in their deployments.

Screen Shot 2017-12-20 at 5.05.57 PM

From Teddy Reed and Mitchell Grenier’s osquery presentation at the 2017 LISA conference

Readable real-time monitoring underpins osquery’s incident-detection capabilities. While scheduled queries can miss system events between queries, real-time event-based monitoring is less prone to false negatives. The absence of this feature in the Windows port greatly degrades osquery’s incident-detection utility for users running Windows machines in their fleets.

Runaway queries and guidance

Respondents reserved some of their harshest criticism for the lack of safeguards against bad queries, especially those that unexpectedly pulled an excessive amount of data. Sometimes, these runaway queries caused major issues for the larger fleet such as degraded endpoint system performance and clogged memory. In addition, malformed queries could flood data logs and rewrite over other data collected, causing other events to pass by undetected.

osquery’s watchdog feature does prevent some performance issues by killing any processes that consume too much CPU or memory. However, this is done without consideration of what’s running at the time. Well-formed audit-based queries often exceed the default quotas, killing processes unnecessarily. As a result, users turned off the feature to avoid missing essential data. A better solution would understand the scale of a user’s query and ask for confirmation.

Users also wanted smarter queries. One interviewee wanted guidance on the right query intervals for different types of system data. He also wanted to save wasted storage from overlapping data within different query packs. Though the issue is relatively cheap, it would be helpful if osquery could de-duplicate this data.

Insufficient debugging/diagnostics at scale

Users struggled with large-scale deployment of osquery, primarily because of difficulty debugging and diagnosing query issues in their fleets. One company reported that roughly 15% of nodes queried persist as pending for unknown reasons. Another reported that certain endpoints would occasionally “fall off the internet” without any apparent cause. Though users can restart osquery with the verbose setting to print information about every action performed, this option is primarily a tool for developers and is not user-friendly.

Deployment and maintenance issues

Every company implementing osquery tackles this ongoing struggle in a different way. We went into great detail in our previous post about the variety of tools and techniques they used to manage this problem. Despite support and documentation improvements, issues persist. One user reported ongoing troubles implementing osquery version updates on endpoints for which employees are admins.


After reading about all of these pain points, you might be wondering why osquery won multiple product awards this year, and why over 1,100 users have engaged with the development community’s Slack channel and GitHub repo. You might be wondering why the five top tech company teams we surveyed for this series reported that they liked osquery better than commercial fleet management tools

It’s simple. The tool has attracted a vibrant development community invested in its success. Every new development brings osquery closer to feature-parity with the equivalent components of competitors’ fully integrated – and higher-priced – security suites. As users commission companies like Trail of Bits to make those improvements, the entire community benefits.

That will be the topic of the third post in this series: osquery’s development requests. If you use osquery today and have requests you’d like to add to our research, please let us know! We’d love to hear from you.

How does your experience with osquery compare to the pains mentioned in this post? Do you have other complaints or issues that you’d like to see addressed in future releases? Tell us! Help us lead the way in improving osquery’s development and implementation.

Announcing the Trail of Bits osquery extension repository

Today, we are releasing access to our maintained repository of osquery extensions. Our first extension takes advantage of the Duo Labs EFIgy API to determine if the EFI firmware on your Mac fleet is up to date.

There are very few examples of publicly released osquery extensions. Very little documentation exists on the topic. This post aims to help future developers in navigating through the process of writing an extension for osquery. The rest of this post describes how we implemented the EFIgy extension for osquery.

About EFIgy

At this year’s Ekoparty, Duo Labs presented the results of its research on the state of support and security in EFI firmwares. These software components are really interesting for attackers. They operate on a privilege level that is out of reach even from operating systems and hypervisors. Duo Labs gathered and analyzed all the publicly released Apple updates from the last three years and verified the information by looking at more than 73,000 Macs across different organizations.

The researchers found that many of these computers were running on outdated firmware, even though the required EFI updates were supposed to be bundled in the same operating system patches that the hosts had installed correctly. Duo Labs followed this finding by creating the EFIgy service, a REST endpoint that can access the latest OS and EFI versions for any known Apple product through the use of details such as logic board id and product name.

Programmatically querying EFIgy

EFIgy expects a JSON object containing the details for the system we wish to query. The JSON request doesn’t require many keys; it boils down to hardware model and software versions:

  "board_id": "Mac-66E35819EE2D0D05",
  "smc_ver": "2.37f21",
  "build_num": "16G07",
  "rom_ver": "MBP132.0226.B20",
  "hw_ver": "MacBookPro12,1",
  "os_ver": "10.12.4",
  "hashed_uuid": ""

The rom_ver key is kind of tricky. It doesn’t include the full EFI version because (according to the author) it includes timestamps that are not necessarily useful or easy to keep track of. Separated by the dot characters, the only fields you need are the first, third and fourth.

As the name suggests, the hashed_uuid field is a SHA256 digest. To compute it correctly, the MAC address of the primary network interface must be prefixed to the system UUID like this: “0x001122334455” + “12345678-1234-1234-1234-1234567890AB”.

The remaining keys are self-explanatory, but keep in mind that they will not be reported correctly when running from a virtual machine. board_id, hw_ver and rom_ver will report information about the virtual components offered by your hypervisor.

Querying the service is simple. The JSON data is sent via an HTTP POST request to the following REST endpoint:

The server response is made of three JSON objects. Compare them with your original values to understand whether your system is fully up to date or not.

  "latest_efi_version" : {
    "msg" : "<version>"

  "latest_os_version" : {
    "msg" : "<version>"

  "latest_build_number" : {
    "msg" : "<error message>",
    "error" : "1"

Developing osquery extensions

The utilities provided by Duo Labs are easy and straightforward to use, but manually running them all on each system in our fleet was not an easy task. We decided to implement an osquery extension that queries EFIgy, an idea we got from Chris Long.

Why write an extension and not a virtual table? We’ve decided to keep native operating system functions in core and convert everything else into an extension. If a new feature uses an external service or a non-native component, we will default to writing an extension.

The only toolset you have available is the standard library and what you manually import into your project. osquery links everything statically. You have to take special care of the libraries you use. Don’t rely on loading dynamic libraries at runtime from your program.

Once your environment is ready, table extensions do not require much work to be implemented. You just have to inherit from the osquery::TablePlugin class and override the two methods used to define the columns and generate the table rows.

class MyTable final : public osquery::TablePlugin {
    osquery::TableColumns columns() const override {
      return {

    osquery::QueryData generate(osquery::QueryContext& request) override {
      osquery::Row row;
      row[“column_name”] = “value”;

      return { row };

The source files must then be placed in a dedicated folder inside osquery/external. Note that you must add the “extension_” prefix to the folder name. Otherwise, the CMake project will ignore it.

For more complex projects, it is also possible to add a CMakeLists.txt file, creating the target using the following helper function:

ADD_OSQUERY_EXTENSION(${PROJECT_NAME} source1.cpp source2.cpp)

You will have access to some of the libraries in osquery such as Boost, but not some other utilities (e.g. the really useful http_client class).

There is no list of recommended steps to take when developing an extension, but if you plan on writing more than one I recommend you bundle your utility functions in headers that can then be easily imported and reused. Keeping all external libraries statically linked is also a good idea, as it will make redistribution easier.

Using our osquery extensions repo

Without anywhere to submit our new feature, we created a new repository for our extensions. The EFIgy extension is the first item available. Expect more to follow.

Using the repository is simple. However, you will have to clone the full source code of osquery first since the SDK is not part of the distributable package. Building the extension is easy. You only have to create a symbolic link of the source folders you want to compile inside the osquery/external folder, taking care to name the link according to the following scheme: extension_<name>. You can then follow the usual build process for your platform, as the default ALL target will also build all extensions.

cd /src/osquery-extensions
ln -s efigy /src/osquery/external/extension_efigy

cd /src/osquery
make sysprep
make deps

make -j `nproc`
make externals

Extensions are easy to use. You can test them (both with the shell and the daemon) by specifying their path with the –extension parameter. Since they are normal executables, you can also start them after osquery. They will automatically connect via Thrift and expose the new functions. The official documentation explains the process very well.

To quickly test the extension, you can either start it from the osqueryi shell, or launch it manually and wait for it to connect to the running osquery instance.

osqueryi --extension /path/to/extension

Take action

If you have a Mac fleet, you can now monitor it with osquery and the EFIgy extension, and ensure all your endpoints have received the required software and firmware updates.

If you’re reading this post some time in the future, you have even more reason to visit our osquery extension repository. We’ll keep it maintained and add to it over time.

Do you have an idea for an osquery extension? Please file an issue on our Github repo for it. Do you need osquery development? Contact us.

Securing Ethereum at Empire Hacking

If you’re building real applications with blockchain technology and are worried about security, consider this meetup essential. Join us on December 12th for a special edition of Empire Hacking focused entirely on the security of Ethereum.

Why attend?

Four blockchain security experts will be sharing how to write secure smart contracts, and hack them. Two speakers come from our team.

We’ve become one of the industry’s most trusted providers of audits, tools, and best practices for securing smart contracts and their adjacent technologies. We’ve secured token launches, decentralized apps, and entire blockchain platforms. As with past Empire Hacking events, we’re excited to share the way forward with the development community.

Who will be speaking?

  • Sophia D’Antoine of Trail of Bits will discuss Solidity and Ethereum challenges in recent CTF competitions and the tools required to exploit them.
  • John Maurelian of Consensys Diligence will discuss his highlights from Devcon3 about the latest developments in Ethereum security.
  • Mark Mossberg will be sharing how Trail of Bits finds bugs in EVM bytecode with our symbolic execution engine, Manticore.
  • Paul Makowski will share his upcoming security-focused Ethereum token, PolySwarm, which uses blockchain technology to address deficiencies in the threat intelligence industry.
  • Amber Baldet and Brian Schroeder of the Enterprise Ethereum Alliance will discuss the threat modeling, confidential transactions, and zero-knowledge proofs that went into the Quorum blockchain.

When and where?

We’ll be meeting December 12th at 6pm. This month’s meetup will take place at the DataDog offices in the New York Times building. RSVP is required. As per usual, light food and beer will be served.

To find out more about Empire Hacking:

How are teams currently using osquery?

In the year since we ported osquery to Windows, the operating system instrumentation and endpoint monitoring agent has attracted a great deal of attention in the open-source community and beyond. In fact, it recently received the 2017 O’Reilly Defender Award for best project.

Many large and leading tech firms have deployed osquery to do totally customizable and cost-effective endpoint monitoring. Their choice and subsequent satisfaction fuels others’ curiosity about making the switch.

But deploying new software to your company’s entire fleet is not a decision to be made lightly. That’s why we sought to take the pulse of the osquery community – to help current and potential users know what to expect. This marks the start of a four-part blog series that sheds light on the current state of osquery, its shortcomings and opportunities for improvement.

Hopefully, the series will help those of you who are sitting on the fence decide if and how to deploy the platform in your companies.

For our research, we interviewed teams of osquery users at five major tech firms. We asked them:

  • How is osquery deployed and used currently?
  • What benefits are your team seeing?
  • What have been your biggest pain points about using osquery?
  • What new features would you most like to see added?

This post will focus on current use of osquery and its benefits.

How are companies using osquery today?

Market Penetration

osquery’s affordability, flexibility, and cross-platform compatibility has quickly established its place in the endpoint monitoring toolkits of top tech firms. Since its debut in October, 2014, over 1,000 users from more than 70 companies have engaged with the development community through its Slack channel and GitHub repo. In August, osquery developers at Facebook began offering bi-weekly office hours to discuss issues, new features, and design direction.

Users have increased due to a number of recent developments. Since contributors like Trail of Bits and Facebook have transformed osquery to support more operating systems (Windows and FreeBSD), a broader number of organizations are now able to install osquery on a greater portion of their endpoints. Multiple supplementary tools, such as Doorman, Kolide, and Uptycs, have emerged to help users deploy and manage the technology. Monitoring of event-based logs (e.g. process auditing and file integrity monitoring) has further enhanced its utility for incident response. Each of these developments has spurred more organizations with unique data and infrastructure needs to use osquery, sometimes in favor of competing commercial products.

Current Use

All the companies surveyed leveraged osquery for high performance and flexible monitoring of their fleets. Interviewees expressed particular interest in just-in-time incident response including initial malware detection and identifying propagation.

Many teams used osquery in conjunction with other open source and commercial technologies. Some used collection and aggregation services such as Splunk to mine data collected by osquery. One innovative team built incident alerting with osquery by piping log data into ElasticSearch and auto-generated Jira tickets through ElastAlert upon anomaly detection. Most of the companies interviewed expected to phase out some paid services, especially costly suites (e.g. Carbon Black, Tripwire, Red Cloak), in favor of the current osquery build or upon addition of new features.

Deployment Maturity

Deployment maturity for osquery varied widely. One company reported being at the phase of testing and setting up infrastructure. Other companies had osquery deployed on most or all endpoints in their fleets, including one team who reported plans to roll out to 17,500 machines. Three out of the five companies we interviewed had osquery deployed on production servers. However, one of these companies reported having installed osquery on production machines but rarely querying these endpoints due to concerns about osquery’s reliability and scalability. Runaway queries on production fleets was a major concern for all companies interviewed though no production performance incidents were reported.

Strategies for Deployment

Most companies used Chef or Puppet to deploy, configure, and manage osquery installations on their endpoints. One company used the fleet management tool Doorman to maintain their fleet of remote endpoints and bypass the need for separate aggregation tools. Many teams leveraged osquery’s TLS documentation to author their own custom deployment tools that granted them both independence from third party applications and freedom to fully customize features/configurations to their native environments.

Multiple teams took precautions while rolling out osquery by deploying in stages. One team avoided potential performance issues by assigning osquery tasks to CGroups with limits on CPU and memory usage.

osquery Governance

Security teams were responsible for initiating the installation of osquery in the fleet. While most teams did so with buy-in and collaboration from other teams, some executed the installation covertly. One team reported that a performance incident had mildly tarnished the osquery reputation within their organization. Some security teams we interviewed collaborated with other internal teams such as Data Analytics and Machine Learning to mine log data and generate actionable insights.

Benefits of osquery

Teams reported that they liked osquery better than other fleet management tools for a variety of reasons, including:

  • simpler to use,
  • more customizable, and
  • exposed new endpoint data that they had never before had access to.

For teams exploring alternatives to their current tools, the open-source technology helped them avoid the bureaucratic friction of buying new commercial security solutions. For one team, osquery also fit into a growing preference for home-built software within their company.

In its current state, osquery appeared to be most powerful when leveraged as a flexible building block within a suite of tools. Where other endpoint monitoring tools expose users to select log data, osquery provided simple, portable access to a far richer variety of endpoint data. For teams who want to roll their own solutions, or who can’t afford expensive commercial comprehensive suites, osquery was the best option.

How it compares to other endpoint monitoring solutions

Our interviewees mentioned having used or evaluated some alternative endpoint monitoring solutions in addition to osquery. We list the highlights of their comments below. While osquery did present a more flexible, affordable solution overall, some paid commercial solutions still offer distinct advantages, especially in integrating automated prevention and incident response. However, as the development community continues to build features in osquery, the capability gap appears to be closing.


OSSEC is an open source system monitoring and management platform. It features essential incident response tools such as file integrity checking, log monitoring, rootkit detection, and automatic incident response. However, OSSEC lacks osquery’s ability to query multiple hosts (Windows, BSD, etc) with a universal syntax. It’s also not as flexible; users of osquery can quickly form new queries with the usability of SQL syntax, while OSSEC requires cumbersome log file decoders and deliberate ahead-of-time configuration. Both the overall simplicity and the on-going development for community contributed tables have often been cited as advantages osquery has over OSSEC.


SysDig provides a commercial container performance monitoring tool and an open source container troubleshooting tool. While osquery is used for security and malicious incident detection, SysDig tools work with real-time data streams (network or file I/O, or tracking errors in running processes) and are best suited for monitoring performance. However, despite significant recent gains in container support including new Docker tables that allow instrumentation at the host level, SysDig maintains the advantage over osquery for performance-sensitive container introspection. Though osquery is capable of running within containers, our respondents indicated that the current version isn’t yet built to support all deployments cleanly. One user reported avoiding deployment of osquery on their Docker-based production fleet for this reason.

Carbon Black

Carbon Black is one of the industry’s leading malware detection, defense, and response packages. In contrast, osquery by itself only provides detection capabilities. However, when combined with alerting systems such as PagerDuty or ElastAlert, osquery can transform into a powerful incident response tool. Finally, interviewees considering Carbon Black remarked on its high price tag and voiced a desire to minimize its use.

Bromium vSentry

Bromium vSentry provides impact containment and introspection powered by micro-virtualization and supported by comprehensive dashboards. While companies can leverage tools like Kolide and Uptycs to access data visualizations similar to osquery, Bromium’s micro-virtualization isolation functionality to quarantine attacks remains an advantage. However, Bromium’s introspection is significantly less flexible and expansive. It can only access data about targeted isolated applications. osquery can be configured to gather data from a growing number of operating-level logs, events, and processes.

Red Cloak

Red Cloak provides automated threat detection as part of a service offering from Dell SecureWorks. It has two advantages over osquery: first, it provides an expert team to help with analysis and response; second, it aggregates endpoint information from all customers to inform and improve its detection and response. For organizations focused solely on breach response, Red Cloak may be worth its cost. However, for IT teams who want direct access to a variety of endpoint data, osquery is a better and cheaper solution.


osquery fills a need in many corporate security teams; its transparency and flexibility make it a great option for rolling bespoke endpoint monitoring solutions. Without any modification, it exposes all the endpoint data an analysis engine needs. We expect (and hope) to hear from more security teams multiplying osquery’s power with their incident response toolkit.

That will happen faster if teams would share their deployment techniques and lessons learned. Much of the Slack and Github discussions focus on codebase issues. Too few users openly discuss innovative implementation strategies. But that isn’t the only reason holding back osquery’s adoption.

The second post in this series will focus on users’ pain points. If you use osquery today and have pain points you’d like to add to our research, please let us know! We’d love to hear from you.

How does your experience with osquery compare to that of the teams mentioned in this post? Do you have other workarounds, deployment strategies, or features you’d like to see built in future releases? Tell us! Help us lead the way in improving osquery’s development and implementation.

Hands on the Ethernaut CTF

Last week Zeppelin released their Ethereum CTF, Ethernaut.

This CTF is a good introduction to discover how to interact with a blockchain and learn the basics of the smart contract vulnerabilities. The CTF is hosted on the ropsten blockchain, and you can receive free ethers for it. The browser developer console is used to interact with the CTF, as well as the metamask plugin.

I was fortunate enough to be the first one to finish the challenges. The following is how I did it.

1. Fallback

This challenge is the first one, and I think it is more of an introduction, to be sure that everyone is able to play with the API. Let us see in detail what a Solidity smart contract looks like.

Challenge description

The contract is composed of one constructor and four functions. The goal is to become the owner of the contract and to withdraw all the money.

The first function is the constructor of the contract, because it has the same name as the contract. The constructor is a specific function which is called only once when the contract is first deployed, and cannot be called later. This function is usually used to set up some parameters (here an initial contribution from the owner).

function Fallback() {
  contributions[msg.sender] = 1000 * (1 ether);

The second function, contribute(), stores the number of ethers (msg.value) sent by the caller in the contributions map. If this value is greater than the contributions of the contract owner, then the caller becomes the owner of the contract.

function contribute() public payable {
  require(msg.value < 0.001 ether);
  contributions[msg.sender] += msg.value;
  if(contributions[msg.sender] > contributions[owner]) {
    owner = msg.sender;

getContribution() is a simple getter:

function getContribution() public constant returns (uint) {
  return contributions[msg.sender];

withdraw() allows the owner of the contract to withdraw all the money. Notice the onlyOwner keyword after the signature of the function. This is a modifier that ensures that this function is only called by the owner.

function withdraw() onlyOwner {

Finally, the last function is the fallback function of the contract. This function can be executed if the caller has previously made a contribution.

function() payable {
  require(msg.value > 0 && contributions[msg.sender] > 0);
  owner = msg.sender;

Fallback function

To understand what a fallback is, we have to understand the function selector and arguments mechanism in ethereum. When you call a function in ethereum, you are in fact sending a transaction to the network. This transaction contains, among other things, the amount of ether sent (msg.value) and a so-called data, which is an array of bytes. This array of bytes holds the id of the function to be called, and the function’s arguments. They choose to use the first four bytes of the keccak256 of the function signature as the function id. For example, if the function signature is transfer(address,uint256), the function id is 0xa9059cbb.

If you want to call transfer(0x41414141, 0x42), the data will be:


During its execution, the first thing that a smart contract does is to check the function id, using a dispatcher. If there is no match, the fallback function is called, if it exists.

You can visualize this dispatcher using our open source disassembler Ethersplay:

ethersplay_largeEthersplay shows the EVM dispatcher structure(*)

(*) For simplification, the Owner inheritance was removed from the source code. You can find the solidity file and the runtime bytecode here: fallback


If we put everything together we have to:

  1. Call contribution to put some initial value inside contributions
  2. Call the fallback function to become the owner of the contract
  3. Call withdraw to get all the money

(1) is easily done by calling contract.contribution({value:1}) in the browser’s developer tool console. A simple way to call the fallback function (2) is just to send to ether directly to the contract using the metamask plugin. Then (3) is achieved by calling contract.withdraw().

2. Fallout

Challenge description

The goal here is to become the owner of the contract.

At first, this contract appears to have one constructor and four functions. But if we look closer at the constructor, we realize that the name of the function is slightly different than the contract’s name:

contract Fallout is Ownable {

  mapping (address => uint) allocations;

  /* constructor */
  function Fal1out() payable {
    owner = msg.sender;
    allocations[owner] = msg.value;

As a result, this function is not a constructor, but a classic public function. Anyone can call it once the contract is deployed!

This may look too simple to be a real vulnerability, but it is real. Using our internal static analyzer Slither, we have found several contracts where this mistake was made (for example, ZiberCrowdsale or PeerBudsToken)!


We only need to call contract.Fal1out() to become the owner, that’s it!

3. Token

Challenge description

Here we are given 20 free tokens in the contract. Our goal is to find a way to hold a very large amount of tokens.

The function called  transfer allows the transfer of tokens between users:

function transfer(address _to, uint _value) public returns (bool) {
  require(balances[msg.sender] - _value >= 0);
  balances[msg.sender] -= _value;
  balances[_to] += _value;
  return true;

At first, the function looks fine, as it seems to check for overflow

require(balances[msg.sender] - _value >= 0);

Usually, when I have to deal with arithmetic computations, I go for the easy way and use our open source symbolic executor manticore to check if I can abuse the contract. We recently added support for evm, a really powerful tool for auditing integer related issues. But here, with a closer look, we realize that _value and balances are unsigned integers, meaning that balances[msg.sender] - _value >= 0 is always true!

So we can produce an underflow in

balances[msg.sender] -= _value;

As a result, balances[msg.sender] will contain a very large number!


To trigger the underflow, we can simply call contract.transfer(0x0, 21).  balances[msg.sender] will then contain 2**256 – 1.


Challenge description

The goal here is to become the owner of the contract Delegation.

There is no direct way to change the owner in this contract. However, it holds another contract, Delegate with this function:

function pwn() {
  owner = msg.sender;

A particularity of Delegation is the use of delegatecall in the fallback function.

function() {
  if(delegate.delegatecall( {

The hint of the challenge is pretty explicit about it:

Usage of delegatecall is particularly risky and has been used as an attack vector on multiple historic hacks. With it, you contract is practically saying “here, -other contract- or -other library-, do whatever you want with my state”. Delegates have complete access to your contract’s state. The delegatecall function is a powerful feature, but a dangerous one, and must be used with extreme care.

Please refer to the The Parity Wallet Hack Explained article for an accurate explanation of how this idea was used to steal 30M USD.

So here we need to call the fallback function, and to put in the the signature of pwn(), so that the delegatecall will execute the function pwn() within the state of Delegation and change the owner of the contract.


As we saw in “Fallback”, we have to put in the function id of pwn(); which is 0xdd365b8b. As a result, Delegate.pwn() will be called within the state of Delegation, and we will become the owner of the contract.

5. Force

Challenge description

Here we have to send ethers to an empty contract. As there is no payable fallback function, a direct send of ether to the contract will fail.

There are other ways to send ether to a contract without executing its code:

  • Calling selfdestruct(address)
  • Specifying the address as the reward mining destination
  • Sending ethers to the address before the creation of the contract


We can create a contract that will simply call selfdestruct(address) to the targeted contract.

contract Selfdestruct{
  function Selfdestruct() payable{}
  function attack(){

Note that we use a payable constructor. Doing so we can directly put some value inside the contract at its construction. This value will then be sent through selfdestruct.

You can easily test and deploy this contract on ropsten using Remix browser.

6. Re-entrancy

Challenge description

The last challenge! You have to send one ether to the contract during its creation and then get your money back.

The contract has four functions. We are interested in two of them.

donate lets you donate ethers to the contract, and the number of ethers sent is stored in balances.

function donate(address _to) public payable {
  balances[_to] += msg.value;

withdraw, the second function allows users to retrieve the ethers that were stored in balances.

function withdraw(uint _amount) public {
  if(balances[msg.sender] >= _amount) {
    if( {
    balances[msg.sender] -= _amount;

The ethers are sent through the call

At first, everything seems fine here, as the value sent decreased in  balances[msg.sender] -= _amount; and there is no way to increase balances without sending ethers.

Now recall the fallback function mechanism explained in “Fallback.” If you send ethers to a contract containing a fallback function, this function will be executed. What is the problem here? You can have a fallback function which calls back withdraw, and thus can be executed twice before balances[msg.sender] -= _amount is executed!

This vulnerability is called a re-entrancy vulnerability and was used during the unfamous DAO hack.


To exploit a re-entrancy vulnerability you have to use another contract as a proxy. This contract will need to:

  1. Have a fallback function which will call withdraw
  2. Call donate to deposit ethers in the vulnerable contract
  3. Call withdraw

In our not-so-smart-contracts database, you will find an example of a generic skeleton to exploit this vulnerability. I’ll leave the exercise of adapting this skeleton to the reader.

Similar to the previous challenge, you can test and deploy the contract on ropsten using Remix browser.


This CTF was really cool. The interface makes it easy to get into smart contract security. Zeppelin did a good job, so thanks to them!

If you are interested in the tools mentioned in the article, or you need a smart contract security assessment, do not hesitate to contact us!

Trail of Bits joins the Enterprise Ethereum Alliance

We’re proud to announce that Trail of Bits has joined the Enterprise Ethereum Alliance (EEA), the world’s largest open source blockchain initiative. As the first information security company to join, and currently one of the industry’s top smart contract auditors, we’re excited to contribute our unparalleled expertise to the EEA.

As companies begin to re-architect their critical systems with blockchain technology, they will need a strong software engineering model that ensures the safety of confidential data and integration with existing security best practices. We already work with many of the world’s largest companies to secure critical systems and products. As businesses rush to make use of this emerging technology, we look forward to designing innovative and pragmatic security solutions for the enterprise Ethereum community.

Ethereum Enterprise Alliance

We’re helping to secure Ethereum with our expertise, tools, and results.

Preparing Ethereum for production enterprise use will take a lot of work. Collaboration with other motivated researchers, developers, and users is the best way to build a secure and useful Enterprise Ethereum ecosystem. By contributing the tools we’re building to help secure public Ethereum applications, and participating in the EEA Technical Steering Committee’s working groups, we will help the EEA to ensure the security model for Enterprise Ethereum meets enterprise requirements.

How we will contribute

  • Novel​ ​research.​ We’ll bring our industry-leading security expertise to help discover, formalize and secure the unexpected behaviors in DApps and Smart Contracts. We’re already accumulating discoveries from the security audits we’ve conducted.
  • Foundational​ ​tools.​ We’ll help other members reduce risk when building on this technology. As our audits uncover fundamental gaps in Ethereum’s tooling, we’ll fill them in with tools like our symbolic executor, Manticore, and others in development.
  • Sharing​ ​attitude.​ We’ll help define a secure development process for smart contracts, share the tools that we create, and warn the community about pitfalls we encounter. Our results will help smart contract developers and auditors find vulnerabilities and decrease risk.

Soon, we’ll release the internal tools and guidance we have adapted and refined over the course of many recent smart contract audits. In the weeks ahead, you can expect posts about:

  • Manticore, a symbolic emulator capable of simulating complex multi-contract and multi-transaction attacks against EVM bytecode.
  • Not So Smart Contracts, a collection of example Ethereum smart contract vulnerabilities, including code from real smart contracts, useful as a reference and a benchmark for security tools.
  • Ethersplay, a graphical Binary Ninja-based EVM disassembler capable of method recovery, dynamic jump computation, source code matching, and bytecode diffing.
  • Slither, a static analyzer for the Solidity AST that detects common security issues in reentrancy, constructors, method access, and more.
  • Echidna, a property-based tester for EVM bytecode with integrated shrinking that can rapidly find bugs in smart contracts in a manner similar to fuzzing.

We’ll also begin publishing case studies of our smart contract audits, how we used those tools, and the results we found.

Get help auditing your smart contracts

Contact us for a demonstration of how we can help your enterprise make the most of Ethereum and blockchain.

Our team is growing

We’ve added five more to our ranks in the last two months, bringing our total size to 32 employees. Their resumes feature words and acronyms like ‘CTO,’ ‘Co-founder’ and ‘Editor.’ You might recognize their names from publications and presentations that advance the field.

We’re excited to offer them a place where they can dig deeper into security research.

Please help us welcome Mike Myers, Chris Evans, Evan Teitelman, Evan Sultanik, and Alessandro Gario to Trail of Bits!

Mike Myers

Leads our L.A. office (of one). Mike brings 15 years of experience in embedded software development, exploitation, malware analysis, code audits, threat modeling, and reverse engineering. Prior to joining Trail of Bits, Mike was CTO at a small security firm that specializes in cyber security research for government agencies and Fortune 500 companies. Before that, as Principal Security Researcher at Crucial Security (acquired by Harris Corporation), he was the most senior engineer in a division providing cyber capabilities and on-site operations support. Mike contributed the Forensics chapter of the CTF Field Guide, and co-authored “Exploiting Weak Shellcode Hashes to Thwart Module Discovery” in POC || GTFO. He’s NOP certified.

Chris Evans

Chris’s background is in operating systems and software development, but he’s also interested in vulnerability research, reverse engineering and security tooling. Prior to joining Trail of Bits, Chris architected the toolchain and core functionality of a host-based embedded defense platform, reversed and exploited ARM TrustZone secure boot vulnerabilities, and recovered data out of a GPIO peripheral with Van Eck phreaking. Chris has experience in traditional frontend and backend development through work at Mic and Foursquare. He activates Vim key binding mode more frequently than could possibly be warranted.

Evan Teitelman

After solving the world’s largest case of lottery fraud, Evan spent the last year leading code audits for over a dozen state lotteries, including their random number generators (RNGs). Prior to that, Evan focused on security tooling for the Android Kernel and SELinux as a vulnerability researcher with Raytheon SI. Also, he brings experience as a circuit board designer and embedded programmer. He founded BlackArch Linux. In his free time he enjoys hiking and climbing in Seattle, Washington.

Evan Sultanik

A computer scientist with extensive experience in both industry and academia, Evan is particularly interested in computer security, AI/ML/NLP, combinatorial optimization, and distributed systems. Evan is an editor and frequent contributor to PoC || GTFO, a journal of offensive security and reverse engineering that follows in the tradition of Phrack and Uninformed. Prior to Trail of Bits, Evan was the Chief Scientist for a small security firm that specializes in providing cyber security research to government agencies and Fortune 500 companies. He obtained his PhD in Computer Science from Drexel University, where he still occasionally moonlights as an adjunct faculty member. A Pennsylvania native, Evan contributes time to Code for Philly and has published research on zoning density changes in Philadelphia over the last five years.

Alessandro Gario

Hailing out of Italy, Alessandro brings ten years of systems software development experience to the team. Prior to Trail of Bits, he worked on building and optimizing networked storage systems and reverse engineering file formats. In his free time, he enjoys playing CTFs and reverse engineering video games. Alessandro’s mustache has made him a local celebrity. He is our second European employee.

We are very excited for the contributions these people will make to the discipline of information security. If their areas of expertise overlap with the challenges your organization faces, please contact us for help.

iOS jailbreak detection toolkit now available

We now offer a library for developers to check if their apps are running on jailbroken phones. It includes the most comprehensive checks in the industry and it is App Store compatible. Contact us now to license the iVerify security library for your app.

Jailbreaks threaten your work

Users like to install jailbreaks on their phones for extra functionality, unaware that they’re increasing their exposure to risks. Jailbreaks disable many of iOS’s security features such as mandatory code signing and application sandboxing. Apps and code found outside the App Store can contain malware and jailbreaks themselves have included backdoors in the past.

Moreover, running an app on a jailbroken phone may indicate a user is attempting to manipulate the app. App developers deserve to know when their apps are installed on such untrustworthy phones.

Developing the security library to do so requires time and knowledge outside the core competency of many development teams. Establishing and maintaining expertise in the depths of iOS security internals and keeping up with new developments in jailbreak tools and techniques requires more time than many developers can afford. Ineffective jailbreak detection can be worse than no jailbreak detection at all.

Why you should use iVerify

Trail of Bits employs some of the world’s best experts in the field of iOS security internals. Our engineers have reviewed the security model of iOS, jailbreak techniques, and the tools that exist to run them today and developed the best checks possible. The resulting library, iVerify, includes checks for known jailbreaks, like Pangu, and checks for anomalies that may indicate unknown or custom jailbreaks.

iVerify easily integrates into your app as an iOS Framework. Your app can read a simple pass/fail result or it can inspect the results of each check individually.

As an optional feature, the raw checks are aggregated into a JSON message that includes the results of each individual check. A helper function adds other identifying information from the device. Developers can match this information with user information from their app, send it to a centralized logging facility for collection and analysis, and then use it to take action.

We continuously explore new versions of iOS for more effective checks capable of finding known and unknown jailbreaks. iVerify detects jailbreaks on iOS 10 and 11, and our expert team will update the library as new versions are released and new checks are developed.

Start protecting your work with superb jailbreak detection

iVerify delivers an easy-to-use solution without heavy-weight dependencies or obligations to a SaaS service. The checks are the best available and are maintained by our team of experts. The iVerify library is suitable for App Store deployment and will integrate into your app easily.

Contact us now to discuss licensing options.

Tracking a stolen code-signing certificate with osquery

Recently, 2.27 million computers running Windows were infected with malware signed with a stolen certificate from the creators of a popular app called CCleaner, and inserted into its software update mechanism. Fortunately, signed malware is now simple to detect with osquery thanks to a pull request submitted by our colleague Alessandro Gario that adds Windows executable code signature verification (also known as Authenticode). This post explains the importance of code signatures in incident response, and demonstrates a use case for this new osquery feature by using it to detect the recent CCleaner malware.

If you are unfamiliar with osquery, take a moment to read our previous blog post in which we explain why we are osquery evangelists, and how we extended it to run on the Windows platform. Part of osquery’s appeal is its flexibility and open-source model – if there’s another feature you need built, let us know!

Code-signed malware

Code signing was intended to be an effective deterrent against maliciously modified executables, and to allow a user (or platform owner) to choose whether to run executables from untrusted sources. Unfortunately, on general-purpose computing platforms like Windows, third-party software vendors are individually responsible for protecting their code-signing certificates. Malicious actors realized that they only needed to steal one of these certificates in order to sign malware and make it appear to be from a legitimate software vendor. This realization (and the high-profile Stuxnet incident) began a trend of malware signed with stolen code-signing certificates. It has become a routine feature of criminal and nation-state malware attacks in the past few years, and most recently happened again with an infected software update to the popular app CCleaner.

So, defenders already know that a trust model based on an assumption that all third-party software vendors can protect their code-signing certificates is untenable, and that on platforms like Windows, code-signing is only a weak trust marker or application whitelisting mechanism. But, there’s another use for code signatures: incident response. Once a particular signing certificate is known to be stolen, it also works as a telltale indicator of compromise. As the defender you can make lemonade out of these lemons: search for other systems on your network with executables that were also signed with this stolen certificate. The malware might have successfully evaded antivirus-type protections, but any code signed with a known-stolen certificate is an easy red flag: signing can be checked with a 0% chance of any false-positives. osquery offers an ideal method for performing such a search.

Verifying Authenticode signatures with osquery

New sensors are added to osquery with the addition of “tables,” maintaining the abstraction of all system information as SQL tables.

To add a table to osquery, you first define its spec, or schema. An osquery table spec is just a short description of the table’s columns, their data types, and short descriptions, as well as a reference to the implementation. In Alessandro’s pull request, he added an ‘authenticode’ virtual table for Windows, containing the following columns: path, original_program_name (from the publisher), serial_number, issuer_name, subject_name, and result.

Alessandro implemented the code to read code signature and certificate information from the system in osquery/tables/system/windows/authenticode.cpp. The verification of signatures is done using a call to the system API, WinVerifyTrust().

Here’s a simplified example of using osquery to check a Windows executable’s code signature:

osquery> SELECT serial_number, issuer_name, subject_name,
    ...> result FROM authenticode
    ...> WHERE path = 'C:\Windows\explorer.exe';


Most of the columns are self-explanatory. The result values aren’t. “Result” could mean:

State Explanation
missing Missing signature.
invalid Invalid signature, caused by missing or broken files.
untrusted Signature that could not be validated.
distrusted Valid signature, explicitly distrusted by the user.
valid Valid signature, but which is not explicitly trusted by the user.
trusted Valid signature, trusted by the user.

Getting focused results with SQL in osquery

To make the most out of this new functionality, perform JOIN queries with other system tables within osquery. We will demonstrate how using SQL queries enhances system monitoring by reducing the amount of noise when listing processes:

osquery> SELECT, process.path, authenticode.result
    ...> FROM processes as process
    ...> LEFT JOIN authenticode
    ...> ON process.path = authenticode.path
    ...> WHERE result = 'missing';

| pid  | path                                                      | result  |
| 3752 | c:\windows\system32\sihost.exe                            | missing |
| 3872 | C:\Windows\system32\notepad.exe                           | missing |
| 4860 | C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe | missing |
| 5200 | C:\Windows\system32\conhost.exe                           | missing |
| 6040 | C:\Windows\osqueryi.exe                                   | missing |

Tracking a stolen signing certificate

Assume that you have just learned of a malware campaign. The malware authors code-signed their executables using a code-signing certificate that they stole from a legitimate software vendor. The vendor has responded to the incident by acquiring a new code-signing certificate and redistributing their application signed with the new certificate. In this example, we will use CCleaner. How can you search a machine for any software signed with this stolen certificate, but filter out software signed with the vendor’s new certificate?

Example 1: Find executables signed with the stolen certificate

osquery> SELECT files.path, authenticode.subject_name,
    ...>        authenticode.serial_number,
    ...>        authenticode.result AS status
    ...> FROM (
    ...>   SELECT * FROM file
    ...>   WHERE directory = "C:\Program Files\CCleaner"
    ...> ) AS files
    ...> LEFT JOIN authenticode
    ...> ON authenticode.path = files.path
    ...> WHERE authenticode.serial_number == "4b48b27c8224fe37b17a6a2ed7a81c9f";


Example 2: Find executables signed by the affected vendor, but not with their new certificate

osquery> SELECT files.path, authenticode.subject_name,
    ...>        authenticode.serial_number,
    ...>        authenticode.result AS status
    ...> FROM (
    ...>   SELECT * FROM file
    ...>   WHERE directory = "C:\Program Files\CCleaner"
    ...> ) AS files
    ...> LEFT JOIN authenticode
    ...> ON authenticode.path = files.path
    ...> WHERE authenticode.subject_name LIKE "%Piriform%"
    ...> AND authenticode.serial_number != "52b6a81474e8048920f1909e454d7fc0";


Example 3: Code signatures and file hashing

Perhaps you would also like to keep a log of hashes, to keep track of what has been installed:

SELECT files.path AS path,
    ...>        authenticode.subject_name AS subject_name,
    ...>        authenticode.serial_number AS serial_number,
    ...>        authenticode.result AS status,
    ...>        hashes.sha256 AS sha256
    ...> FROM (
    ...>   SELECT * FROM file
    ...>   WHERE directory = "C:\Program Files\CCleaner"
    ...> ) AS files
    ...> LEFT JOIN authenticode
    ...> ON authenticode.path = files.path
    ...> LEFT JOIN hash AS hashes
    ...> ON hashes.path = files.path
    ...> WHERE authenticode.subject_name LIKE "%Piriform%"
    ...> AND authenticode.serial_number != "52b6a81474e8048920f1909e454d7fc0"


For the purposes of our examples here, notice that we have restricted the searches to “C:\Program Files\CCleaner”. You could tailor the scope of your search as desired.

The queries we’ve shown have been run in osquery’s interactive shell mode, which is more appropriate for incident response. You could run any of these queries on a schedule – using osquery for detection rather than response. For this, you would install osqueryd (the osquery daemon) on the hosts you wish to monitor, and configure logging infrastructure to collect the output of these queries (feeding the osquery output to, for example, LogStash / ElasticSearch for later analysis).

Future osquery Work

In this post we demonstrated the flexibility of osquery as a system information retrieval tool: using familiar SQL syntax, you can quickly craft custom queries that return only the information relevant to your current objective. The ability to check Authenticode signatures is just one use of osquery as a response tool to search for potential indicators of compromise. Many IT and security teams are using osquery for just-in-time incidence response including initial malware detection and identifying propagation.

Trail of Bits was early to recognize osquery’s potential. For over a year we have been adding various features like this one in response to requests from our clients. If you are already using osquery or considering using it and there’s a feature you need built, let us know! We’re ready to help you tailor osquery to your needs.