“AMD Flaws” Technical Summary

Two weeks ago, we were engaged by CTS Labs as independent consultants at our standard consulting rates to review and confirm the technical accuracy of their preliminary findings. We participated neither in their research nor in their subsequent disclosure process. Our recommendation to CTS was to disclose the vulnerabilities through a CERT.

Our review of the vulnerabilities was based on documentation and proof-of-concept code provided by CTS. We confirmed that the proof-of-concept code worked as described on the hardware we tested, but we will defer to AMD for a final determination of their full impact, patches, and remediation recommendations.

Most of the discussion after the public announcement of the vulnerabilities has been focused on the way they were disclosed rather than their technical impact. In this post, we have tried to extract the relevant technical details from the CTS whitepaper so they can be of use to the security community without the distraction of the surrounding disclosure issues.

Technical Summary

The security architecture of modern computer systems is based on a defense in depth. Security features like Windows Credential Guard, TPMs, and virtualization can be used to prevent access to sensitive data from even an administrator or root.

The AMD Platform Security Processor (PSP) is a security coprocessor that resides inside AMD CPUs and is implemented as a separate ARM CPU. It is similar to Intel ME or the Apple Secure Enclave. It runs applications that provide security features like the TPM or Secure Encrypted Virtualization. The PSP has privileged access to the lowest level of the computer system.

The PSP firmware can be updated through a BIOS update, but it must be cryptographically signed by AMD. Physical access is usually not required to update the BIOS and this can be done with administrator access to the computer. The MASTERKEY vulnerability bypasses the PSP signature checks to update the PSP with the attacker’s firmware. Cfir Cohen on the Google Cloud Security Team discovered a similar issue in an adjacent area of the AMD PSP in September 2017.

The PSP also exposes an API to the host computer. The FALLOUT and RYZENFALL vulnerabilities exploit the PSP APIs to gain code execution in the PSP or the SMM.

The “chipset” is a component on the motherboard used to broker communication between the processor, memory, and peripherals. The chipset has full access to the system memory and devices. The CHIMERA vulnerability abuses exposed interfaces of the AMD Promontory chipset to gain code execution in the chipset processor.

Exploitation requirements

  • All exploits require the ability to run an executable as admin (no physical access is required)
  • MASTERKEY additionally requires issuing a BIOS update + reboot

Potential technical impact

  • Code execution in the PSP and SMM (no visibility to typical security products)
  • Persistence across OS reinstallation and BIOS updates
  • Block or infect further BIOS updates, or brick the device
  • Bypass Windows Credential Guard
  • Bypass Secure Encrypted Virtualization (SEV)
  • Bypass Secure Boot
  • Bypass or attack security features implemented on top of the PSP (e.g., fTPM)

There is no immediate risk of exploitation of these vulnerabilities for most users. Even if the full details were published today, attackers would need to invest significant development efforts to build attack tools that utilize these vulnerabilities. This level of effort is beyond the reach of most attackers (see https://www.usenix.org/system/files/1401_08-12_mickens.pdf, Figure 1)

These types of vulnerabilities should not surprise any security researchers; similar flaws have been found in other embedded systems that have attempted to implement security features. They are the result of simple programming flaws, unclear security boundaries, and insufficient security testing. In contrast, the recent Meltdown and Spectre flaws required previously unknown techniques and novel research advances to discover and exploit.

Echidna, a smart fuzzer for Ethereum

Today we released Echidna, our next-generation EVM smart fuzzer at EthCC. It’s the first-ever fuzzer to target smart contracts, and has powerful features like abstract state-machine modeling and automatic minimal test case generation. We’ve been working on it for quite some time, and are thrilled to finally share it with the world.

Different interfaces for different applications

Echidna ships with an echidna-test executable, which can start finding bugs in your Solidity code in minutes, and a powerful and comprehensive library for writing your own analyses. echidna-test requires nothing other than simple Solidity assertions to find deep bugs and comes with a clear UI to make understanding its output easy. See a video of it in action:

As a library, Echidna provides a myriad of tools to write custom analyses for more complex contracts. Want to model the progression of time to test your token launch contract? Echidna does that. Want to write a symbolic model of your contract’s underlying state machine? Echidna does that. Want to do something even fancier? No promises, but ask around on Empire Hacking and there’s a good chance Echidna does that too.

Batteries included

Echidna isn’t just a powerful fuzzer.

We took care to write a beautiful UI, ensure generated test cases are minimal, test with continuous integration, and provide examples so you can use it more easily. We’re still actively developing it, so there may be some rough edges. On the off chance you run into one, you should file an issue and we’ll do our best to take care of it. We use Echidna on real audits, so we’re committed to it being as correct and powerful as possible.

It’s easy to get started

Echidna uses stack, so setup should be a breeze, and the only dependency you’ll need is whatever version of solc your contract uses (and, of course, stack itself). If you do run into issues, we’ve tried to document the process and potential workarounds.

Once it’s installed, you can simply run echidna-test solidity/cli.sol and you’re off to the races! If you have any issues, open one on Github or ask in #ethereum on the Empire Hacking Slack and we’ll do our best to help.

2017 in review

What a roller coaster of a year! Well, outside of our office. Inside, 2017 was excellent.

We published novel research that advanced – among others – the practices of automated bug discovery, symbolic execution, and binary translation. In the process, we improved many foundational tools that an increasing number of security researchers will come to rely on. We scaled up our work on securing smart contracts and established ourselves as a premiere blockchain security firm. Finally, as in years past, we shared what lessons we could and supported others to do the same.

Whether you’re a client, a long-time follower, or a budding security researcher, thank you for your interest and contribution.

Below, find 12 highlights from 2017; each one is a reason to stick around in 2018.

Novel research

Automated bug discovery entered the real world

This field really picked up momentum in 2017. If you weren’t paying close attention, the flurry of developments was easy to miss. That’s why we gave a tour of the field’s recent advances at IT Defense, BSidesLisbon, and CyCon.

But ‘unused tools don’t find bugs,’ and many roadblocks still stand in the way of widespread adoption. We’re changing that in the defense industry. We won contracts with Lockheed Martin and the Department of Defense’s DIUx to apply and scale our Cyber Reasoning System.

If you’re wondering about the future role of humans in the secure development lifecycle, we maintain that these tools will always require expert operators.

If you’re on a team developing an automated bug discovery tool, you’ll be happy to know that we ported the CGC’s Challenge Binaries to Windows, macOS, and Linux. Now you have an objective benchmark for evaluating your tool’s performance.

Manticore improved the state of accessible symbolic execution tooling

We open-sourced Manticore, to some applause in the community. Manticore is a highly flexible symbolic execution tool, which we rely on for binary analysis and rapid prototyping of new research techniques.

Parts of Manticore underpinned our symbolic execution capabilities in the Cyber Grand Challenge. Since then, it has been an integral component of our research for DARPA’s LADS (Leveraging the Analog Domain for Security) program.

In only one year after Manticore’s public release, we’ve adapted the tool to amplify the abilities of smart contract auditors and contribute to the security of the Ethereum platform. In December, we explained how we use Manticore for our work on Ethereum Virtual Machine (EVM) bytecode. When applied to Ethereum, symbolic execution can automatically discover functions in a compiled contract, generate transactions to trigger contract states, and check for failure states.

McSema 2.0 brought us closer to treating binaries like source code

In early 2017, we decided to give McSema a fresh coat of paint. We cleaned up the code, and made it more portable and easier to install. It ran faster. The code it produced was better. But we knew we could push it further.

Since we released McSema four years ago, programs have adopted modern x86 features at an increasing rate, and our lifting goals have expanded to include AArch64, the architecture used by modern smartphones.

So, we made a series of major enhancements. For example, we completely separated the instruction semantics from the control flow recovery and created Remill. McSema is now a client that uses the Remill library for binary lifting. To borrow an analogy, McSema is to Remill as Clang is to LLVM. If you want access to lifting capabilities in your own app, you should use Remill.

Next, we demonstrated a series of use cases for McSema, including: binary patching and modification, symbolic execution with KLEE, and reuse of existing LLVM-based tools (e.g. libFuzzer).

Foundational Tools

Ethereum’s foundation firmed up

In response to the surge of interest in Ethereum smart contracts and blockchain technology, we launched new services and created tools that offer verifiable security gains to the community. We adapted Manticore into an industry-leading security tool, and developed a suite of additional tools that help others write more secure smart contracts.

In a short period of time, we’ve become one of the industry’s most trusted providers of audits, tools, and best practices for securing smart contracts and their adjacent technologies. We’ve secured token launches, decentralized apps, and entire blockchain platforms. See our public reports for RSK and DappHub’s Sai.

We focused the year’s final Empire Hacking meetup on how to write secure smart contracts, and hack them. Two of the six speakers came from our team. In November, we were the first to finish Zeppelin’s Ethereum CTF, Ethernaut.

We became the first information security company to join the Enterprise Ethereum Alliance (EEA), the world’s largest open source blockchain initiative. As one of the industry’s top smart contract auditors, we’re excited to contribute our unparalleled expertise and unique toolset to the EEA’s working groups.

osquery expanded its reach and abilities

Following our port of Facebook’s open source endpoint instrumentation and monitoring agent to Windows in 2016, we’ve continued to contribute to osquery’s development and adoption.

We made foundational enhancements that increased the framework’s raw capabilities. Adding auditd-based file integrity monitoring required a redesign from the ground up. As a result, end users get better performance, no fake or broken events, and new file integrity monitoring.

Among numerous other improvements, we showed how osquery can find notable industry issues like the CCleaner malware, and contributed the features needed to detect them. For additions that aren’t native operating system functions, we’ve created a maintained repository of osquery extensions.

In an effort to promote osquery’s long-term success, we shared the experiences, pains and wishes of users at five major tech firms. We hope the findings will help the community to chart a course forward, and help the undecided to determine if and how to deploy osquery in their companies.

iVerify satisfied a fundamental need for iPhone users

We released iVerify, an App-Store-compatible library of the most comprehensive iOS jailbreak checks in the industry. The checks are maintained by our team of experts; some of the world’s foremost authorities in iOS security internals.

App developers deserve to know when their apps are installed on jailbroken phones. However, ineffective jailbreak detection can be worse than no jailbreak detection at all.

iVerify detects jailbreaks on iOS 10 and 11 right now. We’re committed to updating the library as new versions of iOS are released, and as more effective checks capable of finding known and unknown jailbreaks are developed.

Algo brought self-hosted VPN services to the masses

In late 2016, we released our self-hosted personal VPN server. Algo is designed for ease of deployment and security, it relies on only modern protocols and ciphers, it includes only the minimal software you need, and it’s free.

Then, in 2017, interest in protecting one’s online activity exploded. We can’t bring ourselves to thank the FCC for relaxing ISP commercialization rules, but we are glad that more people are putting more thought into their digital privacy.

And yes, we are very grateful to:

We’ll continue to work aggressively toward simplifying and automating Algo’s installation so those who lack the technical expertise to build and maintain their own VPNs aren’t left exposed.

Learn & Share

Helped the industry deploy new exploit mitigations

Following our discussions of Control Flow Integrity (CFI) and Control Flow Guard (CFG), we shared our attempt to compare clang’s implementation of CFI against Visual Studio’s Control Flow Guard by applying both to osquery. Instead of a direct comparison, we generated a case study of how seemingly small tradeoffs in security mitigations have serious implications for usability. Our discussion shows developers how to use these mitigations and includes sample programs that showcase the bugs they mitigate.

Months later, when Microsoft was caught on the wrong end of a ‘tradeoff’ with serious implications for its users, we applied AppJailLauncher-rs to Windows Defender on the software giant’s behalf. The result, Flying Sandbox Monster, is the industry’s first sandboxed anti-virus scanner for Windows. We described the process and results of creating the tool, as well as its Rust-based framework to contain untrustworthy apps in AppContainers.

Combining Control Flow Integrity with sandboxing makes for an incredible challenge for attackers. Unfortunately, they’re also a challenge for developers to use! In creating the above materials, we lowered the learning curve for the community.

Shone a spotlight on Binary Ninja

We think that Vector35’s versatile reversing platform doesn’t get the respect it deserves. We worked to help others understand Binary Ninja’s capabilities by:

  • Describing the fundamentals of Binary Ninja’s Low Level IL, and how the Python API can be used to interact with it.
  • Demonstrating how to easily develop platform-agnostic tools harnessing the power of Binary Ninja’s LLIL and its dataflow analysis.
  • Explaining how we analyzed this year’s DEF CON CTF challenges with our own Binary Ninja processor module, now available for anyone interested in trying out the challenges.
  • Sharing at Infiltrate and Summercon how Binary Ninja makes program analysis more accessible and useful.

Sponsored the causes that matter to us

The next generation. We care about giving younger people opportunities to learn and develop skills in the industry, so we continued our sponsorship of capture the flag competitions like UIUC CTF, HSCTF, and CSAW. We contributed both financial support and unique challenges.

The InfoSec community. We want to share our research with a larger audience and help others gain access to it, so we sponsored conferences like GreHack, Infiltrate, and ISSISP. We provided both financial support and workshops on new techniques and Manticore.

The Truth. We care about getting accurate information out there, so we’re always happy to sponsor the industry’s best podcast host: Patrick Gray at Risky Business. We appreciate his cutting commentary on industry news. Listen to our interviews in episodes #449 and #474 on exploit mitigations and security engineering, respectively.

Advanced the public’s understanding of security

As in years past, when we come across something that would improve the state of security, and it isn’t covered under an NDA, we share it. To that end, we:

Grew as a team

This has been another wonderful year for our team. We expanded our numbers. We went to Infiltrate in Miami and Whistler for company retreats. Josselin earned his PhD. We tacked on more NOP certifications, and hosted some wonderful interns.

Well done, everyone!

More in store for 2018

This year, we will continue to publish more of our research, advance our commitment to our open source projects, and share more of the tools we’ve developed in-house. Look for more soon about:

  • DIUx – The Department of Defense’s experimental innovation unit DIUx recently awarded us a seven-figure contract to take our Cyber Reasoning System (CRS) to the next level as part of project Voltron.
  • Blockchain – As this area becomes a larger part of our business, expect to see more of our discoveries about the security of smart contracts, the security implications of the Solidity language and the Ethereum Virtual Machine.
  • Open source support – We are taking new projects under our wing (Google Santa, Google Omaha, and more), in addition to the major contributions we have in the works for osquery.
  • iVerify – We plan to release a standalone version that allows anyone to check whether their phone has been jailbroken. The service is intended for high-risk users like journalists and activists operating in high threat environments.
  • Algo – We’ll be making it easier to use for those who don’t want to use a terminal.
  • Accessible tooling – We’ll make advanced tools and technologies available to greater numbers of software engineers with new releases of DeepState, Manticore, and fcd-remill.
  • And finally, Operation Waking Shark – Keep an eye out for these team fleeces at an upcoming Empire Hacking.

Parity Technologies engages Trail of Bits

We’re helping Parity Technologies secure their Ethereum client. We’ll begin by auditing their codebase, and look forward to publishing results and the knowledge we gained in the future.

Parity Technologies combines cryptography, cellular systems, peer-to-peer technology and decentralized consensus to solve the problems that have gone unaddressed by conventional server-client architecture.

Their Ethereum client is designed for foundational use in enterprise environments, so businesses and organizations can capitalize on the new opportunities blockchain technology presents.

Parity selected us for several reasons

  • Our expert staff brings decades of security knowledge to the field of smart contracts, deep experience with Rust and Solidity, and rapid command of the latest developments in Ethereum security.
  • We can dig deeper into the construction of smart contracts, the security implications of the Solidity language, and the Ethereum Virtual Machine (EVM) than any other team because of our proprietary tools such as Manticore, Ethersplay, Slither, and Echidna.
  • Finally, Parity was attracted to our enthusiasm for jointly publishing discoveries in our audit, and possibly even educational material for the benefit of the broader blockchain community.

Bottom line, we’re one of the leading blockchain security firms–a natural choice for their needs.

What you can expect next

Over the course of the next few weeks, we will audit the beta branch of Parity and the corresponding jsonrpc library. We’ll review Parity’s key generation and storage, RPCs that use private keys and are responsible for permissions, and an assortment of smart contracts. Once the report is made public we plan to write about our lessons learned and results.

We’re excited to work with Parity to help secure the Ethereum ecosystem!

An accessible overview of Meltdown and Spectre, Part 1

In the past few weeks the details of two critical design flaws in modern processors were finally revealed to the public. Much has been written about the impact of Meltdown and Spectre, but there is scant detail about what these attacks are and how they work. We are going to try our best to fix that.

This article is explains how the Meltdown and Spectre attacks work, but in a way that is accessible to people who have never taken a computer architecture course. Some technical details will be greatly simplified or omitted, but you’ll understand the core concepts and why Spectre and Meltdown are so important and technically interesting.

This article is divided into two parts for easier reading. The first part (what you are reading now) starts with a crash course in computer architecture to provide some background and explains what Meltdown actually does. The second part explains both variants of Spectre and discusses why we’re fixing these bugs now, even though they’ve been around for the past 25 years.


First, a lightning-fast overview of some important computer architecture concepts and some basic assumptions about hardware, software, and how they work together. These are necessary to understand the flaws and why they work.

Software is Instructions and Data

All the software that you use (e.g. Chrome, Photoshop, Notepad, Outlook, etc.) is a sequence of small individual instructions executed by your computer’s processor. These instructions operate on data stored in memory (RAM) and also in a small table of special storage locations, called registers. Almost all software assumes that a program’s instructions execute one after another. This assumption is both sound and practical — it is equivalent to assuming that time travel is impossible — and it enables us to write functioning software.


These Intel x86-64 processor instructions in notepad.exe on Windows show what software looks like at the instruction level. The arrows flow from branch instructions to their possible destinations.

Processors are designed to be fast. Very, very fast. A modern Intel processor can execute ~300 billion instructions per second. Speed drives new processor sales. Consumers demand speed. Computer engineers have found some amazingly clever ways to make computers fast. Three of these techniques — caching, speculative execution, and branch prediction — are key to understanding Meltdown and Spectre. As you may have guessed, these optimizations are in conflict with the sequential assumption of how the hardware in your computer executes instructions.


Processors execute instructions very quickly (one instruction every ~2 ns). These instructions need to be stored somewhere, as does the data they operate on. That place is called main memory (i.e. RAM). Reading or writing to RAM is 50-100x slower (~100 ns/operation) than the speed at which processors execute instructions.

Because reading from and writing to memory is slow (relative to the instruction execution speed), a key goal of modern processors is to avoid this slowness. One way to achieve this is to assume a common behavior across most programs: they access the same data over and over again. Modern processors speed up reads and writes to frequently accessed memory locations by storing copies of the contents of those memory locations in a “cache.” This cache is located on-chip, near the processor cores that execute instructions. This closeness makes accessing cached memory locations considerably faster than going off-chip to the main storage in RAM. Cache access times vary by the type of cache and its location, but they are on the order of ~1ns to ~3ns, versus ~100ns for going to RAM.


An image of an Intel Nehalem processor die (first generation Core i7, taken from this press release). There are multiple levels of cache, numbered by how far away they are from the execution circuitry. The L1 and L2 cache are in the core itself (likely the bottom right/bottom left of the core images). The L3 cache is on the die but shared among multiple cores. Cache takes up a lot of expensive processor real estate because the performance gains are worth it.

Cache capacity is tiny compared to main memory capacity. When a cache fills, any new items put in the cache must evict an existing item. Because of the stark difference in access times between memory and cache, it is possible for a program to tell whether or not a memory location it requested was cached by timing how long the access took. We’ll discuss this in depth later, but this cache-based timing side-channel is what Meltdown and Spectre use to observe internal processor state.

Speculative Execution

Executing one instruction at a time is slow. Modern processors don’t wait. They execute a bundle of instructions at once, then re-order the results to pretend that everything executed in sequence. This technique is called out-of-order execution. It makes a lot of sense from a performance standpoint: executing 4 instructions one at a time would take 8ns (4 instructions x 2 ns/instruction). Executing 4 instructions at once (realistic on a modern processor) takes just 2ns — a 75% speedup!

While out-of-order execution and speculative execution have different technical definitions, for the purposes of this blog post we’ll be referring to both as speculative execution. We feel justified in this because out-of-order execution is by nature speculative. Some instructions in a bundle may not need to be executed. For example, an invalid operation like a division by zero may halt execution, thus forcing the processor to roll back operations performed by subsequent instructions in the same bundle.


A visualization of performance gained by speculatively executing instructions. Assuming 4 execution units and instructions that do not depend on each other, in-order execution will take 8ns while out-of-order execution will take 2ns. Please note that this diagram is a vast oversimplification and purposely doesn’t show many important things that happen in real processors (like pipelining).

Sometimes, the processor makes incorrect guesses about what instructions will execute. In those cases, speculatively executed instructions must be “un-executed.” As you may have guessed, researchers have discovered that some side-effects of un-executed instructions remain.

There are many caveats that lead to speculative execution guessing wrong, but we’ll focus on the two that are relevant to Meltdown and Spectre: exceptions and branch instructions. Exceptions happen when the processor detects that a rule is being violated. For example, a divide instruction could divide by zero, or a memory read could access memory without permission. We discuss this more in the section on Meltdown. The second caveat, branch instructions, tell the processor what to execute next. Branch instructions are critical to understanding Spectre and are further described in the next section.

Branch Prediction

Branch instructions control execution flow; they specify where the processor should get the next instruction. For this discussion we are only interested in two kinds of branches: conditional branches and indirect branches. A conditional branch is like an fork in the road because the processor must select one of two choices depending on the value of a condition (e.g. A > B; C = 0; etc. ). An indirect branch is more like a portal because the processor can go anywhere. In an indirect branch, the processor reads a value that tells it where to fetch the next instruction.


A conditional branch and its two potential destinations. If the two operands of the cmp instruction are equal, this branch will be taken (i.e. the processor will execute instructions at the green arrow). Otherwise the branch will not be taken (i.e. the processor will execute instructions at the red arrow). Code taken from notepad.exe


An indirect branch. This branch will redirect execution to whatever address is at memory location 0x10000c5f0. In this case, it will call the initterm function. Code taken from notepad.exe.

Branches happen very frequently, and get in the way of speculative execution. After all, the processor can’t know which code to execute until after the branch condition calculation completes. The way processors get around this dilemma is called branch prediction. The processor guesses the branch destination. When it guesses incorrectly, the already-executed actions are un-executed and new instructions are fetched from the correct location. This is uncommon. Modern branch predictors are easily 96+% accurate on normal workloads.

When the branch predictor is wrong, the processor speculatively executes instructions with the wrong context. Once the mistake is noticed, these phantom instructions are un-executed. As we’ll explain, the Spectre bug shows that it is possible to control both the branch predictor and to determine some effects of those un-executed instructions.


Now let’s apply the above computer architecture knowledge to explain Meltdown. The Meltdown bug is a design flaw in (almost) every Intel processor released since 1995. Meltdown allows a specially crafted program to read core operating system memory that it should not have permission to access.

Processors typically have two privilege modes: user and kernel. The user part is for normal programs you interact with every day. The kernel part is for the core of your operating system. The kernel is shared among all programs on your machine, making sure they can function together and with your computer hardware, and contains sensitive data (keystrokes, network traffic, encryption keys, etc) that you may not want exposed to all of the programs running on your machine. Because of that, user programs are not permitted to read kernel memory. The table that determines what part of memory is user and what part is kernel is also stored in memory.

Imagine a situation where some kernel memory content is in the cache, but its permissions are not. Checking permissions will be much slower than simply reading the value of the content (because it requires a memory read). In these cases, Intel processors check permissions asynchronously: they start the permission check, read the cached value anyway, and abort execution if permission was denied. Because processors are much faster than memory, dozens of instructions may speculatively execute before the permission result arrives. Normally, this is not a problem. Any instructions that happen after a permissions check fails will be thrown away, as if they were never executed.

What researchers figured out was that it was possible to speculatively execute a set of instructions that would leave an observable sign (via a cache timing side-channel), even after un-execution. Furthermore, it was possible to leave a different sign depending on the content of kernel memory — meaning a user application could indirectly observe kernel memory content, without ever having permission to read that memory.

Technical Details


A graphical representation of the core issue in Meltdown, using terminology from the steps described below. It is possible to speculatively read core system memory without permission. The effects of these temporary speculative reads are supposed to be invisible after instruction abort and un-execution. It turns out that cache effects of speculative execution cannot be undone, which creates a cache-based timing side channel that can be used to read core system memory without permission.

At a high level, the attack works as follows:

  1. A user application requests a large block of memory, which we’ll call bigblock, and ensures that none of it is cached. The block is logically divided into 256 pieces (bigblock[0], bigblock[1], bigblock[2], ... bigblock[255]).
  2. Some preparation takes place to ensure that a memory permissions check for a kernel address will take a long time.
  3. The fun begins! The program will read one byte from a kernel memory address — we’ll call this value secret_kernel_byte. As a refresher, a byte can be any number in the range of 0 to 255. This action starts a race between the permissions check and the processor.
  4. Before the permissions check completes, the hardware continues its speculative execution of the program, which uses secret_kernel_byte to read a piece of bigblock (i.e. x = bigblock[secret_kernel_byte]). This use of a piece of bigblock will cache that piece, even if the instruction is later undone.
  5. At this point the permissions check returns permission denied. All speculatively executed instructions are un-executed and the processor pretends it never read memory at bigblock[secret_kernel_byte]. There is just one problem: a piece of bigblock is now in the cache, and it wasn’t before.
  6. The program will time how long it takes to read every piece of bigblock. The piece cached due to speculative execution will be read much faster than the rest.
  7. The index of the piece in bigblock is the value of secret_kernel_byte. For example, if bigblock[42] was read much faster than any other entry, the value of secret_kernel_byte must be 42.
  8. The program has now read one byte from kernel memory via a cache timing side-channel and speculative execution.
  9. The program can now continue to read more bytes. The Meltdown paper authors claim they can read kernel memory at a rate of 503 Kb/s using this technique.

What is the impact?

Malicious software can use Meltdown to more easily gain a permanent foothold on your desktop and to spy on your passwords and network traffic. This is definitely bad. You should go apply the fixes. However, malicious software could already do those things, albeit with more effort.

If you are a cloud provider (like Amazon, Google, or Microsoft) or a company with major internet infrastructure (like Facebook), then this bug is an absolute disaster. It’s hard to underscore just how awful this bug is. Here’s the problem: the cloud works by dividing a massive datacenter into many virtual machines rentable by the minute. A single physical machine can have hundreds of different virtual tenants, each running their own custom code. Meltdown breaks down the walls between tenants: each of those tenants could potentially see everything the other is doing, like their passwords, encryption keys, source code, etc. Note: how the physical hardware was virtualized matters. Meltdown does not apply in some cases. The details are beyond the scope of this post.

The fix for Meltdown incurs a performance penalty. Some sources say it is a 5-30% performance penalty, some say it is negligible, and others say single digits to noticeable. What we know for sure is that older Intel processors are impacted much more than newer ones. For a desktop machine, this is slightly inconvenient. For a large cloud provider or internet company, a 5% performance penalty across their entire infrastructure is an enormous price. For example, Amazon is estimated to have 2 million servers. A 5 to 30% slowdown could mean buying and installing 100,000 (5%) to 600,000 (30%) additional servers to match prior capability.

What should I do?

Please install the latest updates to your operating system (i.e. MacOS, Windows, Linux). All major software vendors have released fixes that should be applied by your automatic updater.

All major cloud providers have deployed fixes internally, and you as a customer have nothing to worry about.

To be continued…

We hope you have a better understanding of computer architecture concepts and the technical details behind Meltdown. In the second half of this blog post we will explain the technical details of Spectre V1 and Spectre V2 and discuss why these bugs managed to stay hidden for the past 25 years. The technical background will get more complicated, but the bugs are also more interesting.

Finally, we’d like to remind our readers that this blog post was written to be accessible to someone without a computer architecture background, and we sincerely hope we succeeded in explaining some difficult concepts. The Meltdown and Spectre papers, and the Project Zero blog post are better sources for the gory details.

Heavy lifting with McSema 2.0

Four years ago, we released McSema, our x86 to LLVM bitcode binary translator. Since then, it has stretched and flexed; we added x86-64 support, put it on a performance-focused diet, and improved its usability and documentation.

McSema wasn’t the only thing improving these past years, though. At the same time, programs were increasingly adopting modern x86 features like the advanced vector extensions (AVX) instructions, which operate on 256-bit wide vector registers. Adjusting to these changes was back-breaking but achievable work. Then our lifting goals expanded to include AArch64, the architecture used by modern smartphones. That’s when we realized that we needed to step back and strengthen McSema’s core. This change in focus paid off; now McSema can transpile AArch64 binaries into x86-64! Keep reading for more details.

Enter the dragon

Today we are announcing the official release of McSema 2.0! This release greatly advances McSema’s core and brings several exciting new developments to our binary lifter:

  • Remill. Instruction semantics are now completely separated into Remill, their own library. McSema is a client that uses the library for binary lifting. To borrow an analogy, McSema is to Remill as Clang is to LLVM. Look out for future projects using Remill.
  • Simplified semantics. The separation of McSema and Remill makes it easier to add support for new instructions. In Remill, instruction semantics can be expressed directly in C++ and are automatically compiled by Clang into LLVM bitcode.
  • AArch64 (64-bit ARMv8). The switch to using Remill as a semantics backend means that McSema 2 supports multiple architectures from the start. Not only does it work on x86 and x86-64 binaries, but it also supports lifting 64-bit ARMv8 programs.
  • SSE3/4 and AVX support. McSema now supports lifting programs that utilize advanced vector instruction sets.
  • Better CFG recovery. A common source of lifting errors is poor control flow recovery. We improved the control flow recovery process to make it simpler, faster, and more accurate. McSema’s CFG recovery is also beginning to incorporate advanced features, like lifting global variables and stack variables.
  • Binary Ninja support. McSema now has beta support for recovering program control flow via Binary Ninja.

McSema 2.0 is under active development and is rapidly improving and gaining features. We hope to make both using and hacking on McSema easier and more accessible than ever.

See it soar: Using McSema 2

The biggest change to McSema is the switch to using Remill for instruction semantics, and the subsequent support for AArch64. A good demonstration of this improvement is to show that McSema can disassemble an AArch64 binary, lift it to bitcode, and then recompile that bitcode to x86-64 machine code. Let’s get to it then!

Getting McSema

The first step is to download and install the code. For now, Linux is the primary platform supported by McSema; however, we are working toward macOS and Windows build support. If your goal is to lift Windows binaries, then no sweat! Linux builds of McSema will happily analyze Windows binaries.

The above linked instructions give more details that you should follow (e.g. getting dependencies, resolving common errors, etc.), but the essential steps to downloading and installing McSema are as follows:

mkdir ~/data
cd ~/data
git clone git@github.com:trailofbits/remill.git
cd ~/data/remill/tools
git clone git@github.com:trailofbits/mcsema.git
cd ~/data
~/data/remill/scripts/build.sh --llvm-version 3.9
cd ~/data/remill-build
sudo make install

These commands will clone Remill and McSema, invoke a common build script that compiles both projects in the ~/data/remill-build directory, and then install the projects onto the system.

Disassembling our first binary

Using McSema is usually a two- or three-step process. The first step is always to disassemble a binary into a “control flow graph” file using the mcsema-disass command-line tool. This file contains all of the program binary’s original code and data, but organized into logical groupings, like variables, functions, blocks of instructions, and references therebetween.

We’ll use Felipe Manzano’s maze, compiled as an AArch64 program binary, as our running example. It’s an interactive, command-line game that asks the user to solve a maze. Precompiled binaries for the maze can be found in the McSema’s examples/Maze/bin directory.

cd ~/data/remill/tools/mcsema/examples/Maze/bin
mcsema-disass --arch aarch64 
  --os linux 
  --binary maze.aarch64 
  --output /tmp/maze.aarch64.cfg 
  --log_file /tmp/maze.aarch64.log 
  --entrypoint main 
  --disassembler /opt/ida-6.9/idal64

The above steps will produce a control flow graph (CFG) file from the maze program, saving the CFG file to /tmp/maze.aarch64.cfg. If you’re following along at home and don’t have a licensed version of IDA Pro, but do have a Binary Ninja license, then you can change the value passed to the --disassembler option to point to the Binary Ninja executable instead (i.e. --disassembler /opt/binaryninja/binaryninja). Finally, if you are one of those radare2 holdouts, then no sweat — we have CFG files for the maze binary already made.

Lifting to bitcode

The second step is to lift the CFG file into LLVM bitcode using the mcsema-lift-3.9 command-line tool. The 3.9 in this case isn’t the McSema version; it’s the LLVM toolchain version. LLVM is a fast-evolving project, which sometimes means that interesting projects (e.g. KLEE) are left behind and only work with older LLVM versions. We’ve tried to make it as simple as possible for users to reap the benefits of using McSema — that’s why McSema works using LLVM versions 3.5 and up. In fact, with McSema 2, you can now have multiple versions of McSema installed on your system, each targeting distinct LLVM versions. Enough about that, time to lift some weights!

mcsema-lift-3.9 --arch aarch64 
  --os linux 
  --cfg /tmp/maze.aarch64.cfg 
  --output /tmp/maze.aarch64.bc 

The above command instructs McSema to save the lifted bitcode to the file /tmp/maze.aarch64.bc. The --explicit_args command-line flag is a new feature of McSema 2 that emulates the original behavior of McSema 1. If your goal is to perform static analysis or symbolic execution of lifted bitcode, then you will want to employ this option. Similarly, if you are compiling bitcode lifted from one architecture (e.g. AArch64) into machine code of another architecture (e.g. x86-64), then you also want this option. On the other hand, if your goal is to compile the lifted bitcode back into an executable program for the same architecture (as is the case for the Cyber Fault-tolerance Attack Recovery program), then you should not use --explicit_args.

Compiling bitcode back to runnable programs

It’s finally time to make the magic happen — we’re going to take bitcode from an AArch64 program, and make it run on x86-64. We have conveniently ensured that a Clang compiler is installed alongside McSema, and in such a way that it does not clash with any other compilers that you already have installed. Here’s how to use that Clang to compile the lifted bitcode into an executable named /tmp/maze.aarch64.lifted.

remill-clang-3.9 -o /tmp/maze.aarch64.lifted /tmp/maze.aarch64.bc

Note: If for some reason remill-clang-3.9 does not work for you, then you can also use ~/data/remill-build/libraries/llvm/bin/clang.

Solving the maze

We’ve now successfully transpiled an AArch64 program binary into a x86-64 program binary. Wait, what? Yes, we really did that. Running the transpiled version shows us the correct output, prompting us with instructions on how to play the game.

$ /tmp/maze.aarch64.lifted
 Maze dimensions: 11x7
 Player position: 1x1
 Iteration no. 0
 Program the player moves with a sequence of 'w', 's', 'a' and 'd'
 Try to reach the price(#)!
 |X| |#|
 | | --+ | |
 | | | | |
 | +-- | | |
 | | |

But what if — try as we might — we’re not able to solve the maze? That won’t be a problem, because we can always use the KLEE symbolic executor to solve the maze for us.

Your new workout routine

We’ve practiced all the moves and your new workout routine is ready. Day 1 in your routine is to disassemble a binary and make a CFG file.

mcsema-disass --arch aarch64 
  --os linux 
  --binary ~/data/remill/tools/mcsema/examples/Maze/bin/maze.aarch64 
  --output /tmp/maze.aarch64.cfg 
  --log_file /tmp/maze.aarch64.log 
  --entrypoint main 
  --disassembler /opt/ida-6.9/idal64

Day 2 is your lift day, where we lift the CFG file into LLVM bitcode.

mcsema-lift-3.9 --arch aarch64 
  --os linux 
  --cfg /tmp/maze.aarch64.cfg 
  --output /tmp/maze.aarch64.bc 

Day 3 ends your week with some intense compiling, producing a new machine code executable from the lifted bitcode.

remill-clang-3.9 -o /tmp/maze.aarch64.lifted /tmp/maze.aarch64.bc

Finally, don’t forget your stretches. We want to make sure those muscles still work.

echo ssssddddwwaawwddddssssddwwww | /tmp/maze.aarch64.lifted

Come with me if you want to lift

Come with me if you want to liftThe Maze transpiling and symbolic execution demos scratch the surface of what you can do with McSema 2. The ultimate goal has always been to enable binaries to be treated like source code. With the numerous improvements in McSema 2, we are getting closer to that ideal. In the coming months we’ll talk more about other exciting features of McSema 2 (like stack and global variable recovery) and how Trail of Bits and others are using McSema.

We’d love to talk to you about McSema and how it can solve your binary analysis and transformation problems. We’re always available at the Empire Hacking Slack and via our contact page.

For now though, put your belt on — it’s time for some heavy lifting. McSema version 2 is ready for your binaries.

Videos from Ethereum-focused Empire Hacking

On December 12, over 150 attendees learned how to write and hack secure smart contracts at the final Empire Hacking meetup of 2017. Thank you to everyone who came, to our superb speakers, and to Datadog for hosting this meetup at their office.

Watch the presentations again

We believe strongly that the community should share what knowledge it can. That’s why we’re posting these recordings from the event. We hope you find them useful.

A brief history of smart contract security

Jon Maurelian of Consensys Diligence reviewed the past, present, and future of Ethereum with an eye for security at each stage.


  • Ethereum was envisioned as a distributed shared computer for the world. High level languages such as Solidity enable developers to write smart contracts.
  • This shared computer where anyone can execute code comes with a number of inherent security issues. Delegate calls, reentrancy, and other idiosyncrasies of Ethereum have been exploited on the public chain for spectacular thefts.
  • Among the most exciting upcoming developments include safer languages like Viper, the promise of on-chain privacy with zk-SNARKs, and security tooling like Manticore and KEVM.

A CTF Field Guide for smart contracts

Sophia D’Antoine of Trail of Bits discussed recent Capture the Flag (CTF) competitions that featured Solidity and Ethereum challenges, and the tools required to exploit them.


  • CTFs have started to include Ethereum challenges. If you want to set up your own Ethereum CTF, reference Sophia’s scripts from CSAW 2017.
  • Become familiar projects from Trail of Bits, like Manticore, Ethersplay, and Not So Smart Contracts to learn about Ethereum security and compete in CTFs.
  • Integer overflows and reentrancy are common flaws to include in challenges. Review how to discover and exploit these flaws in write-ups from past competitions.

Automatic bug finding for the blockchain

Mark Mossberg of Trail of Bits explained practical symbolic execution of EVM bytecode with Manticore.


  • Symbolic execution is a program analysis technique that can achieve high code coverage, and has been used to create effective automated bug finding systems.
  • When applied to Ethereum, symbolic execution can automatically discover functions in a contract, generate transactions to trigger contract states, and check for failure states.
  • Manticore, an open source program analysis tool, uses symbolic execution to analyze EVM smart contracts.

Addressing infosec needs with blockchain technology

Paul Makowski introduced PolySwarm, an upcoming cybersecurity-focused Ethereum token, and explained how it aligns incentives and addresses deficiencies in the threat intelligence industry.


  • The economics of today’s threat intelligence market produce solutions with largely overlapping detection capabilities which result in limited coverage and expose enterprises to innovative threats.
  • Ethereum smart contracts provide a distributed platform for intelligent, programmed market design. They fix the incentives in the threat intelligence space without becoming a middleman.
  • PolySwarm unlocks latent security expertise by removing barriers to participate in tomorrow’s threat-intelligence community. PolySwarm directs this expertise toward the greater good, getting more security experts to create a better collective defense for all.

Learn more about Empire Hacking

Let’s secure your smart contracts

We’ve become one of the industry’s most trusted providers of audits, tools, and best practices for securing smart contracts and their adjacent technologies. We’ve secured token launches, decentralized apps, and entire blockchain platforms.

Contact us for help.

What are the current pain points of osquery?

You’re reading the second post in our four-part series about osquery. Read post number one for a snapshot of the tool’s current use, the reasons for its growing popularity among enterprise security teams, and how it stacks up against commercial alternatives.

osquery shows considerable potential to revolutionize the endpoint monitoring market. (For example, it greatly simplifies the detection of signed malware with Windows executable code signature verification.) However, the teams at five major tech firms and osquery developers whom we interviewed for this series say that the open source tool has room for improvement.

Some of these qualms relate to true limitations. However, we’ve also heard some recent grumbling among prospective users and industry competitors that doesn’t align with osquery’s actual shortcomings.

As with many rapidly improving open source projects, documentation updates lagging behind a blistering development schedule may be to blame for these misconceptions. So, before diving into what true pain points the osquery community should tackle next, let’s shed light on some current myths about osquery’s limitations.

Mythical limitations of osquery

“osquery has no support for containers”

Oh, but it does! Teams like Uptycs have poured significant development support into this much-requested feature within the past year. Currently, osquery can perform container introspection at the management host layer (more efficient) or it can operate in each container (more granularity) without dominating CPU.

“osquery cannot operate in real time”

osquery handles file integrity and process auditing for MacOS and Linux. It can also monitor user access, hardware events, and socket events in select operating systems. It performs these tasks through interaction with the audit kernel API. Unlike its other pull-based queries, these monitoring services create event-based logs in real time and ensure that osquery doesn’t miss important events between queries. These features are essential to osquery’s power in incident detection.

“osquery is high overhead”

osquery is a lightweight solution when correctly deployed and managed. As we’ll touch on later in the post, the leading causes of performance issues come from misconfiguration: scheduling runaway queries, performing event-based queries on high-traffic file paths, or running osquery in a resource-constrained environment without implementing CPU controls. In fact, respondents who implemented safeguards such as Cgroups were so confident in osquery’s performance that they deployed the tool on every endpoint, including production servers.

Current limitations of osquery; the facts

Demand for user support outstrips supply

For an open-source project, osquery offers a lot of user support; a website, an active slack channel, extensive documentation, and lots of blogs (like this one!). That said, the community can do better.

  • Documentation updates have not matched pace with the growing list of project feature updates, leaving users unaware of new functionality or confused about how to use it.
  • Confused users flocking to the osquery slack channel ask similar questions. Experts try to help these individuals instead of creating FAQs, writing comprehensive query packs, and making tutorials.

Users test feature integrity

Facebook has done an excellent job at thoroughly reviewing new code and hosting productive debates about how best to build new features. However, there has been a lack of oversight into the efficacy of new features. So far, this has been the job of users who report edge cases and unexpected behavior to the slack channel or github repo.

A recent example of this issue: Developers and users saw false negatives in file integrity monitoring for osquery. The audit backend contained multiple bugs that caused inaccurate logs. This has persisted unchecked since FIM was first enabled in 2015. Thankfully, our own Alessandro Gario is implementing a fix.

Issues with extensions

This crucial component of osquery has been causing problems for users. The issues stem from insufficient support during their development. osquery’s current SDK only provides the bare minimum needed to integrate with osquery. The documentation and APIs are also limited. Because of these factors, many extensions aren’t well-built or well-maintained, and therefore, introduce unreliable results.

Fortunately, the community has started to resolve the issues. Kolide’s osquery-go provides a rich SDK for developers to create osquery extensions with Go. Last week, we explained how to write an extension for osquery. We also released a repository of our own well-maintained osquery extensions that users can pull from (there’s only one in there right now but more to come, soon!). We intend to help the community navigate the extension-building process and to create a reliable source of updated extensions.

Limited platform support beyond Linux and macOS

Users are eager for osquery to support more platforms and provide better introspection on all endpoints. osquery’s current limit to just a subset of endpoints leaves holes in users’ monitoring capacity. Further, they noted that some supported platforms lacked important features.

For example, macOS and Linux platforms can collect usable real-time data about a variety of event types. Real-time data in osquery for Windows is limited to the Windows Event Log, which only exposes a stream of difficult-to-parse system data. No users whom we interviewed had successfully implemented or parsed these logs in their deployments.

Screen Shot 2017-12-20 at 5.05.57 PM

From Teddy Reed and Mitchell Grenier’s osquery presentation at the 2017 LISA conference

Readable real-time monitoring underpins osquery’s incident-detection capabilities. While scheduled queries can miss system events between queries, real-time event-based monitoring is less prone to false negatives. The absence of this feature in the Windows port greatly degrades osquery’s incident-detection utility for users running Windows machines in their fleets.

Runaway queries and guidance

Respondents reserved some of their harshest criticism for the lack of safeguards against bad queries, especially those that unexpectedly pulled an excessive amount of data. Sometimes, these runaway queries caused major issues for the larger fleet such as degraded endpoint system performance and clogged memory. In addition, malformed queries could flood data logs and rewrite over other data collected, causing other events to pass by undetected.

osquery’s watchdog feature does prevent some performance issues by killing any processes that consume too much CPU or memory. However, this is done without consideration of what’s running at the time. Well-formed audit-based queries often exceed the default quotas, killing processes unnecessarily. As a result, users turned off the feature to avoid missing essential data. A better solution would understand the scale of a user’s query and ask for confirmation.

Users also wanted smarter queries. One interviewee wanted guidance on the right query intervals for different types of system data. He also wanted to save wasted storage from overlapping data within different query packs. Though the issue is relatively cheap, it would be helpful if osquery could de-duplicate this data.

Insufficient debugging/diagnostics at scale

Users struggled with large-scale deployment of osquery, primarily because of difficulty debugging and diagnosing query issues in their fleets. One company reported that roughly 15% of nodes queried persist as pending for unknown reasons. Another reported that certain endpoints would occasionally “fall off the internet” without any apparent cause. Though users can restart osquery with the verbose setting to print information about every action performed, this option is primarily a tool for developers and is not user-friendly.

Deployment and maintenance issues

Every company implementing osquery tackles this ongoing struggle in a different way. We went into great detail in our previous post about the variety of tools and techniques they used to manage this problem. Despite support and documentation improvements, issues persist. One user reported ongoing troubles implementing osquery version updates on endpoints for which employees are admins.


After reading about all of these pain points, you might be wondering why osquery won multiple product awards this year, and why over 1,100 users have engaged with the development community’s Slack channel and GitHub repo. You might be wondering why the five top tech company teams we surveyed for this series reported that they liked osquery better than commercial fleet management tools

It’s simple. The tool has attracted a vibrant development community invested in its success. Every new development brings osquery closer to feature-parity with the equivalent components of competitors’ fully integrated – and higher-priced – security suites. As users commission companies like Trail of Bits to make those improvements, the entire community benefits.

That will be the topic of the third post in this series: osquery’s development requests. If you use osquery today and have requests you’d like to add to our research, please let us know! We’d love to hear from you.

How does your experience with osquery compare to the pains mentioned in this post? Do you have other complaints or issues that you’d like to see addressed in future releases? Tell us! Help us lead the way in improving osquery’s development and implementation.

Announcing the Trail of Bits osquery extension repository

Today, we are releasing access to our maintained repository of osquery extensions. Our first extension takes advantage of the Duo Labs EFIgy API to determine if the EFI firmware on your Mac fleet is up to date.

There are very few examples of publicly released osquery extensions. Very little documentation exists on the topic. This post aims to help future developers in navigating through the process of writing an extension for osquery. The rest of this post describes how we implemented the EFIgy extension for osquery.

About EFIgy

At this year’s Ekoparty, Duo Labs presented the results of its research on the state of support and security in EFI firmwares. These software components are really interesting for attackers. They operate on a privilege level that is out of reach even from operating systems and hypervisors. Duo Labs gathered and analyzed all the publicly released Apple updates from the last three years and verified the information by looking at more than 73,000 Macs across different organizations.

The researchers found that many of these computers were running on outdated firmware, even though the required EFI updates were supposed to be bundled in the same operating system patches that the hosts had installed correctly. Duo Labs followed this finding by creating the EFIgy service, a REST endpoint that can access the latest OS and EFI versions for any known Apple product through the use of details such as logic board id and product name.

Programmatically querying EFIgy

EFIgy expects a JSON object containing the details for the system we wish to query. The JSON request doesn’t require many keys; it boils down to hardware model and software versions:

  "board_id": "Mac-66E35819EE2D0D05",
  "smc_ver": "2.37f21",
  "build_num": "16G07",
  "rom_ver": "MBP132.0226.B20",
  "hw_ver": "MacBookPro12,1",
  "os_ver": "10.12.4",
  "hashed_uuid": ""

The rom_ver key is kind of tricky. It doesn’t include the full EFI version because (according to the author) it includes timestamps that are not necessarily useful or easy to keep track of. Separated by the dot characters, the only fields you need are the first, third and fourth.

As the name suggests, the hashed_uuid field is a SHA256 digest. To compute it correctly, the MAC address of the primary network interface must be prefixed to the system UUID like this: “0x001122334455” + “12345678-1234-1234-1234-1234567890AB”.

The remaining keys are self-explanatory, but keep in mind that they will not be reported correctly when running from a virtual machine. board_id, hw_ver and rom_ver will report information about the virtual components offered by your hypervisor.

Querying the service is simple. The JSON data is sent via an HTTP POST request to the following REST endpoint: https://api.efigy.io/apple/oneshot.

The server response is made of three JSON objects. Compare them with your original values to understand whether your system is fully up to date or not.

  "latest_efi_version" : {
    "msg" : "<version>"

  "latest_os_version" : {
    "msg" : "<version>"

  "latest_build_number" : {
    "msg" : "<error message>",
    "error" : "1"

Developing osquery extensions

The utilities provided by Duo Labs are easy and straightforward to use, but manually running them all on each system in our fleet was not an easy task. We decided to implement an osquery extension that queries EFIgy, an idea we got from Chris Long.

Why write an extension and not a virtual table? We’ve decided to keep native operating system functions in core and convert everything else into an extension. If a new feature uses an external service or a non-native component, we will default to writing an extension.

The only toolset you have available is the standard library and what you manually import into your project. osquery links everything statically. You have to take special care of the libraries you use. Don’t rely on loading dynamic libraries at runtime from your program.

Once your environment is ready, table extensions do not require much work to be implemented. You just have to inherit from the osquery::TablePlugin class and override the two methods used to define the columns and generate the table rows.

class MyTable final : public osquery::TablePlugin {
    osquery::TableColumns columns() const override {
      return {

    osquery::QueryData generate(osquery::QueryContext& request) override {
      osquery::Row row;
      row[“column_name”] = “value”;

      return { row };

The source files must then be placed in a dedicated folder inside osquery/external. Note that you must add the “extension_” prefix to the folder name. Otherwise, the CMake project will ignore it.

For more complex projects, it is also possible to add a CMakeLists.txt file, creating the target using the following helper function:

ADD_OSQUERY_EXTENSION(${PROJECT_NAME} source1.cpp source2.cpp)

You will have access to some of the libraries in osquery such as Boost, but not some other utilities (e.g. the really useful http_client class).

There is no list of recommended steps to take when developing an extension, but if you plan on writing more than one I recommend you bundle your utility functions in headers that can then be easily imported and reused. Keeping all external libraries statically linked is also a good idea, as it will make redistribution easier.

Using our osquery extensions repo

Without anywhere to submit our new feature, we created a new repository for our extensions. The EFIgy extension is the first item available. Expect more to follow.

Using the repository is simple. However, you will have to clone the full source code of osquery first since the SDK is not part of the distributable package. Building the extension is easy. You only have to create a symbolic link of the source folders you want to compile inside the osquery/external folder, taking care to name the link according to the following scheme: extension_<name>. You can then follow the usual build process for your platform, as the default ALL target will also build all extensions.

cd /src/osquery-extensions
ln -s efigy /src/osquery/external/extension_efigy

cd /src/osquery
make sysprep
make deps

make -j `nproc`
make externals

Extensions are easy to use. You can test them (both with the shell and the daemon) by specifying their path with the –extension parameter. Since they are normal executables, you can also start them after osquery. They will automatically connect via Thrift and expose the new functions. The official documentation explains the process very well.

To quickly test the extension, you can either start it from the osqueryi shell, or launch it manually and wait for it to connect to the running osquery instance.

osqueryi --extension /path/to/extension

Take action

If you have a Mac fleet, you can now monitor it with osquery and the EFIgy extension, and ensure all your endpoints have received the required software and firmware updates.

If you’re reading this post some time in the future, you have even more reason to visit our osquery extension repository. We’ll keep it maintained and add to it over time.

Do you have an idea for an osquery extension? Please file an issue on our Github repo for it. Do you need osquery development? Contact us.

Securing Ethereum at Empire Hacking

If you’re building real applications with blockchain technology and are worried about security, consider this meetup essential. Join us on December 12th for a special edition of Empire Hacking focused entirely on the security of Ethereum.

Why attend?

Four blockchain security experts will be sharing how to write secure smart contracts, and hack them. Two speakers come from our team.

We’ve become one of the industry’s most trusted providers of audits, tools, and best practices for securing smart contracts and their adjacent technologies. We’ve secured token launches, decentralized apps, and entire blockchain platforms. As with past Empire Hacking events, we’re excited to share the way forward with the development community.

Who will be speaking?

  • Sophia D’Antoine of Trail of Bits will discuss Solidity and Ethereum challenges in recent CTF competitions and the tools required to exploit them.
  • John Maurelian of Consensys Diligence will discuss his highlights from Devcon3 about the latest developments in Ethereum security.
  • Mark Mossberg will be sharing how Trail of Bits finds bugs in EVM bytecode with our symbolic execution engine, Manticore.
  • Paul Makowski will share his upcoming security-focused Ethereum token, PolySwarm, which uses blockchain technology to address deficiencies in the threat intelligence industry.
  • Amber Baldet and Brian Schroeder of the Enterprise Ethereum Alliance will discuss the threat modeling, confidential transactions, and zero-knowledge proofs that went into the Quorum blockchain.

When and where?

We’ll be meeting December 12th at 6pm. This month’s meetup will take place at the DataDog offices in the New York Times building. RSVP is required. As per usual, light food and beer will be served.

To find out more about Empire Hacking: