The Art of Memory Forensics: Detecting Malware and Threats in Windows, Linux, and Mac Memory
Published by John Wiley & Sons, Inc.10475 Crosspoint BoulevardIndianapolis, IN 46256www.wiley.com
Copyright © 2014 by John Wiley & Sons, Inc., Indianapolis, Indiana
Published simultaneously in Canada
ISBN: 978-1-118-82509-9
ISBN: 978-1-118-82504-4 (ebk)
ISBN: 978-1-118-82499-3 (ebk)
Manufactured in the United States of America
10 9 8 7 6 5 4 3 2 1
No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 646-8600. Requests to the Publisher for permission should be addressed to the Permissions Department, John Wiley & Sons, Inc., 111 River Street, Hoboken, NJ 07030, (201) 748-6011, fax (201) 748-6008, or online at http://www.wiley.com/go/permissions.
Limit of Liability/Disclaimer of Warranty: The publisher and the author make no representations or warranties with respect to the accuracy or completeness of the contents of this work and specifically disclaim all warranties, including without limitation warranties of fitness for a particular purpose. No warranty may be created or extended by sales or promotional materials. The advice and strategies contained herein may not be suitable for every situation. This work is sold with the understanding that the publisher is not engaged in rendering legal, accounting, or other professional services. If professional assistance is required, the services of a competent professional person should be sought. Neither the publisher nor the author shall be liable for damages arising herefrom. The fact that an organization or Web site is referred to in this work as a citation and/or a potential source of further information does not mean that the author or the publisher endorses the information the organization or website may provide or recommendations it may make. Further, readers should be aware that Internet websites listed in this work may have changed or disappeared between when this work was written and when it is read.
For general information on our other products and services please contact our Customer Care Department within the United States at (877) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002.
Wiley publishes in a variety of print and electronic formats and by print-on-demand. Some material included with standard print versions of this book may not be included in e-books or in print-on-demand. If this book refers to media such as a CD or DVD that is not included in the version you purchased, you may download this material at http://booksupport.wiley.com. For more information about Wiley products, visit www.wiley.com.
Library of Congress Control Number: 2014935751
Trademarks: Wiley and the Wiley logo are trademarks or registered trademarks of John Wiley & Sons, Inc. and/or its affiliates, in the United States and other countries, and may not be used without written permission. All other trademarks are the property of their respective owners. John Wiley & Sons, Inc. is not associated with any product or vendor mentioned in this book.
To my three best friends: Suzanne, Ellis, and Miki. If I could take back the time it took to write this book, I’d spend every minute with you. Looking forward to our new house!
—Michael Hale Ligh
I would like to thank my wife, Jennifer, for her patience during my many sleepless nights and long road trips. I would also like to thank my friends and family, both in the physical and digital world, who have helped me get to where I am today.
—Andrew Case
To my family, who made me the person I am today, and especially to my husband, Tomer, the love of my life, without whose support I wouldn’t be here.
—Jamie Levy
To my family for their unconditional support; to my wife, Robyn, for her love and understanding; and to Addisyn and Declan for reminding me what is truly important and creating the only memories that matter.
—AAron Walters
Michael Hale Ligh (@iMHLv2) is author of Malware Analyst’s Cookbook and secretary-treasurer of the Volatility Foundation. As both a developer and reverse engineer, his focus is malware cryptography, memory forensics, and automated analysis. He has taught advanced malware and memory forensics courses to students around the world.
Andrew Case (@attrc) is digital forensics researcher for the Volatility Project responsible for projects related to memory, disk, and network forensics. He is the co-developer of Registry Decoder (a National Institute of Justice–funded forensics application) and was voted Digital Forensics Examiner of the Year in 2013. He has presented original memory forensics research at Black Hat, RSA, and many others.
Jamie Levy (@gleeda) is senior researcher and developer with the Volatility Project. Jamie has taught classes in computer forensics at Queens College and John Jay College. She is an avid contributor to the open-source computer forensics community, and has authored peer-reviewed conference publications and presented at numerous conferences on the topics of memory, network, and malware forensics analysis.
AAron Walters (@4tphi) is founder and lead developer of the Volatility Project, president of the Volatility Foundation, and chair of the Open Memory Forensics Workshop. AAron’s research led to groundbreaking developments that helped shape how digital investigators analyze RAM. He has published peer-reviewed papers in IEEE and Digital Investigation journals, and presented at Black Hat, DoD Cyber Crime Conference, and American Academy of Forensic Sciences.
Golden G. Richard III (@nolaforensix) is currently Professor of Computer Science and Director of the Greater New Orleans Center for Information Assurance at the University of New Orleans. He also owns Arcane Alloy, LLC, a private digital forensics and computer security company.
Nick L. Petroni, Jr., Ph.D., is a computer security researcher in the Washington, DC metro area. He has more than a decade of experience working on problems related to low-level systems security and memory forensics.
Executive Editor
Carol Long
Project Editor
T-Squared Document Services
Technical Editors
Golden G. Richard III
Nick L. Petroni, Jr.
Production Editor
Christine Mugnolo
Copy Editor
Nancy Sixsmith
Manager of Content Development and Assembly
Mary Beth Wakefield
Director of Community Marketing
David Mayhew
Marketing Manager
Dave Allen
Business Manager
Amy Knies
Vice President and Executive Group Publisher
Richard Swadley
Associate Publisher
Jim Minatel
Project Coordinator, Cover
Patrick Redmond
Compositor
Maureen Forys, Happenstance Type-O-Rama
Proofreaders
Jennifer Bennett
Josh Chase
Indexer
Johnna VanHoose Dinse
Cover Designer
© iStock.com/Raycat
Cover Image
Wiley
We would like to thank the memory forensics community at large: those who spend their weekends, nights, and holidays conducting research and creating free, open-source code for practitioners. This includes developers and users, both past and present, that have contributed unique ideas, plugins, and bug fixes to the Volatility Framework. Specifically, for their help on this book, we want to recognize the following:
We also want to thank Maureen Tullis (T-Squared Document Services), Carol Long, and the various teams at Wiley that helped us through the authoring and publishing process.
Memory forensics is arguably the most fruitful, interesting, and provocative realm of digital forensics. Each function performed by an operating system or application results in specific modifications to the computer’s memory (RAM), which can often persist a long time after the action, essentially preserving them. Additionally, memory forensics provides unprecedented visibility into the runtime state of the system, such as which processes were running, open network connections, and recently executed commands. You can extract these artifacts in a manner that is completely independent of the system you are investigating, reducing the chance that malware or rootkits can interfere with your results. Critical data often exists exclusively in memory, such as disk encryption keys, memory-resident injected code fragments, off-the-record chat messages, unencrypted e-mail messages, and non-cacheable Internet history records.
By learning how to capture computer memory and profile its contents, you’ll add an invaluable resource to your incident response, malware analysis, and digital forensics capabilities. Although inspection of hard disks and network packet captures can yield compelling evidence, it is often the contents of RAM that enables the full reconstruction of events and provides the necessary puzzle pieces for determining what happened before, during, and after an infection by malware or an intrusion by advanced threat actors. For example, clues you find in memory can help you correlate traditional forensic artifacts that may appear disparate, allowing you to make associations that would otherwise go unnoticed.
Regarding the title of this book, the authors believe that memory forensics is a form of art. It takes creativity and commitment to develop this art, but anyone can enjoy and utilize it. Like an exquisite painting, some details are immediately obvious the first time you see them, and others may take time for you to notice as you continue to explore and learn. Furthermore, just like art, there is rarely an absolute right or wrong way to perform memory forensics. Along those lines, this book is not meant to be all-encompassing or wholly authoritative. From the plethora of tools and techniques, you can choose the ones that best suit your personal goals. This book will serve as your guide to choosing what type of artist you want to become.
The world’s reliance on computing grows enormously every day. Companies protect themselves with digital defenses such as firewalls, encryption, and signature/heuristic scanning. Additionally, nations plan attacks by targeting power grids, infiltrating military data centers, and stealing trade secrets from both public and private organizations. It is no wonder that detecting, responding, and reporting on these types of intrusions, as well as other incidents involving computer systems, are critical for information security professionals.
As these attack surfaces expand and the sophistication of adversaries grows, defenders must adapt in order to survive. If evidence of compromise is never written to a hard drive, you cannot rely on disk forensics. Memory, on the other hand, has a high potential to contain malicious code from an infection, in whole or in part, even if it’s never written to disk—because it must be loaded in memory to execute. The RAM of a victimized system will also contain evidence that system resources were allocated by, and in support of, the malicious code.
Likewise, if the data exfiltrated from an organization is encrypted across the network, a packet capture is not likely to help you determine which sensitive files were stolen. However, memory forensics can often recover encryption keys and passwords, or even the plain-text contents of files before they were encrypted, giving you an accelerated way to draw conclusions and understand the scope of an attack.
The most compelling reason for writing this book is that the need for memory forensics in digital investigations greatly exceeds the amount of information available on the topic. Aside from journals, short academic papers, blog posts, and Wiki entries, the most thorough documentation on the subject of consists of a few chapters in Malware Analyst’s Cookbook (Wiley, 2010, Chapters 15 through 18). Nearing its fourth birthday, much of the Cookbook’s content is now outdated, and many new capabilities have been developed since then.
The Art of Memory Forensics, and the corresponding Volatility 2.4 Framework code, covers the most recent Windows, Linux, and Mac OS X operating systems. In particular, Windows 8.1 and Server 2012 R2, Linux kernels up to 3.14, and Mac OS X Mavericks, including the 64-bit editions. If your company or clients have a heterogeneous mix of laptops, desktops, and servers running different operating systems, you’ll want to read all parts of this book to learn investigative techniques specific to each platform.
This book is written for practitioners of technical computing disciplines such as digital forensics, malicious code analysis, network security, threat intelligence gathering, and incident response. It is also geared toward law enforcement officers and government agents who pursue powerful new ways to investigate digital crime scenes. Furthermore, we know that many students of colleges and universities are interested in studying similar topics. If you have worked, or desire to work, in any of the aforementioned fields, this book will become a major point of reference for you.
The material we present is intended to appeal to a broad spectrum of readers interested in solving modern digital crimes and fighting advanced malware using memory forensics. While not required, we assume that you have a basic familiarity with C and Python programming languages. In particular, this includes a basic understanding of data structures, functions, and control flow. This familiarity will allow you to realize the full benefit of the code exhibits, which are also presented with detailed explanations.
For those new to the field, we suggest carefully reading the introductory material in the first part, because it will provide the building blocks to help you through the rest of the book. For the experienced reader, you may want to use the first part as reference material and skip to the parts that interest you most. Regardless of the path you take, the book is intended for the digital investigator who constantly strives to build their skills and seeks new ideas for combating sophisticated and creative digital adversaries.
This book is broken down into four major parts. The first part introduces the fundamentals of modern computers (hardware and software) and presents the tools and methodologies you need for acquiring memory and getting started with the Volatility Framework. The next three parts dive deep into the specifics of each major operating system: Windows, Linux, and Mac. The individual chapters for each OS are organized according to the category of artifacts (i.e., networking, rootkits) or where the artifacts are found (i.e., process memory, kernel memory). The order of the chapters is not meant to imply that your investigations should occur in the same order. We suggest reading the entire book to learn all the possibilities and then determine your priorities based on the specifics of each case.
There are a number of conventions used throughout the book, such as the following:
0x31337
, user.ds
, PsCreateProcess
, process_pid = 4
$
sign, that means we were using a UNIX system (Linux or Mac OS X). Otherwise, you’ll see a Windows prompt. For example:
$ echo "typing on UNIX" | grep typing
C:\Users\Mike\Desktop> echo "typing on windows" | findstr typing
Additionally, we typically define analysis objectives before we present the details of a particular subject. We also make an effort to present and explain the underlying operating system or application data structures related to the evidence you’re analyzing. You’ll see these items in the following format:
To facilitate understanding and help associate context with the artifacts, we show practical examples of using memory forensics to detect specific behaviors exhibited by high profile malware samples, rootkits, suspects, and threat groups.
On the book’s website (http://artofmemoryforensics.com) you will find the lab guide and exemplary evidence files. These hands-on exercises are designed to simulate practical investigations and to reinforce the concepts you learn in the text. You can also find any necessary errata (i.e., mistakes, bug fixes) on the website.
To complete the hands-on exercises, you will need at a minimum:
The following tools are not required for memory forensics per se, but they’re mentioned throughout the book and can help complement your memory-related investigations.
Please note that some tools may require third-party libraries or dependencies.
The authors of this book, also the core developers of the Volatility Framework, teach an internationally acclaimed five-day training course: Windows Malware and Memory Forensics Training by The Volatility Project. Although books help us disseminate the information that we feel is critical to the future of digital forensics, they only provide one-way communication. If you prefer a classroom environment with the ability to ask questions and receive one-on-one tutorials, we invite you to bring your curiosity and enthusiasm to this weeklong journey to the center of memory forensics.
Keep an eye on our training website (http://www.memoryanalysis.net) for upcoming announcements regarding the following:
Since launching the course in 2012, we have exposed students to bleeding-edge material and exclusive new capabilities. This course is your opportunity to learn these invaluable skills from the researchers and developers that pioneered the field. This is also the only memory forensics training class authorized to teach Volatility, officially sponsored by the Volatility Project, and taught directly by Volatility developers. For more information, send us an e-mail at voltraining@memoryanalysis.net.