Reverse Engineering Malware (Part 4)

If you would like to read the other parts in this article series please go to:

We saw in part three of this article series that quite often things are not always as they seem. Spammers, Internet bottom feeders, and others with ill intent often try to mask what is in reality malware. In this part of the article series we will continue unmasking their efforts.

Reverse engineering and you, Part IV

Well in part three of this article series we left off at the point where we recognized that the piece of malware was indeed not a zipped archive, but rather in the PE format. We ascertained this by opening up the piece of malware in a hex editor, which allowed us to examine its contents without actually executing it. You will remember that the characters “MZ” in the file told us that this was actually the aforementioned PE file format.

An important point to remember here is that all of this information is free for the having. There is nothing magical or mysterious about it. None of it is squirreled away in some government vault, unlike what the conspiracy theorists would have you believe. What I’m getting at here is that reverse engineering is more about digging deep and applying yourself than it is about some black art. Documentation on things such as the PE header can be easily obtained from the Microsoft website, or Google itself. Reverse engineering methodology and tools can easily be found at sites such as OpenRCE. Everything that you need to further your studies in reverse engineering can be had for free. That said, let’s get back to applying ourselves to the task at hand.

Malware packers

What is the point of compressing malware down to a smaller size? Well it is not really all about simply shrinking the actual size, but more importantly about trying to evade anti-virus scanners. “What do you mean”, you say? Well quite simply put, a lot of anti-virus programs will pick up malware even if it was custom written. That is what the heuristics portion of the anti-virus suite is all about. So the malicious hackers then decided to try different ways of bypassing the anti-virus program. That led to them creating programs such as packers.

Much like the old saying goes “not everything is created equal”, the same applies to hackers. They all have varying skill levels. Some of them write such custom packers as Yoda, while others will simply use one like UPX to pack their malware. Well it would seem I gave myself a nice segue way here so on that note let’s take a look at the screenshot below. It will show us the piece of malware once again opened up via a hex editor. You will see the ascii string “UPX” in the screenshot below.

Figure 1

In case you were wondering where it was that I found this ascii string, it was only a little bit below the “MZ” that signifies, if you recall, that this file is a PE format one. So the MZ was found at the very beginning of the opened executable, and a little bit beneath it was the UPX string. That tells us that this executable was packed using the UPX packer. Now when you see that a file has been packed via a packer, that often means some pain in trying to unpack it. Not so in the case of files packed with UPX. They can be simply unpacked via the very same tool that was used to pack them in the first place. On a side note, some files can be packed with UPX and then deliberately obfuscated to make the unpacking far more difficult. 

Let’s get this party started!

In keeping with having this topic as simple as possible let us now step through the actual process of unpacking the UPX packed file. What you would have to do is download a copy of the UPX program and install it at the root of your C drive where these types of programs are best kept. Once done it is simply a matter of invoking the program itself and filling in the command line information. It is normally helpful to copy the malware itself to your C drive as well, lest you want to type in a long relative path to where it currently resides. Let’s take a look at the screenshot below, which illustrates the help menu provided by UPX once invoked.

Figure 2

Now let’s take a look at the filled in command line information.

Figure 3

Well as you can see, the unpacking of a UPX file is really rather trivial. There is nothing to it, as you simply need to enter in the right information. The trouble comes when you don’t know what it was packed with. That can lead to a lot of headaches and life altering experiences. Then again, that is the fun that is reverse engineering!

Where are we now at?

We are now at the point where the actual analysis of the malware itself can be started. This can be done in a variety of ways that roughly break down to two separate methods. The first is the static analysis of the malware executable, and the second is in the dynamic analysis of the very same executable. What do static and dynamic mean? Well they roughly break down as follows. During the static analysis of a program/executable/malware/… you would open up the aforementioned without actually executing it. That is typically done through the use of hex editors as we have already seen in the article series. It can also be done through the use of various disassemblers.

These disassemblers will allow you to safely view the executables code without executing it, and also allow you to write down the offsets of interesting breakpoints for example. A breakpoint is essentially the address of a computer instruction in a program that you wish to have a disassembler halt the programs execution at. This then allows you to step through the program execution once instruction at a time. Yes, I realize that these last few lines are a bit advanced for this article, but it does cover pertinent ground as it impacts reverse engineering. Dynamic analysis of the file in question is essentially where you execute it and observe what actually happens. This is not as difficult as it may sound. There are many excellent tools out there which will allow you to observe changes to your computer once a piece of malware has been executed. To name but a few there are tools such as filemon, and regmon.


I have taken great care in detailing all of the peripheral topics as they impact reverse engineering for they are very important. These topics, in case you don’t remember, are ones such as file format specifications, file packers, usage of hex editors, amongst others. When approaching a topic such as reverse engineering, slow and steady, is by far the best approach. There is no shortage of information on the Internet today when it comes to reverse engineering. What you need to do is start assimilating it in small byte sized chunks. On that note I shall see you in the next part where we will look at the now unpacked piece of malware. See you then!

If you would like to read the other parts in this article series please go to:

About The Author

Leave a Comment

Your email address will not be published. Required fields are marked *

This site is protected by reCAPTCHA and the Google Privacy Policy and Terms of Service apply.

Scroll to Top