Interlude: The See-Invisibility Exploit

Once in a while I will take a break from the main Tale of Eternity story, and elaborate on a very interesting issue: exploits. I will overview some common hacks, how they work, what solutions are normally implemented, and how we solved the problem. I'll try to generalize more advanced concepts so everyone can understand, but some of it might be unavoidably technical. First, there are many different ways of cheating - but in today's post, I'll just outline one. 

 

See-Invisibility Hack

Otherwise known as the Maya Purple Hackthis is a fairly common hack exists in just about every game that incorporates invisibility. On RO--and probably most other games as well--this hack works because of unnecessary information being sent from the server to the client. There are semi-logical reasons for it, but in the end we have to blame it on poor programming.

 

The Reason

Normally in a visible-player setting, the client needs to tell the server "Hey, I'm at square x,y", and then the server checks for all players within the area (x +/- 14, y +/- 14). For each player it finds, it tells them, "Hey, this player is at square x,y", at which point each recipient's client translates that information and shows the player visually. Pretty simple, and it makes sense.

Of course, location data isn't the only data that's sent. At the same time, sprite data (how the player looks), class data (what class the character is), and a bunch of other data is also sent--including visibility data. With the way the RO client is programmed, received information that isn't recognized by the client results in an error dump--a message with lots of Korean code, followed by a client crash.

I'm guessing it's because of this, that some genius at Gravity Inc. decided to implement invisibility as it works now. With invisible characters, the same aforementioned process happens, with one key exception. After the client receives the location data for the invisible player, it also receives visibility data, informing the client that said player is invisible. In turn, the code tells the client to show nothing, rather than a player. Okay, this solution works as long as no one tries to tamper with the game client.

 

The Problem

There are two generic ways to exploit this: hex editing, and packet filtering. There is a third way, but it's game-specific and not worth discussing at length. 

Hex editing works with the hexadecimal makeup of a program. Just like a program can be represented in binary as a string of 0 and 1, it can be represented in hex. With unencrypted files (such as the RO client), hex editing is extremely easy, and values can be changed in a heartbeat. Unfortunately, such was the case for the RO client. For clarification, imagine the following piece of pseudo code:

if (visibility = no), then {player = not visible}

Essentially, what you're doing when you're hex editing is changing it to:

if (visibility = no), then {player = visible}

Hey, now you can see invisible players. Granted, it's a little more complicated than I've described. 

 

The second method is packet filtering. By ignoring visibility packets and filtering for position packets, a user is able to translate the packet data directly in order to see hidden players. This can be solved with a good encryption technique, but encrypting every packet for real-time gaming proves to be a real problem. Some problems originate from increased server load, and others from mirroring the decryption on the client.

If you use a basic encryption technique, it's going to get cracked; that's a fact you can't avoid. In order to encrypt and decrypt packets on the server, you need related functionality on the client. This ultimately means you're delivering your encryption system to the exploiter. He's going to diff your files, collect packets and analyze them for patterns, and eventually try to solve your encryption. Sadly, the more secure your encryption technique is, the more load it imposes on the server. 

Let me give you an example of server load: On RO, potions can be consumed at a most, 10 per second. With 2,000 players all consuming on average 5 potions per second in Siege War, there are about 20,000 MySQL inserts per second, counting inventory and logs - which is taxing enough by itself. Try to blowfish each of these 10,000 packets, and a nightmare ensues. 

 

The Solution

The traditional methodology in addressing this problem strikes an uncanny resemblance to struggling competing products: a functionality war. The builders keep improving on their old techniques marginally, and after a week or two, the hackers catch up marginally. The cycle just keeps continuing, eating up a ton of time, and not really getting anywhere. You see, when you introduce a technical problem, the exploiters--who are people who do this for fun, just because they can--are excited about the challenge.

To disengage people from breaking the encryption, you can either introduce a revolutionary technology so profound, that it's no longer fun to solve, or find an alternative method that can't be attacked directly. A technical solution was never finalized for the invisibility hack, and there came a point where it became a big problem on Eternity. 

Because we had no reliable way of catching and proving cheating, the bad players started using it - giving them an advantage over people who didn't use it. The good players, feeling the situation was unjust, decided to level the playing field and use it too. Soon, a large portion of the server was using this exploit, and we were receiving a ton of complaints.

Then, genius: Somewhere in the middle of version 2 (I haven't gotten that far yet in my story), I was discussing this issue with my co-admin, Griffin. I rebooted my brain, and proposed an idea for an alternative way of catching these cheaters - we'll codename it Project Stalker to be hip and unique. Project Stalker involved several steps of logic.

  1. There are only a few ways in the game to reveal a hidden character.
  2. By manipulating game mechanics, it is possible to make a "hacked character" undetectable by normal means.
  3. Hence, if anyone sees this character, they must be cheating. But how do we tell?
  4. When a player hovers their mouse over a character, a packet is sent to the server, requesting the name of the selected player.
  5. Thus, if we receive a name request packet on an undetectable character, the requester must be cheating.

Griffin understood immediately, and proceeded to code it into the server. He had the server spit back logs of players who try to request a specific character we hard coded in to be our undetectable character.

Next Siege War, we tested the system. Griffin ran around as Project Stalker, and the server spit back the names and IPs of every player who requested his packet - all the while without letting them know what was going on. The numbers were shocking. Over 50% of the players were cheating! Instead of banning everyone, we alerted the guild masters of cheating players (some of which were cheating themselves) and made a general announcement: "We have a way of catching you. We're letting today slide, but next time, you will be banned." 

We succeeded. People were scared, and they realized we weren't lying. The amount of invisibility-exploiters dropped from over 50% to less than 1%, and we went through several Sieges without catching a single cheater. This remarkable feat gave us a lot of credibility in terms of catching cheaters. 

 

More to come... follow me on Twitter (@zeteg) for updates

6 responses
Not sure if this would apply to your type of game programming, but believe it or not this mistake was actually made by some of the early online poker websites in the late 90s. Then enter encryption, etc. and by the early 2000s there were still some sites online that didn't realize some of their players were getting access to other players' cards by packet sniffing.

The answer of course is that if it's game-critical to prevent an exploit like that (say, if there's real money on the line) then you have to dedicate server-side cycles to figuring out who can see what through some kind of static function that strips the packets based on who's requesting them. That's how we do it. Of course we trap the bejeezus out of it to find out who tried to pull something funny by sending bad data, and then we can ban the player too. But just being sneaky and having a ban threat isn't really enough because you never know what other ways people might have to get around your obstacle. Especially now that you've published it. If they can hack the client to alter visibility behavior, it should be trivial to find your red-flare "I've been hacked" link in a hex editor. And the worst kind of hack is the one you don't know about.

>it should be trivial to find your red-flare "I've been hacked" link in a hex editor.

That's kind of the point, there is no flare to speak of.

>When a player hovers their mouse over a character, a packet is sent to the server, requesting the name of the selected player.

That's a standard command which is part of the core functionality. The analysis is done completely on the server, with no change to the client. I guess they could still add a conditional statement that only stops the name request on invisible players but even then you could catch the ones who attack invisible players. Any interaction is an indication, and not being able to interact defeats the advantages of the hack.

That's nice, but don't you think sending verboten information to the client in any form at all is a terrible idea?
What's the game-specific third way?
@Josh Im assuming they either did not want to try and make large structural changes (fixing the way invisibility is handled so they check first for visibility and only send other data if so, or something) or they wanted something they could have without a client update to "alert" cheaters
1 visitor upvoted this post.