Managing Social Game Data

• Elasticity: Matching infrastructure costs to demand optimizes a game’s profitability. The ability to easily dial up (and dial down) database resources is a critical part of that equation. One should be able to make these capacity changes to a live game so there is never a need to take a game offline, maintaining continuous revenue generation.
• Low latency: Interactive games must be responsive. Making a player wait for feedback leads to abandonment. If the experience is not quick and predictable, users leave … and take their entertainment budget with them. Database technologies must be able to consistently deliver sub-millisecond random reads and writes of data, across the entire scaling spectrum.
• Data format flexibility: The best social games adapt, delaying or preventing boredom and the resulting decline in active user count. The data tier must be flexible enough (even at very large scale, and without downtime) to support the changing data management requirements of a game in transition.

Choosing the Right NoSQL Database

Selecting the right NoSQL database can be difficult. It seems like a new NoSQL database project appears every week. Sorting through the options can be daunting. There are various classes of NoSQL database: key-value, document, graph, columnar. Each data model has pros and cons.

• Big Data. Data is being generated at an unprecedented rate. How can you efficiently analyze these extremely large datasets and identify patterns, trends and opportunities? This is the “Big Data” problem. Technologies like Hadoop, Map-Reduce and Cassandra are solutions built for analyzing very large datasets. They are generally batch-oriented and focused on analysis.
• Big Audience. Social games have user counts measured in the millions. Millions of users put tremendous pressure on a database — regardless of the size of the dataset. Even with only a few bytes per user (and thus a fairly small aggregate dataset size), keeping up with a non-stop stream of random reads and writes from a large number of concurrent users is incredibly hard. This is the Big Audience problem and what NoSQL databases are designed to address.

• Performance. The document data model keeps related data in a single physical location in memory and on disk (a document). This allows consistently low-latency access to the data — reads and writes happen with very little delay. Database latency can result in perceived “lag” by the player of a game and avoiding it is a key success criterion.
• Dynamic elasticity. Because the document approach keeps records “in one place” (a single document in a contiguous physical location), it is much easier to move the data from one server to another while maintaining consistency — and without requiring any game downtime. Moving data between servers is required to add and remove cluster capacity to cost-effectively match the aggregate performance needs of the application to the performance capability of the database. Doing this at any time without stopping the revenue flow of the game can make a material difference in game profitability.
• Schema flexibility. While all NoSQL databases provide schema flexibility, key-value and document-oriented databases enjoy the most flexibility. Column-oriented databases still require maintenance to add new columns and to group them. A key-value or document-oriented database requires no database maintenance to change the database schema (to add and remove “fields” or data elements from a given record).
• Query flexibility. Balancing schema flexibility with query expressiveness (the ability to ask the database questions, for example, “return me a list of all the farms in which a player purchased a black sheep last month”) is important. While a key-value database is completely flexible, allowing a user to put any desired value in the “value” part of the key-value pair, it doesn’t provide the ability to ask questions. It only permits accessing the data record associated with a given key. I can ask for the farm data for user A, B and C to see if they have a black sheep, but I can’t ask the database to do that work on my behalf. Document-databases provide the best balance of schema flexibility without giving up the ability to do sophisticated queries.