Blog

Notes on a Summer Update
6/28/2012

A lot has happened with Age of Empires Online since the release of the Summer Update. While Empire Points, Vanity Gear, and other new features have been well received, they were also overshadowed by the changes to AI and repeatable quest content. So, let's dig right in with the measurable impact of the AI and quest changes.

As with most online games, we maintain a fairly sizable database to track all kinds of data. Number of players, amount of time played, level reached, etc., but not the amount and quality of equipped gear (more on that later).

We also track a lot of data about quests. We know how many were started, how many were dropped or abandoned, how many were completed, how many were lost, and we have this for every single quest since the game launched.

In light of the Summer Update and its impact on difficulty, we've decided to share some global trends:

Over time, repeatable quests have a higher rate of success

Regardless of initial difficulty, almost every single repeatable quest shows an increased completion percentage over time. We assume this is true because players get better, they watch videos, they figure out patterns, or some combination thereof.

Even with the recent difficulty spike due to the Summer Update, we're already seeing some completion levels for high-end quests nearing, or in rare cases exceeding, pre-update levels.

Over time, regular quests have a lower rate of success

This seems slightly counterintuitive. Our assumption is that more experienced players tend to consume quest content extremely quickly, keeping initial completion percentages extremely high. Over time, you get less experienced players finally reaching those quests, and completion rates trend downward.

As for the impact of the Summer Update, our "regular" quest completion rates dropped by an average of 16% across the board, which wasn't its intent and is our current concern. Based on our data Greece and Egypt were disproportionally hurt by the changes; the Celt campaign, however, was largely unaffected.

The more players you have, the lower your rate of success

The March release on Steam saw a large influx of new and returning players, and globally we saw large decreases in completion rates at the end of March through early April, despite few or no AI or difficulty changes.

We've also seen a significant increase in players since the Summer Update, and they likely account for some of the decreases in global completion rates.

If you increase difficulty, you end up with some extreme results

It's kind of obvious, but you probably don't want serious difficulty spikes in all frequently played quests, as your completion rates can quickly crater. You risk alienating your players, which is generally a bad thing.

Completion Percentages

And now for some data. For comparison purposes, here are some and their completion percentages over time. The dates aren't 100% on the nose, as in sometimes "release" is a few days post-release (when there are enough players to have meaningful data), and the "one week later" is similarly tweaked.

Summer Update Outliers

Quest

Day 1

First Week

March Update

Week After March

Pre-Summer Update

Summer Update: Day 1

Week After Summer Update

Defend Mycenae!

82%

53%

40%

46%

40%

11%

16%

Cataclysm

90%

75%

32%

46%

50%

5%

8%

Helping Out a Neighbor

86%

81%

76%

77%

75%

33%

32%

Ace in the Hole

80%

71%

75%

67%

75%

40%

62%

Operation Napata

84%

75%

50%

55%

55%

15%

24%

Vizier's Doom

80%

75%

58%

75%

75%

32%

30%

Defend Egypt!

45%

34%

35%

43%

35%

15%

25%

Most of these quests were been chosen because they were impacted the most by the Summer Update.

In general, we want high completion rates for quests that are required to progress through a campaign, and we want variable completion rates for all side quests and repeatable quests. Excessive failure is perfectly acceptable for a quest labeled "Challenge;" not so much on the one you need to advance to level 5.

Some quests received tuning between launch and March, so you see some wildly variable completion rates even pre-update. For example, "Cataclysm" was revised once before March, resulting in a lower completion rate, and then became a literal cataclysm with the Summer Update, as anyone that tried to play it can attest to.

The issues with "Helping Out a Neighbor" were caused by a last-second addition of Siege, as in "wouldn't it be fun if we threw some Siege in there at the end?" This turned out to be very not fun, as players at its level (8) weren't ready to handle a Siege attack. We screwed this one up.

In general, many of these quests (plus others not listed) will some require hand-tuning to repair, as they remain prohibitively difficult for most players.

Cyprus Legendary Repeatable Quests

Quest

Release

One Week

March Update

Week After March

Pre-Summer Update

Summer Update: Day 1

Week After Summer Update

Legendary: Recapturing Soloi

1%

10%

61%

48%

58%

9%

29%

Legendary: The Renegade Escapade Grand Finale

25%

95%

74%

76%

88%

19%

42%

Legendary: Tamiat Triple Time

42%

65%

85%

68%

82%

32%

50%

Legendary: Recapturing Marion

1%

3%

27%

30%

27%

7%

26%

Legendary: Recapturing Paphos

3%

10%

56%

54%

55%

24%

53%

Cyprus Non-Legendary Repeatable Quests

Quest

Release

One Week

March Update

Week After March

Pre-Summer Update

Summer Update: Day 1

Week After Summer Update

Recapturing Ledrai

28%

36%

68%

80%

85%

57%

73%

The Renegade Escapade Returns

5%

43%

67%

59%

75%

25%

28%

Take That and That, Tamiat Pirates!

6%

11%

76%

38%

86%

53%

57%

Cyprus quests were very difficult at launch, and players figured them out over time. When Steam launched, more players were playing them for the first time and rates decreased a bit before returning to previous levels.

With the Summer Update, they cratered. After a week and a hotfix, they're trending up to their pre-update levels as players get more comfortable with the new tech tree, develop new strategies, and share them via YouTube or the forum.

Alliance Quests

Quest

Summer Update: Day 1

Week After Summer Update

Relentless Canyon

21%

44%

Valley of Discord

23%

40%

A New Pharaoh's Challenge

15%

38%

The Belgica Defense

30%

48%

King of the Wonder

14%

30%

Persian Coercion

5%

14%

Palace Race: Egypt

29%

55%

Palace Race: Crete

28%

56%

Palace Race: Belgica

25%

54%

These are average percentages across all three Alliances. These are challenging quests that are showing a general upward trend in completion percentages as players figure out how to best tackle their challenges.

Skirmish

Quest

Pre-Summer Update

Week After Summer Update

Skirmish Conquest Victory

52%

50%

Skirmish Standard Victory

77%

79%

Skirmish Wonder Victory

95%

94%

Under the hood, Skirmish uses three primary quests for its different victory conditions.

Overall, it's unaffected by the Summer Update. Which made sense, as it uses a unique Skirmish-specific AI and gear system.

Forum Threads

Quest

Pre-Summer Update

Week After Summer Update

Best Friends Forever

33%

27%

Size Matters

10%

6%

We Killed These Guys Already

25%

9%

Good Ol' Peepaw

8%

50%

Popping Up

11%

28%

The Elite Meet in Crete

14%

15%

Defend Egypt!

35%

25%

Some of these quests have been discussed in the forum as being suddenly difficult or impossible, but the data shows mixed results. Some of these quests had ridiculously low completion rates before the update, and they're still ridiculously low.

(OK, Peepaw showed a huge boost. That's because we broke it a while back. It's included for the LULZ.)

Elite Meet in Crete doesn't use any of the AI or scaling improvements, which shows in its consistent completion rate.

Some Random Notes

Our lack of data about player gear—what's equipped, how many units have gear, what level it is, etc.—is ultimately the biggest issue with the Summer Update. In assuming players had more and better gear than in reality, our new scaling difficulties and opponents were doomed to failure. Hopefully, players will understand that once this is worked out, everything will feel similar to how it was previously. Unfortunately, we've created a situation where people are—rightly—suspicious about any and all of these changes, which leads to "rollback the update" threads.

We failed to message how much of an impact the changes in the tech tree would have on player strategies. A lot of players who refined very specific techniques to efficiently complete repeatable quests quickly found that they no longer worked, and were forced to scramble and "re-learn" their civs. Some players enjoy this, others do not.

While it was understandable that players were upset—and rightly so—with the difficulty spike, we were disappointed to see that a previously non-existent chasm between "hardcore" and "casual" PvE players emerged. Some people asked for help, a lot of people were super helpful, and others took it as an opportunity to create additional friction in a community that should be welcoming and not exclusionary. For those of you who remained helpful and constructive, our thanks. It’s that kind of support that makes a community great.

On the plus side, there's been a consistent 20% increase in the total number of players since the Summer Update. Which doesn't mean that we're satisfied and content with everything right now; if anything, it motivates us even more to produce interesting quests for wide varieties of players of varying skill levels. And we need to fix Cataclysm.

(Tune in on Friday, June 29th for more information about the coming AI adjustments.)

Join the conversation! Discuss this blog post in our forums here: http://forums.ageofempiresonline.com/forums/thread/229488.aspx