Raiden Chronicles - Written by Raiden
1 March, 2010
First day of the month and possibly the first day I am working since the launch of 3.4. Even though it is a Monday (my busiest day of the week), it feels like a storm has passed after the weeks that we had.
We have been working against time since a couple of weeks. So many things went wrong during this update that it looked like 3.4 would have been delayed indefinitely. It all started with the making of the server. Evora was one adamant baby who refused to go live no matter what. Without this server, we couldn’t even begin testing the content.
It took us a month and required us to build the server twice from scratch just to install the content on Evora and bring it to login conditions. From there on testing went smoothly. We encountered a major bug that crashed the client and locked test accounts from the server. Through each passing update, we lost few brave accounts to this bug. *maintains a minute of silence*
During Feb, we held a meeting with the Production and the QA team to share updates on the progress of the new contents. It was during this meeting that we set a deadline for ourselves that seemed impossible. 24 February. At this point, we had lost the cash shop on the server and IMC and HBS were trying to revive it.
19 February: We received a huge patch that was supposed to upload the loading screens into the game. Since we had few beta players on Evora, we decided that we will upload it on Montoro first and reduce the Beta Server downtime. Now I am not sure if it was because Montoro was feeling jealous of all the attention given to Evora or global warming, the loading screen patch nuked Montoro. We were not even able to enter the server.
The release was looking impossible during the next maintenance. The crash bug that claimed the test accounts were still existent and the colony wars were broken. Apart from these major issues, there were tons of other bugs that hindered the launch. The QA team was already exhausted due to their 12+ hour schedule everyday and the report at the end of that day suggested that release on 24th was IMPOSSIBLE.
22 February: The Sword team entered the office looking like a bunch of people who just came out from a month long ordeal in prison. By now we had figured out most of the bugs after working over the weekend. However, Faction Wars were still broken and the game breaking bug still existed on the server. Cash shop was pretty much in a mess.
Most of us started looking at 3rd March as the possible launch date. But we were not ready to throw in the towel yet. We stuffed ourselves with coffee and sugar and everything that could keep us awake to get 3.4 out the door.
Pressure was mounting on everyone due to the self declared deadline. Even the guys from IMC were eager to get 3.4 out during the maintenance and they too were working overtime to fix the issues that were being reported on a daily basis. They delivered 5 patches that day that fixed most of the critical issues.
Time spent awake in office: 19 hours
Colony war status: Down!
23 and 24 February: The final call on the release was handed over to Neume with a compiled report of all the existing bugs.
After reviewing them, it was decided that we will go in for an early maintenance, provided we receive a few more patches that were scheduled for that day. With that in mind, we requested the AMs to start making an emergency maintenance notice so that the players could prepare themselves.
While we were an hour away from the maintenance, IMC delivered a fix for the critical crash bug. The breath of relief that we took at the sight of that patch in our inbox was something we cannot express.
It was much better than the feeling you have when you realize that you just won a round in Russian roulette (I know, I am dark and is next in line to rule hell ).
The patch was uploaded on the test server immediately. Obviously the maintenance was put on hold and we began testing the new fixes.
By now most of us were pretty much brain dead due to the lack of sleep. Lucky for us that the AFK feature is activated by space bar. We were prepared for this and aimed our head to hit space bar in case we crashed. Some of the QA guys actually crashed and took an hour or two worth of nap right here in the office when they simply could not take it anymore.
While the testing was in progress, IMC delivered another last minute fix. Due to these fixes, we had to delay the maintenance by 4 hours. Well, technically it was still early since the maintenance was normally an hour later from when we began.
Faction War Status: Down!
I am going off topic for a minute to vent a little. During maintenance, a couple of users had posted some comments such as *puts on a whining voice*’oh…maintenance is late. Another k2 fail.” And a lot of other negative posts by the same two to three all over the forums.
It is not that I didn’t know how to give a proper reply to those. I was just in a great mood and nothing could ruin it for us that day. /facepalm
Disclaimer: The references are fictitious and do not resemble anyone logged in or logged off. The story was illustrated just to prove a point. If it does resemble you, please logout and login with a different account. We are in a virtual world where men are women, women are men and little girls are FBI agents. Batteries not included.
Back to Uncle Raiden’s story…
It took us 7 and half hours to release 3.4 to our community. While maintenance began, Nutc4se was at the hospital tending to his wife who was going into labor. He ran into the office to make a check list of things to be done and hurried back to the hospital. Hours later, he became the father of a son!! What a day! I’m calling the baby 3.4 and Neume is calling him Grenado Espada.
Neume was awake till the wee hours of the morning overseeing the maintenance and keeping the community occupied with cats. It took him a few hours to prepare the patch notes. It was him who told me over the phone that we can inform the community about 3.4 during maintenance instead of keeping that news till the end. My obvious reaction was “BAH HUMBUG!” By about 4 in the morning, he crashed. It is rumored that Senor Gato ate him.
Quinzel was coordinating with IT and IMC to complete the maintenance. While QA was performing the routine checks when servers were online internally, she was at a stage where she PMed me and said that she might crash any minute and if I don’t get any reply from her, know that it is because she is unconscious. But she hung in there and crashed only when IT sent out the e-mail saying “servers are online”.
Everyone took the day off on 24th to simply sleep in and recharge.
Colony War status: Down!
25th and 26th February: Those were the laziest day we had till date. We hardly did anything. I put on my headphones and lazed around in the forum, checking for any issues with the release. It was time to give all our attention to 4.1.
On Thursday, we had one of our weekly production team meetings where we normally decide the fate of the New World. We were all very happy that we were able to deliver on the impossible deadline that we had. But we were so tired that it was reflected on the cheers we gave each other. It was like “WE DID IT!!!” zzzzzz
I was looking like a gorilla with a beard and hair resembling a hot air balloon.
Colony war status: IMC sent an e-mail which explained why the colony wars went down on Evora. This told us how to fix the colony war issue. Fixed!
Even though CW was a critical issue during content release, we went ahead with the release because we had time till Sunday to fix it. I agree. It was a huge risk that we took. But it paid off. 3.4 has been a dream for us and players alike. We were determined that we will not skip the deadline.
Monday: Quinzel is in Korea to meet with IMC in person. The relation with our developers has improved a hundred times since the dark ages. During this visit, she will be concentrating on the future plans we have for sword and speed up the updates even more.
What are our plans from today, you might ask. Well, we are starting all over again.
4.1 will not be as easy as 3.4!