Dear Machinists:

As I promised, I’ve finally obtained information on the network outages that have been affecting us for the past few months. This state of the machine describes our data center troubles and my plan to rectify them.

For context, I have two cabinets of servers with SiteSouth who has a large private cage in the AtlantaNAP. The AtlantaNAP provides secure space, power, network access, and cooling to SiteSouth. SiteSouth provides those four necessary services to Rails Machine. SiteSouth is one of the largest providers within AtlantaNAP. Rails Machine owns its hardware which is manufactured by Cisco, Supermicro, and APC.

Over the past few days, I’ve spoken with the owners of both SiteSouth and the AtlantaNAP. Unfortunately, I’ve received two different descriptions of the problems. Given this discrepancy, it is difficult for me to say what exactly the issues are. The only facts that I have is that there have been several outages of network connectivity between the AtlantaNAP and Rails Machine via SiteSouth. Given the unusual nature of this situation, I can not describe the precise issues and their resolutions. What I can say is that situation is not acceptable and I am working immediately to resolve it.

At this point, my plan is to give you a choice. I would like to split Rails Machine across two data centers. Since the beginning of the network issues a month ago, I’ve been in discussions with several data centers. I have not made a final decision, but I will shortly. I want to make sure that my decisions will restore a high quality of service as quickly as possible.

One option is to move into another space in the AtlantaNAP. This can happen quickly and everyone will be able to keep their IP addresses. The migration should require minimal downtime and can be done during low periods of traffic for your sites.

At the same time, we will install service in another data center. I have not made a final decision, but the current top choice is well known and used by a number of high profile companies. Those customers that wish can migrate to this facility. The downside is that your IP address will change and require some extra downtime. In addition, it will be possible for us to offer data center redundancy. We will work with you during the migration to make it as smooth and seamless as possible.

Specifics and a time line will be forth coming for both data center migrations. It is a significant undertaking and will require time to plan and execute. I am working as fast as I can to evaluate the options and purchase the necessary hardware to make for a smooth transition. Please watch the blog and your inbox for more details in the next week.

During the two outages many of you provided some excellent feedback on how we can improve our communications with you. As announced earlier, I have already launched a server status page with an independent monitoring service. In addition, I am correcting some issues with being able to easily send email notifications to customers who are affected by a server or network outage. In the future, we will be announcing a comprehensive monitoring package for your VPS that will allow you to keep a watchful eye on your servers’ health and connectivity.

Please let me know if you have any questions or concerns about these plans. Since Rails Machine began, I have worked hard to provide a high quality service for Rails hosting. I am sorry that the recent events have had a significant impact on that goal. I appreciate your understanding and patience.

Sincerely, Bradley Taylor Owner, Rails Machine

PS. Thanks to everyone who hung out in the Campfire room! It made the waiting somewhat less torturous. If you still don’t have a Campfire account, please request one via the support system.