Shared Community WiFi Networking Blog From A Toronto Co-op ISP

Monday, December 04, 2006

Continued Server Problems (Monday the 4th, 12 noon)

Continued Server Problems

As most of us know by now, the wireless login system has not been working since midnight, Saturday, December 2, 2006. (Just for member, though, that the wired Internet access should still be working for everyone -- even though the wireless is down, plug in to the router and you will be online!)

It is a frustrating time for everyone, and as we try to fix things we feel we're letting people down when the system is not working.

This is not the first time the wireless has experienced a problem in the last few weeks. In fact, today's problems are directly connected to a problem that began on the 12th of November, when a routine software upgrade on the server started causing all sorts of problems.

Installing the software patch was not really optional -- it was supposed to fix a serious security hole in the server's operating system, and while it appears it did fix that security problem, the update changed many things in strange ways and has proven to be incompatible with the MySQL database, which all sorts of other software on the server that is needed for wireless login access, e-mail, and voicemail.

Several times, now, Steve and Ron have fixed the damage to the MySQL database caused by the software patch, but each time, the database somehow gets damaged again. When the database is damaged, there is no way for the web site to figure out who should login, what passwords match with what usernames, and so on, and things just don't work.

Unfortunately, this means there is a deeper problem with the system, and it will take some real work to get it working properly. No one thought it would get this bad when problem first appeared -- and even after it happened a second time, we figured it was just a minor glitch and could be fixed relatively easily. Obviously, we were wrong, and obviously, it is a more serious problem for the server and for all of us.

All is not lost, however.

First, everyone still has Internet access, but they have to plug into the router to get online. The wired connections use a different system from the wireless, so no one is without Internet access, even though plugging in can be inconvenient.

Second, Steve and Ron will hopefully have the database patched again and working in the next few hours, even though that fix will also be temporary, as we now know.

Third, Steve and Ron will also be tearing the server's software apart looking to see what is actually going wrong deep inside. This will be many hours of work, but is doable with lots of patience and lots of coffee.

In the meantime, if someone needs to reach us urgently (not just to tell us the wireless is working!), send an e-mail to our backup e-mail address which is:

contact -dot- wirelessnomad -at- gmail -dot- com

Sorry for all the trouble, but rest assured fixing this for now -and for real- is our top priority.

Sincerely,
-Damien

10 Comments:

Anonymous Anonymous said...

As much as I can understand your position, the fact remains I am not getting what I paid for. User patience will only go so far before you start losing users, and that is not tenable with tour business model... I hope you are taking this as seriously as your customers!

1:47 p.m., December 04, 2006

 
Anonymous Anonymous said...

I do not want to put more straws on the fire but just something to keep in mind after this problem is fixed. There are some other services that do not work properly too (for a while I may say) that may be related or not with the mysql problem. The ns1.wirelessnomad.com is not synchronized with ns2.wirelessnomad.com (looking from outside wirelessnomad.com). It is stuck now since October. Would you please check is on your to do list or turn ns1.wirelessnomad.com off. Thank you.

4:14 p.m., December 04, 2006

 
Anonymous Anonymous said...

I guess it is easy to speak when things go wrong but not that easy to praise what does work well, so I thought I would contribute by saying that besides this, which to me is just a little glitch, I have always been very happy with the service and very committed to the goals of the co-op. I understand computers don't always behave and in fact I do have lots of patience for WirelessNomad. These guys have taken on a huge project, we can't expect it will always be smooth and easy, I have no doubt this will be solved, so I would encourage others to be patient or be helpful... after all, we are all part of the same co-op, right?

1:26 a.m., December 05, 2006

 
Anonymous Anonymous said...

Two things
1) This should be on the WirelessNomad main page. We can't read the blog unless we've plugged in.

2) If the problem is the database of passwords and IDs, get rid of the database for the time being. Let everyone on.

In fact, you already did this: cap our speeds at the free level if necessary, and let me use my computer at my desk again.

9:13 a.m., December 05, 2006

 
Anonymous Anonymous said...

I understand that outages occur occasionally on all networks. However, this is not Nomad's first glitch and to have no service since Saturday is a huge deal for me. I do not have the ability to plug my desktop into the router because of the layout of my apartment... so essentially this leaves me without any service.
I believe a discount to the paying members for the lack of service and inconvenience should be in order.

Dave C

12:00 p.m., December 05, 2006

 
Anonymous Anonymous said...

May I also suggest that the blog be moved away from blogger and integrated into your pre-login accessible area on wirelessnomad.com. Being a new customer for the past couple of months it is hard to be in the know without status updates, and no router/modem to hardwire into.

If you have a timeline of status updates to let us know how things are coming that would be great. Again, I understand the massive amount of work you have to do and also fielding customer questions that I am sure have been bombarding you.

Thank you and I want to support Wireless Nomad going forward.

2:13 p.m., December 05, 2006

 
Anonymous Anonymous said...

So, what's the status? Is this ship sinking?

10:25 p.m., December 06, 2006

 
Blogger Damien said...

Yes, I/we are just as frustrated as you all are. We are working on it as best we can, at least the DSL is rock-solid.

This blog appears on the home page as well, updating automatically. It is hosted here to be redundant.

Sincerely,
-Damien

2:38 p.m., December 11, 2006

 
Anonymous Anonymous said...

ns2.wirelessnomad.com is gone too.
Time for a dynDNS account. This is an unfortunate situation given Christmas is close and friends are waiting.
Keep up the good work, we are all waiting. Thank you.

1:51 p.m., December 12, 2006

 
Blogger Damien said...

thanks for the encouragement-- its been hard for everyone, can't wait to get it fixed.

3:07 p.m., December 12, 2006

 

Post a Comment

<< Home