[Hi Everyone,
What a weekend from hell!

AfricanCrisis was running so beautifully on the new SQL Server database. On saturday morning it looked as if I was completely through the worst of it! I was delighted.

I was in a rush to convert the remainder of the tables to SQL Server and I worked like crazy… and then in the rush, I made a mistake with a “where” SQL clause… and what should have been the update of a single record turned into an update of 30,000… for EACH person who was logging on. So if 10 people clicked on AfricanCrisis the database was churning away updating 300,000 records! The site ground to a complete halt through all the deadlocks!

But I only discovered this bug by sunday night. But I also decided to start switching off various “behind the scenes” processing so as to make the site run faster. I see this morning its looking quite good.

So what I will do from here onwards is just take it easy with the changes. I’ll give the site a few days to remain stable, before doing smaller sets of changes, to nurse it over on to SQL Server properly.

What is NOT helping me is my own stupid bloody dialup connection. I am so sick of my service provider. I could hardly connect to my own website. So I make changes and I get HTTP 500 errors when I am trying to run programs on it! I can’t even see my own programs! I tell you… there were times when I was wondering if I was the victim of some type of electronic warfare! But I was probably more likely the victim of just declining standards. I spent hours on the phone to my service provider in South Africa last week, and finally just threw my hands up in the air. They basically were ZERO help when it came to explaining why I was getting HTTP 500 errors on all my sites – and even on the Drudge Report.

Now in the middle of March 2006, I had placed an order for an ADSL line from Telkom. I’m still waiting. They say they had to upgrade the exchange in my area and that it is now done. So I *HOPE* ADSL will come soon. I really need it badly.

I also actually signed up for a dialup line with someone else on friday. I was so sick to death of my existing ISP. I’ve stuck with the same ISP for 13 years because of an email address I don’t want to get rid of because there are references to it on websites which can’t be changed… and I still get emails and feedback which I want. But finally I can’t take it any more. So getting a new dialup on friday, with much better response has helped me a lot. Though ADSL is hopefully the final solution. I’m quite efficient, and until recently, ADSL wouldn’t have bought me much, but nowadays I need it especially for backups.

So let me nurse the system along over the next 3 weeks. I hope to be able to resume posting news articles properly from tomorrow. I’ll monitor the errors and performance before making more changes.

There has been some discussion between myself and some friends of mine, several of whom have been talking about converting AfricanCrisis away from ASP to PHP – which means it can run on Linux – and Linux hosting is of course cheaper.

I have been toying with this as a long term strategy. I’m wondering how long ASP will be around – knowing that Microsoft is phasing out VB6. But I originally planned to only start work on moving to PHP later this year. I was thinking of moving AfricanCrisis to PHP AND at the same time rewriting its front-end completely as people have been urging me to do for years.

(If anyone wants to volunteer help with the graphics and the “look” of it – then drop me a line at: [email protected])

I have one volunteer who suggested that we use an existing free Content Management System (Jamoo?) as the basis for it. The problem is I have a *LOT* of back-end functionality in the site which people don’t see and which enables me to do a lot of things by myself without people seeing it. And that is highly customised stuff which can’t be rewritten easily. So, if the site moves, then it will be a long term project.

I am also looking at parts of the site which experience shows are important, and other parts which did not work so well and which won’t be missed much.

What I’ve asked my volunteer to do, is to do a test… and let him put up a prototype on a linux site. All it must contain to begin with are news articles.

Moving to PHP could be good because then finally it allows me to move the Forums on to a “professional” product – which I will need to purchase.

So as soon as some kind of “Proof of Concept” is available, I’ll link to it, and maybe we start running AfricanCrisis and an “African Crisis Mirror” site at the same time, and look at the results.

Then in time we can discuss the way forward, and if one or two volunteers want to help, maybe we can do some things together.

I am also curious about what PERL can do – not as the main driving force of AfricanCrisis, but more in terms of the type of things which .NET can do – I still have a dream of doing automated posting on to other sites by having a script which can log on to other sites, and can post (as if a human had done it).

So it could be that the PHP/PERL/Linux route is one I slowly want to edge towards.

With my web experience though, there is a certain concept I would have liked to try in PHP much along the lines of what I did on www.StraightTalk.co.za – and that is to actually drive the entire site off of a single php program. If you will go to www.StraightTalk.co.za you’ll see the links point to .txt files. In effect they’re really XML files – and it is all highly parameterised.

AfricanCrisis evolved over nights and weekends over the last 5 years, as my technical skills improved. So I have lots of old things – especially Photo Gallery pages – that were thrown together, which actually need to be turned into XML files or entries in a database. I have evolved a single type of table layout (behind the scenes), which can actually handle EVERYTHING … news, photos, video, etc… and I would like to take lots of the old stuff and convert it to the same format. But there is a lot of slog involved in that.

So what we might begin with is a type of mirrror site which we GROW and EVOLVE gradually… and see how it holds up… Linux will be new to me, though PHP is not that unfamiliar to me… it is similar to things I’ve seen before. So what I’ll try to do, is look at keeping both options open, while slowly evolving away from the one on to the other. But at the same time, we can make the RADICAL changes to look and feel which we’ve been wanting for a long time.

I want to discuss two thoughts:-
1. Volunteers: I have been urged to move AfricanCrisis away from a centrally controlled system to having a system of volunteers to help with the admi, replying to letters and general behind-the-scenes management. That is something I think could work well. Many others doing that – with FreeRepublic.com being the most successful volunteer organisation I have ever seen. FreeRepublic is an awesome concept. But to do this, one needs to have a back-end management system where you assign people tasks. BUT… given the kind of SABOTAGE which Adrianna Stuijt at CensorBugBear had – I would want to track the activities of volunteers and control the permissions I give to any people – so that if someone slips through the net and tries to sabotage the site we can identify the person immediately and kick them out. So I want to look at evolving towards more Volunteer control of the Admin.

2. AfricanCrisis “Open Data”: I have said many times that people are welcome to make copies of AfricanCrisis’s data in the event that something happens to me or my site. But maybe I can take this one step further by actually giving people the data in its raw format – for those with technical minds, who may want it. For example, for my volunteer, I gave him an XML file containing the entire news article archive (8,000 articles – 28Mb in size). If there is an interest in the news or photo archives, then I could easily consider making that “Public Domain” in the near future. The INFORMATION on the site has a lot of value, and as it keeps building up, I want to ensure that numerous copies of it exist in the hands of private people around the world in case something one day happens to me or prevents me from carrying on with the site.

So these are thoughts I am throwing around for everyone. Jan]