About irritation, inspiration, motivation and transpiration

In short

Irritation: Webforums and newsgroups are completely separated entities. There is no valid reason why they should be.
Inspiration: I wanted to do something about that
Motivation: Google's Summer of Code and Drupal helped me do it.
Transpiration: A lot has been done, and there's lots more to do.

Irritation

I study computer science at the Dutch Open University (OUNL). OUNL targets distance learning at university level. Naturally, they use the Internet for teacher-student and student-student contact. For years, they used newsgroups (netnews) on a private server. Each course has its own group, and there are various general discussion groups. In the spring of 2005, OUNL quite suddenly introduced a webforum, and announced that in due course it would replace the newsgroups. The main motivation was that accessing newsgroups caused too many problems, especially for new students, but also for some teachers. Many students who were already using the newsgroups protested. Apart from the obvious "why change what is good and what I'm used to arguments", they had more fundamental issues with the webforum: it lacked all facilities for selecting, sorting, filtering, archiving and other functionality that many newsreaders provide. Those in favor of the webforum simply stated that it looked good, or that they knew more webforums and felt happy about them. OUNL apparently had not anticipated the resistance, and did not proceed to close the newsgroups, leaving the students with two unconnected communication facilities.
I found this very unsatisfactory, but instead of chosing in favor of one of two, I wondered why newsgroups and webforums were not integrated, or could not be integrated. Scanning the internet for solutions, I found that several webinterfaces to newsgroups exist, with Google Groups as the obviously best known one. The integration of webforums with newsgroups however, seemed to be an almost (onontgonnen terrein), especially in the open source world.

Inspiration

I did not find a satisfactory solution for integrated webforums and newsgroups. What I did find was several online discussions, both in forums and in newsgroups, about this dichotomy. So the need was there, but so far nobody had created a working solution.
I decided to have a go at this myself, after all, I had to find an outlet for my irritation. As a software engineer and CS student should, I started with a
design. It's actually not that much of a design, it is just an informal sketch that I used to keep me on track.
What I had in mind was a website that lets its users access newsgroups. It should provide facilities for registering users, administering access, allow for posting, and, if I could manage that, allow users to sort and filter the messages that they wish to see.
I did not expect to finish something like this within a year. And though I hoped that OUNL would be interested, I had no guarantee for that. But I had become inspired, since I knew the need for a website like this existed elsewhere as well.

Motivation

I knew it was going to take me a long time to come up with a working solution: I have a fulltime job, study in my spare time, and I also want to spend time away from books and computers. In june 2005 I had the first version of the NNTP interface. That was the easiest part. A lot of similar code existed already, and protocol programming is not new for me. The user interface would be much harder, since I had very little experience with building web sites. Then, some time in june, one of my fellow students, Astrid Nijs posted a message in one of the OUNL newsgroups, mentioning Google's Summer of Code. I read about it, thought about it, gave myself a snowballs chance in hell to qualify, and almost forgot about it. Almost, because on the day of the deadline for submitting proposals, I squeezed out a proposal, and sent it to Google, together with the design sketch. It was much more an inspired idea than a decent specification of what I wanted to build. And I had to choose a mentoring organisation very quickly. Drupal looked nice to me. Well designed, extensible, and from Flemish origin.
A few weeks later, I was more than surprised when I saw the "Congratulations" email from Google. From some 8700 proposals, 412 were selected. Including mine! I was very happy, proud, and worried. It had to be finished within two months, next to work, vacation and study. Study? No, I decided I would not study until september, in spite of the plans I had recently made with my mentor at the university. The pay was a very nice push, the deadline a very good limit. I had started working on the project already, and I wanted to do it anyway, so the incentive from Google was a very welcome extra motivator.

Transpiration

I had to get started quickly. I did not know Drupal at all before I submitted my proposal, so I had to start by unravelling the internal workings of Drupal. At the start, I did not even really know if Drupal would enable me to easily do what I wanted to do. Fortunataly, Drupal proved to be at least as flexible and extensible as I needed. Also, my mentor Karoly Negyesi was very helpful, and got me on track very quickly.
Now, at the beginning of september 2005, there is a working netnews module for Drupal. It does not do all that I want yet, and it definately needs a lot more testing than I was able to give it. But the basic forum - netnews synchronization works.
Much more needs to be done, Googles Summer of Code may be over, but my transpiration phase will continue for months. albeit at a lower pace then during the summer. Deletion of messages should be handled, encoding needs work, attachments need to be handled, and not ignored, as is the case now.