Monday, January 29, 2007

Generate dynamic content with Tomcat and MySQL

Create your own Java-based rotating banner advertisements

Companies like doubleclick.net have made a lot of money serving banner ads on the Web. The service they provide is great, but why pay for something you can do yourself? In this article, enterprise Java consultants Javid Jamae and Kulvir Bhogal demonstrate how to create rotating banner ads using an all open-source environment: Apache Tomcat, MySQL, and the MM MySQL JDBC driver. First, they'll walk you through the necessary setup in Tomcat and MySQL, and then show you how to install the MM MySQL JDBC driver to allow a Java servlet running in Tomcat to communicate with MySQL.

Banner advertising has been around since the Internet was first transformed from an educational and governmental information store into a centerpoint for international commerce. A rotating banner is an allocated space on a Web page, in which the space is used to display an advertisement -- either randomly or based on some business logic -- every time the Web page is loaded or reloaded. The programs that drive rotating banner ads are fairly simple, but they are important advertising tools. Just like their half-minute radio and television counterparts, these dynamic advertising tools allow a single Web page to display advertisements from various sources and enable different companies to create brand recognition for their products and services while targeting the same audience.

Regardless of how you view them (yes, we all find them annoying at times), Web banners have become a way of life on the Internet. The fact is that the Web constitutes a gargantuan audience of consumers, whose money keeps the wheels of e-business turning. In the short history of Internet marketing, e-business owners have shown they are willing to pay top dollar to place their banner ads on high-traffic sites.

Some companies, such as doubleclick.net, have capitalized on the Web banner phenomenon, acting as an application service provider (ASP) to perform such services as tracking clicks for specific banner advertisements. These ASPs then inform advertisers on how effective their Web advertisement campaigns are.

Of course, ASPs such as doubleclick.net cost money. If you're like us, you don't like spending money when you could get the service for free. Wait a second -- you've probably heard that nothing in this world is free. But don't worry. All this article will cost you is your time. In fact, we'll show you how to tie together an open source (that is, free) environment to set up your own, powerful Web banner tracking system. To pull off this task, our weapons of choice will be Tomcat, MySQL, a single Java servlet, and a few helper classes. Excited? Well then, let's jump into the software installation.

Setting up Tomcat and MySQL

In this section, we will walk through the setup for both Tomcat and MySQL. We'll then show you how to install the driver you need to enable these two applications to communicate.
Setting up Tomcat

Download and install Tomcat. For this article, we used Tomcat 4.1 for Windows, which comes with a nice install package and creates icons and a Start menu folder for you. It also creates a Windows Service to start and stop the Tomcat server. Installation should be very straightforward, but if you have trouble, refer to the Tomcat documentation. Because of the popularity of Tomcat, there is also ample help available on newsgroups and on the Web, some of which we've listed in Resources.

After you install Tomcat, there are a few steps you need to complete to set up our rotating banner Web application. First, we will create a subdirectory called banner under the [installdir]\webapps directory. Under the banner subdirectory, we will then create the standard Web application directory structure:


[installdir]\webapps\banner
[installdir]\webapps\banner\WEB-INF
[installdir]\webapps\banner\WEB-INF\classes
[installdir]\webapps\banner\WEB-INF\lib

Next, we'll add a context to point to our Web application. A context is merely an alias, telling Tomcat where to access our Web application. Our context path will be /banner, and it will point to the banner subdirectory we just created. When a user enters http://localhost:8080/banner, he will be taken to our top-level banner directory under webapps. If he wants to run our BannerServlet, which will exist in the WEB-INF/classes directory, he would use http://localhost:8080/banner/servlet/BannerServlet.

To add the /banner context, we first need to edit the server.xml file in the Tomcat conf directory. Go nearly to the bottom of the file where you will see several context tags. There should be one for /admin and one for /examples. Add the following context tag:



debug="0" reloadable="true" crossContext="true"/>

After you've added the context tag, restart Tomcat so that the changes to the server.xml file take effect (in our case, we just restarted the Windows Service that Tomcat installed).
Setting up MySQL

MySQL is a powerful database for the price you pay for it (nothing), and many companies use MySQL to handle their data. The number is growing daily as companies with low budgets enter the Web market. The open source community has greeted MySQL with open arms. The documentation about this powerful database is abundant, and there are both Linux and Windows versions.

Download and install MySQL with the "Typical" setup option (for this article, we'll assume you are working with the WinNT version of MySQL). After you've completed the installation, you will notice one of the annoying aspects of MySQL: it doesn't place anything in your Start menu. You will need to go to the directory where you installed the database (c:\mysql\, by default), then go to the bin directory, where you will find the executables to run MySQL.

First, double-click the winmysqladmin.exe file. The first time you open this file, you will be asked to enter a username and password. Next, right-click on the street light icon that will appear in your taskbar. Go to WinNT and choose "Start the service" to keep MySQL running in the background. Finally, double-click the "mysql.exe" icon to start "MySQL Monitor," where you will work with MySQL.
Getting MySQL and Tomcat to work together

Getting MySQL and Tomcat to communicate with one another can be difficult. With the JDBC API, however, we'll be able to use SQL to talk to a MySQL database from our Java classes with relative ease.

We'll use the MM MySQL JDBC driver, an open source driver, to facilitate communication between MySQL and Tomcat. (At the time of this writing, 2.0.14 is the latest version.)

Unfortunately, setting up this driver is a little tricky. First, download the appropriate JAR file for the driver from here. We downloaded the file called mm.mysql-2.0.14-you-must-unjar-me.jar. Next, unjar (or unzip) the file to a temporary directory. Finally, copy the file that contains the driver from the unzipped directory structure into your WEBAPPS/BANNER/WEB-INF/lib directory and restart Tomcat. In the version of the driver we downloaded, the file is called mm.mysql-2.0.14-bin.jar.

We could have used the JDBC/ODBC bridge driver to communicate with MySQL, but we've assumed that a native driver would provide more of a performance advantage (though we haven't run any benchmarks to prove our assumption). For this application, it probably wouldn't make much of a difference in performance, but we decided to demonstrate how to use the native JDBC driver so you wouldn't have to figure it out when you are designing a larger application.

The rotating banner application

Now that we have installed all of our software, we'll go over what our application is capable of and how we have architected and developed it.

There are essentially two actions you can perform using our banner servlet. First, you can use it to view a random banner image on a Web page, which occurs every time a Web page is loaded that contains the banner ad. Second, you can click on the banner image, which will forward you to the link corresponding to the image that was loaded.

In terms of HTML, the code looks similar to this:







If we want to load a random image, our image tag obviously can't point to a static image file, so we will direct it to run a servlet, which we'll call BannerServlet. We will use an HTTP GET method parameter to direct our servlet to give us an image. So our image tag will look like this:




This tag calls our servlet and passes in the parameter key-value pair type=image. The servlet's service() method interprets this request and returns a random image to the browser. Of course, the servlet must somehow remember which image it sent to the client so it knows where to go when the client clicks on it. We'll store the metadata related to the image that we sent on the client's session so that when the user clicks the image, the metadata from his session will load, redirecting him to the appropriate URL.

Our link tag will look almost the same as our image tag:







When the servlet is called with the type=link key-value pair, the servlet grabs the banner's metadata and reroutes the user to the appropriate URL.
Code and CLASSPATH setup

To use the code provided with this article, you must first unzip the zip file (in Resources) and compile the .java files using the command line javac compiler or your favorite IDE. To compile the code, set your CLASSPATH with these two JAR files:

* mm.mySQL-2.0.14-bin.jar (or whatever version you get from the MM MySQL Web site)

* servlet.jar (if it doesn't come packaged with the JDK you are using)

Copy the compiled .class files into the [tomcat_installdir]\webapps\banner\WEB-INF\classes directory we created earlier. The image files and the .htm file we provided as an example must go in the [tomcat_installdir]\webapps\banner directory.
Database setup

The database portion of our application is merely used to persist metadata about each banner in our system. In other words, we are not actually going to store the image files in our database, but rather a reference to each image file. In our database, we will use seven columns to describe each banner ad.

The descriptions in Table 1 show you what each record will contain. We will actually only use five out of the seven database columns in the application. CustomerName and NumberOfClicksPurchased aren't used in our version, but we put them in as placeholders for expandability. You could very easily extend our application and use it as a real-world business application where customers would pay for a certain number of clicks per banner.

Of course, in a real-world environment, you would have more than one site banner. Depending on how much your banner "sponsor" paid you in comparison to other sponsors, you might want his banner to be shown less or more. The BannerWeight fields will be used to implement this functionality. We have implemented a very simple weight system where the percent chance for each banner to be displayed is:


(BannerWeight / Sum of all BannerWeights) * 100

Translating what was just stated in SQL, you would issue the following statements using the MySQL Monitor:


mysql> create database BANNER;

To connect to the database, you enter:


mysql> use BANNER;


Next, we create our table:


mysql> create table ADS
(IMAGEFILE VARCHAR(50) NOT NULL,
URL VARCHAR(50) NOT NULL,
CUSTOMERNAME VARCHAR(50),
NUMBEROFCLICKSPURCHASED INT(4),
NUMBEROFCLICKSREMAINING INT(4) NOT NULL,
NUMBEROFIMPRESSIONS INT(4) NOT NULL,
BANNERWEIGHT INT(4) NOT NULL);

Friday, January 26, 2007

Boffins analyse open source development process

US researchers have secured a $750,000 three-year grant to study how open source software such as the Apache web server is built.

The group from UC Davis, which was awarded the funding from the National Science Foundation, pointed out that, unlike the construction of a cathedral or an airliner, there is no 'blueprint' for a piece of open source software that shows all the parts in relation to each other.

Computer software systems are now among the most complex, expensive items ever created by humans, and some of the most sophisticated are being built by teams of volunteers as 'open source' projects where any programmer can read the code and suggest improvements.

Premkumar Devanbu, a professor of computer science at UC Davis, explained that the open source development process typically sees a small group of programmers look after the core elements of the system with the rest being broken up into modules that are attacked by a floating group of volunteers.

He added that open source defies conventional wisdom about collaborative projects. For example, most office workers know that the slowest member of the team sets the pace for everybody else.

But in open source projects, work moves at the speed of the fastest member of the team, and adding more hands accelerates rather slows the project.

The researchers will focus on the Apache Web Server, the PostgreSQL database and the Python scripting language.

They will collect information from message boards, bug reports and email discussions to understand how design teams organise and interact.

Professor Devanbu believes that the way teams are organised is reflected in the resulting software. At the same time, the structure of the software will have an effect on how teams of programmers are put together.

For example, software broken into large chunks of code might need a different approach than a structure of smaller chunks.

Thursday, January 25, 2007

Why Use Open Source Technology

Introduction

You have probably heard something about "open source" and "free" software solutions such as Linux, Perl or Apache. Many people have the mistaken impression that open source software is not for serious or large web sites. Let's dispel that myth right now. Etoys.com was the third busiest e-commerce site during the 2000 Christmas season. They served over 2.5 million page views and processed 20,000 orders per hour. Etoys was built using Perl running the Apache server under the Linux operating system, all open source software tools. Interested in learning more? This article will teach you about the benefits of using open source software.



What is Open Source Software?

Open Source software generally is distributed under a license that guarantees the right to read, modify, redistribute and use the software freely. Open source software may be developed by community of programmers interested in developing a software application for a specific purpose. Companies may also develop open source software. These companies will distribute their software for free and make their money from support contracts and customized development.

Much of the open source software is distributed under the GNU General Public License (GNU GPL). The GNU GPL allows you to copy, use, modify, redistribute the software but prohibits companies or individuals from making modified versions proprietary. Richard Stallman, a McArthur genius award recipient, developed this license, in order to encourage the development of a software sharing community. Tens of thousands of developers and several large corporations such as IBM, Sun Microsystems, and Intel have chosen to participate in the open source software movement. Some of most successful and robust software on the web today has been developed under this license.

Open Source Software Dominates the Web

The web is dominated by open source software solutions.

Did you know that over 58% of the web is using the Apache Web Server. This is compared to 28% for Microsoft’s IIS Web Server.

Linux and Free BSD, both open source flavors of UNIX, are the dominant operating system for web servers, not Microsoft Windows 2000 or Windows NT. Linux has a 34% market share for web server operating systems. Microsoft Windows has a 23% market share (source: Linux Today: www.linuxtoday.com)

The most widely used language for web programming is Perl, not Microsoft’s Active Server Pages. Perl has been often referred to as the "glue" that holds the Internet together.

Over 60% to 80% of e-mail travels across the Internet using the open source program, Send Mail.


Source: E-soft inc.: www.securityspace.com

Open Source Software is Used By Some of the Largest Web Sites

Some of the most popular and most successful high traffic web sites use open source software solutions. (source: Netcraft: www.netcraft.com)
yahoo.com is running free BSD (free Berkley Standard Unix)
amazon.com is running Linux, the Apache Server, and the e-commerce system is written in Perl.

google.com (the worlds largest search engine) is running Linux
altavista.com is running Linux
mp3.com is running Linux and Apache
aol.com uses AOL server which is open source.
cnn.com uses Perl

Why do these large web sites use open source solutions? Because many open source software solutions such as Linux and Apache are rock solid. Others such as MySQL database, PHP and Perl programming languages have a large user community with wide range of support.

Benefits of Open Source

Cost: Open Source software available under the GNU GPL license is free. In some cases, you may choose to pay for the distribution (compiled version that contains installable executable software on CD ROM). The cost of the distribution is generally trivial compared to the cost of many enterprise level commercial offerings. In addition, the developers of many of the open source solutions offer support contracts that are suitable to all levels of business or organization.

Software Source Code: When you purchase a license to use most commercial software, you are dependent on the software designer to add features or customize the software for the needs of your business or organization. The software manufacturer provides you with only the executable program. You do not have access to the source code. With open source software, you are free to modify the software and customize it in order to suit your application.

Scalability and Robustness: a large community of highly skilled software developers has created open source solutions, such as Linux, Perl, and Apache. As you can see from our examples, open source software is used across a full spectrum of web sites. Open source UNIX based operating systems such as Linux and FreeBSD are extremely robust and efficient as they are suitable for both small and large organizations.

Large Support Community: a large community of developers that communicate through on-line discussion groups supports many open source offerings. This allows common problems to be easily solved and bugs to be quickly exposed and fixed.

Security and Protection of Proprietary Data: There is a myth that open source software is more vulnerable to attack than proprietary solutions. Actually, the opposite is often true. Because the source code is exposed, it is often easier for a security minded software community to close security holes or breeches.

What Development Platforms are Used By Celestial Graphics?

Celestial Graphics uses a combination of commercial development tools and Open Source Software. We have chosen a combination of tools that will help us develop the best and most cost effective web site designs for our clients.
Web Site Development Tools

We use Macromedia Dreamweaver, Fireworks and Flash. Dreamweaver is used by over 70% of web design professionals (source: PC Data).

Web Programming Language

Celestial Graphics uses PHP for Server Side application development. PHP is an open source solution built from the ground up specifically for web applications. PHP enjoys a large developers community. PHP runs under Linux and can be compiled as an Apache module. This makes PHP very fast. There are over PHP is one of the fastest growing languages on the web. Over 5 million domains (about 800,000 IP addresses) are running PHP (source: http://www.php.net/usage.php). We choose PHP because it has the flexibility of Perl but was built from the ground up for web application development. PHP is fast, robust and scalable.
Database

We use MySQL. MySQL is an open source high performance relational database management system (RDMS). Other rational database management systems include Oracle, Informix, Sybase, and SQL Server. MySQL is widely used on the web and has Interfaces (APIs) to most programming languages including C++, Java, PHP, Perl as well as OBDC.

How does MySQL stack-up against commercial offerings?
In terms of raw speed, MySQL benchmarks faster than many other databases such as Microsoft’s SQL Server and performs favorably against industry heavyweights such as Oracle. MySQL is fast because it was designed primarily as a web based relational database management system.

Who Uses MySQL?

MySQL is used Yahoo, Slash dot, and Linux Today. In addition, MySQL is used as a backend database at NASA and NASA Kids web sites.
E-Commerce Solutions and Content Management Systems

Interchange: Red Hat’s Interchange is a widely used e-commerce shopping cart and web based order management system. Originally called Minivend, this software has features rivaling systems costing well over $100,000 yet is available for free under the GNU GPL license. Interchange contains full-featured web based administration of order entry, inventory, product, content and customer management. Interchange connects to a wide variety of databases including MySQL and Oracle.

EZ Publish: This is a Norwegian based highly flexible and complete content management system. EZ Publish allows you to separate your content from your web site design. The web site design is completely template based for flexibility. Content can be uploaded through a web interface. Here are some of the features:

Community Building: User Polling, Moderated Discussion Groups, User Registration and Management, and Calendar of Events.
Commerce Module: Internationalized on-line shopping system.
Banner Advertising
News Feeds
On Line Statistics
Database Driven with Search Engine Friendly URLs

Tuesday, January 23, 2007

User Driven Modelling - Detailed Explanation - Part 3 - Criteria Necessary

Why a different approach is needed

Translating concepts into an implementation is difficult. The difficulty of trying to explain the subjects of interest in a call for papers, or proposals illustrates this problem. Because of the ambiguity of words, there is always going to be a problem of interpretation between those who specify the requirements, and those who need to understand and interpret them.

For software development, a good way to reduce the level of misunderstandings is to go through the loop from concept to design to implementation quickly and efficiently so that feedback can be returned from the software model. Then mistakes can be seen and corrected quickly. It becomes much easier to achieve this high speed development, if the interface for development is made sufficiently easy to understand, so that a domain expert can use it to create the software, or at least a simple prototype that a developer can then work with and improve. Even if the aim of the users is to specify requirements rather than create programs, creating working programs conveys requirements much better than any other form of requirement specification.

It may also prove possible to work in reverse from implementation to design, or design to conceptual model. A UWE paper explains how ontologies could be mapped to conceptual models, El-Ghalayini Et al. [1]. This process can be made easier if the same open standard software representations, languages, and structures are used throughout this process. This would be useful for checking software is designed well or re-using software designs.

User involvement is important in the development of software but a domain expert does not necessarily possess expertise in software development, and a software developer cannot have expertise in every domain to which software might apply. So it is important to make it possible for software to be created using methods as close as possible to that which the domain expert normally uses. The proportion of domain experts in a particular domain (aerospace engineering) for example who can develop their own programs is fairly low, but the proportion that are computer literate in the everyday use of computers is much higher. If this computer literacy is harnessed to allow the domain experts to develop and share models, the productivity for software development will be increased and the proportion of misunderstandings between domain experts and developers reduced. The domain experts can then explore a problem they are trying to solve and produce code to solve it. The role of developers would then become more that of a mentor and enabler rather than someone who has to translate all the ideas of experts into code themselves. Other developers may work at providing better translation software for the experts.

User Driven Model Development

The intention of the research into User Driven Modelling (UDM) and more widely User Driven Programming (UDP) is to enable non-programmers to create software, from a user interface that allows them to model a particular problem or scenario. This involves a user entering information visually in the form of a tree diagram. The research involves developing ways of automatically translating this information into program code in a variety of computer languages. This is very important and useful for many employees that have insufficient time to learn programming languages. To achieve this, visual editors are used to create and edit taxonomies to be translated into code. To make this possible, it is also important to examine visualisation, and visualisation techniques to create a human computer interface that allows non-experts to create software.

The research mainly concentrates on using the above technique for modelling, searching and sorting. The technique should be usable for other types of program development. Research relevant to User Driven Programming in general is covered, as this could be applied to the problem in future.

This research unites approaches of object orientation, the semantic web, relational databases, and event driven programming. Tim Berners-Lee defined the semantic web as 'a web of data that can be processed directly or indirectly by machines' [2]. The research examines ways of structuring information, and enabling processing and searching of the information to provide a modelling capability.

UDM could help increase user involvement in software, by providing templates to enable non-programmers to develop modelling software for the purposes that interest them. If more users of software are involved in creation of software and the source of the code is open, this allows for the creation of development communities that can share ideas and code and learn form each other. These communities could include both software experts, and domain experts who would be much more able to attain the expertise to develop their own models than they are using current software languages.

Criteria necessary for User Driven Model Development

This section explains the theory behind the User Driven Modelling approach, and the factors necessary to make this approach possible. For this research the focus is on combining the development of dynamic software created in response to user actions, with object oriented, rule based and semantic web techniques. Research has examined ways of structuring information, processing and searching this information to provide a modelling capability. Research by Aziz et al. [1] examines how open standards software can assist in an organisations collaborative product development, and Wang et al. [2] outline an approach for integrating distributed relational database systems. Our automated production of software containing recursive Structured Query Language (SQL) queries enables this. This approach is a type of very high level Meta-programming. Meta-programming, and structured language is explained by Dmitriev [3] and Mens et al. [4]. The approach proposed is intended to solve the problems of cost and time over-run, and failure to achieve objectives that are the common malaise of software development projects. The creation of a web based visual representation of the information will allow people to examine and agree on information structures.

Firstly it is necessary to find a way for people with little programming expertise, to use an alternative form of software creation, that can later be translated into program code. The main approach taken was the use of visual metaphors to enable this creation process, although others may investigate a natural language approach. The decision on what combination of diagrammatic or natural language to use in the representation may be influenced by the type of user and the domain to be modelled. Engineers usually deal with diagrams as a regular part of their work, so understand this representation particularly well. In fact developers also use metaphors from engineering diagrams in order to provide a user interface for software design. This is explained in Tollis [5].

A translation method can then be provided that converts this representation into program code in a number of languages, or into a Meta-language that can then be further translated. In order to achieve this, it is necessary for the translator to understand and interpret equations that relate objects in the visual definition and obtain the results. In order for the user to understand the translation that has been performed it is then important to visualise the translated code, and this must be accessible to others who use the translated implementation. Web pages are a useful mechanism for this as they are widely accessible.

This visualisation of results is essential to express clearly their meaning. Words in a report document can be ambiguous. So the relationship of results to inputs must be clearly shown.

1 El-Ghalayini, H., Odeh, M., McClatchey, R., Solomonides, T. (2005). Reverse engineering ontology to conceptual data models, http://www.uwe.ac.uk/cems/graduateschool/news/posters/conference/conference2005.pdf, Graduate School Conference, 114-119.

2 Berners-Lee, T. (1999). Weaving the Web, Orion business - now Texere, http://www.w3.org/People/Berners-Lee/Weaving/Overview.html.

3 Aziz, H., Gao, J., Maropoulos, P., Cheung, W. M. (2005). Open standard, open source and peer-to-peer tools and methods for collaborative product development, Computers in Industry, 56: 260-271.

4 Wang, C.-B., Chen, T.-Y., Chen, Y.-M., Chu, H.-C. (2005). Design of a Meta Model for integrating enterprise systems, Computers in Industry, 56: 205-322.

5 Dmitriev, S. (2004). Language Oriented Programming: The Next Programming Paradigm, http://www.onboard.jetbrains.com/is1/articles/04/10/lop/.

6 Mens, K., Michiels, I., Wuyts, R. (2002). Supporting Software Development through Declaratively Codified Programming Patterns, Expert Systems with Applications, 23: 405-413.

7 Tollis, I. G. (1996). Graph Drawing and Information Visualization, ACM Computing Surveys, 28A(4).

Understanding OsCommerce

The Internet has become a popular place to do business. Now that there are several options available to everyone, everywhere, the need for viable options is imminent. Likewise, companies offering their services online, whether exclusively, or in conjunction with their traditional means of operation, need ways to quickly and efficiently do business, while making sure their customer information is safe and secure. This is why it has become necessary to find an OsCommerce solution.

OsCommerce is an online solution for E-Commerce that is currently under ongoing development by the open source community. This means that it is continually worked on by those in the industry who understand its need and are able to experiment to find out the best solutions. Through the open source community, information is shared in order to provide optimal assistance toward solving both common and uncommon problems that may arise. It also allows store owners to set up, run, and maintain their online solutions with little to no effort, which enables them to concentrate more on the business aspect than the technical issues.

The best part of OsCommerce is that it is free. There are no licensing fees or costs involved, which is often one of the most attractive characteristics of open source solutions. OsCommerce combines open source solutions together to provide a free and open E-Commerce platform, which includes the PHP web scripting language, a stable Apache web server, and the MySQL database server. This scripting language is powerful and the server fast, which allows for more seamless transactions to take place.

There are no restrictions or special requirements with OsCommerce, and it is, therefore, able to run on any PHP enabled web server, along with a variety of supported environments. This broadens the spectrum for OsCommerce to work on several operating system environments and in multiple situations.

OsCommerce began in 2000, and has now matured into a solution that is currently powering a wide variety of online shops around the globe. The services it offers are helping business owners in many ways, and are making it easier for them to concentrate on keeping their business up and running, while serving a constantly growing number of customers on a daily basis. It has also made it easier for potential customers to find businesses, and to purchase products and services more quickly and efficiently. It is serving both the business owner and the customer in ways that will only help E-commerce get better.

What's the Fuss over FOSS?

What’s the fuss over FOSS?

IN an industry where acronyms are de rigueur, it seemed inevitable that an old computing concept would get a new name.

Last November at the United Nations World Summit on the Information Society (which goes by its own unwieldy acronym WSIS), FOSS was on everyone’s lips as a way of bridging the divide between technology haves and have-nots.

FOSS stands for “free and open source software,” and it’s increasingly seen as the answer to everything from piracy to the lack of computing resources in Third World countries.

Free software that just anyone can copy or download over the Internet? What a crazy concept! Would businesses want to use software that nobody sold or supported?

The surprising answer is, many of them already do—without knowing it.

Indirectly, anyone who sends e-mail or browses the Web is using free and open source software because that is what powers most of the Internet.

In June 2006, more than 6 of 10 Web sites were on servers using Apache HTTP Server, an open source program running on Linux, which is also open source. That means if a company has a Web site, chances are good that it’s hosted on an open source system.

Anyone using Firefox (186 million downloads and counting) to surf the Internet is also using open source software.

Despite these success stories, few people understand why or how free software works.

The concept of free software isn’t all that new. In fact, before software became a commodity, Unix hackers in universities would routinely share their source code – human-readable instructions—with each other, copying and adapting them freely. Programmers expected to be paid for their work, not for the programs themselves.

Out of this environment came Richard Stallman, a programmer at the Artificial Intelligence Lab at the Massachusetts Institute of Technology. Stallman valued this collegial approach and argued that software users should have the freedom to share programs with others and be able to study and make changes to the software that they used. Restricting these freedoms, he believed, would hurt the larger community by limiting the benefits that the software could bring.

Dissatisfied by the limits that companies were putting on software, including Unix, he set out in 1983 to create a free operating system that he called GNU – a recursive acronym for Gnu’s Not Unix—and invited other programmers to help. In 1985, he created the Free Software Foundation to support the development of free software.

Out of these efforts came a complete set of programming tools but the core of the operating system – called the kernel – proved difficult to complete. Into this gap stepped Linus Torvalds, a student at the University of Helsinki. In 1991, Torvalds released a kernel that used libraries and tools developed by the GNU project and released it on the Internet, inviting feedback and help from other programmers. Remarkably, developers from all over the world responded and helped Torvalds create and refine his free operating system.

Today, Linux and other free or open source programs have made significant inroads in a world accustomed to proprietary software.

“The Codebreakers,” a BBC documentary in May 2006, shows how different countries have used FOSS to narrow the digital divide. These include school projects in Namibia, Spain and India, computer kiosks in South Africa, a disaster management system in Sri Lanka, and an environmental database system for the Galapagos.

One of the most high-profile efforts, however, is Brazil’s program to support the use and development of free and open source software in its government offices. The move saves Brazil $150 million a year in software licensing costs, which means more money to buy computer hardware for technologically deprived areas. Nobody expects FOSS to replace proprietary software soon, but given the substantial savings and benefits it would bring, shouldn’t the Philippine government take a closer look?

Open Source Content Management Systems - An Overview

Publishing documents on the World Wide Web can seem daunting for the self proclaimed ludites out there. Let's face it HTML, XML, CSS, JavaScript to name just a few of the technologies that you would need to be familiar with in order to develop a truly professional looking website for yourself or your business. Fortunately there is a solution to the problem, it is called a content management system and it is an easier way for just about anyone willing to learn to publish to the Internet.

Some of these content management systems have stemmed from the need of publishing data to the web. CNet for example created one of their own in house content management systems which they later spun off into a separate company called Vignette. Since publishing to the web began to take off in 1995 the development of these systems has really begun to progress. However my focus will be only on open source versions of the web content management systems.

The first open source system that I want to mention is called DotNetNuke (http://www.dotnetnuke.com) which is a AsP.Net based system. By far one of the more popular open source projects out there today DotNetNuke has an avid following of hundreds of thousands of users. The benefits of using this system is the rapid updates to the platform and the intense support you will receive from it community. These are two very important factors when considering an open source system, the level of updates and the support you will receive from its creators or users of the platform. Further the innovations that are built into this system make it one powerful application by any measure. For instance the skinning mechanism is by far one of the easiest to modify and customize and the scheduling processor reduces some of the more mundane tasks of maintaining any website.

However DotNetNuke does not come without any drawbacks. Since this system was developed in ASP.Net this means that you will need at least Visual Studio 2003 Professional or greater in order to truly modify the source and customize the system as you see fit. This can be a serious drawback as the cost of this development platform can run into the hundreds of dollars.

The second open source system that I have some experience in using is called AXCMS.Net (http://www.axcms.net) which is again built upon the .Net platform. This system is as feature rich as other open source systems such as DotNetNuke. However, this system has some definite drawbacks. First is its somewhat difficult setup and deployment problems that can seriously hinder any project. Also, there does not seem to be as an avid user base as DotNetNuke. However the system is as "solid" as they get and you will definitely be rewarded by your efforts once you have the system fully setup and ready to use. Also since this system is really a neat way to get introduced to the development team any updates or customizations you may need will be for a fee from the creators of the system. This really hinders the community support of the system and makes adoption that much harder.

There are many other open source systems out there with some more being developed as I write this article. The main factors when adopting any open source content management systems you should consider are: how well is this system supported by the community and how often is this system updated by its core development team. Even an open source system can have some hidden cost that must be considered before adoption into any business or enterprise.

A Corporate Web Site At An Affordable Price

To have a web site which is flexible, professional in design, can have additional functionality added in wherever is needed, is compliant in web development standards and can be controlled by the owner is no longer a corporate reality but is an affordable reality.

By using open source software the cost of such an affordable website has dramatically reduced. Open source software is free, built by an ethically sound and forward thinking community determined to better the functionality of the Internet. Whilst it is free, one should always try help those helping you by donating to open source software so that everybody benefits.

Just because the software is free, this does not mean that the website is free, the web site designer must still have knowledge of the website system and will be able to add a design and functionality tailored to your requirements. Building a website to deliver what you need is never an easy task.

The benefits of using open source software included affordable design changes without disturbing the content. Regular upgrades to improve the speed and security of the web site, additional modules allow a plug in mechanism for additional functionality to improve the use of the website, web site portability, reduced bespoke programming costs and probably most importantly, a support network which can aide in system issues and troubleshooting.

Easy Design Changes

Most open source software systems provide compliant website standards, this invariably means that the content is separated from the presentation giving two main benefits. The first of these is that the web site can provide slick content to be provided to the search engines and secondly since the design is independent of the content then the design of the whole site can be changed more affordably.

Regular upgrades

These open source software systems just grow from strength to strength, the development team expands, new ideas come into play and the software evolves version after version.

Support forums

Support forums of all levels up for the more popular open source solutions and communities develop within them. Whether you want to know how to do something in an administration panel, need help on best practices or require technical information on coding, more often than not, there is an answer for you on one of the many forums by people willing to share information and help you.

Additional functionality

The core functionality of many of the open source software systems is probably more than most people require. But a huge benefit Is the ability to add additional functionality. If there is something your website needs to do, very likely the more popular open source software has a configurable module or component of some sort with the ability to do it. If it does not then a bespoke coder will be able to do it at an affordable cost, as it is just one part of the site integrating into the central system not a rewrite of everything.

Web site Portability

Being able to move a website between one website company and another without loss of support or "the getting to know the code" time is an excellent facility to have. Because the open source software is known to many of the web design companies, taking a new web site on board would be no problem.

For now and for the future why have the bespoke built web site at a large cost and even larger support costs, as opposed to an affordable flexible system which gives you the power to use the best minds of the Internet at an affordable price, with corporate functionality.

Following the Movement to Open Source

When it comes to security, versatility and stability, there is no better solution in information technology than open source. From operating systems to Web development to just about any common application, the open-source revolution finally has taken hold.

Businesses have started to truly understand the benefits of opting for open source, not only to save money, but also for the peace of mind it brings to their IT environment.

The term “open source” has a variety of meanings in the IT world. It is used to describe initiatives and collaborations ranging from software projects to enterprise environments with source code that is accessible to anyone. It stands as perhaps the ultimate example of the process of continuous improvement by encouraging everyone to make it better – and share how they did it.

Many open-source-based solutions like RedHat, Ubuntu and Suse are becoming down-right mainstream, drawing offers for proprietary partnerships from hardware giants like Dell, HP and IBM. So far, each has maintained its autonomy and reiterated its commitment to furthering “the movement” rather than its bottom line.

The issue of support no longer keeps businesses from buying in to open-source technology either. Most are fully supported, either through packages by their distributors, as in the case of RedHat and Suse, or via the well-rounded Internet network.

Open-source now is possible - and pragmatic - for every new Web development project. By using LAMP – that’s Linux, Apache, MySQL and PHP – an open-source solution can be created that is versatile and secure.

Java Goes Open Source

In November of this year Sun Microsystems moved to "open source" status for Java, after a decade of maintaining proprietary status for the portable programming language. Specifically, Sun has placed Java into the public domain by putting it under GPL - an acronym for General Public License. What this means is that software programmers will have vastly increased freedom to develop programs based on Java and to develop modifications for the language itself.

It also puts Sun into the mainstream with other major platform developers such as Linux. While the company had put its Solaris operating system into open source status some time ago, Java is a highly distributed consumer platform and providing open source access to it gives the company a real boost in its standing among its peers. Perhaps more important, it will stimulate further development of consumer oriented Java-based programs. It is estimated that eight out of every ten cell phones have a Java application running on them.

A GPL use requires that any product developed under such licensure be returned to the "open source community" and remain, in effect accessible to all. Sun's variation on this principle has an exception for applications built on the Java "Virtual Machine," a platform that the company made available to software developers some time ago.

What this exception does is allow continued development of proprietary software written for Java, which keeps the language viable as a platform for revenue producing products. Prior to the switch to GPL status, Java program developers had to pay a licensing fee to Sun.

IBM has been after Sun to take Java to open source status for years. Their Works Projects has been a center for the development of open source products, primarily based on Linux. From their perspective, Sun's decision to grant GPL status for Java is viewed as an opportunity to unite with Linux and provide a stronger platform to challenge Microsoft. The politics of software can be enormously complicated, especially when there's an elephant like Microsoft in the house. But what Sun has accomplished with this move is provide an opportunity for programmers to zero in on Java products as potentially large revenue sources.

Unlike Linux, which was spun off of UNIX to provide an alternative to Windows, Java stands in a class of its own. While Linux has survived in the marketplace, it has never mounted a major challenge to Windows. Java's unique qualities and the intellectual property that protects those qualities will now be an open book for programmers developing new applications.

It will also provide the opportunity to bundle Java products with Linux based software. Sun's internal interest in this move is to stimulate more developers to use the language, in order to revive its own internal software business. Since taking a huge hit in their high-end server market, Sun has been struggling to find a new path and has increasingly looked to software as an opportunity.

Sun's EVP for software summed up the value of the move for the company and its product. "People have been hesitant to distribute Java worldwide with Linux (distributions) because of (concerns over) license alignment," Green said. "This is the last gate to ensure that Java will be distributed worldwide."

A Practical Use of Open Source Code Softwares

As we started the project of our web site, we knew that the proprietary software costs would be too high for our financial resources. Our only option then was to make use of Open Source Code softwares.

However, within the Open Source Code there are lots of softwares to choose from, and it´s up to the system analyst/programmer to pick the ones that best suit the project goals.

In our case, we picked Linux as our operating system. However we faced a big question: Which Linux distribution ? There are dozens of linux distributions, each one with specific advantages, sometimes with or without paid services.

We chose linux Slackware because we thought that it would suit better our needs, in terms of stability and reliability. Slackware is a conservative distribution, meaning that it only offers in its distribution package kernel versions and third party libraries, once it´s been proved to be stable and reliable. As a result, Slackware is one the most stable Linux operating system on the market. If the user wishes the most recent Linux kernel and libraries, we recommend to look for a different distribution.

Another reason for us picking Slackware, was because of its text interface, which at the beginning may become a bit difficult, in the long run however it offers a better understanding of the Linux environment. This way most of Slackware configurations are done thru text scripts.

Once it was set the operating system, we picked Apache as our web server. Apache is one of the most popular web server in the internet. For server programming language, we chose PHP. Also, a very popular programming language. Our experience with Apache/PHP proves that this setup is very reliable, working extremely well in our production environment. For workstation programming language, we chose javascript, which besides of having a simple syntax, it offers lots of resources.

Apache web server is really a complete web server. Also, it can become very complex with many configuration setups.

For our database, we picked Postgresql. This database has been made available to the public for more than 15 years, and besides of being well debugged, it offers so many resources that it has become one of the best database options from Open Source Code softwares. Extremely reliable, Postgresql has reported cases where there have been databases in production with over 4 terabytes. Besides all of that, Postresql is not the most popular Open Source Code database. A matter of marketing ? Probably.

As for the programming language PHP, we had to use an object oriented application structure. There are many PHP application structures being offered with Open Source Code license. Object oriented application is very important, because it makes programming very easy and scalable up to the point where the system may grow without many problems. Also, it helps to keep PHP code separated from HTML, with the help of some specific libraries, such as Smarty. We make use of Smarty library and we believe that it is a very important resource, which can even make web pages faster thru the use of its cache functions.

Also a very important set of libraries worth mentioning for PHP systems, is the PEAR libraries. In PEAR web site, among the many libraries being offered, there are the database interface DB and the MAIL_QUEUE, which sends emails thru the use of database.

As we came to the end of our project, we had to use some sort of real time database replication. For that we chose the DRBD and Heartbeat libraries. The use of database replication is very important and should not be postponed. Just after 3 months our system got into production, our server hard disk became bad. Thru the use of DRBD/Heartbeat we were able to put the system back into production within seconds, without loosing any data. DRBD/Heartbeat replication is done thru the use of a lan network between the server and slave nodes.

Our project also makes use of ImageMagick (enlarge and reduce images), Iptables (firewall), Tsearch2(text search in portgresql) and Mon (linux monitoring tasks).

Also a very important library worth mentioning is the pseudo APC compiler for PHP. Speed is always a critical factor for internet sites.

Our web site is already in production and based upon our experience we can say that the Open Source Code softwares we picked, proved to be extremely reliable and stable. Open Source Code softwares can certainly be an excellent option for developing web sites.

Roberto Sedycias IT Consultant

This article is under GNU FDL license and can be distributed without any previous authorization from the author. However the author´s name and all the URL´s (links) mentioned in the article and biography must be kept.

Build Faster with This Turnkey SWT Project Template

The Eclipse Standard Widget Toolkit (SWT) is growing in popularity, as evidenced by its use in applications like Azureus and RSSOwl. In this 10-minute solution, you will learn how to make use of an SWT project template that can greatly reduce the time required to build small and medium-sized applications. In particular, the template offers the following features:


* A UI that Works Out of the Box—The project already has a MainWindow class that comes complete with a menubar, toolbar, client area, status bar, and about dialog. The Open toolbar button and File menu item have code that opens a file dialog box, and the File | Exit menu option closes the app. (See Figure 1.)

Figure 1. The basic template application, awaiting your modifications, is shown.
  • Executables—The project has an Ant task that makes it trivial to build executable JARs, Windows .exe's, and Windows installers.
  • Better Help—Context-sensitive SWT Javadocs are integrated into the template.
  • Preferences—The MainWindow saves its window size and location in a preferences file that your application can use.
  • TextFile—The project includes a class that makes reading and writing text files trivial.
  • Cursors—A default (arrow) cursor and a waiting (hourglass) cursor have been added to the MainWindow class.
  • Portable—You will be able to take your project to other computers and it will just work because the swt.jar and associated .dll are included in the project.

To give you an idea of the time savings you can expect to see using the template, consider a utility that I wrote recently. The application allows the user to select a directory for source code analysis. The application then parses all source code files in the directory and its sub-tree, and compiles information on the number of lines of code, comments, and white space for each file. From the time I began working on the application, to the time that the application was being installed on a colleague's computer, 50 minutes had passed. Of this, just under 40 minutes were spent on the tree-walking and source code parsing logic. Only 10 minutes were spent working on the user interface, building the executables, and creating an installer—tasks that could easily have taken hours to complete without the template.

The specific environment used to develop the SWT Project Template was Eclipse 3.2 (which includes Ant), JDK 6.0 and Windows XP (however, the basic steps can be adapted for use on *NIX and MacOS systems). Optional, but recommended, components include Jigloo GUI Builder 3.9, Launch4j 3.0, and NSIS 2.0.

Inherent AJAX Security with Java and JSF

With the popular adoption of AJAX techniques mushrooming on the web today, real-life security issues were sure to follow, as illustrated by the Yamanner and MySpace worms. The debate over AJAX security now rages, with widely differing opinions. Just google "AJAX security" and you will receive more information than you can sort through—let alone absorb—in a week. Never mind making intelligent strategic decisions based on it and producing secure AJAX-based applications as a result. However, to help you jump-start your research, CGISecurity has conveniently organized some AJAX security content here, and the Open Web Application Security Project (OWASP) has begun to assemble some formal documentation here.

The Common Themes of AJAX Security

As you wade through the plethora of available AJAX security information, you will discover three common themes:

1. AJAX, or more specifically, the use of the XMLHttpRequest object (XHR), does not corrupt the web security model on its own, provided that a strong security architecture is defined and enforced during application development.
2. Defining and enforcing a strong security architecture is a non-trivial exercise, and virtually none of the multitude of JavaScript-laden AJAX libraries that exist today will help you do it. This is an exercise left to the developer—don't blow it, or blow it off for that matter.
3. Most obviously: the client is untrusted, SO DON'T TRUST IT!!

Let's assess what each of these themes means to an enterprise that is motivated to leverage the benefits of AJAX, but needs to maintain high levels of security.

AJAX on Its Own Doesn't Corrupt Web Security

This is the good news. It means nothing about the use of XHR in JavaScript is fundamentally insecure. In fact, XHR is restricted to on-domain HTTP requests, which makes it safe to use provided proper access control is established. And if the contents of an XHR response contain sensitive data, the whole connection can be secured with SSL, so you essentially have a secure foundation on which to build.

From a security perspective, calls in the industry to extend XHR to allow cross-domain requests are ludicrous, as this would compromise the security foundation. The concept of an autonomous JavaScript client freely accessing multiple domains in some sort of SOA/mashup architecture cannot be achieved in a secure fashion, and should be discarded.

Enforce a Strong Security Architecture

If you have security at the foundation of AJAX, then building secure AJAX applications is just a matter of ensuring that you develop your applications in a secure manner. The dos and don'ts associated with this are wide ranging and have been enumerated and described in articles all over the web (at WhiteHatSecurity.com and OWASP.org, just to name a couple). So, it is not whether you can in theory, it is whether you can, and will, in practice.

Don't expect your favorite JavaScript library to provide inherent security. You need to be conscious of security issues at every step of development. In fact, depending on which libraries you use, you may not be able to ensure security at all. Frameworks that rely on client-side interpreters and/or dynamic script injection are particularly vulnerable to attack, and they could introduce inherent insecurities to any application that uses them. Also, because of the complexity of most JavaScript AJAX libraries, security audits and thorough security testing may be next to impossible.

Don't Trust the Untrusted Client

It is as plain and simple as it can be: the client is untrusted so don't trust it. This means that you cannot trust client-side validation, you cannot trust client-side state management of sensitive information, and you cannot trust client-side business logic. But what do most client-centric AJAX libraries promote? You guessed it, all of the above. So if you are going to be security minded, you might not be able to fully leverage the features of your client-centric AJAX technology of choice.

Intertwined with all of this is the blurring of development roles. Client-centric AJAX approaches have already blurred the lines between application developers and web designers, and now the AJAX developer clearly is going to have to be a security expert as well. This blurring of roles compromises proficiency and stifles creativity at all levels. And of course, when you consider all aspects of cross-domain expertise, skill sets start to go from scarce to nonexistent.

All in all, the challenges involved in delivering secure AJAX applications appear to be fairly daunting. What is really needed is an approach that provides inherent security, and maintains role distinction so that security experts can ensure security, while developers and designers focus on creative application solutions.

The Rules of Open-Source Programming

1. Rules must be either funny, whimsical or true.
2. You are not expected to understand these rules.
3. The rules as a whole may be self-contradictory.
4. Rules may have exceptions.
5. Rules may be false.

And now the rules themselves:

The Rules of Open-Source Programming


1. Don't whine unless you are going to implement it yourself.

4. If you don't work on your project, chances are that no one will.

5. A project is never finished.

6. The user is always right unless proven otherwise by the developer.

7. Release early, release often. Clean compilation is optional.

8. Open-Source is not a panacea.

9. Give me refactoring or give me death!

11. When a developer says he will work on something, he or she means
"maybe".

13. Your first release can always be improved upon.

15. If you like it, let the author know. If you hate it, let the author
know why.

20. Open Code != Good Code

22. Backward compatiblity is your worst enemy.

23. Backward compatiblity is your users' best friend.

31. You are your best tester

32. Your users are your second best testers

33. Your test-cases and test-scripts are your worst testers

34. Every successful project will eventually spawn a sub-project

37. Duplicate effort is inevitable. Live with it.

48. The number of items on a project's to-do list always grows or
remains constant.

Modern GUI toolkits make for easy system-level GUIs

As a system programmer, you habitually work with command-line interfaces. Perhaps you've lost track of how easy it can be to wrap existing functionality with a lightweight graphical user interface (GUI). New Perl/Tk releases make it timely to remember that sometimes high quality accompanies ease of use.

You can keep your focus on highly productive server-side programming, and still choose to jazz up your interfaces occasionally. What's more, lightweight toolkits such as Perl/Tk make it possible to do this without the costs of higher-profile GUI approaches.

Toolkit landscape

What are the costs? GUIs are generally harder to program, test, port, and deploy than their corresponding command-line-oriented relatives. Microsoft Visual Basic (VB) is widely recognized for its simplification of GUI construction. However, effective isolation to the Windows world and its clumsiness as a programming language constrain VB; see Resources later in this article for details on these disadvantages. Although Java, GTK+, and Qt have become accepted for Linux and related open source development, it takes a lot of study and lines of code to begin to use them; their "barriers to entry" discourage low-key transitions from character-oriented to WIMP (windows-icons-menus-pointers) programming.

As simplistic as the Web's model of user interaction is, it suffices for plenty of little jobs. In fact, most of my own GUI programming is of Web applications.

Sometimes, though, the Web's not enough. You need more sophisticated programmability or a different security model. Sometimes you just need a "second source"; a Web-based control panel for your Web server makes your system too fragile. Sometimes the Web's too much: the usual servers and clients have big memory footprints and demand configuration.

There's good news, though. Keep your highly-productive system programming habits. Stir in such lightweight high-level language bindings as Perl/Tk or Tkinter. You'll end up with portable, maintainable, surprisingly well-performing, "GUIfied" versions of the programs you've already developed.

Whether your favorite language is Ruby or Fortran, the right accessories will give you similar opportunities. Here's an example of how it can work for a programmer already comfortable with Perl.
Warming up to Perl/Tk

You know Perl. Its clever succinctness lets you code in five minutes the report generators, network probes, text filters, and process monitors that demand day-long efforts with other languages.

What you might not know, though, is that with just a few lines more, you can turn your Perl applications into well-behaved, good-looking GUI ones. Teodor Zlatanov's article on developerWorks describing cfengine and related Perl utilities (see Resources for a link to the article) shows how to write handy tools such as this tiny script, which shuts down all inetd processes:
Listing 1. Source code for kill_inetd.pl



use Proc::ProcessTable;

$t = new Proc::ProcessTable;

foreach $p (@{$t->table}) {
# note that we will also kill "xinetd" and all processes
# whose command line contains "inetd"
kill 9, $p->pid if $p->cmndline =~ 'inetd';
}


What if you need to shut down several rogue CGI processes? Or what if you want more graceful signaling than the rather brutal "kill 9"? What if your boss or your backup needs to use this script while you're away? The traditional answer is just to fire up an editor, substitute cgi for inetd, and so on, and re-run perl kill_inetd.pl from the command line.

Transform that first listing, though, to Listing 2:
Listing 2. Source code for modified kill_inetd.pl


use Proc::ProcessTable;
use Tk;


sub kill_them {
$t = new Proc::ProcessTable;
foreach $p (@{$t->table}) {
kill $signal, $p->pid if $p->cmndline =~ $keyword;
}
$result_text = "Signal $signal sent to all '$keyword' processes.";
}


$main = MainWindow->new();
$main->Label(-text =>
"Control panel for killing processes\n enter keyword here")->pack();
$main->Entry(-textvariable => \$keyword)->pack();
$radiobutton_frame = $main->Frame->pack();
$radiobutton_frame->Radiobutton(-text => "KILL",
-variable => \$signal,
-value => 9)->pack(-side => left);
$radiobutton_frame->Radiobutton(-text => "TERM",
-variable => \$signal,
-value => 15)->pack(-side => left);
$main->Button(-text => "Send kill signal to named processes",
-command => \&kill_them)->pack();
$main->Label(-textvariable => \$result_text)->pack();

# Set reasonable defaults.
$keyword = "inetd";
$signal = 9;

MainLoop();


Simple signal-sending control panel
Simple signal-sending control panel

Notice, though, what you do not have with this twenty-line script: you don't have pages of declarations and configuration. You don't have any license restriction beyond the familiar Perl ones. You don't need to learn a new language.

New horizons

This is exciting stuff. You can wrap up all your existing Perl-coded applications with skinny GUI scripts to make them into similarly interactive and "user-friendly" programs. In fact, you can easily rewrite utilities you have now so that they can continue to operate in a normal "batch" mode when given appropriate command-line arguments, but they can branch and construct a GUI window otherwise:
Listing 3. Select between command-line and GUI operation





sub batch_operation {
}

sub start_GUI {
use Tk;
...
MainLoop();
}

if (0 == scalar(@ARGV)) {
start_GUI();
}
if (errors_detected_in_@ARGV()) {
start_GUI();
}
batch_operation();


Remember: it's just a few extra lines of the Perl (or Python, or ...) code you already know to give these results. What's more, because Perl acts as "glue" to external applications, these same techniques work to wrap legacy Fortran or C applications that you don't want to touch. Just wrap them up in a bit of Perl with backticks or exec, and you're in business.

Perl/Tk scales well, too. Substantial Perl/Tk applications are in everyday use around the world, including a biochemical simulator, geographic mapping programs, a flight management system, and much more. It's difficult to convey the feel of large-scale Perl/Tk programming. The examples above demonstrate how straightforward it is to "GUIfy" with a few extra lines of Perl. For a fuller sense of Perl/Tk programming "in the large," follow the references in Resources below, and study for yourself the source code of more complex programs.

One program I haven't written yet, except in a crude prototype, is a graphical version of Zlatanov's genetic algorithm modeler (read about it in his developerWorks article listed in Resources). I leave a more satisfying version as an exercise to the reader. My own experiments suggested it'll make a showcase for the approach this column recommends: its batch-mode operation will really come alive when visualized on the convenient Perl/Tk Canvas widget.

If you tried Perl/Tk a few years ago, and were unimpressed, it's time to give it another chance. The Perl/Tk processor has improved a great deal lately. It's far easier for Perl/Tk beginners to install binaries for popular platforms, Windows portability is much better, and the latest releases simply exhibit higher quality. Perhaps even more important, more books on Perl/Tk are available. July's second edition of O'Reilly's Perl in a Nutshell includes a valuable Perl/Tk reference. Best of all, O'Reilly kicked off 2002 with publication of Mastering Perl/Tk by Nancy Walsh and Stephen Lidie. If you plan to work for more than a couple of hours with Perk/Tk, Mastering Perl/Tk is the one reference you need to buy.

Conclusion

GUI work can seem mysterious to systems programmers and administrators. It looks like a specialized pursuit that's rather removed from the kinds of development we normally do in "Server clinic."

It doesn't have to be that way, though. Modern GUI toolkits like Perl/Tk offer good performance and functionality, along with the productivity and ease-of-learning this column most often emphasizes. Take an hour or two this month and give Perl/Tk a chance to help you wrap up your programs more attractively.

How Linux and open-source development could change the way we get things done

An army of disheveled computer programmers has built an operating system called Linux based on a business model that seems to have been written with everything but business in mind. Instead of charging customers as much as the market can bear, Linux is given away for free; instead of hiding information from competitors, Linux programmers share their work with the world; instead of working for money, Linux developers are motivated primarily by adrenaline, altruism, and the respect of their peers.

Despite this unusual foundation, Linux is booming and even beginning to challenge Microsoft's control of the operating system industry. Linux may eventually pull the rug out from under the richest company in the world. It may not. But no matter what happens, it has already shown that money doesn't have to make the world, even the business world, go round. In fact, as technology improves and computers connect and create even more of our society, the principles of cooperation and collaboration that drive Linux may well spread to other fields: from computers, to medicine, to the law.

The Source

The Linux movement kick-started in 1991 when Linus Torvalds, a puckish graduate student at the University of Helsinki, got frustrated with his rickety computer. Refusing to buy another one, he wrote a new operating system--the core programs by which applications (like Microsoft Word) talk to hardware (like microprocessors). When finished, instead of running down to the patent office, he posted his code on the Internet and urged other programmers to download it and work with him to improve it. A few emailed back suggestions, some of which Torvalds took. A few more wrote the next day and a couple more the day after that. Torvalds worked constantly with these new colleagues, publicly posting each improvement and delegating responsibility to more and more programmers as the system grew. By 1994, Linux (a combination of "Linus" and "Unix," another operating system) had 100,000 users. Today, it has between 10 and 20 million users and is the fastest growing operating system in the world.

But Linux (rhymes with 'cynics') is different from almost every other operating system available. For one thing, it's downloadable for free straight off the Web. It's also open source, meaning that the source code, the program's all-important DNA, is open for anyone to look at, test, and modify. Most software is developed so that only the original authors can examine and change the code; with open-source models, however, anyone can do it if they have a computer and the right intuition.

To see the power of this model, consider what happens when you're running Microsoft Windows or Macintosh OS and your computer crashes: You stamp your feet and poke a twisted paper clip into a tiny reset button. You probably don't know what happened and it's probably going to happen again. Since you've never seen the source code, it probably doesn't even occur to you that you could fix the problem at its root. With Linux, everything's transparent and, even if you aren't an expert, you can simply post your question on a Linux-help Web page and other users can usually find solutions within hours, if not minutes. (The amorphous Linux community recently won InfoWorld's Product of the Year award for Best Technical Support.) It's also entirely possible that someone--perhaps you--will write some new code that fixes the problem permanently and that Linux developers, led by Torvalds, will incorporate into the next release. Presto, that problem's fixed and no one will need paper clips to fix it again.

To make another analogy, fixing an error caused by a normal software product is like trying to fix a car with the hood welded shut. With Linux, not only can you easily pop the hood open, there is extensive documentation telling you how everything works and how it all was developed; there's also a community of thousands of mechanics who will help you put in a new fuel pump if asked. In fact, the whole car was built by mechanics piecing it together in their spare time while emailing back and forth across the Web.

The obvious threat to this type of open development is appropriation. What if someone lifts all the clever code off the Web, copyrights it, and sells it? What if someone takes the code that you wrote to fix your crashed computer (or busted fuel pump), copyrights it, and markets it for $19.95? Well, they can't. When Torvalds created Linux, he protected it under the GNU General Public License, an intriguing form of copyright commonly known as copyleft. Under copyleft, anyone who redistributes the program, with or without changes, must pass along the freedom to further copy, change, and distribute it. Theoretically one can download Linux off the Web, add a string of useful features, and try to sell it for $50. But anyone who buys this new version can just copy it, give it away, or sell it for a dollar, thus destroying the incentive for piracy. An equivalent would be if I were to write the following at the end of this article: "Verbatim copying and redistribution of this entire article is permitted in any medium provided this notice remains at the end."

Use the Source

From the Oxford English Dictionary to hip-hop music, open-source development has always been with us to some degree. Since the mid-19th century, contributors to the OED have defined words and sent them to a centralized location to be gathered, criticized, sorted, and published. With music, as long as you stay within certain copyright laws, you can take chunks of other people's compositions and work them into your own. Most academic research is also built on open-source cooperation. Even the compact fluorescent light bulb above my head comes from data shared by researchers over centuries about electricity, properties of glass, and centralized power.

But still, most business isn't done with open source. Coca-Cola keeps its formula secret, Microsoft won't tell you how it builds its programs, and when a researcher for Ford suddenly stumbles upon the means to a more efficient fuel pump, she doesn't reflexively email her friend at Honda with a precise description of her discovery. A great deal of scientific and medical research is also done through closed source as individual laboratories race each other to determine who'll be the first to find the answer and earn the patent.

But two extraordinary changes are making open source substantially more plausible as a development and research model for 2000 than it was for 1990--and they'll make it even more so for 2010. First, the Internet. Today, I can open my Web browser and communicate instantly with Burmese refugees or writers working on projects similar to mine. Secondly, computer power has been increasing exponentially for generations and will probably continue to do so--in large part because every time you build a faster computer it allows you to build a faster one still. It's difficult to overestimate the change. The standard laptop to which I'm now dictating this article (with technology unavailable just two years ago) has more power than almost any computer owned by the government a decade ago. In four years, it could well be obsolete. As author Ray Kurzweil and others have pointed out, if cars had improved as much over the past 50 years as computers, they'd cost less than a nickel and go faster than the speed of light.

Intellectual and Physical Properties

This rate of progress is critical because the advantages of open-source development depend on the powers of technology and the balance between what can be done through thinking and what has to be done by building. Every product has certain intellectual components and certain physical components built into it. With a car, for example, the intellectual component is the thought about how to build it, how to set up the assembly lines, and how to process data you have on different kinds of tires. The physical components include the actual rubber in the tires, the machines that ran the tests, the operation and maintenance of factories.

Faster computers and increased connectivity are drastically changing the relationship between these components in two ways. First, some things that used to require physical components no longer do. You may not have to buy rubber to test the tires if you can just run a simulator online. (The 777 project at Boeing was designed so that nothing physical was produced before the plans hit the factory floor.) Second, connectivity makes the flow of information much faster and smoother and greatly facilitates the sharing of ideas and data. There is a saying known as "Linus' law" that "given enough eyes, all bugs are shallow." In other words, given enough people working on them, all problems are solvable. And the Internet has not only helped coordinate a lot more eyes: in some ways, it's given everyone glasses.

Open-source development benefits from this transition because its advantages are almost all in the realm of intellectual property. Open source improves communication and facilitates sharing ideas. But it doesn't mean that you can buy a ton of concrete for free or drive a nail into a wall without a hammer. This is why open source has come first and most prominently to computer programming: a profession where almost all of the development is intellectual, not physical, and where people have been connected over the Internet for more than 20 years. Programming also employs highly specific, common tools--computers--that a fairly large number of people have access to and that some people, such as university students, can even access for free. If a solution or improvement is found, nothing additional needs to be built; code just needs to be entered and then downloaded.

But there is still one great problem standing firmly in the way of even the most modern open-source software development project. As a 21-year-old Bill Gates asked in an angry letter to open-source programmers in 1976: "One thing you do is prevent good software from being written. Who can afford to do professional work for nothing?"

Microsoft's empire, and a great deal of the rest of our society, is built upon the assumption that there isn't an answer to this rhetorical question. To survive, organizations need to patent their information, protect their secrets, and earn as much money as they can. But the success of Linux shows that there's a way around that model. Something extraordinary has been built with a completely different set of rules. I asked David Soergel, a researcher in the department of genetics at Stanford, whether people could use the open-source model to develop medicines. "My first reaction: they'd rather get paid, and if they were any good at it, then they would be paid by somebody. My second reaction: wait a minute, that argument obviously fails for software."

Money for Nothing

Naive as it may be to think that people aren't motivated by money, it is just as naive to think that people are only motivated by money. People are motivated by a variety of factors: money, recognition, enjoyment, a belief that one is doing something good for the world, and so on. We each weigh these factors and make decisions based on our perceptions of their relative importance. At different points in our lives, we give different factors different weights. When we're poor, we tend to value simply high-paying work more than we do when we're well-off; it usually takes a high-paying job to get someone to do something really boring and it generally takes a very fulfilling job to get someone to work for less than what he should normally be able to earn.

Since people working on open-source projects generally earn nothing, or very little, there need to be other incentives. In Linux, there seem to be principally three. First, enjoyment. Computer programming can be addictive, exciting, and extraordinarily intense. Linux can be particularly enjoyable because almost every problem solved is a new one. If your Windows machine crashes, fixing the problem generally entails tediously working through scores of repair procedures which you may have used a thousand times. If A fails, try B. If B fails, try C. If a Linux computer crashes, anyone who repairs it is not only working on that one specific machine, he's finding a solution for the whole Linux community.

Eric Roberts, a computer science professor at Stanford, once explained to The New York Times that people in the profession must be "well trained to do work that is mind-numbingly boring." This is particularly true of work on most closed-source systems where programmers must continually reinvent the wheel. But with open-source projects, the things that need to be done haven't been done before. According to one only slightly hyperbolic programmer, Ali Abdin, writing to a Linux news group about how he felt after creating his first open-source project: "The feeling I got inside when I knew that I had some code out there that I can share with people is indescribable... I felt on top of the world, that I can program anything...I felt as mother would feel giving birth to a child, giving it life, for the first time."

Secondly, and similarly, Linux programmers are motivated by a feeling that they are changing the world and developing an operating system that really works. Torvalds laid out this philosophy well in a speech this summer: "I don't resent Microsoft for making lots of money. I resent them for making bad software."

Thirdly, but most significantly, Linux programmers seem motivated by prestige and, in particular, respect from their peers. Having "hacked the kernel" (contributed to the core of the operating system) gives programmers a certain stature--much as completing a four-minute mile does among runners--and, since the program is open source, everyone knows exactly who contributed what. I was once introduced to a programmer described as "the guy who wrote all the Ethernet device drivers!" as though I was meeting Jonas Salk "who came up with the cure for polio!" And, in fact, Linux programmers often discuss their work as falling in the tradition of eminent scientists. As three well-known programmers put it in the introduction to their book Open Sources: "It would be shortsighted of those in the computer industry to believe that monetary reward is the primary concern of open source's best programmers... These people are involved in a reputation game and history has shown that scientific success outlives financial success... When the history of this time is written a hundred years from now, people will perhaps remember the name of Bill Gates, but few other computer industrialists. They are much more likely to remember names like... Linus Torvalds."

Importantly, this philosophy may well be helping Linux develop creatively. There is a great deal of psychological research that shows that people actually do more creative work when they aren't motivated primarily by money. Tell a child that you'll pay her for reading a book and she'll read it with little imagination. Have one group of college poets think about getting rich and famous through their writing, according to research done by Harvard Professor Teresa Amabile, and they tend to turn out less creative work than a second group that's just asked to write poems. Is it possible that Linux programmers created such an extraordinary operating system in part because they were driven by other factors and weren't doing it for the money? I asked Professor Amabile if the implications of her research cross over to open-source programming and whether it could explain some of the remarkable innovations that have come from people working without pay. "Yes," she responded, "this [would be] entirely consistent."

Making free software affordable

Still, Linux programmers are not completely locking themselves out of the economy and there's a second response to Gates' rhetorical question: If your core open-source project is successful enough, it's possible to eventually make money off of it indirectly. No, it's not possible to make as much money as a proprietary company can--open source and copyleft will ensure this--and there's always going to be an astounding amount of work that has to be done without financial reward. But open-source programmers don't have to starve.

The trick to making money off Linux, or any open-source development, is to profit off of derivatives. No one actually makes money off the Linux code, but they can make money by offering technical support or helping customers install the program. Companies that do this follow a well-trodden path: give something away in order to sell something else. This is what cellular phone companies do when they give you the actual telephone handset for free if you agree to pay for using it a certain number of minutes a month. We do the same thing with the Monthly's Web page. We post some articles (giving them to readers for free) in the hope that visitors' interest will be piqued and that they'll subscribe.

Red Hat, the best-known Linux company, sells the operating system and other open-source programs in a box for $80, though you can download their product for free from redhat.com. Their revenue comes from the technical support they offer and from consumers' need for trust. It's much less unsettling to buy a program like Linux if you get it shrink-wrapped with a manual than if you have to download both. VA Linux, another well-known company, sells Linux hardware: You choose the memory and the motherboard; it builds you a computer.

Has the money these companies brought into the open-source movement corrupted it and made it more like the traditional model that Microsoft uses? Surely, there are a lot of people doing promotional and administrative work at Red Hat for the money and there are probably even some people working on Linux for Red Hat because they get paid a lot to do it (the company hires programmers to write code that anyone, including Red Hat's competitors, can use). But programmers mostly see the money as an added and surprising plus, and Linux is still driven by adrenaline, altruism, and peer recognition.

While it is possible that this could change, it hasn't so far. I asked Richard Stallman--the creator of copyleft, as well as many of the programs that run with the Linux through a system called GNU, and the man often considered to be the father of the open-source movement--whether he thought that money would change the attitudes of people who used to work on GNU/Linux without being paid. "In general, I don't think that wealth will make a hacker into a worse person. It would be more likely to enable the hacker to spend more time volunteering for free software instead of on work for pay."

This point is particularly germane because most open-source programmers have always had to work other jobs, and many have only been able to contribute to the project during the evenings or when their employers weren't looking. Linus Torvalds, for example, helps design microprocessors for a company called Transmeta in the daytime and does all his Linux coding after work. When I asked John Hall, vice president of VA Linux, what motivates programmers, he responded: "For some, it's altruism. For some, it's fame. For some, it's religion. For a very few, it's money."

So what's next?

To determine where open source is likely to move next, one has to imagine a scenario where these obstacles can be overcome. A project would need to be fun, or at least rewarding, to get going and it would have to primarily pose intellectual, not physical, challenges. Once it began to move, derivative financial incentives could help push it along. There also has to be a certain amount of luck, particularly with regard to organization. It's hard enough to get six people to agree to a restaurant for dinner; it's much harder to coordinate thousands of people known to each other only by email addresses. Linux has gotten around this latter problem in no small part because of Torvalds himself. He is a benevolent dictator who has earned the trust and respect of virtually everyone on the project. He's a relaxed, friendly, funny guy whose strength of character keeps his distended organization free from the internal battles that sink so many others. He also has learned how to delegate and has developed a core of equally well-respected close associates who coordinate different parts of the project.

One intriguing possibility for future open-source development comes from medicine, an area where people can become passionate and where intellectual components can far exceed physical components. Consider a smart doctor who loses a friend to rare disease X and decides to devote her life to finding a cure. Ten years ago, trying to develop anything more than a local network to collaboratively design a drug to cure the disease would have been extremely difficult. Communication would have had to be done over the phone or with photocopies slowly and expensively mailed. It made much more sense to have small groups of developers working together in laboratories, foundations, or universities.

Today the possibilities for open collaboration have improved. An ambitious doctor can network online with other researchers interested in disease X and, at the least, can quickly exchange data about the newest research techniques. In fact, there are already medical networks that pass around information about acute medical cases, using email and computers that can automatically send out patient files over a network and put X-rays into the overnight mail.

Now think another decade ahead when everyone will have high-speed Internet lines at least 500 times as fast as standard connections today (this will probably happen in three years), where it is fairly likely that we all will be able to simulate the movement of disease X online, and where it will surely be possible for medical students to run tests that approximate the human immune system on high-powered laboratory computers. Now the same doctor can farm out parts of the project to interested collaborators, many of whom have also lost friends to X and are passionate about finding a cure. If the coordinator is a good organizer and can hold people together the way that Torvalds has, the organization could grow, attracting even more people to join the collaborative effort with each success. Every breakthrough or improvement in the model could be posted online so that other participants could begin work on the next challenge. If a sample test is performed, data could be transferred to the Web simultaneously.

Eventually a prototype could be developed and adopted by an established drug company (or perhaps even a non-profit company, funded by foundations, that specializes in distributing open-source drugs and selling them at minimal costs) that licenses the product with the FDA, runs it through the necessary tests, and then manufactures, distributes and sells it--keeping prices relatively low both because no company would have exclusive copyrights and because research costs (drug companies' largest expense) would be drastically reduced.

Law

A real-life example of another possible opportunity for open source comes from Harvard where law Professors Larry Lessig and Charles Nesson have started the Open Law Project, an attempt to try cases using the open-source model. Interested people sign into the Website, read what other contributors have written, and help to develop arguments and briefs. According to the site description, "what we lose in secrecy, we expect to regain in depth of sources and breadth of argument." The program is run under the same sort of benevolent dictatorship model as Linux, with Lessig serving as chief. People brainstorm, debate, and then Lessig synthesizes and writes the briefs. Currently, the group is constructing arguments challenging, appropriately enough, the United States Copyright Extension Act.

There are great advantages to this model for law: The problems faced by lawyers are mostly intellectual, not physical; there is an abundance of people (especially law students) who are potentially willing to work on the projects for free; and there is the powerful draw of doing something you believe is a public service. If you don't agree with current copyright laws, join the group and figure out how to change them.

Of course, open-law will never be able to handle certain kinds of cases. As Nesson said to me, "open-law is not conducive to ambush." If you need to rely on secret arguments or evidence that the other side doesn't know about, you're not going to post everything on the Net. But if you just need to develop the best argument and rely on increased information flows, the model could work quite well.

Future

It is very difficult to determine exactly where open-source projects will take off next. So much depends on the personalities of the coordinators and the excitement they are able to generate; a great deal also depends on the different ways that technology develops and the way that different markets and research fields change. For some organizations, open-source development will make more sense in a couple of years than it does now. For others, it will make less. But the overall trends of technology are likely to push open source closer and closer to the mainstream.

Imagine a scale with all the advantages of a proprietary model on the left and all the advantages of an open-source model on the right. Pretend everybody who wants to solve a problem or build a project has a scale like this. If it tips to the left, the proprietary model is chosen; if it tips to the right, the open model is chosen. Now, as connectivity increases with the Internet, and computer power increases exponentially, more and more weight accumulates on the right. Every time computer power increases, another household gets wired, or a new simulator is built online, a little more weight is added to the right. Having the example of Linux to learn from adds some more weight to the right; the next successful open-source project will add even more.

Not enough is added to the right side to tip the scale for everybody and everything, but open source is presently growing and it should only continue that way. Netscape has made its Web browser open source. Sendmail, the program that routes most of our email, is open source. Most Web sites use an open-source program called Apache at their core. Even some microchip developers are starting to use open source.

Perhaps the next boom in open source will come from the law; perhaps from drug X; perhaps it will be something entirely different. Although it's difficult to tell, it is quite likely that the scale is going to tip for some projects and that there will be serious efforts at open-source development in the next decade. Moreover, it's quite likely some of these projects will work. Open source has created the fastest growing operating system in the world and it's done so by capitalizing on changes in technology that will almost certainly seem commonplace in a decade or two. Linux will continue to grow; but 10 years from now, it will probably no longer be the largest open-source project in the world.