XFree86 has been the standard implementation of the X Windows System [1] for PC Unix systems for quite a while now; The XFree86 Project [2] was started in 1992 and incorporated in 1994. From the very beginning XFree86 was an open source project (even though back then the term "open source" wasn't widely used), still the first supported platforms were proprietary commercial Unix implementations. Open source simply came as a natural consequence of extending the X Window System. And with the success of Linux, XFree86 became one of the most often used implementations of the X Window System and to some extend the new driving force behind the development of X11.
Having started soon after the official release of X11R5 in 1991, XFree86 was in its design and source layout based on the original X386 work by Thomas Röll which was donated into X11R5 by SGCS.
This had many implications for the fundamental assumptions upon which this design was based. One of the more important (and more devastating) ones was the fact that there was no real design document. While Jim Gettys et al. in [3] describe the fundamental design of X11, and while Elias Israel and Erik Fortune in [4] explain the fundamental X Server design, there is no document that defines the design behind X386 and, subsequently, behind XFree86 versions 1 through 3.
The device dependent X (ddx) code that handles the programming of the hardware was always an area where companies tried to maintain some intellectual property outside the (open source) sample implementation of X11. This was true both for companies like SGCS as well as for the traditional Unix vendors. Therefore, little progress in the ddx code became part of the sample implementation
While X11R6 changed some of the source layout and some of the structure of the device independent X (dix) code, the design of the ddx was largely untouched. It was based on late-80s / early-90s design decisions that, to make matters worse, were not documented.
So in oder to develop drivers for the older versions of XFree86 one had to sit down and read the code. And we are talking about quite a large amount of code. The xfree86 ddx of XFree86-3.2, for example, contains about 1000 source files and roughly 350000 lines of code. For version 3.3.6 this increased to about 1400 source files with about 520000 lines of code.
Early on David Wexelblat, one of the founders of XFree86 and a member of the XFree86 Core Team provided a sceleton driver [5], but even using that it was still necessary to understand very subtle interdependencies and many assumptions that the overall code made about the type of hardware used, in order to successfully write new code for XFree86.
One of the more involved assumptions was that the graphics board in a PC would be VGA compatible. The code loading color maps, for example, relied on the fact that the default way to do so on a VGA board would work. This and other similar assumptions of course turned out to be problematic as new more advanced architectures for graphics card started to become available.
But aside from the technical details of the driver implementation, there were some logistical consequences to the design as well. The hardware drivers were part of the X server binary. This monolithic architecture of XFree86 before version 4 forced (in most cases, there were exceptions for some types of extensions and for Xinput drivers) the release of a complete server binary if a new driver (or an updated version of a driver) was to be released. At first glance this doesn't sound so bad. But the logistical problem stems from the fact that XFree86 supports more than a dozen OSs, including Linux, FreeBSD, NetBSD, OpenBSD, Solaris, SCO, QNX and even OS/2, to name a few. Several of these are supported on multiple hardware architectures.
Releasing even a simple driver therefore implied to get hold of all these platforms (or at least the more popular ones), to create server binaries and installation instructions, and to provide those to the users. And of course there's the added complexity if more that one group is working on improvements (so if you want the new extension from group A, e.g., VMware's new DGA version, and the new driver from group B, e.g. SuSE's new Rage128 driver, then you had to rely on these two groups talking to each other and importing each other's new features).