Berkeley UNIX Yesterday, Today and Tomorrow
Keith Bostic
Marshall Kirk McKusick
Michael J. Karels
Computer Systems Research Group
Computer Science Division
Department of Electrical Engineering and Computer Science
University of California, Berkeley
Berkeley, California 94720
ABSTRACT
This paper presents a brief overview of the
historic Berkeley releases and the role that
Berkeley has played in the UNIX- world and
discusses the role that University and research
releases should play in the future.
1. A Technical History of the Berkeley Project
This is a brief technical history of Berkeley UNIX. For
more details, [McKusick85] is strongly recommended along
with the other papers listed in the Reference section.
Specifically, we apologize that space does not permit men-
tioning most of the major contributors and influences on the
Berkeley system!
The University of California, Berkeley, first ran Bell
Laboratories Version 4 UNIX on a PDP-11/45, in January of
1974. Later that spring, a PDP-11/40 was configured with the
newly available Version 5. Version 6 was running on a PDP-
11/70 by the fall of 1975, with the arrival of two new gra-
duate students, Bill Joy and Chuck Haley. They initially
concentrating on making improvements to a Pascal system
written by Ken Thompson while on sabbatical at Berkeley.
With this completed, they turned their attention to the ed
editor, eventually producing a new version they called ex.
By the end of the summer of 1976, Joy and Haley began to
take an interest in exploring the internals of the UNIX
_________________________
-UNIX is a registered trademark of UNIX System Labora-
tories.
April 27, 2013
- 2 -
kernel.
In 1977, a steady stream of requests for the enhanced
Pascal system had begun. Early that year, Joy put together
the first ``Berkeley Software Distribution'', including the
Pascal system and the source to ex. Over the next year,
about thirty copies of this distribution were sent out. At
around the same time, Joy began work on what was to become
vi. By mid-1978, the software distribution clearly needed to
be updated. The Pascal system had been made markedly more
robust through feedback from its expanding user community,
and had been split into two passes so that it could be run
on PDP-11/34s. The result of the update was the ``Second
Berkeley Software Distribution'' a name that was quickly
abbreviated to 2BSD. Along with the Pascal system, vi and
termcap for several terminals were included. Once again Bill
Joy single-handedly put together distributions, answered the
phone, and incorporated user feedback into the system. Over
the next year nearly seventy-five tapes were shipped.
Although Joy soon moved on to other projects, the 2BSD dis-
tribution continued to expand. Today the latest version of
this distribution, 2.10BSD is still alive and being used by
people all around the world on PDP-11's.
Early in 1978, Berkeley obtained a newly announced
Digital Equipment Corp. (DEC) VAX= 11-780's. Shortly after
the arrival of the VAX, Bell Laboratories provided Berkeley
with a copy of their 32/V port of UNIX to the VAX. Although
32/V supported a Version 7 UNIX environment on the VAX, it
did not take advantage of the virtual memory capability of
the VAX hardware. Like its predecessors on the PDP-11, it
was a swapping system. Ozalp Babaoglu, a Berkeley graduate
student, set about finding a way of implementing a working
set paging system on the VAX. As Babaoglu neared the comple-
tion of his first attempt at an implementation, he
approached Bill Joy for some help in understanding the
intricacies of the UNIX kernel. Intrigued by Babaoglu's
approach, Joy joined in helping to integrate the code into
32/V and then with the ensuing debugging.
Joy realized that the 32-bit VAX would soon make the
16-bit PDP-11 obsolete, and he began to port the 2BSD
software to the VAX. By the end of 1979, a complete distri-
bution had been created. This distribution included the vir-
tual memory kernel, the standard 32/V utilities, and the
2BSD additions, which by now included the Pascal system,
ex/vi, the Cshell and several other utilities. In December,
1979, Joy shipped the first of nearly a hundred copies of
3BSD, the first VAX distribution from Berkeley.
_________________________
=VAX is a registered trademark of Digital Equipment
Corporation.
April 27, 2013
- 3 -
In the fall of 1979, the Defense Advanced Research Pro-
jects Agency, DARPA, accepted a Berkeley proposal to develop
an enhanced version of 3BSD for the DARPA community. With
the already good reputation of 3BSD supporting the proposal,
the Berkeley project was funded. Joy took charge of the pro-
ject, which was to become the Computer Systems Research
Group (CSRG). Joy soon incorporated job control, added auto
reboot, a 1K block file system, and support for the latest
VAX machine, the VAX-11/750. By October, 1980, a polished
distribution that also included the Franz Lisp system and an
enhanced mail handling system was released as 4BSD. During
its nine-month lifetime, nearly 150 copies were shipped.
With the increasingly wide distribution and visibility
of Berkeley UNIX, several critics began to emerge. The major
objection cited was the performance of various benchmarks as
compared to the DEC VMS system. Over the course of several
months, Joy systematically tuned the kernel, soon matching
VMS's performance. Rather than continue shipping 4BSD, the
tuned system, with the addition of Robert Elz's auto confi-
guration code, was released as 4.1BSD in June, 1981. Over
its two year lifetime about 400 distributions were shipped.
With the release of 4.1BSD, much of the debate over
performance died down. DARPA again funded Berkeley, this
time with the intention of adding new features to the BSD
system. These new features eventually included Berkeley
reliable signals, the fast filesystem, disk quotas, and the
socket interface with TCP/IP networking support. Around this
time Joy left the CSRG for Sun Microsystems, and Sam Leffler
took over responsibility for completing the project. In
August, 1983, this system was released as 4.2BSD [Joy83].
The popularity of 4.2BSD was impressive; within eighteen
months more copies of 4.2BSD had been shipped than of all
the previous Berkeley software distributions combined.
As with 4BSD, the major criticism of 4.2BSD was perfor-
mance. The problem, not surprisingly, was that the new
facilities had not been tuned and that many of the kernel
data structures were not well suited to their new uses. In
addition, many of the interfaces, particularly in the net-
working area had been left unfinished. As Sam Leffler had
left the CSRG for Lucasfilm, the tuning and enhancement of
4.2BSD was largely done under the direction of Michael
Karels and Kirk McKusick [Leffler84] [McKusick85]. This sys-
tem was released in April of 1986 as 4.3BSD, and had greatly
enhanced performance and reliability over 4.2BSD, along with
several new features.
As it had done in 4.2BSD, the CSRG then embarked on a
new development phase to update other major components of
the system, and design and integrate new functionality. The
4.4BSD release, scheduled for fall of 1992, will contain the
results of several major new projects. Among these projects
April 27, 2013
- 4 -
are an OSI network protocol suite integrated with existing
ISO applications, an IEEE POSIX 1003.1 standard interface, a
highly tuned TCP/IP networking interface, support for Sun
Microsystem's Network File System, the integration of a
log-structured file system, an integration of the MACH vir-
tual memory system, volume labels and user-level database
support.
There will have been four interim releases made by the
CSRG between 4.3BSD and the upcoming 4.4BSD release. The
first two of these releases, 4.3BSD-Tahoe and 4.3BSD-Reno
were intended to distribute a subset of the new functional-
ity found in 4.4BSD available to vendors. The 4.3BSD-Tahoe
release, made in the summer of 1988, was the first Berkeley
release to support two architectures. This goal was made
possible by the reimplementation of much of the machine
specific kernel source and a fundamental restructuring of
the source code pool so that binaries for more than one
architecture could be constructed from a single source pool.
The two supported architectures were the VAX and the Com-
puter Consoles Inc. Power 6/32 (the Tahoe). Since this
release, architecture support for the Intel 386/486, the Sun
Microsystems SPARCstation- I and II, the DECstation 3100 and
5000 and the Hewlett-Packard 300 have been added as well.
The 4.3BSD-Reno release, made in the summer of 1990, was
intended to make the Network File System code available to
vendors using Berkeley-derived systems, such as The Open
Software Foundation (OSF). This code had been written by
Rick Macklem at the University of Guelph and integrated by
the CSRG, under a new version of the kernel file system
switch.
Two other interim releases, the first and second
release of the ``Berkeley Network Software Distribution'',
usually abbreviated as NET/1 and NET/2, were intended to
make the source code of the 4BSD system available to and
redistributable by anyone. Over the years of development by
the CSRG and others, an increasingly larger percentage of
the system was not derived from the original AT&T 32/V dis-
tribution. In the spring of 1988 Berkeley made its first
distribution not requiring an AT&T source license, NET/1.
This distribution primarily contained the networking por-
tions of the system, from the utilities all the way through
to the kernel device drivers, although other items such as
login and other files were included for various reasons.
This release was extremely popular with many vendors with
their own versions of UNIX but who wished to run the Berke-
ley TCP/IP code and with vendors wishing to create smart
networking cards, not to mention the users that wanted
_________________________
-All SPARC trademarks are trademarks or registered
trademarks of SPARC International, Inc. SPARCstation is
licensed exclusively to Sun Microsystems, Inc.
April 27, 2013
- 5 -
access to the source code for class work or other research
purposes.
Around this time, the CSRG also began to search out
freely redistributable versions of the UNIX utilities and to
rewrite, or encourage BSD users to rewrite, those that were
not available elsewhere. This was an immensely time-
consuming task, involving contributions by hundreds of pro-
grammers from all around the world. In the summer of 1991,
Berkeley released NET/2, which, like NET/1, did not require
an AT&T source license. The NET/2 release included about 80%
of the source code found in the 4.3BSD-Reno release. This
release has proved to be immensely popular, with hundreds of
thousands of copies taken from the public network archives
and an unknown number redistributed by other organizations.
2. The Role of the Berkeley Project in the UNIX World
The role that Berkeley has played in the UNIX world has
been a constantly changing one. In the 1970's, Berkeley was
among the first participants in the UNIX research community,
acting as host to several researchers on sabbatical from
Bell Laboratories. This cooperation typified the harmony
that was characteristic of the early UNIX community, as led
by Bell Laboratories. Work that was contributed to the
Laboratories by different members of the community, Berkeley
among them, helped produce a rapidly expanding set of tools
and facilities. With the commercialization of UNIX, the Bell
Laboratories researchers were no longer able to act as a
clearinghouse for the ongoing UNIX research. As the research
community continued to modify the UNIX system, it found that
it needed an organization that could produce leading edge
research releases. Because of its early involvement in UNIX
and its history of releasing UNIX-based tools, the CSRG
quickly filled this role.
For the first half of the 1980's, Berkeley served as
the focus of the leading edge of UNIX research. The Berkeley
system was widely used, ported and considered the arbiter of
what should comprise a UNIX system. By the mid-1980's,
largely because the networking component of the Berkeley
system was unique and unavailable from vendors for a period
of time, Berkeley was forced into the role of a vendor
[McKusick89]. This role expanded to the point that there
were two major variants of UNIX, System V and BSD, and
resulted in a breach in the UNIX world that is only gradu-
ally being healed. Acting as a vendor required an immense
amount of time, money and effort by the CSRG. Thousands of
hours were devoted to release engineering, thousands more to
participation in the emerging UNIX standards and thousands
more in distribution and user support. Over the years it
became increasingly clear to the people associated with the
Berkeley UNIX project that its limited funding and manpower
were insufficient to complete its historical task of
April 27, 2013
- 6 -
designing, implementing and supporting a complete, reliable,
leading edge system. As each portion of the system became
more complex and additional features were added, more and
more effort had to be expended to keep the system at a high
level of quality, and less and less effort was available to
move the system technically forward. Fortunately, during the
last half of the 1980's, as the UNIX interface became the
consensus choice for an industry standard, and the number of
vendors marketing, selling and supporting UNIX systems grew,
Berkeley has been able to start to return to its historical
orientation of doing leading-edge research instead of custo-
mer support.
3. Berkeley UNIX Tomorrow
For UNIX to become the system of choice for a large
segment of the industry, potential customers must have con-
fidence that the product is supported, that future versions
will continue to be developed and enhanced, and that future
versions will be upwardly compatible with all past applica-
tions. In addition, vendors desiring to maximize their
return on investment require that the source code for their
systems be proprietary and are unwilling to make it avail-
able to users under any but the most onerous restrictions.
Many of these changes, while acceptable for most users, are
diametrically opposed to what has made UNIX the research
platform of choice: low cost, wide availability of source
code, and leading edge technology.
System development can be likened to the process of
evolution. While gene mutation is critical to the advance-
ment of the species, only one in 100 mutations produces a
useful feature; the rest result in needless or detrimental
changes. The mere existence of an environment for mutation
is not enough -- some organization must bear responsibility
for brutally pruning the weak, outdated and useless ideas.
UNIX was fortunate in this sense. Unlike other projects
beset by competing groups jealously guarding their work from
one another, UNIX thrived in an open and cooperative commun-
ity willing to channel its ideas through a central clearing-
house (first Bell Laboratories and later the CSRG), in spite
of the clearinghouse's reputation for selective technical
scrutiny.
Here one must distinguish between the selection process
provided by research and commercial organizations. Research
organizations can base pruning decisions strictly on the
coherence of the system and the technical merit of the idea.
They need not concern themselves with how changes might
affect past variants of the system. Commercial organiza-
tions, though, must ensure that changes will not affect pro-
grams built to an obsolete interface. For example, paging
might be a great idea, but it will cause problems for
software that depends on the execution predictability of a
April 27, 2013
- 7 -
swap-based system, making it impossible for paging to
replace swapping. As a result, both schemes must be main-
tained, dramatically increasing the complexity of the sys-
tem. As the system becomes more complex, its evolutionary
paths will become increasingly restricted.
Here the role of a dynamic research version of UNIX
becomes clear. While it is only directly used by a small
group of people, it provides an important role as the
feedstock for the commercial versions of UNIX. Over the long
term, it is reasonable to expect that the most useful func-
tionality of the research systems will be grafted into the
commercial versions. Examples of ideas that began with BSD
and moved into commercial systems include the fast filesys-
tem, TCP/IP networking, and nearly half of the commands and
utilities.
The CSRG spends a significant amount of time collecting
prototypes of projects throughout the research world and
molding them together into a coherent and usable system.
Many of the ideas do not work out and are dropped in later
releases. The ability to experiment without concern for past
applications is critical. The resulting system is a third
the size and a fraction of the complexity of its roughly
equivalently functioned commercial brethren. This lean and
mean approach allows the system to evolve rapidly (the
nightmare of every commercial user, but the dream of every
researcher). A recent example of this type of experimenta-
tion is the prototyping of various proposed POSIX utilities
and interfaces by the CSRG. When drafts of the standard were
implemented, basic flaws in the specification became
apparent. These flaws and suggested solutions were presented
to the standards committees, resulting in changes to the
standard ensuring that the ratified standard could be effi-
ciently and correctly implemented. The research system users
also benefit from having a reference implementation of the
standard almost from the day that it is finalized.
Another major influence on the UNIX systems of the
future will be the NET/2 release. At least three separate
groups (two in the U.S. and one in Europe) have added the
necessary source code to the NET/2 release to make it a
fully functional UNIX system. As the NET/2 release was not
proprietary to any person or organization other than the
University of California and may be freely redistributed,
the cost of a UNIX system with source code will be less in
the future than in the current UNIX market by two orders of
magnitude. The UNIX single-server release by the Carnegie
Mellon University MACH group will also use the NET/2 release
as a starting point, making their release freely redistri-
butable without a UNIX source license. The advent of cheaply
available sources will make it far easier than ever before
for research groups and users to develop and exchange
software.
April 27, 2013
- 8 -
The role of designing and implementing a leading-edge
research version of UNIX is one that Berkeley is uniquely
equipped to fill. Future Berkeley releases will be oriented,
as they were in the early days of Berkeley UNIX, toward the
development and integration of a few well-chosen pieces of
new research into a leading-edge system.
4. References:
Joy83
Joy, W., E. Cooper, R. Fabry, S. Leffler, M. McKusick,
D. Mosher, ``4.2BSD System Manual,'' 4.2BSD UNIX
Programmer's Manual, Vol 2c, Document #68 August 1983.
Leffler84
Leffler, S., M.K. McKusick, M. Karels, ``Measuring and
Improving the Performance of 4.2BSD,'' Usenix Confer-
ence Proceedings, pp. 237-252, June, 1984.
McKusick85
McKusick, M.K., M. Karels, S. Leffler, ``Performance
Improvements and Functional Enhancements in 4.3BSD,''
Usenix Conference Proceedings, pp. 519-531, June, 1985.
McKusick87
M. McKusick, M. Karels, ``Directions of UNIX at Berke-
ley'', Digest of Papers of the Thirty-second IEEE Com-
puter Society International Conference, Compcon, San
Francisco, pp. 196-199, February 23-27, 1987.
McKusick89
M. McKusick, M. Karels, K. Bostic, ``The Release
Engineering of 4.3BSD'', Proceedings of the New Orleans
Usenix Workshop on Software Management, pp. 95-100,
April 1989.
April 27, 2013
Generated on 2013-04-27 00:20:00 by $MirOS: src/scripts/roff2htm,v 1.77 2013/01/01 20:49:09 tg Exp $
These manual pages and other documentation are copyrighted by their respective writers;
their source is available at our CVSweb,
AnonCVS, and other mirrors. The rest is Copyright © 2002‒2013 The MirOS Project, Germany.
This product includes material
provided by Thorsten Glaser.
This manual page’s HTML representation is supposed to be valid XHTML/1.1; if not, please send a bug report – diffs preferred.