Email Tom
Tom's Website
Email Brian
Brian's Website
|
Email Tom Tom's Website |
|
Email Brian Brian's Website |
Did you read the Preface? Thanks!
Gather ten system administrators together in a room and ask them to define their job. You'll probably get ten distinct, diverse answers. Why? Because system administration has a lot of aspects to it: some concrete and tangible, and some abstract and intangible. On the concrete side of the equation, today's typical administrator needs to have a strong grasp of all things hardware-related, a good understanding of network topologies and protocols, and how to make everything "play nice together" in the same "sandbox." The job frequently demands in-depth, hands-on experience with at least two or three operating systems. There's software to install and update, programs to reconfigure, and kernels to tweak. User accounts need to be created and removed. File systems require ongoing maintenance. Backups need to be scheduled and tested. The list goes on and on.
Now factor in the intangibles. Users have needs, and sometime those needs directly contradict system integrity or security. Did someone mention training (usually for a product the administrator has little or no direct experience with)? Network or filesystem design anyone? How about that new VPN that needed to be in place yesterday?
This chapter is takes a sweeping look at some of the concepts of system administration. While the balance of this book deals with the hands-on implementation of installing, configuring, and maintaining a Linux system, the following topics are meant to provoke thought and reflection. Keep in mind if there's any rule to system administration, it's that there are no rules per se; that there's more than one way to successfully implement something or solve a problem.
Several topics discussed in the following pages might at first glance appear to apply only to large networks with hundreds of users. We disagree. Whether you're tasked with the care and feeding of one standalone machine, or a complex network fed by a diverse range of operating systems, we believe the underlying principles are the same.
The job of a system administrator is complex. It's demanding, frequently requires the patience of Job, and definitely requires a good sense of humor. In addition, you need to be able to switch hats quickly and without breaking stride. System administration is a lot like working in the emergency ward of a hospital. You need to be a doctor, a psychologist, and - when your instruments fail - a mechanic. Often you need to work with limited resources. You need to be inventive in a crisis; you need to know a lot of facts and figures about the way computers work. You need to recognize that the answers are not always written down for you to copy, and that machines do not always behave the way you think they should. And, on top of everything else, you need to learn hundreds of new things a year.
While each administrator's knowledge base will vary according the hardware, software, and number of users they're supporting, one aspect of the job is consistent - attitude. Being a system administrator is as much a state of mind as it is about being knowledgeable. You need to be ready for the unexpected, resigned to an uncertain future, and able to plan for the unknown. It requires organization and the ability to be systematic. There is often no right answer, but there always a wrong answer. It's about making something robust that works as advertised, with a minimum amount of fuss, muss, or expense. In spite of stereotypes, today's system administrator is neither haphazard nor messy. Modern computing systems demand the very best of organizational skills and a professional attitude.
System administration also demands a certain degree of self-confidence. In the beginning, the novice is typically too nervous or worried about breaking something to get any real work done. Later, as experience is gained, aspects of the job become routine and second-nature. It's like learning to ski - there are three stages:
To get going, you need to know your stuff and have confidence in your abilities - but you also need to know your limitations.
A lot of people think that system administration is begins and ends with installing an operating system and creating a few user accounts. In fact, this is only the beginning. Correctly installing an OS and a handful of software programs is the easy part; keeping that system running, smoothly, without interruption or incident, for months at a time, is the challenge.
The days when one could fly a system by the seat of one's pants are over. System administration demands practical skills but nothing can replace real understanding. Today, keeping up with the theory is as important as upgrading the software. Successful system administration means approaching problems with the appropriate mindset. The following attitudes are ill-advised:
In contrast it is recommended to begin by:
System administrators need to be able to read documentation, to be able to communicate clearly with others, and to ask the appropriate questions when necessary.
It is important to start by being systematic from the beginning. Every time you read something, or learn some isolated fact, think: how does this apply to me? Do yourself a favor and take the time to write down your experiences. Start now. And try to find your own method for remembering what you have learned.
System administration is not just about installing operating systems. It's about planning, designing, and maintaining an efficient network of computers that are going to be used by people on an ongoing basis. On the computer side of the equation, this means:
Some system administrators are responsible for both the network hardware and the computers that it connects, that is, the cables as well as the computers. Some are only responsible for the computers. Either way, an understanding of how data flows from machine to machine is essential as well as an understanding of how each machine affects every other one. A central axiom for network administration is as follows:
Once a computer is plugged into the network, it no longer belongs to any one individual. It becomes part of a community of machines that share resources and communicate with the whole. What your machine does affects other machines. What other machines do affects your machine.
The ethical issues associated with connecting to the network are not trivial. Administrators are ultimately responsible for how an organization conducts itself electronically. This is a responsibility that must be exercised wisely and with care.
A system administrator wields great power. He or she has the means to read everyone's mail, change files, and to start and kill processes. This power can easily be abused and at times the temptation to play divine ruler is very alluring.
Tip
Another danger an administrator often gets caught up in is thinking a system exists primarily for their own entertainment and/or use, and that users are simply a nuisance to the smooth operation of a network. We'd like to gently remind all administrators that their role is defined by user needs, not the other way around.
The ethical integrity of a system administrator is clearly an important issue. Administrators for top secret government organizations and administrators for small businesses have the same responsibilities towards their users and their organizations. A quick glance at the governing institutions around the world should quickly tell you that power corrupts. And despite noble intentions, few individuals are immune to the temptations of such power at one time or other.
Administrators "watch over" backups, e-mail, private communications, and they have access to everyone's files. While it is almost never necessary to look at a user's private files, it is possible at any time and users do not usually consider the fact that their files are available to other individuals in this way. Users need to be able to trust the system and its administrator. Here are some thinks to think about as you mediate the systems for your users:
Objectivity of the administrator means avoiding taking sides in ethical, moral, religious or political debates. Personal views should be kept separate from professional views. However, the extent to which this is possible depends strongly on the individual and organizations have to be aware of this. Some organizations dictate policy for their employees. This is also an issue to be cautious with: if a policy is too loose it can lead to laziness and unprofessional behavior; if it is too paranoid or restrictive it can lead to hostile feelings within the organization. Historically, unhappy employees have been responsible for the largest cases of computer crime.
Note
Almost all the material in this chapter is generic to all variants of Unix. Feel free to substitute the word "Linux" for "Unix" anywhere it's used.
The following is a list of Tom and Brian's NEVER-DO's regarding root access and administrative privileges. Most of these were learned directly from the school of hard knocks - in other words, we've made our share of errors in the past and (in most cases) learned our lessons.
su command to gain root privileges when you need them and quit immediately after completing the task at hand. And never run ordinary programs with root privileges - doing so increases the risk of a system being compromised by a virus or a program accessing data with inappropriate permissions. System administration requires its operatives to know a lot of facts about hardware and software. The road to real knowledge is long and winding; where's the best place to start?
A top-down approach is useful for understanding a network's interrelationships. The best place to start is at the network level. In most daily situations, one starts with a network already in place - that is, one doesn't have to build one from scratch. It is important to know what hardware one has to work with and where everything is to be found; how it is organized (or not), and so on. Here's a general checklist:
Having thought about the network as a whole, now think about individual hosts/machines. First there's a hardware list.
Then there's the network implementation list:
And there are many other such questions, such as... Which machines provide the key services on the network? What kind of file sharing services are in use? (NFS? Samba shares? NT shares?) What permissions are allowed on the shares? Has anyone checked these permissions for accuracy lately? Does the network have its own DNS server? If not, who provides DNS services? Does the company run its own web server? Is it secured? Which machines provide firewall services? When's the last time the firewall was tested against an attack scenario?
Accurately documenting both machines and the network itself is extremely important to all installations, large and small. Reviewing (or creating documentation if it doesn't exist) is a good way for an administrator to become familiar with the tools at hand, how they're currently being utilized, and how they might be better deployed.
Finding and recording the above information is not only an important learning process, but an accurately documented network is crucial The information changes as time goes by. Networks are not static; they grow and evolve with time.
All operating systems are based on some form of hierarchical file system, and use directories and subdirectories to sort and organize the data they contain. Disks can also be divided up into logical partitions. The key reason behind directories and partitions is to logically distinguish files separately from each other. For example, you can have the following directory structure:
One of the challenges of system design is to find a directory structure for data that's simple, flexible, and allows users to immediately grasp where to find data and where the administrator expects them to save new files.
Hint
Data, programs, and operating system files should always be segregated at the very least by directory tree, and preferably by disk (or at least by disk partition). Mixing data and programs files with the operating system tree makes upgrading or re-installing an OS unnecessarily difficult.
It makes no sense to mix logically separate file trees. Operating systems usually have a special place for installed software. Regrettably, some vendors break this rule and mix software with the operating system's file tree. On Unix machines, the place for installed software is traditionally /usr/local; fortunately separate disk partitions can be placed anywhere in the file tree on a directory boundary, so this is not a practical problem as long as everything lies under a common directory. Under NT, software is often installed in the same directory as the operating system itself; also, thanks to Microsoft's concept of a central "Registry," re-installation of NT means re-installation of all program software as well.
The Filesystem Hierarchy Standard
The locations for the installation of various file types is delineated by the Linux Filesystem Hierarchy Standard (FHS), which is found at http://www.pathname.com/fhs/. One area of confusion is the location of locally installed software (as opposed to software that comes with the distribution. Our reading of the FHS indicates that small tools, utilities and assorted add-in software should indeed be placed in /usr/local. The /opt subdirectory is reserved for application level software, like the office suites that we reviewed in Chapter 11.
Data files installed or created locally are not usually subject to any location constraints; they can reside anywhere. One can therefore find a naming scheme that gives the system logical clarity. This benefits users and management issues. Again, directories are typically used for this purpose. Operating systems that are descended from DOS also have the concept of drive designators like A:, B:, C: and so on. These are assigned to different disk partitions. Some Unix operating systems have virtual file systems that allow administrators to add disks transparently without ever reaching a practical limit. Users never see partition boundaries. This has both advantages and disadvantages since small partitions are a cheap way to contain groups of misbehaving users, without resorting to disk quotas.
Each operating system has a model for laying out its files in a standard pattern, but user files are usually left unspecified. The naming convention and layout of these files is generally left to the discretion of the system administrator. Choosing a sound layout for data can make the difference between incomprehensible chaos and a neat orderly structure. An orderly structure is beneficial not only to the users of the system, but also when making backups. Some relevant issues are as follows:
It is good practice to consolidate file storage into a few special locations rather than spreading it out all over the network. Data kept on many machines can be difficult to manage compared to data collected on a few dedicated file servers. Also, insecure operating systems offer no protection for files on a local disk.
The site/host/purpose model of organizing a filesystem is but one of many. It has an advantage over some other schemes in that anyone who knows the naming rules can quickly identify the host and function of a network resource. Also, it falls nicely into a hierarchical directory pattern. A simple but effective implementation is to use a three level mount-point for adding disks: each user disk is mapped onto a directory with a name of the form: /site/host/purpose .This approach works well even for large organizations and can be extended in obvious ways.
Within an organization, using this structure provides a global naming scheme, like those used in true network filesystems like AFS, NFS, and DFS. These use the name of the host on which a resource is physically located to provide a point of reference. This is also an excellent way of labeling backups of partitions since it is then immediately clear where the data belongs. A few rules of thumb allow this naming scheme to live painlessly alongside traditional Unix naming schemes.
It doesn't matter whether software compiles in the path names of special directories into software as long as you follow the previous points. The first link in the mount point is the part of the organization or site that the host belongs to, the second link is the name of the host to which the disk is physically connected, and the third and final link is a name that reflects the contents of the partition. Some examples are as follows:
/syroidmanor/janus/local/usr
/syroidmanor/hydras/home
/syroidmanor/hydras/storage
/orbdesigns/gryphon/home
/orbdesigns/grendel/data
/orbdesigns/grendel/db
The problem of drive names under NT and Windows is awkward, especially if your goal is Linux/NT interoperability. In practice, many networks based on NT and Windows use Microsoft's model throughout, and while it might not gleam with elegance, it does the job. The problem of backups is confined to the domain servers, so the fact that Windows is not a fully distributed operating system restricts the problem to manageable proportions.
Unlike many other systems, Unix and its newer sibling Linux are often used by people who write their own software rather than relying on off-the-shelf products. The Internet contains gigabytes of software for Unix systems that are completely free. Large companies like the oil industry and newspapers can afford off-the-shelf software for Unix, but most individuals can't.
There are therefore two kinds of software installation: the installation of commercial software and the installation of freeware. (Either free as in the public domain, or free as in licensed under a sharing license, like the GNU GPL). Commercial software is usually installed from a CD by running a simple script and by following the instructions carefully; the only decision you need to make is where you want to install the software. Freeware usually comes as source code and must therefore be compiled. Unix programmers have gone to great lengths to make this process as simple as possible for system administrators.
The first step in installing software is to decide where you want to keep it. You can, of course, put the software anywhere but keep the following in mind:
The directory traditionally chosen for installed software is called /usr/local. Within that sub-hierarchy, create the sub-directories /usr/local/bin and /usr/local/lib, and so on. Linux has a de-facto naming standard (set out by the aforementioned FHS) for directories that you should try to stick to so others can quickly grasp the structure of a system.
bin - Binaries or executables for normal user programssbin - Binaries or executables for programs that only system administrators require. Those files in /sbin are often statically linked to avoid problems with libraries which lie on unmounted disks during system bootlib - Libraries and support files for special softwareetc - Configuration filesshare - Files which might be shared by several programs or hosts: For instance, databases or help information; other common resourcesBelow is one suggestion for structuring installed software.
/usr/local
|
---------------------------------------------
| | |
bin/ gnu/bin site/bin
lib/ gnu/lib site/lib
etc/ gnu/etc site/etc
sbin/ gnu/sbin site/sbin
share/ gnu/share site/share
In this example, we use three categories: regular installed software, GNU software, and site software. The reasons for this are as follows:
ls and cp. GNU software has its own system of installation and set of standards. GNU will also eventually become an operating system in its own right, and should therefore be kept separate. When installing software, you will usually be asked for the name of a prefix or location for the package. The prefix in the above cases is /usr/local for ordinary software, /usr/local/gnu for GNU software, and /usr/local/sitename for site specific software. Most software installation scripts place executables under bin and lib automatically.
To begin compiling software, you should always start by looking for a file called README or INSTALL. This will tell you what you have to do to compile and install the software. In most cases, you will only have to type a couple of commands, as in the following example.
The following example illustrates the GNU method of installing software from sources. The steps are:
ncftp for painless anonymous login.tar zxvf tar software.tar.gz.cd software../configure [options].make.make install. This should be enough to install the software. Some installation scripts leave files with the wrong permissions so ordinary users cannot access the files. Always check new program installations and make sure the executable binaries (the ones you want average users to run, that is) have file permission of 755. Again, most installation scripts ensure the correct permissions bits are set, and if for some reason they are not, this is usually noted in the program's documentation.
This procedure should be more or less the same for just about any software an administrator encounters. Older software packages sometimes provide only Makefiles that you must customize yourself. Some X11-based windowing software requires you to use the xmkmf X-make-makefiles command instead of configure. You should always consult a program's INSTALL file.
Systems that use shared libraries or shared objects sometimes need to be reconfigured as new binaries are added to the system. This is because the names of the libraries are cached for fast access. The system will not look for a library if it is not in the cache file.
To register a library directory with the operating system, add it to /etc/ld.so.conf. Then run ldconfig to update the /etc/ld.so.cache file. ld.so is the Linux utility that loads the shared code libraries that a program needs. It looks in several default locations for the libraries, such as /lib and /usr/lib, before reading /etc/ld.so.cache.
Beating you about the head...
Always scour a program's README file before installation - especially when you're updating an existing version. On occasion, new libraries are required by program updates and failure to install them will either break the product completely, or leave it in a very unstable state. This is especially relevant to administrators who prefer to compile programs from source code, as opposed to package releases that incorporate automatic dependency resolution or notification.
Some software (especially some free software) suffers from frequent update cycles. You could easily spend your entire life just chasing the latest versions of your favorite software packages. Don't!
On the other hand, some program updates contain important security fixes or features that benefit user productivity. There's a fine line here, of course, and each administrator has to weigh an update's benefits against the time invoked in applying it and potential user re-training or re-familiarization.
We do not recommend upgrading front-line server applications "cold turkey." Ideally, install the update on a second tier server and test it thoroughly before putting it into active duty. If extra hardware is at a premium or simply unavailable, consider installing the product in a different location for testing purposes and run it in conjunction with your existing version. If neither of these options works for you, at the very least don't install an upgrade the day it hits the streets - monitor the product's mailing lists, and let other people do the testing for you.
In this chapter we examined several philosophical [CT9]issues pertaining to the role of system administrator, plus looked at three generic topics regarding efficient system design and implementation: Getting to know the topology of a network, planning a file system, and installing software.
Go to the Table Of Contents