Date: Sat, 07 Sep 2002 19:40:46 GMT Newsgroups: linux.redhat.install, comp.os.linux.security, comp.os.linux.questions, comp.os.linux.admin
Date: Sat, 07 Sep 2002 19:40:46 GMT Newsgroups: linux.redhat.install, comp.os.linux.security, comp.os.linux.questions, comp.os.linux.admin
> I am conducting a survey of what top administrators and hackers consider > BEST practices for disk partitioning on linux systems. Answers should > be non-distribution specific and should examine journaling v. > non-journaling file systems. > Consensus answers will be used to set corporate policy for future > systems integration projects.
Ok, let's start with something.
Goals:
separate read/write and read-mostly/read-only filesystems
minimise root fs; minimise writes to root fs
disallow public writes to root fs
Rationale:
in events of disaster, partitions with the most disk write activity are the most prone to corruption
try to get as much static (read-only) data off from the actively changed filesystems to minimise the loss in disasters
minimise size of actively changed filesystems to speed up post-disaster recovery
a minimally sized root fs is less prone to corruption, and in the case of corruption fsck has better chances to succeed
minimising changes on root fs help this, too
tolerant root fs with necessary tools to resue rest of the system after a disaster
filling up root fs can cause loss of log information and loss of configuration data
So, fs list: / (/boot) /tmp /var /usr /home swap
The /boot is more or less optional; I see it mostly beneficial when sharing the machine with another OS; it's then possible to just allocate a minimalistic /boot on the boot drive, and have rest of the linux installation on a separate drive.
For more resilience, add
/var/tmp
(same rationale as with /tmp; in this case check also other ways users can maliciously fill up /var)
For mail servers, add
/var/spool/mail
(and possibly add also /var/spool/imap, if using Cyrus imapd)
(again to restrict most disk activity to a more limited partition, also to segregate disk consumption by mail content and mail logs)
For news servers, add
/var/spool/news
For web cache servers, add some fs for web cache content (/var/spool/squid, f.ex.)
Then the next question is, how these should be located on a disk or shared across disks (when having the luxury of multiple independent disks — and note that IDE drives sharing a channel are not independent, whereas SCSI drives sharing a channel are)
I tend to group the actively updated filesystems as closely together as I can, with read-mostly filesystems residing on both sides of this "write zone". This might just be a superstition, though, but I expect some added performance by minimising the average seek distance.
From the list above, the various filesystems beneath /var/spool are likely to benefit from a disk of their own (if the servers are really utilised).
Wolf a.k.a. Juha Laiho Espoo, Finland
OK, here goes my description of how I partitioned a new 31.5GB harddisk half a year ago and the rationale behind it:
hda1: 31MB FAT16 C: hda2: 31MB ext2 /boot hda3: 2047MB FAT16 D: hda4: 29GB Linux etended hda5: 478MB ext3 / hda6: 509MB ext3 /home hda7: 1529MB swap hda7: 1529MB swap hda8: 25GB ext3 misc
The two FAT partitions just exist in case I some time want to run some of my old DOS programs in the real DOS environment. This far I have only used it once to run some timing critical parallel port communication program.
/boot is always a nice thing to have. I keep it small and simple. Most people can do with just 8MB, but since I do a lot of kernel hacking, I like to have it slightly larger. And 31MB is really no big deal with a harddisk this size.
The size of /home was choosen so that it can always be backed up on a single CD, and I can use the rest of the space for redundant encoding. This means the backup will still be usable in case of a few bad sectors on the CD. Of course a multiuser system would need a larger /home and other backup procedures.
The size of / was choosen so that it can also be backed up on a single CD together with /boot. I wanted to make the CD bootable. This turned out to be quite tricky, I had to do some modifications of startup scripts, to make it work nicely.
I have /usr on my misc partition and /usr/local is on my / partition. This means I had to use bindmounts to get /usr/local in place. I first bindmount / onto /mnt/root (in fact I don't use the bind option, but rather mount the device again, bind mounts of the root of a filesystem confuses some startup scripts.) then I mount /usr and bindmount /mnt/root/usr/local onto /usr/local.
The /mnt/root is also usable for easy creation of backups of the root filesystem, therefore I also bindmount /boot onto /mnt/root/boot, because I want it on the same backup.
The misc partition is mounted on /mnt/misc, from there I bindmount /mnt/misc/tmp to /tmp and /mnt/misc/usr to /usr. The rationale behind this is, that both directories are places where I want lot of space, and backup is not necesarry. People have suggested this was a bad idea, because /usr should be mounted readonly. Well I have f*cked up a system by forgetting to remount /usr readonly before installing/removing packages with rpm.
I have to swap partitions, because I occationally want to do some experiments with one of them.
I want to be able to use rpm without /usr mounted, unfortunately that doesn't work right away. So I copied /usr/lib/rpm and a few needed libraries to /mnt/root/usr. I also copied a few executables from /usr/bin to /usr/local/bin, because I use them in my customized startup scripts.
All I have written here just serves as inspiration. I was able to get a system working like I wanted, and I do have some bootable bakcups.
But I found this not to be a perfect setup. Because when I finally had a diskcrash on this harddisk earlier this week, I found I still didn't want to simply forget about the misc partition. Well, fortunately I just lost my / partition, and I had a backup only a few days old.
Another problem is if I eventually run into unrecoverable logical errors in a filesystem. (I did fear that has happened to one of my partitions when rpm -K gave a lot of random errors. It turned out to be caused by a filled up /var/tmp.) With this in mind I decided that on my new harddisk no partition was allowed to be larger than one third of the disk. So when I had created the usual small partitions and found only 10% used, I split the rest into 3 equal sized partitions.
I guess I should use one of those for /tmp, then I will always be able to just use mkfs on the tmp partition and copy the bad filesystem there. Then mkfs the old and then switch the roles in fstab. I should also get /var/tmp located somewhere else.
I hope some of this will be helpfull to some people.
Kasper Dupont
> > >> Note that e.g. on Solaris (7 & 8, at least) /bin & /sbin are > > >> symlinks to /usr/bin, /usr/sbin resp. by default. > > > > > > If that is the case Solaris does not comply with FHS. > > > > Heh. Of course it doesn't: it is not linux.
Oops. It's only /bin -> /usr/bin, they have enough sense to leave /sbin as separate directory. But you knew that.
> And with the question being about how to partition a Linux > system, what does it really matter if Solaris complies with > FHS or not. > > > That doesn't mean their way is inherently worse -- or better. > > The FHS does describe some principles and reasons behind > them. Is there a description of how Solaris does, and why?
I don't have the cite; my personal impression is that linux standards tend to over-specify things while solaris leaves more to the sysadmin's discretion. Which is fair enough considering that Solaris shops tend to have professional sysadmins whereas most linux installations…
BTW, the tendency to overspecify things makes linux standards an amusing read at times: e.g. check out the difference between /bin, /sbin, and /usr/bin in FHS:
/usr/bin is for binaries that are "not needed in single-user mode", /bin is for binaries that may be used by "all users", /sbin is for "root-only commands".
Which means that /bin is for all binaries that can be run by non-root in single-user mode. Suddently /bin -> /usr/bin symlink is beginning to look very sensible indeed…
Dima
> Which means that /bin is for all binaries that can be run by > non-root in single-user mode.
Actually not. It is for executables that is needed by all users AND is needed in single user mode. It is not about all users in single user mode, that wouldn't make any sense. One example is ls, which any user want to use on a usual working system, and which is also needed by root when the system is in single user mode.
So the way to decide is:
First find out if the executable is needed in single user mode, and from that choose between / and /usr.
Second find out if this command is usable by anybody but root, and from that choose between bin and sbin.
> Suddently /bin -> /usr/bin symlink > is beginning to look very sensible indeed...
I don't think so. I mean no ls command in single user mode. How am I supposed to do any system maintainance without access to the ls command?
Kasper Dupont
> Especially for security reasons there are partitions that > would be nice to have mounted read-only. Only problem is > there are too many things in /usr and /var that are written > to that would also be nice to have read-only.
The /usr and /var directories was introduced to split the readonly parts from the readwrite parts. Everything that is readonly should go in /usr while everything that is readwrite should go in /var. The only case where writes should happen to /usr is when you install/uninstall software. OTOH writes to /var could happen all the time.
Kasper Dupont
I would definitively have the following as partitions of their own:
/ /usr /var /home
If I distributed /usr over a netowor or had it on a CD-ROM… or intended to add lots of programs I probably also would make a /usr/local.
With large packages like WordPerfect and StarOffice, it would be useful to put them in /opt.
/usr/src and/or /usr/local/src may be put on seperate partition, if for no other reason, than for allowing writing to the directory while /usr and /usr/local were mounted read-only (which they ought to be). Another reason, could be if you had very much source- code floating about.
It could also be useful to put /var/mail (formarly /var/spool/mail) and /var/spool/news on partitions of their own… either because you wanted to distribute these directories over a network, or because they would take up an unreasonable amount of /var… which they probarbly would on a dedicated news and/or mail-server.
It could perhaps also be an idea to seperate /var/spool/lp as a partition on system where many would attempt to print very large documents.
Koppe
Newsgroups: gmane.linux.debian.user Date: Fri, 13 Aug 2004 16:34:25 +0800
>2. Is it better for the swap to be on a different drive? If so, how does one >move it?
Don't move it. I recommend a swap _file_ not a parition. It's more flexible wrt resizing if you need. and if/when you use a 2.6 kernel there are no peformance benefits to swap partitions. Iff there ever were on desktops.
Swap belongs on your least busy drive, in your busiest partition.
Least busy drive reduces contention. Busiest parition reduces average seek.
John Summerfield
>4. Swap file vs swap partition--I did not know there was such an alternative >in linux. IF a file is better, how do I change over? (Swap is almost never >used, it seems.)
A word about swap space.
Although you can create swap space as a file, instead of the kernel handling a raw partition to swap, all swapping must go through the file system, which slows swapping considerably. Ideally, you would always want swap as a partition.
With that said, if you have 2 disks, it's important to have them both installed as IDE primaries, never having two disks on the same IDE channel as primary and secondary. The idea is that your system can read/write to the two primaries at the same time, whereas with a primary and secondary setup, the read/write must alternate between the primary and secondary.
Now, if you have your two disks installed and IDE primaries, and you create a swap partition on both disks, and both disks are set to the same priority, say 1, then the net effect is that when swapping occurs, data is written to both swap areas interleaved. This is the fastest way of handling swap space.
With two disk set as IDE primaries, further speed gains can be had by splitting your busiest partitions on the two different drives. As an example, I would say that /usr and /var are probably the system's busiest partitions. If the /usr partition is on /dev/hda and /var is on /dev/hdc, you would probably see slightly better system performance.
Mike
> > Although you can create swap space as a file, instead of the kernel > > handling a raw partition to swap, all swapping must go through the > > file system, which slows swapping considerably. > > Not so in 2.6. > > This doesn't address this particular issue, but it does explain why it > might be so: > http://www.osdl.org/docs/linux_journaling_filesystems_and_workloads.pdf[] > > I found it looking for a comparison of filesystems; that's about page 28. > > > Ideally, you would always want swap as a partition. > > I do not believe that is so on single-drive systems. Consider the > amount of time seeking between data and swap paritions, I can't see > how any optimisations can overcome the laws of physics.
What laws of physics pertain to the issue of swap being a file versus swap being a partition? It really doesn't matter which type of file system you use. One may be faster that the other. One may be better for recovering from a system crash. One may be better for logical volume management. But none are faster than the kernel using a partition for swap in raw mode. That's why AIX, Solaris, HPUX and Linux, by default, use swap partitions, not files.
> > With that said, if you have 2 disks, it's important to have them both > > installed as IDE primaries, never having two disks on the same IDE > > channel as primary and secondary. The idea is that your system can > > read/write to the two primaries at the same time, whereas with a > > primary and secondary setup, the read/write must alternate between > > the primary and secondary. > > > > Now, if you have your two disks installed and IDE primaries, and you > > create a swap partition on both disks, and both disks are set to the > > same priority, say 1, then the net effect is that when swapping > > occurs, data is written to both swap areas interleaved. This is the > > fastest way of handling swap space. > > NO NO NO. > best place for swap, least-used drive. > Best parition - busiest.
Give more of an explanation than this. I suppose that setting a single disk to be used solely for a swap partition would be good although a bit impractical.
> > With two disk set as IDE primaries, further speed gains can be had by > > splitting your busiest partitions on the two different drives. As an > > example, I would say that /usr and /var are probably the system's > > busiest partitions. If the /usr partition is on /dev/hda and /var is > > on /dev/hdc, you would probably see slightly better system performance. > > You'd be surprised. > > _I_'d guess /usr isn't used all that much, and /home gets thrashed.
Well I suppose that depends, now, doesn't it Where are all the system commands a user would use, /usr/bin, /usr/sbin and possibly /usr/local/bin(not used that much in Debian). Where is X, /usr/X11R6. Where are the libraries and header files for compiling, /usr/lib and /usr/include. Where is most of the system documentation, /usr/share. Another location for user commands could be /opt. But then, Debian doesn't really utilize /opt. All logs are in /var. Most all configurations are in /etc. So what's thrashing around in /home?
> I was really amazed on OS/2 some years ago when I put swap on my data > drive. The system just about died.
Well, UNIX and Linux is not OS2. All MS Windows machines put swap in a paging file, and any MS Windows machine is the worst example of a tuned VMM.
Mike
> With that said, if you have 2 disks, it's important to have them both > installed as IDE primaries, never having two disks on the same IDE > channel as primary and secondary. The idea is that your system can > read/write to the two primaries at the same time, whereas with a primary > and secondary setup, the read/write must alternate between the primary > and secondary.
I thought this as well. But I was also told that if two differeing devices share and IDE, the slower one sets the rate. (This may be the cause of all those error-51 messages.) So if I put the two disks on two IDEs as primaries and the CDs as secondaries, the results may not be good.
However, I originally set things up similarly and had no complaints. So … what is the truth here.