Newsgroups: comp.os.linux.misc Date: 9 Apr 2003 14:38:40 GMT
Newsgroups: comp.os.linux.misc Date: 9 Apr 2003 14:38:40 GMT
> > Probably you can mount ext3 with some options enhancing performance, > > like "noatime". 'man mount' > > On a side note, what is the ramification of noatime? I've been told > that will stop the disk accessing every few seconds, which I'd like to > do on my home computer to reduce wear.
Modern disks are built to run 24/7. Including "noatime" won't increase the lifespan of your disks, it'll just make seeks and writes less frequent, speeding up disk access in most cases. If you want to make your disks last longer, leave them on all the time. Most disk failures occur at spinup (just as most light-bulb failures occur at initial power-on.)
> But I don't really understand what the ramification of stopping inode > access time means?
It doesn't mean much to the normal user. If you do "ls /bin", every file in /bin gets its inode accessed. Typically, the files you're interested in will have been accessed constantly, so the atime doesn't really convey much useful information. If you mount a filesystem with "noatime", then neither the -atime option to find nor the -u option to ls will give you valid data. Typically, you want the last modification time (-mtime) anyway.
Matt G
> On a side note, what is the ramification of noatime?
My laptop has had everything set to "noatime" for several years and I haven't noticed any problems. It is possible for programs that care about atime to fail if it isn't updated, but that seems fairly rare in practice.
> I've been told that will stop the disk accessing every few seconds,
Atime updates cause inode updates whenever you read a file. This causes writes to disk even if you haven't explicitly written anything. I turn off atime on my laptop because if I don't the disk won't spin down.
Bob Hauck
Posted by: Linegod on Aug 30, 2001
After reading various articles on the virtues of different Journalled Files Systems, I decided to take the plunge and convert my test box over to ext3. Armed with the latest Mandrake kernel and e2fsprogs I set out to bring my drives into the 21st Century :)
The process itself could not have been simpler, and as the title of the article suggest, was done with the following 4 steps:
unmount 'not in use' file systems
run 'tune2fs -j /dev/hdxx' against all file systems
edit '/etc/fstab' and change 'ext2' to 'ext3'
reboot
That's all there was to it. The reason for unmounting 'not in use' file systems is that it will hide the '.journal' entry after creation, where on mounted filesystems it will be visable.
To 'shock test' the system (not recommended), I simply cut the power to the box and watch it boot up. All but the '/' file system, which still ran fsck, came back up without a hitch. I believe the '/' problem is due to the new 'devfsd', but will do some tests to confirm this.
If you are looking for a quick and easy solution for journalled file systems, I'd say ext3 is the way to go.
> did u manage to mount it data=writeback? wouldn't work for me....but should > be a lot faster.
Haven't tried that yet, but I will…..
No luck here either. Must tinker some more….
pasnak
The /sbin/elvtune program can be used to trade off between throughput and latency. For example, you might run: /sbin/elvtune -r 1024 -w 2048 /dev/sdd to change the elevator settings for the device /dev/sdd (including all the partitions on /dev/sdd). Changes to the elevator settings for a partition will apply to the elevator for the device the partition is on; all partitions on a device share the same elevator.
Once you have found elvtune settings that give you your most satisfactory mix of latency and throughput for your application set, you can add the calls to the /sbin/elvtune program to the end of your /etc/rc.d/rc.local script so that they are set again to your chosen values at every boot.
There are some characteristic loads that show very significant speed improvement with the data=writeback option. In those cases, the data consistency guarantees are essentially the same as the ext2 file system; the difference is that the file system integrity is maintained continuously during normal operation.
One of these cases involves heavy syncronous writes.
Other cases involve creating and deleting large numbers of small files heavily, such as delivering a very large flow of small email messages.
If you switch from ext2 to ext3 and find that your application performance drops substantially, the data=writeback option is likely to give you a significant amount of performance back; you will still have some of the availability benefits of ext3 (file system is always consistent) even if you do not have the more expensive data consistency guarantees.
Tong
It is because / is already mounted as read only at boot. And remounting with data=writeback is not possible because at boot it is mounted as data=ordered. So, now to mount as writeback at boot?
Danny
Converting a ext3 partition back to ext2 is an easy task, as shown in the following 3 easy steps. However, it can also be extremely troublesome. The following steps will make conversion hassle free. For the reasons behind it, and cases that may go wrong (and aftermath), read on the following articles.
changed /etc/fstab to mount as ext2
Boot with 'linux single'
mount / -o remount,ro umount <everything else>
run 'tune2fs -O ^has_journal /dev/hda[x]'
That's it.
Optionally you can use parted to confirm the partitions are as ext2.
Tong
Newsgroups: comp.os.linux.misc Date: Sat, 12 Apr 2003 15:30:35 GMT
> The filesystem HOWTO says that it is "easily" done, but doesn't mention > how to do it. Havn't been too successful in googling for an answer. > Somebody give me a pointer please?
Well, it's in the man page (tune2fs), but even some fellow instructors have had trouble figuring it out:
tune2fs -O ^has_journal /dev/hda5
will turn off (and, I believe, remove) the journal. As OP has stated, an ext3 fs can probably be mounted as ext2, so it's not mandatory to turn off the journal to use the fs, but it could be confused if the journal is still there and stale the next time the fs is mounted.
Also, some utilities (specifically, previous versions of Partition Magic) cannot countenance an ext3 fs and won't deal with it. Turning off the journal satisfies PM.
Dave Brown
Newsgroups: linux.debian.user Date: 2003-02-13 13:10:12 PST
> how can I convert a ext3 partition to ext2? Is it only to change the > type in fstab and reboot? How can I synchronize the modification stored > in the journal file?
Simply mounting it is the easy way and should be ok if your filesystem has been correctly unmounted. If you want to delete the journal itself, run "tune2fs -O ^has_journal /dev/xxx", run e2fsck and delete the ".journal" file.
Dennis
Yep. Just change the fstab entry from ext3 to ext2, and then reboot. The journal is flushed on a clean umount: it only needs to replay if your computer crashes.
Once you've rebooted, you can use tune2fs to remove the journal itself.
Mark Ferlatte
How to move back from ext3 to ext2
It's quite easy. If you unmount an ext3 filesystem cleanly, then you can remount it as ext2 without any other commands. If you crash and are left with an unclean ext3 filesystem, on the other hand, the filesystem will prevent you from mounting it as ext2: it is not safe to mount it until you have recovered the journal, and the only way to do that for now is to mount it as ext3.
However, if for any reason you do have an ext3 filesystem which you want to convert permanently back to ext2, whether it was cleanly unmounted or not, you can use "debugfs" from e2fsprogs-1.17 or later to do it. First, run debugfs and open the filesystem (the -w flag means open for write, and the -f flag forces it to open the filesystem even if there are unknown journal flags set):
[root@sarek /root]# debugfs debugfs 1.18, 11-Nov-1999 for EXT2 FS 0.5b, 95/08/09 debugfs: open -f -w /dev/sdb1
Now, use "features" to see which feature bits are set on the filesystem:
debugfs: features Filesystem features: has_journal filetype sparse_super
We want to clear the journal bits, then we can quit:
debugfs: features -has_journal -needs_recovery Filesystem features: filetype sparse_super debugfs: quit [root@sarek /root]# debugfs
That's it!
Levi Waldron
Ok, what I did was booted with 'linux single' then ran 'umount /' and 'umount /boot'. Then I ran 'tune2fs -O ^has_journal /dev/hda[x]' on both drives and changed /etc/fstab to mount as ext2.
After that the system wouldn't boot complaining about a missing journal. What am I doing wrong? I have to use the CD to rescue and go back a recreate the journal.
How can I get rid of ext3 permanently and have the system boot up to ext2?
Warren Bell
> > How can I get rid of ext3 permanently and have the system boot up to > > ext2? > > Edit /etc/fstab, that's all (and you can delete the journal file if > you want to)
I did that and the EXT3 stuff at boot was still showing up. And when I deleted the journal file the system wouldn't boot.
I read that the only real advantage of ext3 is skipping the fsck at startup. But when my system locked and I rebooted it gave me the option to do a system check. I choose yes and it's the same speed doing the fsck as ext2 was.
Now I think I have ext3 disabled. I had to reinstall the kernel wile running in ext2, then edit lilo.conf and remove the initrd line, then go into single mode and remove the journal with tune2fs -O ^has_journal /dev/…
But there's still the choice to skip fsck at boot, the one where it counts down from 5 seconds. Is this part of ext3 or is redhat just giving a choice now to fsck or not?
Also, after getting everything back to ext2 I shut the computer off in X just to test the fsck out. When I boot and X first starts I still get a flash on the screen of the desktop in the state it was when I powered off with no shutdown. I noticed it doing this when I first installed 7.2 with ext3 and thought it might be somthing to do with the journal saving the last state of the system. Since I'm still seeing this, it ext3 still active on the system?
> But there's still the choice to skip fsck at boot, the one where it > counts down from 5 seconds. Is this part of ext3 or is redhat just > giving a choice now to fsck or not?
I got everything working except that there's still the chioce at boot to skip the fsck or choose yes. But now that I've removed ext3, if I choose yes to fsck it does it. If I let the time run out to skip it it does it anyway. Does anyone know how to remove this prompt to fsck? Without ext3 it doesn't do any good exept slow the boot time.
Warren Bell
Newsgroups: comp.os.linux.questions Date: 2002-05-17 22:24:03 PST
i have redhat 7.2 on a 40 gb disk… / is /dev/hda10 about 5 gb. it was ext3 till recently.
i wanted to make it ext2, so i used tune2fs to remove the journal. (tune2fs -O^has_journal /)
then i edited /etc/fstab.
i used parted to see the partition list and parted shows / now as a ext2 partition.
till here things r fine. but when i reboot, linux kernel tries mounting / read-only .. it still thinks / is ext3 and ext3 module gives up, says no journal, so kernel hangs up coz no /'s been found.
sanketh
Might be the reason that he did not mount the / filesystem in read only mode when changing it… nothing revealed in the newsgroup postings.
Newsgroups: comp.os.linux.security Date: 2002-01-04 13:14:13 PST
> Does anybody know how to convert a partition (without loosing data) from > 3->2 format ?
Yup.
Mount it as ext2, and then remove the journal inode by doing
tune2fs -O"^has_journal"
and the inode will be freed up & everything should be back to normal ext2.
~Tim
> > Well after fighting with this I finaly got it to work. It's not > > sufficient to just remove the journal and remount as ext2. The boot image > > loads support for ext3 and jbd and then complains that it can't find a > > journal file, kernel panic. > > > > I had to boot from CD, linux rescue, vi the > > /lib/modules/2.4.9-13/modules.dep and remove EXT3 & JBD. Then mkinitrd a > > new /boot/initrd-2.4.9-13.img. > > What a crap idea. What happened with the `tune2fs' command? Why did it > think any journal was required if you were mounting as ext2?! > > ~Tim
No one want to do things the hard way. He did it because he had to.
yes. mount it as ext2. it's fully backwards compatible, though of course you won't get the journalling features.
[snip] > If drive image 5.0 recognises ext2, in principle ext3 > shouldn't have made any difference.
You might still need to change a few flags in the superblock - things expecting ext2 don't always cope with ext3 just because it's *compatible*.
~Tim
> > You might still need to change a few flags in the superblock > > You might. I would be wary of continuing to use some magic tool on my > partitions that was evidently incompatible with the FS. Cart before the > horse if you start deciding that your DOS programs are the measure of > what ext2 looks like.
Well, the tune2fs -O"^has_journal" that I suggested earlier was there to overcome exactly this problem. It's been known to flush all the ext3- specific bits in my superblocks before now.
I wouldn't worry, anyway; I've been known to fling filesystems around between ext2, ext3, reiserfs and XFS (massive copying across, obviously) with much abandon before now :8)
~Tim
Michael K. Johnson
Why do you want to migrate from ext2 to ext3? Four main reasons: availability, data integrity, speed, and easy transition.
After an unclean system shutdown (unexpected power failure, system crash), each ext2 file system cannot be mounted until its consistency has been checked by the e2fsck program. The amount of time that the e2fsck program takes is determined primarily by the size of the file system, and for today's relatively large (many tens of gigabytes) file systems, this takes a long time. Also, the more files you have on the file system, the longer the consistency check takes. File systems several hundreds of gigabytes in size may take an hour or more to check. This severely limits availability.
By contrast, ext3 does not require a file system check even after an unclean system shutdown, except for certain rare hardware failure cases (e.g. hard drive failures), because the data is written to disk in such a way that the file system is always consistent. The time to recover an ext3 file system after an unclean system shutdown does not depend on the size of the file system or the number of files; rather, it depends on the size of the "journal" used to maintain consistency. The default journal size takes about a second to recover (depends on the speed of the hardware).
Using the ext3 file system can provide stronger guarantees about data integrity in case of an unclean system shutdown. You have a choice of how carefully to protect your data. Essentially, you can choose either to keep the file system consistent but allow for damage to data on the file system in the case of unclean system shutdown (for a modest speed up under some but not all circumstances) or to ensure that the data is consistent with the state of the file system (which means that you will never see garbage data in recently-written files after a crash.) The more safe choice to keep the data consistent with the state of the file system is the default.
Despite writing some data more than once, ext3 is often faster (higher throughput) than ext2 because ext3's journaling optimizes hard drive head motion. You can choose from three journaling modes to optimize speed, optionally choosing to trade off some data integrity.
ne mode, data=writeback, limits the data integrity guarantees, allowing old data to show up in files after a crash, for a potential increase in speed under some circumstances. This mode, which is the default journaling mode for most journaling file systems, essentially provides the more limited data integrity guarantees of the ext2 file system and merely avoids the long file system check at boot time.
The second mode, data=ordered (the default mode), guarantees that the data is consistent with the file system: recently-written files will never show up with garbage contents after a crash.
The last mode, data=journal, requires a larger journal for reasonable speed in most cases and therefore takes longer to recover in case of unclean shutdown, but is sometimes faster for certain database operations. The default mode is recommended for all general-purpose computing needs.
It is easy to change from ext2 to ext3 and gain the benefits of a robust journaling file system, without reformatting. That's right, no need to do a long, tedious, and error-prone backup, reformat, restore operation in order to experience the advantages of ext3. There are two ways to do the transition:
The Red Hat Linux installer program will offer to transition your file systems when you upgrade your system. All you have to do is check one checkbox per file system.
The tune2fs program can add a journal to an existing ext2 file system. If the file system is already mounted when it is being transitioned, the journal will be visible as the file ".journal" in the root directory of the file system. If the file system is not mounted, the journal will be hidden and will not appear in the file system. Just run tune2fs -j /dev/hda1 (or whatever device holds the file system you are transitioning) and change "ext2" to "ext3" on the matching lines in /etc/fstab. If you are transitioning your root file system, you will have to use an initrd to boot; run the "mkinitrd" program as described in the manual and make sure that your lilo or grub configuration loads the initrd. (If you fail to make that change, the system will still boot, but the root file system will be mounted as ext2 instead of ext3 — you can tell this by looking at the output of the command "cat /proc/mounts") More information on tune2fs can be found in the tune2fs man page.
A list of reasons Red Hat chose ext3 for our first supported journaling file system follows. Note that these reasons are not necessarily each unique to ext3 (some other journaling file systems share several of the points here) but the whole set of reasons taken together is unique to ext3.
ext3 is forwards and backwards compatible with ext2, allowing users to keep existing file systems while very simply adding journaling capability. Any user who wishes to un-journal a file system can do so easily. (Not that we expect many to do so…) Furthermore, an ext3 file system can be mounted as ext2 without even removing the journal, as long as a recent version of e2fsprogs (such as the one shipped in this release) is installed.
ext3 benefits from the long history of fixes and enhancements to the ext2 file system, and will continue to do so. This means that ext3 shares ext2's well-known robustness, but also that new features are added to ext2, they can be carried over to ext3 with little difficulty. When, for example, extended attributes or HTrees are added to ext2, it will be relatively easy to add them to ext3. (The extended attributes feature will enable things like access control lists; HTrees make directory operations extremely fast and highly scalable to very large directories.)
ext3, like ext2, has a multi-vendor team of developers who develop it and understand it well; its development does not depend on any one person or organisation.
ext3 provides and makes use of a generic journaling layer (jbd) which can be used in other contexts, and can journal not only within the file system, but also to other devices, so as NVRAM devices become available and supported under Linux, ext3 will be able to support them.
ext3 has multiple journaling modes. It can journal all file data and metadata (data=journal), or it can journal metadata but not data (data=ordered or data=writeback). When not journaling file data, you can choose whether to write file system data before metadata (data=ordered; causes all metadata to point to valid data) or not handle file data specially at all (data=writeback; file system will be consistent, but old data may appear in files after an unclean system shutdown). This gives the administrator the power to make the trade off between speed and file data consistency, and to tune speed for specialized usage patterns.
ext3 has broad cross-platform compatibility, working on 32 and 64 bit architectures, and on both little-endian and big-endian systems. Any system (currently including many Unix clones and variants, BeOS, and Windows) capable of accessing files on an ext2 file system will also be able to access files on an ext3 file system.
ext3 does not require extensive core kernel changes and requires no new system calls, thus presenting Linus no challenges to integrating ext3 into his official Linux kernel releases; ext3 is already integrated into Alan Cox's -ac kernels, slated for migration to Linus's official kernel soon.
The e2fsck file system recovery program has a long and proven track record of successful data recovery when software or hardware faults corrupt a file system. ext3 uses this same e2fsck code for salvaging the file system after such corruption so it has the same robustness against catastrophic data loss as ext2 in the presence of data-corruption faults.
Again, we don't claim that every one of these points are unique to ext3. Most of them are shared by at least one other filesystem. We merely claim that the set of all of them together is true only for ext3.
Here are some of the things Red Hat has done to ensure that ext3 is safe for users to use for their data:
We have done extensive stress testing under a large set of configurations. This has involved many thousands of hours of "contrived" load testing on a wide variety of hardware and file system configurations, as well as many use case tests.
We have audited ext3 for multiple conditions, including memory allocation errors happening at any point. We have tested that by forcing false errors and testing file system consistency.
We audited and tested ext3 for poor interactions with the VM subsystem, finding and fixing several interactions. A journaling file system puts more stress on the VM subsystem, and we found and fixed bugs both in ext3 and in the VM subsystem in the process of this audit and these tests. After thousands of hours of this testing, we are extremely confident in the robustness of the ext3 file system.
We have done an extensive year-long-plus beta program, starting with ext3 on the 2.2 kernel series, and then moving forwards to the 2.4 kernel series. Even before the official beta program, ext3 was put into production use in some circumstances; ext3 has been in production use on some widely-accessed servers, including the rpmfind.net servers, for over two years.